WO2025022706A1 - Object detection system, object detection device, and object detection method - Google Patents

Abstract

The present invention pertains to object detection based on point cloud data acquired by scanning a monitoring range, and provides a mechanism whereby both a stationary object and a moving object can be suitably detected. An object detection system according to one embodiment of the present invention comprises: a LiDAR 10 that acquires point cloud data by scanning a monitoring range; and an object detection server 20 that executes an object detection process on the basis of the point cloud data acquired by the LiDAR 10. In accordance with the setting of a number of accumulated frames for each area obtained by dividing the monitoring range, the object detection server 20 accumulates point cloud data of each area in the monitoring range for each among the number of accumulated frames set for said area and executes the object detection process on the basis of the accumulated point cloud data.

Description

OBJECT DETECTION SYSTEM, OBJECT DETECTION DEVICE, AND OBJECT DETECTION METHOD

The present invention relates to an object detection system based on point cloud data acquired by scanning a monitoring area.

Conventionally, object detection systems have been developed that use LiDAR (Light Detection and Ranging) to detect objects that exist within a surveillance area. Figure 1 shows an example of the configuration of an object detection system. Figure 2 shows an image of how the object detection system works.

The object detection system in FIG. 1 includes a LiDAR 10 and an object detection server 20. As shown in FIG. 2, the LiDAR 10 acquires point cloud data representing the three-dimensional position and shape of an object 30 present in the monitoring range by scanning and irradiating a laser light onto the monitoring range and measuring the arrival time of the reflected light. The point cloud data acquired by the LiDAR 10 is transmitted to the object detection server 20 via a network NW. The object detection server 20 executes an object detection process to detect an object 30 present in the monitoring range based on the point cloud data received from the LiDAR 10.

Ken Nishida and 3 others, "Verification of LIDAR specifications for autonomous driving", Proceedings of the 32nd Fuzzy Systems Society Fuzzy System Symposium FSS (Saga University, 2016), pp. 147-150

The resolution of the laser light emitted from a LiDAR is determined by the LiDAR's irradiation pattern and the individual LiDAR. The higher the resolution of the laser light, the higher the density of the point cloud data that can be acquired, making it possible to detect smaller or more distant objects. On the other hand, if the resolution of the laser light is low, the density of the point cloud data that can be acquired will be low, making it difficult to detect small or distant objects. This is because conventional object detection systems detect objects using one frame of point cloud data acquired by LiDAR at a time.

Figure 3 shows an example of one frame of point cloud data obtained by LiDAR for an object. In the example of Figure 3, the density of the point cloud data is low, so the object cannot be accurately identified. As a solution to this problem, rather than processing the point cloud data obtained by LiDAR on a frame-by-frame basis, it has been considered to accumulate and overlay multiple frames of point cloud data before processing (see, for example, Non-Patent Document 1).

Object detection through accumulation (overlapping) of point cloud data will be explained with reference to Figures 4 and 5. Figure 4 shows an example of point cloud data from the first to fourth frames obtained for an object by LiDAR. The point cloud data obtained by LiDAR is acquired with a slight shift in the coordinates of the point cloud data in successive frames, as shown in Figure 4, even when the object is stationary. Therefore, by accumulating and overlaying point cloud data from multiple frames, as shown in Figure 5, the density of the point cloud data can be artificially increased, allowing for more accurate object detection and discrimination.

The accumulation (overlay) of point cloud data described above is effective for stationary objects (hereafter referred to as "stationary objects"), but is less effective for moving objects, especially objects that move a lot (hereafter referred to as "moving objects"). This is because when observing a moving object with LiDAR, only point cloud data with a scattered distribution area can be obtained in each frame, so even if multiple frames are accumulated and overlaid, only point cloud data distributed over a wide area can be obtained. This problem can be addressed by using high-performance (high-resolution) LiDAR or by increasing the number of LiDAR units installed, but this is hardware-dependent and expensive.

The present invention was made in consideration of the above-mentioned conventional circumstances, and aims to provide a mechanism for object detection based on point cloud data acquired by scanning a monitoring range, capable of effectively detecting both stationary and moving objects.

In order to achieve the above object, an object detection system according to one aspect of the present invention is configured as follows. That is, in an object detection system including a point cloud data acquisition device that scans a monitoring range to acquire point cloud data, and an object detection device that executes object detection processing based on the point cloud data acquired by the point cloud data acquisition device, the object detection device accumulates point cloud data for each area within the monitoring range by the number of accumulation frames set for that area, in accordance with the setting of the number of accumulation frames for each area into which the monitoring range is divided, and executes the object detection processing based on the accumulated point cloud data.

Here, in the above object detection system, the number of accumulated frames for each area can be set to a value according to the expected speed of the object to be detected in each area.

In addition, in the above object detection system, the number of accumulated frames for each area can be set to a different value between the time period when the presence of an object to be detected in each area is expected and other time periods.

Furthermore, in the above object detection system, the object detection device can be configured to re-execute the object detection process for an area in which an object has been detected by the object detection process, based on point cloud data that has accumulated a number of accumulated frames that is greater than the number set for that area.

An object detection device according to another aspect of the present invention is configured as follows. That is, in an object detection device that performs object detection processing based on point cloud data acquired by scanning a monitoring range, the device accumulates point cloud data for each area within the monitoring range by the number of accumulated frames set for that area according to the setting of the number of accumulated frames for each area into which the monitoring range is divided, and performs the object detection processing based on the accumulated point cloud data.

An object detection method according to yet another aspect of the present invention is configured as follows. That is, in the object detection method based on point cloud data acquired by scanning a monitoring range, an object detection device accumulates point cloud data for each area within the monitoring range by the number of accumulated frames set for that area in accordance with the setting of the number of accumulated frames for each area into which the monitoring range is divided, and performs object detection processing based on the accumulated point cloud data.

The present invention provides a mechanism for object detection based on point cloud data acquired by scanning a monitoring range, which can effectively detect both stationary and moving objects.

FIG. 1 is a diagram illustrating an example of the configuration of an object detection system. FIG. 2 is a diagram illustrating an operation image of the object detection system illustrated in FIG. 1 . FIG. 2 is a diagram showing an example of one frame of point cloud data obtained regarding an object. FIG. 2 is a diagram showing an example of point cloud data of the first to fourth frames obtained regarding an object. FIG. 13 is a diagram showing an example of point cloud data obtained by overlapping four frames. FIG. 2 is a diagram showing an example of division of a monitoring range in the proposed method. FIG. 11 is a diagram showing an example of data on the number of accumulated frames by area in the proposed method. FIG. 2 is a diagram illustrating an example of a processing flow in the proposed method.

One embodiment of the present invention will be described with reference to the drawings. The general configuration of an object detection system according to one embodiment of the present invention is the same as the object detection system shown in FIG. 1. That is, the object detection system of this example has a LiDAR 10, which is an example of a point cloud data acquisition device according to the present invention, and an object detection server 20, which is an example of an object detection device according to the present invention.

LiDAR10 acquires point cloud data representing the three-dimensional position and shape of object 30 present within the monitoring range by scanning and irradiating the monitoring range with laser light and measuring the arrival time of the reflected light. The point cloud data acquired by LiDAR10 is transmitted to object detection server 20 via network NW. Based on the point cloud data received from LiDAR10, object detection server 20 executes object detection processing to detect object 30 present within the monitoring range. In the object detection processing, for example, the point cloud data is processed using background subtraction to determine the size, shape, etc. of the detected object, and the detected object is identified based on the results. In addition, the detected object may be identified by matching with previously registered object shapes or by an AI model that has learned the object shape.

In addition, instead of LiDAR 10, other point cloud data acquisition devices such as 3D sensors capable of acquiring point cloud data representing the three-dimensional position and shape of objects present in the monitoring range may be used. Also, although only one LiDAR 10 is shown in FIG. 1, multiple LiDAR 10 may be provided and the object detection server 20 may execute object detection processing based on the point cloud data acquired by each LiDAR 10. The object detection server 20 may be realized, for example, by a computer equipped with hardware resources such as a processor and memory, and may be configured to read programs related to each function according to the present invention from the memory and execute them by the processor. Also, the object detection server 20 may be realized by one computer, or by multiple computers connected to each other so that they can communicate with each other.

In this proposed method, the object detection server 20 divides the monitoring range into multiple areas and executes object detection processing with a different number of accumulated frames for each area. For example, as shown in FIG. 6, the monitoring range is divided into an air area whose main purpose is to detect stationary objects (flying objects caught on utility poles or electric wires) and a ground area whose main purpose is to detect moving objects (passing people and vehicles).

FIG. 7 shows an example of area-specific accumulated frame number data used for the above control. In the example of FIG. 7, the number of accumulated frames for the "sky area" identified by area ID="1" is set to "4", and the number of accumulated frames for the "ground area" identified by area ID="2" is set to "1". Such area-specific accumulated frame number data is stored in the internal memory of the object detection server 20 or in an external device accessible to the object detection server 20. The area-specific accumulated frame number data may be set manually by a user of the object detection system, or may be set automatically by a device such as the object detection server 20 according to the distribution of stationary and moving objects analyzed from past point cloud data.

The object detection server 20 accumulates point cloud data for each area within the monitoring range by the number of accumulated frames set for that area according to the area-specific accumulated frame count data described above, and performs object detection processing based on the accumulated point cloud data. When using the area-specific accumulated frame count data of FIG. 7, the object detection server 20 will perform object detection processing for the sky area based on point cloud data accumulated and superimposed over four frames, and for the ground area based on one frame of point cloud data.

FIG. 8 shows an example of a processing flow in the proposed method.
First, the object detection server 20 initializes (sets to 0) the frame counter for each area and initializes (clears) the accumulated point cloud data for each area (step S11). After that, when the object detection server 20 receives one frame of point cloud data from the LiDAR 10 (step S12), it performs the following processing (steps S13 to S17) for each area.

First, the frame counter of area n (where 1≦n≦number of areas) is incremented (added by 1) (step S13), and the corresponding portion of the received point cloud data is added to the accumulated point cloud data of area n (step S14). Next, it is determined whether the frame counter of area n has reached the accumulated frame number (step S15). If the frame counter of area n has reached the accumulated frame number (step S15; Yes), object detection processing is performed based on the accumulated point cloud data of area n (step S16), and the frame counter and accumulated point cloud data of area n are reset (step S17), and processing of the next area is started. On the other hand, if the frame counter of area n has not reached the accumulated frame number (step S15; Yes), processing of steps S16 to S17 is not performed, and processing of the next area is started. If processing of all areas is completed, the system waits until the reception of the next frame of point cloud data (step S12).

In the processing flow of Figure 8, object detection processing in the sky area is performed every four frames, but by storing point cloud data for the number of accumulated frames as history, it is also possible to perform object detection processing for each frame (each time point cloud data is received from LiDAR) by overlaying the point cloud data from the past four frames.

As described above, the object detection system of this example includes LiDAR 10 that scans the monitoring range to acquire point cloud data, and object detection server 20 that executes object detection processing based on the point cloud data acquired by LiDAR 10. Object detection server 20 accumulates point cloud data for each area within the monitoring range by the number of accumulation frames set for that area according to the setting of the number of accumulation frames for each area into which the monitoring range is divided, and executes object detection processing based on the accumulated point cloud data.

As a result, for example, in the air area, object detection processing can be performed based on point cloud data accumulated and overlaid over four frames, and in the ground area, object detection processing can be performed based on point cloud data for one frame. As a result, in the air area, flying objects caught on utility poles or power lines can be detected with high accuracy, and in the ground area, people and vehicles passing by can be detected with high accuracy. In this way, it becomes possible to effectively detect both stationary and moving objects. Moreover, because this can be achieved through ingenuity in software rather than hardware, more advanced detection is possible while keeping costs down for the entire system.

In the above explanation, the number of accumulated frames in the air area, where the main purpose is to detect stationary objects, is set to 4, and the number of accumulated frames in the ground area, where the main purpose is to detect moving objects, is set to 1, but this is merely an example. For example, the number of accumulated frames for each area may be set to a value according to the expected speed of the object to be detected in each area. As one example, the number of accumulated frames in an area, where the main purpose is to detect slow-moving moving objects, may be set to 2, and the number of accumulated frames in an area, where the main purpose is to detect fast-moving moving objects, may be set to 1.

For example, the number of accumulated frames for each area may be set to a different value depending on the time of day when the presence of an object to be detected in each area is expected and other time of day. As one example, in each area, the number of accumulated frames may be set to a low value during time periods when pedestrians are expected to pass by, and to a high value during time periods when pedestrians are not expected to pass by.

For example, the number of accumulated frames for each area may be set to a value according to the LiDAR sampling period. As an example, the number of accumulated frames may be increased when the LiDAR sampling period is short, and decreased when the sampling period is long.

For example, the number of accumulated frames for each area may be set to a value according to the urgency of detection in each area (degree of need for immediate detection). As one example, the number of accumulated frames may be increased in areas where the urgency of detection is low, and decreased in areas where the urgency of detection is high.

In the above explanation, the monitoring range is divided into two areas, but it may be divided into three or more areas. For example, when monitoring the inside of a station, it may be divided into an overhead area, an area where passengers walk (platform area), and an area where trains run (railroad track area), and a different number of accumulated frames may be set for each area.

As a variation of the object detection system described above, the object detection server 20 may re-execute the object detection process for an area in which an object has been detected by the object detection process, based on point cloud data that has accumulated a number of frames greater than the number set for that area. For example, if an object is detected by executing the object detection process for a ground area based on one frame of point cloud data, the object detection process is re-executed for the ground area based on four frames of point cloud data. This makes it possible to further improve the accuracy of object detection.

The above describes the embodiments of the present invention, but these embodiments are merely illustrative and do not limit the technical scope of the present invention. The present invention can take various other embodiments, and various modifications such as omissions and substitutions can be made without departing from the gist of the present invention. These embodiments and their modifications are included in the scope and gist of the invention described in this specification, etc., and are included in the scope of the invention described in the claims and their equivalents.

In addition, the present invention can be provided not only as the devices described above or as systems composed of these devices, but also as methods executed by these devices, programs for implementing the functions of these devices using a processor, and storage media for storing such programs in a computer-readable format.

The present invention can be used in an object detection system based on point cloud data acquired by scanning a monitoring area.

10: LiDAR, 20: Object detection server, 30: Object

Claims (6)

a point cloud data acquisition device that scans a monitoring range and acquires point cloud data;
and an object detection device that performs object detection processing based on the point cloud data acquired by the point cloud data acquisition device,
The object detection system is characterized in that the object detection device accumulates point cloud data for each area within the monitoring range by the number of accumulated frames set for that area in accordance with a setting of the number of accumulated frames for each area into which the monitoring range is divided, and performs the object detection process based on the accumulated point cloud data. 2. The object detection system according to claim 1,
An object detection system, characterized in that the number of accumulated frames for each area is set to a value corresponding to the estimated speed of an object to be detected in each area. 2. The object detection system according to claim 1,
An object detection system characterized in that the number of accumulated frames for each area is set to a different value between a time period during which an object to be detected in each area is expected to exist and other time periods. 2. The object detection system according to claim 1,
The object detection system is characterized in that the object detection device re-executes the object detection process for an area in which an object is detected by the object detection process based on point cloud data that has accumulated a number of accumulated frames that is greater than the number set for that area. 1. An object detection device that performs object detection processing based on point cloud data acquired by scanning a monitoring range,
An object detection device characterized by accumulating point cloud data for each area within the monitoring range by the number of accumulated frames set for that area in accordance with a setting of the number of accumulated frames for each area into which the monitoring range is divided, and performing the object detection process based on the accumulated point cloud data. 1. An object detection method based on point cloud data acquired by scanning a monitoring range, comprising:
An object detection method characterized in that an object detection device accumulates point cloud data for each area within the monitoring range by the number of accumulated frames set for that area, in accordance with a setting of the number of accumulated frames for each area into which the monitoring range is divided, and performs object detection processing based on the accumulated point cloud data.

Images