WO2018120038A1 - Method and device for target detection - Google Patents

Abstract

A method and device for target detection, related to the technical field of artificial intelligence, applicable in a process of target detection. To solve the problem of limited precision in target detection, the method comprises: acquiring an image to be detected and the depth values of multiple pixel points in the image to be detected (101); and combining the depth values of the pixel points for target detection with respect to the image to be detected (102).

Description

Method and device for detecting target Technical field

The present application relates to the field of artificial intelligence technologies, and in particular, to a method and apparatus for target detection.

Background technique

The existing method of target detection is generally divided into three stages: first, candidate regions are selected in a given image; then feature extraction is performed on these candidate regions, and finally the candidate regions are classified using the trained classifier, and then all the detected regions are detected. Possible target object. When the candidate region is selected in the first stage, the current mainstream target detection method generally uses information such as texture, edge, color, and the like in the image to find out possible locations of the target in the image, and extract hundreds of points. Thousands of candidate areas. The above candidate region-based target detection method has the following disadvantages:

1. Subsequent feature extraction for hundreds to thousands of candidate regions is not only computationally intensive, but the number of candidate regions generally exceeds the number of objects contained in the image, making many calculations redundant.

2. The two-dimensional spatial positions of some objects in the image are adjacent, but their depth values are different. Because the colors and textures are similar, it is likely to be extracted as a whole into the same candidate region, which affects the accuracy of target detection.

Summary of the invention

The embodiment of the present application provides a target detection method and device, which mainly solves the problem that the target detection method in the prior art has limited target detection precision and large calculation amount.

To achieve the above objective, the embodiment of the present application adopts the following technical solutions:

In a first aspect, the present application provides a target detection method, including: acquiring a to-be-detected image and a depth value of a plurality of pixel points in the image to be detected; and performing target detection on the detected image in combination with the depth value of each pixel.

In a second aspect, the present application provides a target detecting apparatus, including: an acquiring unit, a depth value for acquiring a plurality of pixels in the image to be detected and the image to be detected; and a target detecting unit configured to perform target detection on the image to be detected in combination with the depth value of each pixel acquired by the acquiring unit.

In a third aspect, the present application provides a computer storage medium for storing computer software instructions, comprising program code designed to perform the object detection method of the first aspect.

In a fourth aspect, the present application provides a computer program product that can be directly loaded into an internal memory of a computer and includes software code, and the computer program can be loaded and executed by a computer to implement the target detection method according to the first aspect. .

In a fifth aspect, the present application provides an electronic device, including: a memory, a communication interface, and a processor, the memory is configured to store computer executable code, and the processor is configured to execute the computer executable code control to perform the first aspect The target detecting method is used for data transmission between the electronic device and an external device.

In a sixth aspect, the present application provides a robot, including the electronic device of the fifth aspect.

The solution provided by the present application first acquires a depth value of a plurality of pixels in the image to be detected and the image to be detected when performing target detection; and performs target detection on the detected image in combination with the depth value of each pixel. Compared with the prior art image-based texture, edge, color and other information for target detection, the present application considers the depth value when performing target detection, so the color and texture can be similar but different in depth when the target is detected. The target object is distinguished, which in turn can improve the accuracy of the target detection.

DRAWINGS

In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings to be used in the embodiments or the prior art description will be briefly described below. Obviously, the drawings in the following description are only It is a certain embodiment of the present application, and other drawings can be obtained according to the drawings without any creative work for those skilled in the art.

FIG. 1 is a schematic structural diagram of a target detection system according to an embodiment of the present application;

2 is a schematic flowchart of a target detection method according to an embodiment of the present application;

FIG. 3 is a schematic diagram of an image to be detected, a depth image, and an initial candidate region obtained by using the method shown in FIG. 2 according to an embodiment of the present disclosure;

FIG. 4 is a schematic flowchart diagram of another object detection method according to an embodiment of the present application;

5 is a schematic diagram of splitting an initial candidate region shown in FIG. 3 to obtain a target candidate region by using the method shown in FIG. 4 according to an embodiment of the present application;

FIG. 6 is a schematic flowchart diagram of still another object detection method according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a target detecting device according to an embodiment of the present application;

FIG. 8 is a schematic structural diagram of another object detecting apparatus according to an embodiment of the present application;

FIG. 9 is a schematic structural diagram of an electronic device according to an embodiment of the present application.

detailed description

The system architecture and the service scenario described in the embodiments of the present application are for the purpose of more clearly explaining the technical solutions of the embodiments of the present application, and do not constitute a limitation of the technical solutions provided by the embodiments of the present application. The technical solutions provided by the embodiments of the present application are equally applicable to similar technical problems.

It should be noted that, in the embodiments of the present application, the words "exemplary" or "such as" are used to mean an example, an illustration, or a description. Any embodiment or design described as "exemplary" or "for example" in the embodiments of the present application should not be construed as preferred or advantageous over other embodiments or designs. Rather, the use of the words "exemplary" or "such as" is intended to present the concepts in a particular manner.

It should be noted that, in the embodiment of the present application, "(English: of)", "corresponding (relevant)" and "corresponding" may sometimes be mixed. It should be noted that When the difference is not emphasized, the meaning to be expressed is the same.

The embodiment of the present application provides a target detection system. As shown in FIG. 1 , the system includes: an image collection device 11 and a target detection device 12 . The image capturing device 11 is configured to perform image collection on the area to be detected and send the collected image to the target detection. Device 12, Illustratively, the image acquisition device 11 may be a single or multiple cameras capable of acquiring two-dimensional information of an image or a binocular camera capable of acquiring three-dimensional information or the like. The target detecting device 12 is configured to perform an analysis process on the image received from the image capturing device 11 to perform target detection using the received image, and the target detecting device 12 may be a device having a processing function, such as a server or the like. The specific implementation of the image capturing device 11 and the target detecting device 12 can refer to the prior art, and details are not described herein again.

The embodiment of the present application provides a target detection method, which can be applied to the system shown in FIG. 1. When applied to the system shown in FIG. 1, the execution body of the method may be the target detection device 12 shown in FIG. . The following description is made with the execution subject as the target detecting device 12, as shown in FIG. 2, the method includes:

Step 101: Acquire an image to be detected and a depth value of a plurality of pixel points in the image to be detected.

The image to be detected is an image obtained by photographing the area to be detected, and may be an image obtained directly, or may be an image obtained by performing gradation, denoising, or the like on the directly obtained color image.

Exemplarily, an image obtained by photographing a detection area using an ordinary camera or a mobile phone having an imaging function can be used as an image to be detected referred to in the present application.

The depth value may be obtained by acquiring a depth image corresponding to the image to be detected; and determining a depth value of each pixel of the image to be detected from the depth image.

The depth image is also referred to as a range image, and refers to a distance (or depth) from an image collector, such as a binocular camera, to each point in the area to be detected as a pixel value. The image can directly reflect the geometry of the visible surface of the object, that is, the contour of each object can be directly determined. In the depth image, each pixel represents the distance from the object to the camera plane at the particular (x,y) coordinate in the field of view of the image collector. Therefore, each pixel point in the depth image corresponds to the depth value and is used to represent the depth value of each object in the area to be detected. Common depth image acquisition methods include lidar depth imaging, computer stereo vision imaging, coordinate measuring machine method, moiré fringe method, structured light method, etc. The existing technology is tested and will not be described here.

Step 102: Perform target detection on the detected image in combination with the depth value of each pixel.

In an implementation manner of step 102, pixels in the image to be detected whose depth values are in the same range may be divided into the same candidate region.

Exemplarily, as shown in FIG. 3, the present application respectively shows an image to be detected, a depth image, and a candidate region 1 and a candidate region 2 which are divided after performing step 102.

After the candidate region is determined according to the depth value of the pixel, the candidate region may be directly extracted, and finally the candidate region is classified by using the trained classifier, thereby achieving target detection. For the specific implementation process of this step, reference may be made to the prior art, and details are not described herein again.

In another implementation manner of the step 102, the candidate region determined according to the depth value may be used as an initial candidate region, and the initial candidate region is further split to obtain a target candidate region, as shown in FIG. Can be implemented as:

Step 201: Divide pixel points whose depth values are in the same range into the same initial candidate area.

Step 202: Split the initial candidate region into target candidate regions according to image features.

Step 203: Determine a target in the target candidate area.

The image feature includes any one or any of the following features: any one or more of a color feature, a texture feature, a structural feature, a face feature, or a contour feature. For the specific implementation of splitting the initial candidate region according to the image feature to obtain the target candidate region, reference may be made to the implementation process of region proposal for one region in the prior art.

For example, when the image feature is a contour feature, the step 202 may be specifically implemented to: respectively detect contour lines in each initial candidate region; and include at least two independent closed contours in the presence of at least one initial candidate region. At the time of the line, each of the at least one initial candidate region is split to obtain at least two target candidate regions such that at most one closed contour is included in each of the target candidate regions.

Referring to the plurality of initial candidate regions obtained in FIG. 3, after the initial candidate regions are split according to the contour features, the initial candidate region 1 can be split into two target candidate regions, and the target candidate regions as shown in FIG. 5 are obtained. .

For example, when the image feature is a contour feature, the step 202 may be specifically implemented to detect a color feature in each initial candidate region. When there are at least two colors included in the at least one initial candidate region, each of the at least one initial candidate region is respectively split into at least two target candidate regions such that each target candidate region includes one Color characteristics.

It should be noted that the above description only takes the outline feature and the color feature as an example. In the actual application, when the target candidate region is divided, a plurality of features such as color and texture may be combined.

In still another implementation of step 102, as shown in FIG. 6, the method includes:

Step 301: Divide the image to be detected into initial candidate regions according to image features.

The image feature includes any one or any of the following features: any one or more of a color feature, a texture feature, a structural feature, a face feature, or a contour feature. The specific implementation of the method for dividing the image to be detected according to the image feature to obtain the initial candidate region may refer to the implementation process of region proposal for one region in the prior art.

Step 302: Remove the initial candidate region where the depth value of the pixel is not in the same range.

Step 303: Determine a target in the remaining initial candidate regions.

The solution provided by the present application first acquires a depth value of a plurality of pixels in the image to be detected and the image to be detected when performing target detection; and performs target detection on the detected image in combination with the depth value of each pixel. Compared with the prior art image-based texture, edge, color and other information for target detection, the present application considers the depth value when performing target detection, so the color and texture can be similar but different in depth when the target is detected. The target object is distinguished, which in turn can improve the accuracy of the target detection.

In addition, in general, the depth information contained in the area to be detected is in most cases smaller than the color information contained therein, for example, an object may include multiple colors, but the depth value corresponding to the entire object may be only one, and therefore, according to Depth value division The amount of calculation of the candidate area is also greatly reduced.

Optionally, after the target is identified, the target detecting device 12 may also output information such as a category, an outline, and the like of the target.

The method provided by the embodiment of the present application can be used in a scenario in which target recognition is required, for example, in a mobile robot, and the mobile robot can automatically identify an object in its environment to make a decision by using the method of the present application. It can also be applied in the process of assisting users in finding a specific target object. The method provided by this application can be applied to any process that requires detection and identification of an object.

Those skilled in the art will readily appreciate that the present application can be implemented in a combination of hardware or hardware and computer software in combination with the elements and algorithm steps of the various examples described in the embodiments disclosed herein. Whether a function is implemented in hardware or computer software to drive hardware depends on the specific application and design constraints of the solution. A person skilled in the art can use different methods to implement the described functions for each particular application, but such implementation should not be considered to be beyond the scope of the present application.

The embodiment of the present application may perform the division of the function module on the target detection device or the like according to the above method example. For example, each function module may be divided according to each function, or two or more functions may be integrated into one processing module. The above integrated modules can be implemented in the form of hardware or in the form of software functional modules. It should be noted that the division of the module in the embodiment of the present application is schematic, and is only a logical function division, and the actual implementation may have another division manner.

FIG. 7 is a schematic diagram showing a possible structure of the target detecting device involved in the foregoing embodiment. The target detecting device includes: an obtaining unit 401, and a target detecting unit 402. . The obtaining unit 401 is configured to support the target detecting device to perform the process 101 in FIG. 2; the target detecting unit 402 is configured to support the target detecting device to perform the process 102 in FIG. 2, the processes 201, 202, 203 in FIG. 4, in FIG. Processes 301, 302, 303. All the related content of the steps involved in the foregoing method embodiments may be referred to the functional descriptions of the corresponding functional modules, and details are not described herein again.

In the case of using an integrated unit, FIG. 8 shows the involved in the above embodiment. A possible structural schematic of the target detection device. The target detecting device includes a processing module 501 and a communication module 502. The processing module 501 is configured to perform control management on the action of the target detecting device. For example, the processing module 501 is configured to support the target detecting device to perform the processes 101, 102, and 103 in FIG. 2, and the processes 201, 202, and 203 in FIG. Processes 301, 302, 303 in 6, and/or other processes for the techniques described herein. The communication module 502 is configured to support communication of the target detection device with other network entities, such as with the functional modules or network entities shown in FIG. The target detecting device may further include a storage module 503 for storing program codes and data of the target detecting device.

The processing module 501 can be a processor or a controller, for example, a central processing unit (CPU), a general-purpose processor, a digital signal processor (DSP), and an application-specific integrated circuit (Application-Specific Integrated Circuit (ASIC), Field Programmable Gate Array (FPGA) or other programmable logic device, transistor logic device, hardware component, or any combination thereof. It is possible to implement or carry out the various illustrative logical blocks, modules and circuits described in connection with the present disclosure. The processor may also be a combination of computing functions, for example, including one or more microprocessor combinations, a combination of a DSP and a microprocessor, and the like. The communication module 502 can be a transceiver, a transceiver circuit, a communication interface, or the like. The storage module 503 can be a memory.

When the processing module 501 is a processor, the communication module 502 is a communication interface, and the storage module 503 is a memory, the target detection device according to the embodiment of the present application may be the electronic device shown in FIG.

Referring to FIG. 9, the electronic device includes a processor 601, a communication interface 602, a memory 603, and a bus 604. The communication interface 602, the processor 601, and the memory 603 are connected to each other through a bus 604. The bus 604 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. Wait. The bus can be divided into an address bus, a data bus, a control bus, and the like. For ease of representation, only one thick line is shown in Figure 9, but it does not mean that there is only one bus or one type. bus.

The steps of a method or algorithm described in connection with the present disclosure may be implemented in a hardware or may be implemented by a processor executing software instructions. The software instructions may be composed of corresponding software modules, which may be stored in a random access memory (RAM), a flash memory, a read only memory (ROM), an erasable programmable read only memory ( Erasable Programmable ROM (EPROM), electrically erasable programmable read only memory (EEPROM), registers, hard disk, removable hard disk, compact disk read only (CD-ROM) or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor to enable the processor to read information from, and write information to, the storage medium. Of course, the storage medium can also be an integral part of the processor. The processor and the storage medium can be located in an ASIC.

Those skilled in the art will appreciate that in one or more examples described above, the functions described herein can be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the functions may be stored in a computer readable medium or transmitted as one or more instructions or code on a computer readable medium. Computer readable media includes both computer storage media and communication media including any medium that facilitates transfer of a computer program from one location to another. A storage medium may be any available media that can be accessed by a general purpose or special purpose computer.

The specific embodiments of the present invention have been described in detail with reference to the specific embodiments of the present application. It is to be understood that the foregoing description is only The scope of protection, any modifications, equivalent substitutions, improvements, etc. made on the basis of the technical solutions of the present application are included in the scope of protection of the present application.

Claims (14)

A target detection method, comprising: Obtaining a to-be-detected image and a depth value of a plurality of pixel points in the image to be detected; Target detection is performed on the detected image in combination with the depth value of each pixel. The method according to claim 1, wherein the obtaining the depth values of the plurality of pixel points in the image to be detected comprises: Obtaining a depth image corresponding to the image to be detected; A depth value of each pixel point of the image to be detected is determined from the depth image. The method according to claim 1, wherein the detecting the image to be detected by combining the depth values of the respective pixel points comprises: Pixel points whose depth values are in the same range are divided into the same initial candidate area; Splitting the initial candidate region into target candidate regions according to image features; The target is determined in the target candidate area. The method according to claim 1, wherein the detecting the image to be detected by combining the depth values of the respective pixel points comprises: Dividing the image to be detected into initial candidate regions according to image features; Removing the initial candidate region where the depth value of the pixel is not in the same range; The target is determined in the remaining initial candidate regions. The method according to claim 3 or 4, wherein the image feature comprises any one or any of the following features: any of a color feature, a texture feature, a structural feature, a face feature, or a contour feature. One or more. A target detecting device, comprising: An acquiring unit, configured to acquire an image to be detected and a depth value of a plurality of pixel points in the image to be detected; The target detecting unit is configured to perform target detection on the image to be detected in combination with the depth value of each pixel point acquired by the acquiring unit. The apparatus according to claim 6, wherein the acquiring unit is configured to acquire a depth image corresponding to the image to be detected, and determine a depth value of each pixel of the image to be detected from the depth image. The device of claim 6 wherein: The target detecting unit is further configured to divide pixel points whose depth values are within the same range into the same initial candidate area; split the initial candidate area into target candidate areas according to image features; and determine targets in the target candidate areas. The device of claim 6 wherein: The target detecting unit is further configured to divide the image to be detected into an initial candidate region according to the image feature; remove the initial candidate region in which the depth value of the pixel is not in the same range; and determine the target in the remaining initial candidate region. The apparatus according to claim 8 or 9, wherein the image feature comprises any one or any of the following features: any one of a color feature, a texture feature, a structural feature, a face feature, or a contour feature. One or more. An electronic device, comprising: a memory, a communication interface for storing computer executable code, and a processor for executing the computer executable code control to perform any of claims 1-5 A target detection method, wherein the communication interface is used for data transmission between the electronic device and an external device. A robot comprising the electronic device of claim 11. A computer storage medium, characterized by storing computer software instructions comprising program code designed to perform the object detection method of any of claims 1-5. A computer program product, which can be directly loaded into an internal memory of a computer and containing software code, which can be loaded and executed by a computer to achieve the object of any one of claims 1-5 Detection method.

Images