CN114611635A - Object identification method and device, storage medium and electronic device - Google Patents

Abstract

Embodiments of the present invention provide an object recognition method, device, storage medium, and electronic device, wherein the method includes: determining a target point cloud obtained by a first device photographing a target area at a target moment; The target sub-data included in the target data collected at the target moment, and the data characteristics of the target sub-data are determined; each pixel point included in the target sub-data is mapped to the point cloud coordinate system where the target point cloud is located, and multiple Target pixel points; determine point cloud features based on multiple target pixel points and target point clouds; fuse data features and point cloud features based on dimension parameters of data features to obtain fusion features; identify target objects based on fusion features. The invention solves the problem of inaccurate identification of objects existing in the related art, and achieves the effect of improving the accuracy of identification of objects.

Description

Object recognition method, device, storage medium and electronic device

technical field

Embodiments of the present invention relate to the field of computers, and in particular, to an object identification method, device, storage medium, and electronic device.

Background technique

In the related art, image recognition technology under monocular vision is usually used to realize object recognition. However, monocular vision technology also has some limitations, such as the lack of ability to measure distance, size of objects, and performance in low light, resulting in inaccurate recognition results.

It can be seen from this that there is a problem of inaccurate identification of objects in the related art.

For the above problems existing in the related art, no effective solution has been proposed yet.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide an object identification method, device, storage medium, and electronic device, so as to at least solve the problem of inaccurate identification of objects existing in the related art.

According to an embodiment of the present invention, an object recognition method is provided, including: determining a target point cloud obtained by photographing a target area by a first device at a target time; determining a target collected by a second device at the target time The target sub-data included in the data, and the data characteristics for determining the target sub-data, wherein the target data is the data obtained by the second device photographing the target area, and the second device The angle at which the target area is photographed is the same as the angle at which the first device photographs the target area, and the target sub-data is the data of the target object included in the target data; Each pixel is mapped to the point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points; point cloud features are determined based on a plurality of the target pixel points and the target point cloud; based on the data features The dimensional parameters of the data feature and the point cloud feature are fused to obtain a fused feature; the target object is identified based on the fused feature.

According to another embodiment of the present invention, an object identification device is provided, comprising: a first determination module for determining a target point cloud obtained by photographing a target area by a first device at a target moment; a second determination module, Used to determine the target sub-data included in the target data collected by the second device at the target moment, and determine the data characteristics of the target sub-data, wherein the target data is the second device to the target The data obtained by shooting the target area, the angle at which the second device shoots the target area is the same as the angle at which the first device shoots the target area, and the target sub-data is the target data including The data of the target object; the mapping module is used to map each pixel point included in the target sub-data to the point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points; the third determination module , for determining the point cloud feature based on a plurality of the target pixel points and the target point cloud; the fusion module, for fusing the data feature and the point cloud feature based on the dimension parameters of the data feature to obtain the fusion feature ; Recognition module for recognizing the target object based on the fusion feature.

According to yet another embodiment of the present invention, there is also provided a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, wherein, when the computer program is executed by a processor, any one of the above-mentioned items is implemented. the steps of the method.

According to yet another embodiment of the present invention, there is also provided an electronic device comprising a memory and a processor, wherein the memory stores a computer program, the processor is configured to run the computer program to execute any of the above Steps in Method Examples.

Through the present invention, the target point cloud obtained by the first device shooting the target area at the target time is determined, the target sub-data included in the target data collected by the second device in the target device is determined, and the target feature of the target sub-data is determined , where the target data is the data obtained by the second device shooting the target area, the angle at which the second device shoots the target area is the same as the angle at which the first device shoots the target area, and the target sub-data is the target included in the target data The data of the object, map each pixel included in the target data to the point cloud coordinate system where the target point cloud is located to obtain multiple target pixel points; determine the point cloud features according to the multiple target pixel points and the target point cloud, according to The dimensional parameters of the data features fuse the data features and point cloud features to obtain the fused features, and identify the target object according to the fused features. Since the target object can be identified according to the fusion feature that combines the point cloud feature and the data feature, that is, when identifying the target object, the data collected by multiple devices is fused, so the problem of inaccurate object recognition in the related technology can be solved. To achieve the effect of improving the accuracy of object recognition.

Description of drawings

1 is a block diagram of a hardware structure of a mobile terminal of an object identification method according to an embodiment of the present invention;

2 is a flowchart of an object identification method according to an embodiment of the present invention;

3 is a schematic diagram of a secondary network module according to an exemplary embodiment of the present invention;

4 is a flowchart of an object identification method according to a specific embodiment of the present invention;

FIG. 5 is a structural block diagram of an object identification apparatus according to an embodiment of the present invention.

Detailed ways

Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings and in conjunction with the embodiments.

It should be noted that the terms "first", "second" and the like in the description and claims of the present invention and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence.

The method embodiments provided in the embodiments of this application may be executed in a mobile terminal, a computer terminal, or a similar computing device. Taking running on a mobile terminal as an example, FIG. 1 is a block diagram of a hardware structure of a mobile terminal according to an object identification method according to an embodiment of the present invention. As shown in FIG. 1 , the mobile terminal may include one or more (only one is shown in FIG. 1 ) processors 102 (the processors 102 may include but are not limited to processing devices such as a microprocessor MCU or a programmable logic device FPGA) and a memory 104 for storing data, wherein the above-mentioned mobile terminal may also include a transmission device 106 and an input and output device 108 for communication functions. Those of ordinary skill in the art can understand that the structure shown in FIG. 1 is only a schematic diagram, which does not limit the structure of the above-mentioned mobile terminal. For example, the mobile terminal may also include more or fewer components than those shown in FIG. 1 , or have a different configuration than that shown in FIG. 1 .

The memory 104 can be used to store computer programs, for example, software programs and modules of application software, such as computer programs corresponding to the object identification method in the embodiment of the present invention. The processor 102 executes the computer programs stored in the memory 104 by running the computer programs. Various functional applications and data processing implement the above method. Memory 104 may include high-speed random access memory, and may also include non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory 104 may further include memory located remotely from the processor 102, and these remote memories may be connected to the mobile terminal through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

Transmission device 106 is used to receive or transmit data via a network. The specific example of the above-mentioned network may include a wireless network provided by a communication provider of the mobile terminal. In one example, the transmission device 106 includes a network adapter (Network Interface Controller, NIC for short), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device 106 may be a radio frequency (Radio Frequency, RF for short) module, which is used to communicate with the Internet in a wireless manner.

A method for recognizing an object is provided in this embodiment. FIG. 2 is a flowchart of the method for recognizing an object according to an embodiment of the present invention. As shown in FIG. 2 , the process includes the following steps:

Step S202, determining the target point cloud obtained by the first device photographing the target area at the target moment;

Step S204: Determine the target sub-data included in the target data collected by the second device at the target moment, and determine the data characteristics of the target sub-data, wherein the target data is the second device to the target data. The data obtained by shooting the target area, the angle at which the second device shoots the target area is the same as the angle at which the first device shoots the target area, and the target sub-data is in the target data Include data on the target object;

Step S206, mapping each pixel point included in the target sub-data to the point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points;

Step S208, determining point cloud features based on a plurality of the target pixel points and the target point cloud;

Step S210, fuse the data feature and the point cloud feature based on the dimension parameter of the data feature to obtain a fusion feature;

Step S212, identifying the target object based on the fusion feature.

In the above embodiment, the first device may be a radar, such as a laser radar, a microwave radar, a millimeter-wave radar, or the like. The second device may be a camera device, such as a camera (monocular camera, multi-eye camera), a video recorder, and the like. The target data may be images, videos, etc. collected by a camera device. The first device and the second device may be installed at the same height, the same orientation, and adjacent positions, so that the first device and the second device have the same angle when photographing the target area. Of course, it is also possible to set the first device and the second device to have the same orientation and different positions. By adjusting the shooting angles of the first device and the second device, the shooting angles of the first device and the second device when shooting the target area are the same. .

，其中，

）为第i个对象的矩形框的目标点的像素坐标位置，

）为第i辆车的矩形框像素宽高。其中，目标点可以是矩形框的中心点、顶点等。In the above embodiment, the target sub-data may be data of the target object. When determining the target sub-data, an object detection algorithm may be used to perform 2D frame detection on the target data collected by the second device, and a rectangular frame is determined to be framed in the target data. The target object is selected, and the data included in the rectangular frame is determined as the target sub-data. The target sub-data may include position information of the rectangular frame, pixel size information of the rectangular frame, and the like. That is, the target subdata can be expressed as

,in,

) is the pixel coordinate position of the target point of the rectangular frame of the i-th object,

) is the pixel width and height of the rectangle frame of the i-th vehicle. Wherein, the target point may be the center point, vertex, etc. of the rectangular frame.

In the above embodiment, the first device and the second device may be devices that have completed joint calibration in advance, and may pre-calculate the camera extrinsic parameters that map the lidar coordinate system to the monocular camera coordinate system, so that the target point cloud obtained by the lidar scanning Can be correctly projected onto the camera image frame and mapped correctly. The target point cloud collected by the first device may include point clouds of multiple objects, that is, the target point cloud may include multiple sub-point clouds. The target data may include data of multiple objects, that is, the target data may include multiple target sub-data. Each pixel point included in the target sub-data can be mapped to the point cloud coordinate system where the target point cloud is located to obtain multiple target pixel points.

In the above embodiment, the point cloud feature may be determined according to the target pixel point and the target point cloud, and the data feature and the point cloud feature may be fused according to the dimension parameter of the data feature of the target sub-data to obtain the fusion feature. Input the fusion feature into the recognition network to recognize the target object. Wherein, identifying the target object includes identifying attribute information of the target object. Target objects may include vehicles, people, and the like. When the target object is a vehicle, the attribute information of the target object may include a vehicle model, a license plate, and the like. When the target object is a person, the attribute information of the target object may include information such as face information, ID number and the like.

。In the above embodiment, the fusion feature may be determined by the target network model, and the fusion feature may be transmitted to the auxiliary network model and the classifier included in the target network model to identify the target object. That is, after the fusion feature Fusion_Fea _i , the auxiliary network module and classifier can be connected, and the sgd stochastic gradient descent optimizer can be used to train and optimize the overall network. The auxiliary network module may include three consecutive 3x3 convolutional layers, followed by a 1x1 convolutional layer and an fc fully connected layer. For a schematic diagram of the auxiliary network module, see Figure 3. The classifier can be the cross entropy loss function loss

.

Optionally, the execution subject of the above steps may be a background processor, or other equipment with similar processing capabilities, or may be a machine at least integrated with data processing equipment, wherein the data processing equipment may include terminals such as computers and mobile phones, But not limited to this.

Through the present invention, the target point cloud obtained by the first device shooting the target area at the target time is determined, the target sub-data included in the target data collected by the second device in the target device is determined, and the target feature of the target sub-data is determined , where the target data is the data obtained by the second device shooting the target area, the angle at which the second device shoots the target area is the same as the angle at which the first device shoots the target area, and the target sub-data is the target included in the target data The data of the object, map each pixel included in the target data to the point cloud coordinate system where the target point cloud is located to obtain multiple target pixel points; determine the point cloud features according to the multiple target pixel points and the target point cloud, according to The dimensional parameters of the data features fuse the data features and point cloud features to obtain the fused features, and identify the target object according to the fused features. Since the target object can be identified according to the fusion feature that combines the point cloud feature and the data feature, that is, when identifying the target object, the data collected by multiple devices is fused, so the problem of inaccurate object recognition in the related technology can be solved. To achieve the effect of improving the accuracy of object recognition.

，其中，x、y、z为第j个点云在点In an exemplary embodiment, determining a point cloud feature based on a plurality of the target pixel points and the target point cloud includes: performing the following operations on each of the target pixel points to obtain a feature corresponding to each target pixel point Value: determine whether the target point cloud includes a first point with the same coordinates as the target pixel point; if there is a first point, the vertical coordinate of the first point and the corresponding The response intensity is determined as the characteristic value of the target pixel point; in the absence of the first point, the second point in the target point cloud that is closest to the target pixel point is determined, and the second point is The vertical coordinate of and the response intensity corresponding to the second point are determined as the eigenvalues of the target pixel point; a matrix formed by a plurality of the eigenvalues is determined as the point cloud feature. In this embodiment, the target point cloud may be a three-dimensional point cloud, and each point included in the target point cloud may be expressed as

, where x, y, and z are the jth point cloud at the point

维的三维矩阵，并按照被映射的像素坐标位置存储对应的点云点的z值和a值，对于那些没有被映射到的像素点，则用最近邻的z值和a值进行填充，即得到了每个对象的点云特征矩阵

。将点云特征矩阵确定为点云特征。The three-dimensional coordinates in the cloud coordinate system, a is the laser response intensity value of the jth point cloud. For each target object, you can build a

Dimensional three-dimensional matrix, and store the z value and a value of the corresponding point cloud point according to the mapped pixel coordinate position. For those pixels that are not mapped, use the nearest neighbor z value and a value to fill, that is Got the point cloud feature matrix for each object

. The point cloud feature matrix is determined as the point cloud feature.

In an exemplary embodiment, in the presence of a first point, determining the vertical coordinate of the first point and the response intensity corresponding to the first point as the feature value of the target pixel point includes: In the case that the number of the first point is one, the vertical coordinate of the first point and the response intensity corresponding to the first point are determined as the characteristic value of the target pixel point; in the number of the first point In the case of more than one, determine the third point with the smallest vertical coordinate included in the multiple first points, and determine the vertical coordinate of the third point and the response intensity corresponding to the third point as the target The eigenvalues of the pixels. In this embodiment, when mapping each pixel point to the point cloud coordinate system, each target sub-data will be mapped to several discrete point cloud points. If there are multiple point clouds mapped to the same image pixel, just take the point cloud with the smallest z value, and discard the others.

In an exemplary embodiment, determining the data features of the target sub-data includes: inputting the target sub-data into a target network model, and determining the features extracted by the target network model; The extracted feature is determined as the data feature, wherein the network layer is a network layer included in the backbone network of the target network model. In this embodiment, the target sub-data may be input into the target network model, such as convolutional neural network models CNN, VGG, ResNet, Mobilenet, etc., as the backbone network of the target network model. The deep network feature Camera_Fea _i of each target sub-data is extracted using the backbone network.

In the above embodiment, when the size of the input data is required by the backbone network, if the input size of the backbone network is set to 224*224*3, the target sub-data can be uniformly resized to 224*224*3.

In the above embodiment, the deep network feature _{Camera_Feai} extracted from any layer of the target network model may be selected as the data feature of the target sub-data. For example, the output of the network block layer with 16 times downsampling of the network can be used as Camera_Fea _i , that is, the data feature, and the dimension size is 7*7*d, where d is the number of channels of the feature of this layer, and 7 is 224⁄16.

In an exemplary embodiment, fusing the data feature and the point cloud feature based on the dimension parameter of the data feature to obtain the fusion feature includes: adjusting the point cloud feature according to the dimension parameter to obtain the target point cloud feature ; fuse the data feature and the target point cloud feature to obtain the fusion feature. In this embodiment, when determining the fusion feature, the point cloud feature and the data feature can be unified into the same size, for example, the point cloud feature can be adjusted according to the dimension parameter of the data feature to obtain the target point cloud feature. Fusion of target point cloud features and data features to obtain fusion features.

In the above embodiment, the data features can also be adjusted according to the dimension parameters of the point cloud features to obtain the target data features, and the target data features and the point cloud features are fused to obtain the fusion features.

It should be noted that the size relationship between the dimension parameter of the point cloud feature and the dimension parameter of the data feature can be determined, and the feature with a larger dimension parameter can be adjusted according to the dimension parameter with a smaller dimension parameter.

In an exemplary embodiment, fusing the data feature and the target point cloud feature to obtain the fused feature includes: connecting the data feature and the target point cloud feature according to the channel dimension included in the dimension parameter , to obtain the fusion feature. In this embodiment, when fusing data features and target point cloud features, features of the same dimension can be directly connected to obtain fused features. For example, the target point cloud feature Lidar_Fea _i and the data feature Camera_Fea _i can be concat together according to the channel dimension to obtain the fusion feature Fusion_Fea _i , whose dimension size is 7*7*(d+2).

In an exemplary embodiment, adjusting the point cloud feature according to the dimension parameter to obtain the target point cloud feature includes: adjusting the point cloud feature by using a linear interpolation algorithm, so that the dimension of the adjusted point cloud feature is the Dimension parameter; determine the adjusted point cloud feature as the target point cloud feature. In this embodiment, the point cloud feature Lidar_Fea _i can be resized to the dimension of the data feature through nearest neighbor interpolation, for example, 7*7*2 dimensions, to obtain features consistent with the first two dimensions of the data feature Camera_Fea _i .

The method for identifying objects is described below in conjunction with specific embodiments:

FIG. 4 is a flowchart of an object recognition method according to a specific embodiment of the present invention. As shown in FIG. 4 , the method includes:

In step S402, the monocular camera collects the color image, and the lidar collects the laser point cloud. The monocular camera and lidar have been jointly calibrated.

In step S404, vehicle detection is performed to obtain a vehicle sub-image.

Step S406, mapping to obtain vehicle radar features.

Step S408 , input the vehicle sub-image (corresponding to the above-mentioned number target sub-data) to the main network module to obtain image features (corresponding to the above-mentioned data characteristics).

Step S410: Perform linear interpolation on the vehicle radar feature (corresponding to the above point cloud feature) to obtain the radar feature (corresponding to the above target point cloud feature).

Step S412, concat image features and radar features to obtain fusion features.

Step S414, input the fusion feature into the auxiliary network model and the classifier to obtain the recognition result.

In the foregoing embodiment, the data collected in two different modalities of lidar and monocular camera and collected under the same time stamp are calculated separately, and the three-dimensional features of the former are added to the intermediate feature layer of the deep learning main network module of the latter. After fusion, the final vehicle recognition result is obtained by training with an auxiliary network module and a classifier. Use the vehicle detection model to detect the 2D vehicle frame on the camera color image; register the lidar with the monocular camera, correctly map the 3D laser point cloud to the 2D camera plane, and obtain the 3D point cloud features in each vehicle frame; For each vehicle color sub-image, the image features of the main network module are extracted, and concat is connected with the point cloud features of the vehicle; then the auxiliary network module and classifier are connected for classification and training. Due to the technical limitations of monocular vision, it is inevitable that some problems in vehicle identification cannot be well solved. However, with the addition of 3D point cloud features scanned by LiDAR, LiDAR has an innate ability in distance measurement. For this reason, it can be used as a powerful supplement to the monocular camera. By combining the two with radar vision, a more accurate recognition effect can be obtained. It can make up for the shortcomings of pure visual technology and greatly improve the accuracy of vehicle recognition.

From the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is better implementation. Based on this understanding, the technical solutions of the present invention essentially or the parts that contribute to the prior art can be embodied in the form of software products, and the computer software products are stored in a storage medium (such as ROM/RAM, magnetic disk, CD-ROM), including several instructions to make a terminal device (which may be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in the various embodiments of the present invention.

In this embodiment, an object identification device is also provided, and the device is used to implement the above-mentioned embodiments and preferred implementations, and the descriptions that have been described will not be repeated. As used below, the term "module" may be a combination of software and/or hardware that implements a predetermined function. Although the apparatus described in the following embodiments is preferably implemented in software, implementations in hardware, or a combination of software and hardware, are also possible and contemplated.

FIG. 5 is a structural block diagram of an object identification device according to an embodiment of the present invention. As shown in FIG. 5 , the device includes:

The first determination module 502 is used to determine the target point cloud obtained by the first device photographing the target area at the target moment;

The second determining module 504 is configured to determine target sub-data included in the target data collected by the second device at the target moment, and determine data characteristics of the target sub-data, wherein the target data is the first The data obtained by the second device photographing the target area, the angle at which the second device photographs the target area is the same as the angle at which the first device photographs the target area, and the target sub-data is: the data of the target object included in the target data;

a mapping module 506, configured to map each pixel point included in the target sub-data to the point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points;

A third determining module 508, configured to determine point cloud features based on a plurality of the target pixel points and the target point cloud;

A fusion module 510, configured to fuse the data feature and the point cloud feature based on the dimension parameter of the data feature to obtain a fusion feature;

The identification module 512 is configured to identify the target object based on the fusion feature.

In an exemplary embodiment, the third determining module 508 may determine the point cloud feature based on a plurality of the target pixel points and the target point cloud in the following manner: for each of the target pixel points, the following operations are performed: Obtain the feature value corresponding to each target pixel point: determine whether the target point cloud includes the first point with the same coordinates as the target pixel point; The coordinates and the response intensity corresponding to the first point are determined as the characteristic value of the target pixel point; in the case where the first point does not exist, determine the first point closest to the target pixel point in the target point cloud. Two points, the vertical coordinate of the second point and the response intensity corresponding to the second point are determined as the eigenvalues of the target pixel point; the matrix formed by a plurality of the eigenvalues is determined as the point cloud feature .

In an exemplary embodiment, the third determining module 508 may implement, in the presence of a first point, determining the vertical coordinate of the first point and the response intensity corresponding to the first point as the The eigenvalue of the target pixel point: when the number of the first point is one, determine the vertical coordinate of the first point and the response intensity corresponding to the first point as the eigenvalue of the target pixel point ; In the case where the number of the first points is multiple, determine the third point with the smallest vertical coordinate included in the multiple first points, and combine the vertical coordinates of the third point and the third point The corresponding response intensity is determined as the feature value of the target pixel point.

In an exemplary embodiment, the second determining module 504 may determine the data characteristics of the target sub-data in the following manner: inputting the target sub-data into the target network model, and determining the data characteristics extracted by the target network model. feature; the feature extracted from any network layer is determined as the data feature, wherein the network layer is a network layer included in the backbone network of the target network model.

In an exemplary embodiment, the fusion module 510 may implement fusion of the data feature and the point cloud feature based on the dimension parameter of the data feature to obtain a fusion feature by: adjusting the point cloud according to the dimension parameter feature to obtain the target point cloud feature; fuse the data feature and the target point cloud feature to obtain the fusion feature.

In an exemplary embodiment, the fusion module 510 may fuse the data features and the target point cloud features to obtain the fusion features by: connecting the data features according to the channel dimensions included in the dimension parameters and the target point cloud feature to obtain the fusion feature.

In an exemplary embodiment, the fusion module 510 may adjust the point cloud feature according to the dimension parameter to obtain the target point cloud feature by using a linear interpolation algorithm to adjust the point cloud feature so that the adjusted point cloud features The dimension of the cloud feature is the dimension parameter; the adjusted point cloud feature is determined as the target point cloud feature.

It should be noted that the above modules can be implemented by software or hardware, and the latter can be implemented in the following ways, but not limited to this: the above modules are all located in the same processor; or, the above modules can be combined in any combination The forms are located in different processors.

An embodiment of the present invention also provides a computer-readable storage medium, where a computer program is stored in the computer-readable storage medium, wherein the computer program implements any of the methods described above when executed by a processor A step of.

In an exemplary embodiment, the above-mentioned computer-readable storage medium may include, but is not limited to, a USB flash drive, a read-only memory (Read-Only Memory, referred to as ROM for short), and a random access memory (Random Access Memory, referred to as RAM for short) , mobile hard disk, magnetic disk or CD-ROM and other media that can store computer programs.

An embodiment of the present invention also provides an electronic device, comprising a memory and a processor, where a computer program is stored in the memory, and the processor is configured to run the computer program to execute the steps in any of the above method embodiments.

In an exemplary embodiment, the above-mentioned electronic device may further include a transmission device and an input-output device, wherein the transmission device is connected to the above-mentioned processor, and the input-output device is connected to the above-mentioned processor.

For specific examples in this embodiment, reference may be made to the examples described in the foregoing embodiments and exemplary implementation manners, and details are not described herein again in this embodiment.

Obviously, those skilled in the art should understand that the above-mentioned modules or steps of the present invention can be implemented by a general-purpose computing device, which can be centralized on a single computing device, or distributed in a network composed of multiple computing devices On the other hand, they can be implemented in program code executable by a computing device, so that they can be stored in a storage device and executed by the computing device, and in some cases, can be performed in a different order than shown here. Or the described steps, or they are respectively made into individual integrated circuit modules, or a plurality of modules or steps in them are made into a single integrated circuit module to realize. As such, the present invention is not limited to any particular combination of hardware and software.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and changes. Any modification, equivalent replacement, improvement, etc. made within the principle of the present invention shall be included within the protection scope of the present invention.

Claims (10)

1. a recognition method of an object, is characterized in that, comprises: Determine the target point cloud obtained by the first device photographing the target area at the target moment; Determine the target sub-data included in the target data collected by the second device at the target time, and determine the data characteristics of the target sub-data, wherein the target data is the second device performing the target area on the target area. Shooting the obtained data, the angle at which the second device shoots the target area is the same as the angle at which the first device shoots the target area, and the target sub-data is the target included in the target data object data; mapping each pixel point included in the target sub-data to the point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points; Determine point cloud features based on a plurality of the target pixel points and the target point cloud; Fusion of the data feature and the point cloud feature based on the dimension parameter of the data feature to obtain a fusion feature; The target object is identified based on the fusion feature. 2. The method according to claim 1, wherein determining a point cloud feature based on a plurality of the target pixel points and the target point cloud comprises: The following operations are performed for each target pixel point to obtain the feature value corresponding to each target pixel point: determine whether the target point cloud includes a first point with the same coordinates as the target pixel point; In the case of a point, the vertical coordinate of the first point and the response intensity corresponding to the first point are determined as the characteristic value of the target pixel point; in the absence of the first point, determine the The second point closest to the target pixel point in the target point cloud, the vertical coordinate of the second point and the response intensity corresponding to the second point are determined as the feature value of the target pixel point; A matrix formed by a plurality of the eigenvalues is determined as the point cloud feature. 3 . The method according to claim 2 , wherein, in the presence of a first point, the vertical coordinate of the first point and the response intensity corresponding to the first point are determined as the target pixel point. 4 . The eigenvalues include: When the number of the first point is one, the vertical coordinate of the first point and the response intensity corresponding to the first point are determined as the characteristic value of the target pixel point; When the number of the first points is multiple, determine a third point with the smallest vertical coordinate included in the multiple first points, and map the vertical coordinate of the third point to the third point The response intensity of is determined as the feature value of the target pixel point. 4. The method according to claim 1, wherein determining the data characteristics of the target sub-data comprises: Inputting the target sub-data into the target network model, and determining the features extracted by the target network model; A feature extracted from any network layer is determined as the data feature, wherein the network layer is a network layer included in the backbone network of the target network model. 5. The method according to claim 1, wherein the data feature and the point cloud feature are fused based on the dimensional parameters of the data feature, and obtaining the fusion feature comprises: Adjust the point cloud feature according to the dimension parameter to obtain the target point cloud feature; The data feature and the target point cloud feature are fused to obtain the fusion feature. 6. The method according to claim 5, wherein, fusing the data feature and the target point cloud feature to obtain the fusion feature comprises: Connect the data feature and the target point cloud feature according to the channel dimension included in the dimension parameter to obtain the fusion feature. 7. The method according to claim 5, wherein adjusting the point cloud feature according to the dimension parameter, and obtaining the target point cloud feature comprises: Adjust the point cloud feature by using a linear interpolation algorithm, so that the dimension of the adjusted point cloud feature is the dimension parameter; The adjusted point cloud feature is determined as the target point cloud feature. 8. A device for identifying an object, comprising: a first determination module, configured to determine the target point cloud obtained by the first device photographing the target area at the target moment; The second determination module is configured to determine the target sub-data included in the target data collected by the second device at the target moment, and determine the data characteristics of the target sub-data, wherein the target data is the second The data obtained by the device photographing the target area, the angle at which the second device photographs the target area is the same as the angle at which the first device photographs the target area, and the target sub-data is the the data of the target object included in the target data; a mapping module, configured to map each pixel point included in the target sub-data to the point cloud coordinate system where the target point cloud is located to obtain a plurality of target pixel points; a third determination module, configured to determine point cloud features based on a plurality of the target pixel points and the target point cloud; a fusion module, configured to fuse the data feature and the point cloud feature based on the dimension parameter of the data feature to obtain a fusion feature; An identification module, configured to identify the target object based on the fusion feature. 9. A computer-readable storage medium, wherein a computer program is stored in the computer-readable storage medium, wherein, when the computer program is executed by a processor, any one of the claims 1 to 7 is implemented the steps of the method. 10. An electronic device comprising a memory and a processor, wherein a computer program is stored in the memory, and the processor is configured to run the computer program to execute any one of claims 1 to 7 method described in.

Images