CN113933818A - Method, device, storage medium and program product for calibration of external parameters of lidar - Google Patents

Abstract

The method, device, storage medium and program product for calibrating lidar external parameters provided by the present disclosure relate to high-precision maps and automatic driving technologies. The solution provided by the present disclosure is based on the trajectory of the inertial measurement unit. Based on the relative pose of the radar between adjacent frames, the relative pose of the radar between adjacent frames can be estimated based on the relative pose of the inertial measurement unit and the relative pose of the radar between the adjacent data frames. The unit is calibrated, and the rotation parameters and translation parameters between the radar and the inertial measurement unit can be calibrated through the pose change, which can improve the calibration success rate.

Description

Method, device, storage medium and program product for calibration of external parameters of lidar

technical field

The present disclosure relates to high-precision maps and automatic driving technologies in data processing technology, and in particular, to a method, device, storage medium and program product for calibrating external parameters of lidar.

Background technique

With the development of unmanned driving and intelligent driving technology, higher requirements are put forward for the scale and scene of high-precision maps. When making a high-precision map, it is necessary to use the lidar on the vehicle to collect point cloud data, and then perform 3D reconstruction based on the point cloud data to obtain a high-precision map. In the process of 3D reconstruction, the accuracy of the external parameters of the vehicle radar is relatively high.

There is an external parameter estimation scheme based on a B-spline continuous trajectory in the prior art. This scheme uses a B-spline parameterized trajectory to derive the parameterized equation to construct an objective function to estimate the IMU (Inertial Measurement Unit, inertial measurement unit) of the vehicle. ) trajectory, and use the radar odometry to estimate the radar trajectory, and then calibrate the radar external parameters based on the IMU trajectory and the radar trajectory.

In this calibration method, only the rotating external parameters of the radar can be estimated, which leads to a large error in the calibration system and reduces the success rate of calibration.

SUMMARY OF THE INVENTION

The present disclosure provides a method, device, storage medium and program product for calibrating external parameters of a laser radar, so as to improve the accuracy and success rate of external parameter calibration of the radar.

According to a first aspect of the present disclosure, a method for calibrating external parameters of a lidar is provided, including:

acquiring radar data collected by a radar sensor set on the vehicle, inertial navigation data collected by an inertial measurement unit set on the vehicle, and positioning data collected by a positioning system set on the vehicle;

determining the inertial navigation trajectory of the inertial measurement unit according to the positioning data and the inertial navigation data;

Determine the inertial relative pose of the inertial measurement unit between adjacent frames according to the inertial navigation trajectory, and determine the radar relative pose of the radar between adjacent frames according to the radar data;

According to the relative pose of the inertial navigation and the relative pose of the radar, the calibration parameters of the radar relative to the inertial measurement unit are determined.

According to a second aspect of the present disclosure, a device for calibrating external parameters of a lidar is provided, including:

a data acquisition unit, configured to acquire radar data collected by a radar sensor set on the vehicle, inertial navigation data collected by an inertial measurement unit set on the vehicle, and positioning data collected by a positioning system set on the vehicle;

a trajectory determination unit, configured to determine the inertial navigation trajectory of the inertial measurement unit according to the positioning data and the inertial navigation data;

a pose determination unit, configured to determine the inertial navigation relative pose of the inertial measurement unit between adjacent frames according to the inertial navigation trajectory, and determine the radar relative pose of the radar between adjacent frames according to the radar data;

A calibration unit, configured to determine calibration parameters of the radar relative to the inertial measurement unit according to the inertial navigation relative pose and the radar relative pose.

According to a third aspect of the present disclosure, there is provided an electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

The memory stores instructions executable by the at least one processor, the instructions being executed by the at least one processor to enable the at least one processor to perform the method of the first aspect.

According to a fourth aspect of the present disclosure, there is provided a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to cause the computer to perform the method of the first aspect.

According to a fifth aspect of the present disclosure, there is provided a computer program product, the computer program product comprising: a computer program stored in a readable storage medium, from which at least one processor of an electronic device can Reading the storage medium reads the computer program, and executing the computer program by the at least one processor causes the electronic device to perform the method of the first aspect.

The method, device, storage medium and program product for calibrating external parameters of lidar provided by the present disclosure can infer the relative pose of the inertial measurement unit between adjacent data frames based on the trajectory of the inertial measurement unit, and estimate the relative pose of the inertial measurement unit between adjacent data frames according to the radar data. The relative pose between adjacent frames, so that the radar and the inertial measurement unit can be calibrated based on the relative pose of the inertial measurement unit and the relative pose of the radar between adjacent data frames. The rotation parameters and translation parameters between the inertial measurement units can improve the calibration success rate.

It should be understood that what is described in this section is not intended to identify key or critical features of embodiments of the disclosure, nor is it intended to limit the scope of the disclosure. Other features of the present disclosure will become readily understood from the following description.

Description of drawings

The accompanying drawings are used for better understanding of the present solution, and do not constitute a limitation to the present disclosure. in:

FIG. 1 is a schematic diagram of a vehicle shown in an exemplary embodiment;

FIG. 2 is a schematic flowchart of a method for calibrating external parameters of a lidar according to an exemplary embodiment of the present disclosure;

FIG. 3 is a schematic flowchart of a method for calibrating external parameters of a lidar according to another exemplary embodiment of the present disclosure;

FIG. 4 is a schematic structural diagram of an apparatus for calibrating external parameters of a lidar according to an exemplary embodiment of the present disclosure;

FIG. 5 is a schematic structural diagram of an apparatus for calibrating external parameters of a lidar according to another exemplary embodiment of the present disclosure;

6 is a block diagram of an electronic device used to implement the method of an embodiment of the present disclosure.

Detailed ways

Exemplary embodiments of the present disclosure are described below with reference to the accompanying drawings, which include various details of the embodiments of the present disclosure to facilitate understanding and should be considered as exemplary only. Accordingly, those of ordinary skill in the art will recognize that various changes and modifications of the embodiments described herein can be made without departing from the scope and spirit of the present disclosure. Also, descriptions of well-known functions and constructions are omitted from the following description for clarity and conciseness.

FIG. 1 is a schematic diagram of a vehicle according to an exemplary embodiment.

A radar sensor 11 and an inertial measurement unit IMU12 are provided on the vehicle, the radar sensor 11 has a coordinate system O1X1Y1Z1, and the inertial measurement unit 12 also has a coordinate system O2X2Y2Z2. There are installation error angles and position errors between the lidar and the IMU, so the three-dimensional coordinates of the same set of marker points measured by the two sensors are different.

When collecting road data with a vehicle equipped with a radar and an IMU, it is necessary to perform 3D reconstruction according to the point cloud data collected by the radar, and it is also necessary to determine the location information of the 3D reconstructed road environment. When determining the location information, a combined positioning method of lidar and IMU can be used. Since the coordinate systems of the lidar and the IMU are not exactly the same, the three-dimensional coordinates of the same set of marker points measured by the two sensors are different. Therefore, in order to meet the positioning accuracy of the positioning system, it is necessary to calibrate the parameters between the radar and the IMU.

There is an external parameter estimation scheme based on a B-spline continuous trajectory in the prior art. This scheme uses a B-spline parameterized trajectory, derives the parameterized equation to construct an objective function to estimate the trajectory of the vehicle's IMU, and uses a radar odometer. The trajectory of the radar is estimated, and the external parameters of the radar are calibrated based on the trajectory of the IMU and the trajectory of the radar. In this calibration method, only the rotating external parameters of the radar can be estimated, which leads to a large error in the calibration system and reduces the success rate of calibration.

In order to solve the above technical problems, the solution provided by the present disclosure utilizes the relative pose of the IMU and the relative pose of the radar between adjacent data frames to calibrate the radar and the IMU, and the relative pose is used to characterize the pose of the sensor in the two frames of data. Change, since the radar and the IMU are set on the same vehicle, the position transformation of the two sensors should be the same during the driving process of the vehicle. Based on this, the parameters between the radar and the IMU can be calibrated. This method can Calibration of the rotation parameters and translation parameters between the radar and the IMU can improve the success rate of calibration.

FIG. 2 is a schematic flowchart of a method for calibrating external parameters of a lidar according to an exemplary embodiment of the present disclosure.

As shown in FIG. 2 , the method for calibrating external parameters of lidar provided by the present disclosure includes:

Step 201 , acquiring radar data collected by a radar sensor set on the vehicle, inertial navigation data collected by an inertial measurement unit set on the vehicle, and positioning data collected by a positioning system set on the vehicle.

Wherein, the method provided by the present disclosure may be executed by an electronic device with computing capability, and the electronic device may be, for example, a vehicle-mounted device.

Specifically, a radar sensor is provided on the vehicle, and an IMU and a positioning system may also be provided. Further, when the vehicle is running, the radar sensor can collect point cloud data, the IMU can collect inertial navigation data, and the positioning system can collect positioning data. The positioning system can be, for example, GPS (Global Positioning System, global positioning system).

Further, the working principle of the radar sensor is to transmit a detection signal (laser beam) to the target, and then compare the received signal (target echo) reflected from the target with the transmitted signal to obtain point cloud data. Point cloud data can include information about the target, such as target distance, orientation, altitude, speed, attitude, and even shape and other parameters.

In practical applications, the IMU can include three single-axis accelerometers and three single-axis gyroscopes. The accelerometer detects the three-axis acceleration signal, and the gyroscope detects the angular velocity signal to measure the angular velocity and acceleration of the object in three-dimensional space. Thus, the pose of the object can be determined.

Among them, the radar sensor, IMU and positioning system can respectively send the collected data to the electronic device, so that the electronic device can obtain inertial navigation data, radar data and positioning data.

In an optional implementation manner, the vehicle may move in a straight line or in a curve while driving, and a certain amount of data of linear motion and a certain amount of data of curvilinear motion may be screened out from the data collected by each sensor.

Step 202: Determine the inertial navigation trajectory of the inertial measurement unit according to the positioning data and the inertial navigation data.

Specifically, the electronic device can determine the inertial navigation trajectory of the IMU by using the positioning data and the inertial navigation data. The current position of the vehicle is recorded in the positioning data, and the attitude change of the vehicle is recorded in the inertial navigation data. Therefore, the inertial navigation trajectory can be determined by combining the positioning data and the inertial navigation data.

Further, in some specific cases when the signal of the positioning system is not good, the obtained positioning data will also be inaccurate. Therefore, the positioning data with better signal can be combined with the inertial navigation data to determine the driving trajectory of the vehicle. Then determine the inertial navigation trajectory of the IMU according to the driving trajectory. For example, when the signal of the positioning system at point A is good, the position of the vehicle can be determined according to the positioning data of point A. After that, when the signal of the positioning system is not good, the attitude change of the vehicle can be determined according to the inertial navigation data, and the driving trajectory of the vehicle can be generated according to the position of point A. .

In practical applications, the relative position of the IMU and the positioning system can be pre-calibrated, and then the inertial navigation trajectory can be generated according to the relative position of the IMU and the positioning system and the driving trajectory of the vehicle. The inertial navigation trajectory is used to characterize the position change of the inertial navigation when the vehicle is driving.

Step 203: Determine the inertial navigation relative pose of the inertial measurement unit between adjacent frames according to the inertial navigation trajectory, and determine the radar relative pose of the radar between adjacent frames according to radar data.

Among them, the inertial navigation trajectory includes multiple frames of data, and these data constitute multiple position information of the inertial navigation within a period of time. The electronic device can determine the relative pose of the inertial navigation between adjacent data frames according to the multiple frames of data in the inertial navigation trajectory.

Specifically, according to the position information of the inertial navigation in the inertial navigation data of two consecutive frames, the pose transformation information of the inertial navigation is determined. For example, compared with the inertial navigation data of the first frame, the inertial navigation data of the second frame is The X-axis direction of the inertial navigation coordinate system has moved a distance L.

Further, the electronic device may also determine the radar relative pose and attitude of the radar between adjacent frames according to the radar data.

In practical application, the radar data acquired by the electronic device includes multiple frames of data, and the electronic device can determine the pose transformation information of the radar according to the pose information in the radar data of two consecutive frames. For example, compared with the first frame of radar data in the second frame of radar data, the radar moves a distance H in the X-axis direction of the radar coordinate system.

Among them, both the inertial navigation data and the radar data may have frame identification information, so that the inertial navigation data and radar data collected at the same time can be determined according to the frame identification, and then the inertial navigation data and radar data collected at the same time can be used to calibrate parameters. The frame identification information may be, for example, a frame number or time information.

Step 204: Determine the calibration parameters of the radar relative to the inertial measurement unit according to the relative pose of the inertial navigation and the relative pose of the radar.

Specifically, the electronic device can calibrate the radar according to the inertial navigation relative pose and the radar relative pose, so as to determine the parameters of the radar relative to the IMU.

Further, the radar and the IMU are set on the vehicle, so the relative position between the radar and the IMU does not change, and the calibration parameters of the radar relative to the inertial measurement unit will not change, and the radar and IMU are set on the same vehicle, The pose changes of the two are also the same.

In practical applications, the attitude changes of the radar and the IMU should be the same at two adjacent moments during the driving process of the vehicle. Therefore, the relative attitude of the radar relative to the IMU can be determined according to the relative pose of the radar and the relative pose of the IMU between adjacent frames. parameter.

For example, the first parameter of the radar relative to the IMU can be determined according to the relative pose of the radar and the relative pose of the IMU between the first group of adjacent frames, and the relative pose of the radar and the relative position of the IMU between the second group of adjacent frames can also be determined. The attitude determines the second parameter of the radar relative to the IMU. In this way, the parameters corresponding to multiple sets of adjacent frame data can be obtained, and these parameters can be fitted to obtain the parameters of the radar relative to the IMU.

The calibration method provided by the present disclosure can avoid the cumbersome and strict calibration process, and can perform external parameter calibration during the three-dimensional reconstruction process, and use the data collected during the three-dimensional reconstruction to automatically optimize and compensate the external parameters, which is conducive to the improvement of high-precision production and acquisition efficiency .

The method for calibrating external parameters of lidar provided by the present disclosure includes: acquiring radar data collected by a radar sensor set on a vehicle, inertial navigation data collected by an inertial measurement unit set on the vehicle, and collected by a positioning system set on the vehicle According to the positioning data and inertial navigation data, the inertial navigation trajectory of the inertial measurement unit is determined; according to the inertial navigation trajectory, the inertial navigation relative pose of the inertial measurement unit between adjacent frames is determined, and the radar is determined according to the radar data. The relative pose of the radar between frames; according to the relative pose of the inertial navigation and the relative pose of the radar, the calibration parameters of the radar relative to the inertial measurement unit are determined. In this embodiment, the electronic device can infer the relative pose of the IMU between adjacent data frames based on the IMU trajectory, and infer the relative pose of the radar between adjacent frames based on the radar data, so that the relative pose of the radar between adjacent data frames can be inferred based on the difference between the adjacent data frames. The relative pose of the IMU and the relative pose of the radar are used to calibrate the radar and the IMU. The rotation parameters and translation parameters between the radar and the IMU can be calibrated through the change of the pose, which can improve the success rate of calibration.

FIG. 3 is a schematic flowchart of a method for calibrating external parameters of a lidar according to another exemplary embodiment of the present disclosure.

As shown in FIG. 3 , the method for calibrating external parameters of lidar provided by the present disclosure includes:

Step 301: Acquire initial radar data collected by a radar sensor when the vehicle is running, initial inertial navigation data collected by an inertial measurement unit, and initial positioning data collected by a positioning system.

Among them, the data collected by the radar sensor when the vehicle is running is the initial radar data. For example, if the vehicle is running for 20 minutes, the data collected by the radar sensor during this period can be used as the initial radar data.

Specifically, the data collected by the IMU when the vehicle is running is the initial inertial navigation data. For example, if the vehicle runs for 20 minutes, the data collected by the IMU during this period can be used as the initial inertial navigation data.

Further, the data collected by the positioning system when the vehicle is running is the initial positioning data. For example, when the vehicle is running for 20 minutes, the data collected by the positioning system during this period can be used as the initial positioning data.

In practical applications, the electronic device can obtain the initial radar data, initial inertial navigation data and initial positioning data collected by the radar sensor, IMU and positioning system when the vehicle is running.

Step 302: Select the radar data satisfying the preset driving state from the initial radar data, select the inertial navigation data satisfying the preset driving state from the initial inertial navigation data, and select the positioning data satisfying the preset driving state from the initial positioning data .

In practical application, the preset driving state can be set in advance. There may be various driving states when a vehicle is driving on the road, and the data collected by the sensors may be different in different driving states. Therefore, in order to improve the accuracy of parameter calibration, the radar data, inertial navigation data and positioning data that satisfy the preset driving state can be selected from the initial radar data, initial inertial navigation data and initial positioning data.

Here, the preset driving state may include, for example, a linear movement and a rotational movement. More linear motion and rotational motion data can be filtered out from initial radar data, initial inertial navigation data and initial positioning data. However, if there is too much data, the amount of calculation may be too large. Therefore, radar data, inertial navigation data and positioning data of a continuous trajectory can be selected, and this continuous trajectory can include straight driving trajectory and rotating driving trajectory.

The filtered radar data, inertial navigation data, and positioning data are data of the same time period, for example, data from time t1 to t2 in the collected data. The continuous trajectory during this time includes both linear motion and rotational motion.

Step 303: Screen out reliable positioning data whose confidence reaches a preset value from the positioning data, and determine the initial driving trajectory of the vehicle according to the reliable positioning data.

Specifically, after obtaining the positioning data, the electronic device can screen out reliable positioning data with better signals. For example, reliable positioning data can be determined in the positioning data according to the confidence of the positioning data output by the positioning system. If the confidence of the positioning data reaches a preset value, it can be determined that the positioning data is reliable positioning data.

Further, the initial driving trajectory of the vehicle can be determined according to the reliable positioning data. For example, a continuous driving trajectory of the vehicle can be determined according to the reliable positioning data, and used as the initial driving trajectory. For example, the position of the vehicle can be determined according to the reliable positioning data, and then the driving trajectory of the vehicle can be obtained, and a continuous initial driving trajectory can also be determined in this driving trajectory.

Step 304: Determine the inertial navigation trajectory of the inertial measurement unit according to the initial driving trajectory and the inertial navigation data.

In practical applications, the inertial navigation data is used to characterize the pose state of the IMU, and the IMU is set on the vehicle. Therefore, the inertial navigation data can represent the pose state of the vehicle. After the positioning information of the vehicle at one time is determined, the subsequent driving trajectory of the vehicle can be inferred based on the posture state of the vehicle after the time.

Among them, it can be considered that the initial driving trajectory obtained by reliable positioning data is accurate. Therefore, the complete driving trajectory of the vehicle can be inferred by combining the initial driving trajectory and inertial navigation data, and then the accurate inertial navigation trajectory of the IMU can be determined.

In this embodiment, the electronic device can restore the accurate driving trajectory of the vehicle according to the relatively accurate positioning data and inertial navigation data, and then infer the inertial navigation trajectory of the IMU. In this way, a relatively accurate inertial navigation trajectory can be obtained.

For example, the complete driving trajectory of the vehicle can be determined according to the initial driving trajectory and inertial navigation data; according to the relative position between the positioning system and the inertial measurement unit, the complete driving trajectory can be offset to obtain the inertial navigation trajectory of the inertial measurement unit.

Wherein, the initial driving trajectory can be generated according to the positioning data with better signal, and then the complete driving trajectory of the vehicle can be generated according to the inertial navigation data after the vehicle passes through the initial driving trajectory. For example, the tail end of the initial driving trajectory is position A. After passing through this driving trajectory, it is possible to infer the change of the vehicle's position and attitude according to the inertial navigation data. The next position, based on this, the complete travel trajectory of the vehicle can be obtained.

Specifically, the electronic device can perform offset processing on the complete driving trajectory according to the relative position of the positioning system and the IMU to obtain the inertial navigation trajectory. For example, if the IMU is set at 20 centimeters in the first direction of the positioning system, the complete driving trajectory can be shifted by 20 centimeters in the first direction to obtain the inertial navigation trajectory.

In this embodiment, the trajectory of the IMU can be obtained according to the positioning data and inertial navigation data with a high degree of confidence. This method does not use the method of deriving parameters to obtain the trajectory of the IMU, and the determined inertial navigation trajectory is more accurate. The calibration parameters are also more accurate.

Step 305: Determine the inertial navigation relative pose of the inertial measurement unit between adjacent frames according to the inertial navigation trajectory.

The implementation manner of step 305 is similar to that of step 203, and details are not repeated here.

Step 306 , register the point cloud data of adjacent frames, and obtain the radar relative pose between adjacent frames; wherein, the radar data includes point cloud data.

Further, the radar data acquired by the electronic device includes point cloud data, and the electronic device can determine the change of the relative pose of the radar according to the point cloud data of consecutive multiple frames.

Point cloud registration refers to determining a transformation matrix so that one point cloud can coincide with another point as high as possible. The pose of the radar sensor changes when the vehicle is driving, so the point clouds of two consecutive frames collected by the radar do not overlap. By registering two consecutive frames of point cloud data, the pose transformation of the radar in the two frames of data can be determined, and then the relative pose of the radar can be obtained. The attitude change process of the radar can be accurately determined by means of point cloud registration.

In practical application, point cloud registration is an iterative process, and the transformation matrix is optimized by continuous iteration, so that the two point clouds are close to coincidence. In order to determine the transformation matrix between two consecutive frames of point cloud data more quickly, the solution provided by the present disclosure adopts a hierarchical registration method to perform optimization iteration, thereby improving the registration speed.

Among them, for any two adjacent frames of point cloud data, a hierarchical registration method can be adopted.

Specifically, multi-level sampling processing may be performed on point cloud data of adjacent frames to obtain multi-level point cloud sampling data with different precisions of adjacent frames.

The adjacent two frames of point cloud data include the first point cloud and the second point cloud. The first point cloud and the second point cloud may be processed in multiple stages respectively, and specifically, down-sampling processing may be performed according to different resolutions. For example, the first point cloud and the second point cloud can be down-sampled according to the three-level resolution of low resolution, medium resolution and high resolution, respectively, to obtain the first low point cloud, the first middle point cloud, the first point cloud and the second point cloud. A high point cloud, and a second low point cloud, a second mid point cloud, and a second high point cloud.

For example, point cloud data may be sampled with a probability of 60%, point cloud data may be sampled with a probability of 80%, and point cloud data may be sampled with a probability of 90%.

Further, the electronic device can register the point cloud sampling data of adjacent frames in order from low-level accuracy to high-level accuracy, and then register the point cloud data with the highest accuracy. Data refers to point cloud data before sampling.

For example, the electronic device can first register the low-resolution point cloud sampling data to obtain a registration result. When registering the low-resolution point cloud sampling data, an initial value can be randomly generated, and then based on the initial value Registration is performed. Then, the low-resolution registration result is used as the initial value, and the registration is performed when the medium-resolution point cloud sampling data is registered, and the registration result is obtained. Then, the high-resolution registration result is used as the initial value, and the high-resolution point cloud sampling data is registered to obtain the registration result. After that, the high-resolution registration result can be used as the initial value to register the point cloud data before sampling to obtain the relative pose of the radar between adjacent frames.

In practical application, the electronic device can acquire the first point cloud data and the second point cloud data of the same precision level of adjacent frames. The first point cloud data and the second point cloud data may be point cloud sampling data, or may be point cloud data included in radar data. For example, in the registration process, when registering the point cloud sampling data at all levels, the point cloud sampling data of adjacent frames is obtained. After the sequential registration of the sampling data, the point cloud data before sampling needs to be registered. , which is the point cloud data of adjacent frames before sampling.

The first point cloud data and the second point cloud data are both three-dimensional data, and the three-dimensional first point cloud data and the second point cloud data can be projected onto a two-dimensional plane to obtain two-dimensional point cloud data, and then The two-dimensional point cloud can be registered to improve the registration speed.

Specifically, the first three-dimensional point cloud can be mapped to the two-dimensional plane according to the first point cloud data to obtain the first two-dimensional point cloud, and the second three-dimensional point cloud can be mapped to the two-dimensional plane according to the second point cloud data to obtain the first two-dimensional point cloud. Two-dimensional point cloud. For example, the angle of the 3D point cloud relative to the center of the radar sensor can be used as the coordinate to project the point cloud onto the 2D plane. For example, the point cloud can be projected by means of cylindrical projection.

According to the positions of the first two-dimensional point cloud and the second two-dimensional point cloud on the two-dimensional plane, the point cloud sampling data or point cloud data of adjacent frames are registered. When the acquired first point cloud data and the second point cloud data are point cloud sampling data, then according to the positions of the one-dimensional point cloud and the second two-dimensional point cloud on the two-dimensional plane, the points of adjacent frames can be Cloud sampling data for registration. When the acquired first point cloud data and the second point cloud data are point cloud data, then according to the positions of the first two-dimensional point cloud and the second two-dimensional point cloud on the two-dimensional plane, the points of adjacent frames can be Cloud data for registration.

Further, an ICP algorithm (Iterative Closest Point, iterative closest point algorithm) may be used to register the first two-dimensional point cloud and the second two-dimensional point cloud. Specifically, according to the positions of the first two-dimensional point cloud and the second two-dimensional point cloud on the two-dimensional plane, the point corresponding to the first two-dimensional point cloud is determined in the second two-dimensional point cloud, and then the corresponding relationship is determined. The registration result between the first 2D point cloud and the second 2D point cloud. Then use the obtained registration result to process the first two-dimensional point cloud, determine the error based on the processing result and the second two-dimensional point cloud, and adjust the correspondence between points according to the error, so as to obtain the first two-dimensional point through multiple iterations The final registration result of the cloud and the second 2D point cloud.

In this way, the points with the corresponding relationship in the first two-dimensional point cloud and the second two-dimensional point cloud can be determined, and then according to the position of the two-dimensional point cloud with the corresponding relationship in the three-dimensional space, the Point cloud sampling data or registration result of point cloud data. Specifically, the offset parameter and the rotation parameter can be determined, and through these parameters, the points with the corresponding relationship in the three-dimensional space can be made to overlap as much as possible.

Among them, when registering the point cloud data of adjacent frames, since the registration process between adjacent frames of different groups is independent, the parallel computing architecture of the graphics card in the electronic device can be used to parallelize multiple groups of phase data. The point cloud data of adjacent frames are registered to improve the registration speed.

Specifically, the electronic device may also acquire the local point cloud map constructed according to the point cloud data and/or the traveling speed information of the vehicle. Therefore, the point cloud data is obtained by performing motion compensation on the point cloud data according to the local point cloud map and/or the vehicle's traveling speed information.

The electronic device can construct a local point cloud map based on the point cloud data collected by the radar. During the driving process of the vehicle, the radar sensor can collect multiple frames of point cloud data, and the electronic device can process the multiple frames of point cloud data, and accumulate the point cloud data to obtain a local point cloud map.

The point cloud data collected by the radar sensor may be distorted when the vehicle is driving. Therefore, the collected point cloud data can be compensated according to the local point cloud map to obtain accurate point cloud data.

Further, the electronic device can also obtain the driving speed of the vehicle, and then perform motion compensation on the point cloud data collected by the radar through the driving speed, so as to obtain more accurate point cloud data.

In practical applications, the point cloud data of adjacent frames can be registered according to the compensated point cloud data. Since the compensated point cloud data is more accurate, the relative pose of the radar obtained is also more accurate by registering the compensated point cloud data.

Step 307, according to the radar rotation parameter and radar translation parameter between adjacent frames, and the inertial navigation rotation parameter and inertial navigation translation parameter between adjacent frames, determine the calibration parameters of the radar corresponding to the adjacent frame relative to the inertial measurement unit; wherein, The radar relative pose includes radar rotation parameters and radar translation parameters; the inertial navigation relative pose includes inertial rotation parameters and inertial translation parameters.

和雷达平移参数

通过惯导轨迹确定的惯导相对位姿包括惯导旋转参数

和惯导平移参数

Among them, the radar relative pose obtained by point cloud registration includes the radar rotation parameters

and radar panning parameters

The inertial navigation relative pose determined by the inertial navigation trajectory includes the inertial navigation rotation parameters

and INS translation parameters

用于表征雷达传感器在第j帧数据相对于第i帧数据的旋转参数；

用于表征雷达传感器在第j帧数据相对于第i帧数据的平移参数；

用于表征IMU在第j帧数据相对于第i帧数据的旋转参数，

用于表征IMU在第j帧数据相对于第i帧数据的平移参数。specific,

It is used to characterize the rotation parameter of the radar sensor in the jth frame data relative to the ith frame data;

It is used to characterize the translation parameter of the radar sensor in the jth frame of data relative to the ith frame of data;

It is used to characterize the rotation parameter of the IMU in the jth frame data relative to the ith frame data,

It is used to characterize the translation parameter of the IMU in the jth frame of data relative to the ith frame of data.

Further, at the moment corresponding to the adjacent frame, the pose changes of the radar sensor and the IMU should be the same. Therefore, the following formula can be constructed based on this to determine the calibration parameter R _BL of the radar corresponding to the adjacent frame relative to the IMU. and t _BL . R _BL refers to the rotation parameter of the radar relative to the inertial measurement unit, and t _BL refers to the translation parameter of the radar relative to the inertial measurement unit.

In practical application, R _BL and t _BL corresponding to each group of adjacent frames can be determined based on the above formula.

Step 308: Fit the calibration parameters of the radar relative to the inertial measurement unit corresponding to multiple groups of adjacent frames to obtain the calibration parameters of the radar relative to the inertial measurement unit; the calibration parameters include the rotation parameters and translation parameters of the radar relative to the inertial measurement unit .

In practical applications, the calibration parameters of the radar relative to the IMU can be determined for each group of adjacent frames, and these calibration parameters can be fitted to obtain the final calibration parameters of the radar relative to the IMU.

Among them, the rotation parameters of the multiple calibration parameters can be fitted to obtain the final rotation parameter R _BL of the radar relative to the IMU, and the translation parameters of the multiple calibration parameters can also be fitted to obtain the final radar relative to the IMU. Translation parameter t _BL .

In this embodiment, the radar and the IMU can be calibrated based on the relative pose of the IMU and the relative pose of the radar between adjacent data frames, and the rotation parameters and translation parameters between the radar and the IMU can be calibrated by changing the pose, This can improve the success rate of calibration.

In an optional implementation manner, a plurality of radar sensors may be provided in the vehicle. In this implementation manner, each radar sensor may be calibrated according to the above method to obtain the difference between each radar sensor and the IMU. For the calibration parameters, the final calibration parameters of each radar sensor can also be optimized according to the calibration parameters of each radar sensor and multiple groups of adjacent frames.

Step 309, according to the calibration parameter of the first radar corresponding to the adjacent frame relative to the inertial measurement unit, and the calibration parameter of the second radar corresponding to the adjacent frame relative to the inertial measurement unit, determine the first corresponding to the adjacent frame. The relative pose of the radar with respect to the second radar.

Wherein, based on the above step 308, the calibration parameters of the radar relative to the inertial measurement unit corresponding to each group of adjacent frames can be obtained, and the calibration parameters of the adjacent frames can be determined according to any two sensors in the plurality of radar sensors. The relative pose of each radar sensor. The first radar and the second radar here refer to radar sensors.

和

以及第二雷达N相较于IMU的

和

可以根据

和

和

确定出与第i、j帧对应的第一雷达M相较于第二雷达N的相对位姿

Specifically, for example, according to the data of the ith frame and the jth frame, it is possible to determine the difference between the first radar M and the IMU.

and

and the second radar N compared to the IMU's

and

can be based on

and

and

Determine the relative pose of the first radar M corresponding to the ith and jth frames compared to the second radar N

Further, for each group of adjacent frames, the relative pose of the first radar M compared to the second radar N can be determined. Since the installation positions of the first radar M and the second radar N on the vehicle are fixed, Therefore, the relative poses of the two are also fixed, and the obtained radar calibration parameters can be optimized by using the relative poses of the first radar M corresponding to each group of adjacent frames compared to the second radar N. Then the accurate calibration parameters of the radar can be obtained.

Step 310: Optimize the calibration parameters of the first radar and the calibration parameters of the second radar according to the relative poses of the first radar relative to the second radar of the plurality of sets of adjacent frames.

The calibration parameters of the first radar and the calibration parameters of the second radar can be calibrated based on the following equations:

是第二雷达N的标定参数，

是第一雷达M的标定参数，

是最小误差项。in,

is the calibration parameter of the second radar N,

is the calibration parameter of the first radar M,

is the minimum error term.

For each group of adjacent frames, the corresponding minimum error term can be determined. By adjusting the calibration parameters of the first radar M and the second radar N, the minimum error terms corresponding to each adjacent frame can meet the requirements. For example, each group of phase When the minimum error terms corresponding to adjacent frames are all smaller than the threshold, the optimized calibration parameters of the first radar M and the second radar N can be obtained.

After step 308 or 310, it may also include:

Step 311 , according to the calibration parameters of the radar corresponding to the adjacent frame relative to the inertial measurement unit, determine the relative pose of the radar between adjacent frames after registration, and the to-be-optimized state of the radar in each frame.

Steps 312 and 313 may be performed for each radar to optimize the to-be-optimized state of the radar.

Wherein, based on step 308, the calibration parameter of the radar corresponding to the adjacent frame relative to the inertial measurement unit can be determined, and the relative pose of the radar between adjacent frames after registration can be determined according to the calibration parameter.

Specifically, the pose of the radar at the ith frame and the pose of the radar at the jth frame can be determined according to the calibration parameter. For example, the ith frame can be determined according to the calibration parameter and the inertial navigation data of the ith frame. The pose of the radar is determined according to the calibration parameters and the inertial navigation data of the jth frame to determine the radar pose of the jth frame. Since the calibration parameters of the radar are relatively accurate parameters at this time, a relatively accurate radar pose can be determined based on the calibration parameters.

进而得到更加准确的雷达相对位姿。Further, the relative pose of the radar can be determined according to the registered radar pose

Then, a more accurate relative pose of the radar can be obtained.

In practical applications, the state to be optimized refers to the position and attitude of the radar compared to the global coordinate system. For example, the global coordinate system can be the UTM (Universal Transverse Mercator Grid System) coordinate system, which can be determined according to the The pose state of the radar in each frame and the pose state of the global coordinate system determine the to-be-optimized state of the radar in each frame.

For example, the rotation parameters and translation parameters of the radar compared to the global coordinate system.

Step 312: Optimize the state to be optimized according to the relative pose of the radar between adjacent frames after registration and the calibration parameters of the radar.

Among them, the to-be-optimized state of the radar can be optimized based on the following formula.

Specifically, through the above steps, accurate calibration parameters of the radar can be obtained, and the calibration parameters can be used to optimize the pose of the radar compared to the global coordinate system.

是计算得到的最小误差项，通过优化R_i、t_i、R_j、t_j使得最小误差项满足要求，从而得到优化后的R_i、t_i、R_j、t_j。Specifically, R _c , t _c refer to the calibration parameters of any radar, R _i , t _i refer to the to-be-optimized state of the radar in the ith frame, R _j , t _j refer to the to-be-optimized state of the radar in the jth frame optimized state.

is the calculated minimum error term. By optimizing R _i , t _i , R _j , and t _j , the minimum error term satisfies the requirements, so as to obtain the optimized Ri , t _i , R _j , and _{t j .}

Through this implementation, the state of the radar can be optimized according to the calibration parameters of the radar, and then the accurate state of the radar can be obtained.

FIG. 4 is a schematic structural diagram of an apparatus for calibrating external parameters of a lidar according to an exemplary embodiment of the present disclosure.

As shown in FIG. 4 , the apparatus 400 for calibrating external parameters of lidar provided by the present disclosure includes:

A data acquisition unit 410 is configured to acquire radar data collected by a radar sensor set on the vehicle, inertial navigation data collected by an inertial measurement unit set on the vehicle, and positioning data collected by a positioning system set on the vehicle ;

a trajectory determination unit 420, configured to determine the inertial navigation trajectory of the inertial measurement unit according to the positioning data and the inertial navigation data;

a pose determination unit 430, configured to determine the inertial navigation relative pose of the inertial measurement unit between adjacent frames according to the inertial navigation trajectory, and determine the radar relative pose of the radar between adjacent frames according to the radar data;

The calibration unit 440 is configured to determine calibration parameters of the radar relative to the inertial measurement unit according to the inertial navigation relative pose and the radar relative pose.

The device for calibrating the external parameters of the lidar provided by the present disclosure can estimate the relative pose of the IMU between adjacent data frames based on the IMU trajectory, and estimate the relative pose of the radar between adjacent frames based on the radar data, so that the relative pose of the radar between adjacent frames can be estimated based on the phase The relative pose of the IMU and the relative pose of the radar between adjacent data frames are used to calibrate the radar and the IMU. The rotation parameters and translation parameters between the radar and the IMU can be calibrated through the change of the pose, which can improve the success rate of calibration.

FIG. 5 is a schematic structural diagram of an apparatus for calibrating external parameters of a lidar according to another exemplary embodiment of the present disclosure.

As shown in FIG. 5 , in the apparatus 500 for calibrating external parameters of lidar provided by the present disclosure, the data acquisition unit 510 is similar to the data acquisition unit 410 shown in FIG. 4 , and the trajectory determination unit 520 is similar to the trajectory determination unit 520 shown in FIG. 4 Similar to unit 420, pose determination unit 530 is similar to pose determination unit 430 shown in FIG. 4, and calibration unit 540 is similar to calibration unit 440 shown in FIG.

In an optional implementation manner, the radar data includes point cloud data;

The pose determination unit 530 includes a radar pose determination module 531 for including:

The point cloud data of adjacent frames are registered to obtain the relative pose of the radar between adjacent frames.

In an optional implementation manner, the radar pose determination module 531 includes:

The sampling sub-module 5311 is used to perform multi-level sampling processing on the point cloud data of adjacent frames to obtain multi-level point cloud sampling data with different precisions of adjacent frames;

Hierarchical registration sub-module 5312 is used to register the point cloud sampling data and point cloud data of the adjacent frames in order from low-level precision to high-level precision; wherein, the low-level point cloud sampling data The registration result of , as the initial value of high-level point cloud sampling data registration;

The result of registering the point cloud data with the highest accuracy is the relative pose of the radar between the adjacent frames.

In an optional implementation manner, when the hierarchical registration sub-module 5312 registers the point cloud sampling data and point cloud data of the adjacent frames, it is specifically used for:

Obtain the first point cloud data and the second point cloud data of the same precision level of adjacent frames; wherein, the first point cloud data and the second point cloud data are point cloud sampling data, or the radar data the point cloud data included;

The first three-dimensional point cloud is mapped to the two-dimensional plane according to the first point cloud data to obtain the first two-dimensional point cloud, and the second three-dimensional point cloud is mapped to the two-dimensional plane according to the second point cloud data to obtain the first two-dimensional point cloud. 2D point cloud;

According to the positions of the one-dimensional point cloud and the second two-dimensional point cloud on the two-dimensional plane, the point cloud sampling data or point cloud data of the adjacent frames are registered.

In an optional implementation manner, the radar pose determination module 531 further includes a compensation sub-module 5313, which is used for the radar pose determination module 531 to perform registration on the point cloud data of adjacent frames before:

obtaining a local point cloud map constructed according to the point cloud data and/or the driving speed of the vehicle;

Perform motion compensation on the point cloud data according to the local point cloud map and/or the driving speed of the vehicle to obtain compensated point cloud data;

The radar pose determination module 531 is specifically used for:

The point cloud data of adjacent frames are registered according to the compensated point cloud data.

In an optional implementation manner, the radar pose determination module 531 specifically uses the parallel computing architecture of the graphics card in the electronic device to register the point cloud data of multiple groups of adjacent frames in parallel.

In an optional implementation manner, the radar relative pose includes a radar rotation parameter and a radar translation parameter; the inertial navigation relative pose includes an inertial rotation parameter and an inertial translation parameter;

The calibration unit 540 includes:

The inter-frame calibration module 541 is configured to determine the radar corresponding to the adjacent frame according to the radar rotation parameter and the radar translation parameter between adjacent frames, as well as the inertial navigation rotation parameter and the inertial navigation translation parameter between the adjacent frames calibration parameters relative to the inertial measurement unit;

The fitting module 542 is configured to fit the calibration parameters of the radar relative to the inertial measurement unit corresponding to multiple groups of adjacent frames to obtain the calibration parameters of the radar relative to the inertial measurement unit; the calibration Parameters include rotational and translational parameters of the radar relative to the inertial measurement unit.

In an optional embodiment, at least two of the radar sensors are provided on the vehicle;

The calibration unit 540 further includes an optimization module 543, which is configured to, after the fitting module 542 fits the calibration parameters of the radar relative to the inertial measurement unit corresponding to multiple groups of adjacent frames:

According to the calibration parameters of the first radar corresponding to the adjacent frame relative to the inertial measurement unit, and the calibration parameters of the second radar corresponding to the adjacent frame relative to the inertial measurement unit, it is determined that the adjacent the relative pose of the first radar relative to the second radar corresponding to the frame;

The calibration parameters of the first radar and the calibration parameters of the second radar are optimized according to a plurality of sets of relative poses of the first radar relative to the second radar corresponding to adjacent frames.

In an optional implementation manner, the apparatus further includes a state optimization unit 550 for:

According to the calibration parameters of the radar corresponding to the adjacent frame relative to the inertial measurement unit, determine the relative pose of the radar between adjacent frames after registration, and the to-be-optimized state of the radar in each frame;

The to-be-optimized state is optimized according to the relative pose of the radar between adjacent frames after registration and the calibration parameters of the radar.

In an optional implementation manner, the data acquisition unit 510 includes:

An initial data acquisition module 511, configured to acquire initial radar data collected by the radar sensor when the vehicle is running, initial inertial navigation data collected by the inertial measurement unit, and initial positioning data collected by a positioning system;

A data screening unit 512 is configured to screen out radar data satisfying a preset driving state from the initial radar data, filter out inertial navigation data satisfying the predetermined driving state from the initial inertial navigation data, and select the inertial navigation data satisfying the predetermined driving state in the initial The positioning data is filtered to select the positioning data that satisfies the preset driving state.

In an optional implementation manner, the trajectory determination unit 520 includes:

An initial trajectory determination module 521, configured to filter out reliable positioning data whose confidence reaches a preset value from the positioning data, and determine the initial driving trajectory of the vehicle according to the reliable positioning data;

The inertial navigation trajectory determination module 522 is configured to determine the inertial navigation trajectory of the inertial measurement unit according to the initial driving trajectory and the inertial navigation data.

In an optional implementation manner, the inertial navigation trajectory determination module 522 is specifically configured to:

determining the complete driving trajectory of the vehicle according to the initial driving trajectory and the inertial navigation data;

According to the relative position between the positioning system and the inertial measurement unit, offset processing is performed on the complete driving trajectory to obtain the inertial navigation trajectory of the inertial measurement unit.

The present disclosure provides a method, device, storage medium and program product for calibrating lidar external parameters, which are applied to high-precision maps and automatic driving technologies in data processing technology to improve the accuracy and success rate of radar external parameter calibration.

In the technical solutions of the present disclosure, the collection, storage, use, processing, transmission, provision, and disclosure of the user's personal information involved are all in compliance with relevant laws and regulations, and do not violate public order and good customs.

According to embodiments of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

According to an embodiment of the present disclosure, the present disclosure also provides a computer program product, the computer program product includes: a computer program, the computer program is stored in a readable storage medium, and at least one processor of the electronic device can read from the readable storage medium A computer program is taken, and at least one processor executes the computer program so that the electronic device executes the solution provided by any of the foregoing embodiments.

FIG. 6 shows a schematic block diagram of an example electronic device 600 that may be used to implement embodiments of the present disclosure. Electronic devices are intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. Electronic devices may also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are by way of example only, and are not intended to limit implementations of the disclosure described and/or claimed herein.

As shown in FIG. 6 , the device 600 includes a computing unit 601 that can be executed according to a computer program stored in a read only memory (ROM) 602 or a computer program loaded from a storage unit 608 into a random access memory (RAM) 603 Various appropriate actions and handling. In the RAM 603, various programs and data necessary for the operation of the device 600 can also be stored. The computing unit 601 , the ROM 602 , and the RAM 603 are connected to each other through a bus 604 . An input/output (I/O) interface 605 is also connected to bus 604 .

Various components in the device 600 are connected to the I/O interface 605, including: an input unit 606, such as a keyboard, mouse, etc.; an output unit 607, such as various types of displays, speakers, etc.; a storage unit 608, such as a magnetic disk, an optical disk, etc. ; and a communication unit 609, such as a network card, a modem, a wireless communication transceiver, and the like. The communication unit 609 allows the device 600 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

Computing unit 601 may be various general-purpose and/or special-purpose processing components with processing and computing capabilities. Some examples of computing units 601 include, but are not limited to, central processing units (CPUs), graphics processing units (GPUs), various specialized artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processing processor (DSP), and any suitable processor, controller, microcontroller, etc. The computing unit 601 executes the various methods and processes described above, such as the method for calibrating the external parameters of the lidar. For example, in some embodiments, the method of calibration of lidar extrinsic parameters may be implemented as a computer software program tangibly embodied on a machine-readable medium, such as storage unit 608 . In some embodiments, part or all of the computer program may be loaded and/or installed on device 600 via ROM 602 and/or communication unit 609 . When the computer program is loaded into the RAM 603 and executed by the computing unit 601, one or more steps of the method for calibrating the extrinsic parameters of the lidar described above may be performed. Alternatively, in other embodiments, the computing unit 601 may be configured in any other suitable manner (eg, by means of firmware) to perform a method of calibrating lidar extrinsic parameters.

Various implementations of the systems and techniques described herein above may be implemented in digital electronic circuitry, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips system (SOC), load programmable logic device (CPLD), computer hardware, firmware, software, and/or combinations thereof. These various embodiments may include being implemented in one or more computer programs executable and/or interpretable on a programmable system including at least one programmable processor that The processor, which may be a special purpose or general-purpose programmable processor, may receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device an output device.

Program code for implementing the methods of the present disclosure may be written in any combination of one or more programming languages. These program codes may be provided to a processor or controller of a general purpose computer, special purpose computer or other programmable data processing apparatus, such that the program code, when executed by the processor or controller, performs the functions/functions specified in the flowcharts and/or block diagrams. Action is implemented. The program code may execute entirely on the machine, partly on the machine, partly on the machine and partly on a remote machine as a stand-alone software package or entirely on the remote machine or server.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in connection with the instruction execution system, apparatus or device. The machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. Machine-readable media may include, but are not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), fiber optics, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

To provide interaction with a user, the systems and techniques described herein may be implemented on a computer having a display device (eg, a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user ); and a keyboard and pointing device (eg, a mouse or trackball) through which a user can provide input to the computer. Other kinds of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (eg, visual feedback, auditory feedback, or tactile feedback); and can be in any form (including acoustic input, voice input, or tactile input) to receive input from the user.

The systems and techniques described herein may be implemented on a computing system that includes back-end components (eg, as a data server), or a computing system that includes middleware components (eg, an application server), or a computing system that includes front-end components (eg, a user's computer having a graphical user interface or web browser through which a user may interact with implementations of the systems and techniques described herein), or including such backend components, middleware components, Or any combination of front-end components in a computing system. The components of the system may be interconnected by any form or medium of digital data communication (eg, a communication network). Examples of communication networks include: Local Area Networks (LANs), Wide Area Networks (WANs), and the Internet.

A computer system can include clients and servers. Clients and servers are generally remote from each other and usually interact through a communication network. The relationship of client and server arises by computer programs running on the respective computers and having a client-server relationship to each other. The server can be a cloud server, also known as a cloud computing server or a cloud host. It is a host product in the cloud computing service system to solve the traditional physical host and VPS service ("Virtual Private Server", or "VPS" for short). , there are the defects of difficult management and weak business expansion. The server can also be a server of a distributed system, or a server combined with a blockchain.

It should be understood that steps may be reordered, added or deleted using the various forms of flow shown above. For example, the steps described in the present disclosure can be executed in parallel, sequentially, or in different orders. As long as the desired results of the technical solutions disclosed in the present disclosure can be achieved, there is no limitation herein.

The above-mentioned specific embodiments do not constitute a limitation on the protection scope of the present disclosure. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions may occur depending on design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present disclosure should be included within the protection scope of the present disclosure.

Claims (27)

1. A calibration method of laser radar external parameters comprises the following steps:

acquiring radar data acquired by a radar sensor arranged on a vehicle, inertial navigation data acquired by an inertial measurement unit arranged on the vehicle and positioning data acquired by a positioning system arranged on the vehicle;

determining an inertial navigation track of the inertial measurement unit according to the positioning data and the inertial navigation data;

determining the inertial navigation relative pose of the inertial measurement unit between adjacent frames according to the inertial navigation track, and determining the radar relative pose of the radar between adjacent frames according to the radar data;

and determining calibration parameters of the radar relative to the inertial measurement unit according to the inertial navigation relative pose and the radar relative pose.

2. The method of claim 1, wherein the radar data includes point cloud data;

determining the relative pose of the radar between adjacent frames, comprising:

and registering the point cloud data of the adjacent frames to obtain the relative position and posture of the radar between the adjacent frames.

3. The method of claim 2, wherein the registering the point cloud data of adjacent frames to obtain the relative radar pose between adjacent frames comprises:

performing multi-level sampling processing on the point cloud data of adjacent frames to obtain multi-level point cloud sampling data with different accuracies of the adjacent frames;

sequentially registering the point cloud sampling data and the point cloud data of the adjacent frames according to the sequence from low-level precision to high-level precision; wherein, the registration result of the low-level point cloud sampling data is used as an initial value when the high-level point cloud sampling data is registered;

and the result of registering the point cloud data with the highest precision is the relative position and posture of the radar between the adjacent frames.

4. The method of claim 3, wherein the point cloud sampled data and point cloud data of the adjacent frames are registered by:

acquiring first point cloud data and second point cloud data of the same precision level of adjacent frames; wherein the first point cloud data and the second point cloud data are point cloud sampling data or the point cloud data included in the radar data;

mapping the first three-dimensional point cloud to a two-dimensional plane according to the first point cloud data to obtain a first two-dimensional point cloud, and mapping the second three-dimensional point cloud to the two-dimensional plane according to the second point cloud data to obtain a second two-dimensional point cloud;

and registering the point cloud sampling data or the point cloud data of the adjacent frames according to the positions of the two-dimensional point cloud and the second two-dimensional point cloud on a two-dimensional plane.

5. The method of claim 2, wherein prior to registering the point cloud data of adjacent frames, further comprising:

acquiring a local point cloud map constructed according to the point cloud data and/or the driving speed of the vehicle;

performing motion compensation on the point cloud data according to the local point cloud map and/or the driving speed of the vehicle to obtain compensated point cloud data;

the registering of the point cloud data of the adjacent frames comprises:

and registering the point cloud data of the adjacent frames according to the compensated point cloud data.

6. The method of any one of claims 2-5, wherein the registration of the sets of point cloud data of adjacent frames is performed in parallel using a parallel computing architecture of a graphics card in the electronic device.

7. The method of any of claims 1-6, wherein the radar relative pose comprises a radar rotation parameter and a radar translation parameter; the inertial navigation relative pose comprises inertial navigation rotation parameters and inertial navigation translation parameters;

determining calibration parameters of the radar relative to the inertial measurement unit according to the inertial navigation relative pose and the radar relative pose, wherein the calibration parameters comprise:

according to the radar rotation parameter and the radar translation parameter between adjacent frames and the inertial navigation rotation parameter and the inertial navigation translation parameter between adjacent frames, determining the calibration parameter of the radar corresponding to the adjacent frames relative to the inertial measurement unit;

fitting calibration parameters of the radar corresponding to a plurality of groups of adjacent frames relative to the inertial measurement unit to obtain the calibration parameters of the radar relative to the inertial measurement unit; the calibration parameters comprise rotation parameters and translation parameters of the radar relative to the inertial measurement unit.

8. The method of claim 7, wherein at least two of the radar sensors are disposed on the vehicle;

after the radar corresponding to the multiple groups of adjacent frames is fitted with respect to the calibration parameters of the inertial measurement unit, the method further includes:

determining the relative pose of a first radar corresponding to an adjacent frame relative to a second radar according to the calibration parameters of the first radar corresponding to the adjacent frame relative to the inertial measurement unit and the calibration parameters of the second radar corresponding to the adjacent frame relative to the inertial measurement unit;

and optimizing calibration parameters of the first radar and calibration parameters of the second radar according to the relative poses of the first radar and the second radar corresponding to the adjacent frames.

9. The method of claim 7, further comprising:

determining the relative pose of the radar between adjacent frames after registration and the state of the radar to be optimized in each frame according to the calibration parameters of the radar corresponding to the adjacent frames relative to the inertial measurement unit;

and optimizing the state to be optimized according to the relative pose of the radar between adjacent frames after registration and the calibration parameters of the radar.

10. The method of any one of claims 1-9, wherein the acquiring radar data collected by a radar sensor disposed on a vehicle, inertial navigation data collected by an inertial measurement unit disposed on the vehicle, and positioning data collected by a positioning system disposed on the vehicle comprises:

acquiring initial radar data acquired by the radar sensor, initial inertial navigation data acquired by the inertial measurement unit and initial positioning data acquired by a positioning system when the vehicle runs;

and screening out radar data meeting a preset driving state from the initial radar data, screening out inertial navigation data meeting the preset driving state from the initial inertial navigation data, and screening out positioning data meeting the preset driving state from the initial positioning data.

11. The method according to any one of claims 1-9, wherein said determining an inertial navigation trajectory of said inertial measurement unit from said positioning data, said inertial navigation data, comprises:

screening reliable positioning data with confidence coefficient reaching a preset value from the positioning data, and determining an initial running track of the vehicle according to the reliable positioning data;

and determining the inertial navigation track of the inertial measurement unit according to the initial driving track and the inertial navigation data.

12. The method of claim 11, wherein the determining an inertial navigation trajectory of the inertial measurement unit from the initial travel trajectory, the inertial navigation data, comprises:

determining a complete driving track of the vehicle according to the initial driving track and the inertial navigation data;

and carrying out deviation processing on the complete running track according to the relative position between the positioning system and the inertial measurement unit to obtain an inertial navigation track of the inertial measurement unit.

13. A calibration device for laser radar external parameters comprises:

the system comprises a data acquisition unit, a data acquisition unit and a data processing unit, wherein the data acquisition unit is used for acquiring radar data acquired by a radar sensor arranged on a vehicle, inertial navigation data acquired by an inertial measurement unit arranged on the vehicle and positioning data acquired by a positioning system arranged on the vehicle;

the track determining unit is used for determining an inertial navigation track of the inertial measurement unit according to the positioning data and the inertial navigation data;

the position and pose determining unit is used for determining the inertial navigation relative position and pose of the inertial measuring unit between adjacent frames according to the inertial navigation track and determining the radar relative position and pose of the radar between adjacent frames according to the radar data;

and the calibration unit is used for determining calibration parameters of the radar relative to the inertial measurement unit according to the inertial navigation relative pose and the radar relative pose.

14. The apparatus of claim 13, wherein the radar data comprises point cloud data;

the pose determination unit comprises a radar pose determination module for:

and registering the point cloud data of the adjacent frames to obtain the relative position and posture of the radar between the adjacent frames.

15. The apparatus of claim 14, wherein the radar pose determination module comprises:

the sampling submodule is used for carrying out multi-level sampling processing on the point cloud data of the adjacent frames to obtain multi-level point cloud sampling data with different accuracies of the adjacent frames;

the hierarchical registration submodule is used for sequentially registering the point cloud sampling data and the point cloud data of the adjacent frames according to the sequence from low-level precision to high-level precision; wherein, the registration result of the low-level point cloud sampling data is used as an initial value when the high-level point cloud sampling data is registered;

and the result of registering the point cloud data with the highest precision is the relative position and posture of the radar between the adjacent frames.

16. The apparatus of claim 15, wherein the hierarchical registration sub-module, when registering the point cloud sample data and the point cloud data of the adjacent frames, is specifically configured to:

acquiring first point cloud data and second point cloud data of the same precision level of adjacent frames; wherein the first point cloud data and the second point cloud data are point cloud sampling data or the point cloud data included in the radar data;

mapping the first three-dimensional point cloud to a two-dimensional plane according to the first point cloud data to obtain a first two-dimensional point cloud, and mapping the second three-dimensional point cloud to the two-dimensional plane according to the second point cloud data to obtain a second two-dimensional point cloud;

and registering the point cloud sampling data or the point cloud data of the adjacent frames according to the positions of the two-dimensional point cloud and the second two-dimensional point cloud on a two-dimensional plane.

17. The apparatus of claim 14, wherein the radar pose determination module further comprises a compensation sub-module to, prior to registering the point cloud data of adjacent frames:

acquiring a local point cloud map constructed according to the point cloud data and/or the driving speed of the vehicle;

performing motion compensation on the point cloud data according to the local point cloud map and/or the driving speed of the vehicle to obtain compensated point cloud data;

the radar pose determination module is specifically configured to:

and registering the point cloud data of the adjacent frames according to the compensated point cloud data.

18. The apparatus according to any one of claims 14-17, wherein the radar pose determination module is configured to register sets of point cloud data of adjacent frames in parallel, in particular using a parallel computing architecture of a graphics card in the electronic device.

19. The apparatus of any of claims 13-18, wherein the radar relative pose comprises a radar rotation parameter and a radar translation parameter; the inertial navigation relative pose comprises inertial navigation rotation parameters and inertial navigation translation parameters;

the calibration unit comprises:

the inter-frame calibration module is used for determining calibration parameters of the radar corresponding to the adjacent frames relative to the inertial measurement unit according to the radar rotation parameters and the radar translation parameters between the adjacent frames and the inertial navigation rotation parameters and the inertial navigation translation parameters between the adjacent frames;

the fitting module is used for fitting calibration parameters of the radar corresponding to a plurality of groups of adjacent frames relative to the inertial measurement unit to obtain the calibration parameters of the radar relative to the inertial measurement unit; the calibration parameters comprise rotation parameters and translation parameters of the radar relative to the inertial measurement unit.

20. The apparatus of claim 19, wherein at least two of the radar sensors are disposed on the vehicle;

the calibration unit further comprises an optimization module, configured to, after the fitting module fits calibration parameters of the radar relative to the inertial measurement unit corresponding to multiple sets of adjacent frames:

determining the relative pose of a first radar corresponding to an adjacent frame relative to a second radar according to the calibration parameters of the first radar corresponding to the adjacent frame relative to the inertial measurement unit and the calibration parameters of the second radar corresponding to the adjacent frame relative to the inertial measurement unit;

and optimizing calibration parameters of the first radar and calibration parameters of the second radar according to the relative poses of the first radar and the second radar corresponding to the adjacent frames.

21. The apparatus of claim 19, further comprising a state optimization unit to:

determining the relative pose of the radar between adjacent frames after registration and the state of the radar to be optimized in each frame according to the calibration parameters of the radar corresponding to the adjacent frames relative to the inertial measurement unit;

and optimizing the state to be optimized according to the relative pose of the radar between adjacent frames after registration and the calibration parameters of the radar.

22. The apparatus according to any one of claims 13-21, wherein the data acquisition unit comprises:

the initial data acquisition module is used for acquiring initial radar data acquired by the radar sensor, initial inertial navigation data acquired by the inertial measurement unit and initial positioning data acquired by a positioning system when the vehicle runs;

and the data screening unit is used for screening out radar data meeting a preset driving state from the initial radar data, screening out inertial navigation data meeting the preset driving state from the initial inertial navigation data, and screening out positioning data meeting the preset driving state from the initial positioning data.

23. The apparatus of any one of claims 13-21, wherein the trajectory determination unit comprises:

the initial track determining module is used for screening reliable positioning data with confidence coefficient reaching a preset value from the positioning data and determining an initial running track of the vehicle according to the reliable positioning data;

and the inertial navigation track determining module is used for determining the inertial navigation track of the inertial measurement unit according to the initial driving track and the inertial navigation data.

24. The apparatus of claim 23, wherein the inertial navigation trajectory determination module is specifically configured to:

determining a complete driving track of the vehicle according to the initial driving track and the inertial navigation data;

and carrying out deviation processing on the complete running track according to the relative position between the positioning system and the inertial measurement unit to obtain an inertial navigation track of the inertial measurement unit.

25. An electronic device, comprising:

at least one processor; and

a memory communicatively coupled to the at least one processor; wherein,

the memory stores instructions executable by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-12.

26. A non-transitory computer readable storage medium having stored thereon computer instructions for causing the computer to perform the method of any one of claims 1-12.

27. A computer program product comprising a computer program which, when executed by a processor, carries out the steps of the method of any one of claims 1 to 12.

Images