CN118405130A - A method, system and medium for off-road environment perception and tracking and guiding vehicles - Google Patents

Abstract

The present invention discloses a method, system and medium for sensing and tracking a guiding vehicle in an off-road environment, and relates to the technical field of following vehicle control. The method is applied to an off-road environment without positioning signals such as GPS, to accurately sense a guiding vehicle in front, and on this basis, to realize autonomous following vehicle by taking the guiding vehicle in front as a moving following target. The method comprises the following steps: arranging a sensing sensor installed on an autonomous following vehicle, wherein the sensing sensor comprises a camera, a 4D millimeter wave radar and an FMCW laser radar; acquiring detection information of the sensing sensor; and executing an initial recognition process, a continuous tracking process or a target retrieval process for the guiding vehicle according to the detection information. The present invention tracks the guiding vehicle in a manner combining a camera, a 4D millimeter wave radar and an FMCW laser radar, and can effectively follow the vehicle in difficult working conditions or harsh environments such as without GPS, at high vehicle speed, at close distance and under dust, thereby making up for the deficiencies of the prior art.

Description

A method, system and medium for off-road environment perception and tracking and guiding vehicles

Technical Field

The present invention relates to the technical field of vehicle following control, and in particular to a method, system and medium for sensing an off-road environment and tracking and guiding a vehicle.

Background technique

The unmanned vehicle target recognition and following system is an intelligent vehicle control system that uses advanced sensors, computer vision, artificial intelligence and other technologies to enable unmanned vehicles to autonomously follow the guide vehicle. The autonomous following function can be applied to autonomous logistics and transportation support, effectively reducing the number of drivers or remote control personnel, while improving the intelligence level of long-distance transportation, and is widely used in the field of special transportation.

In the existing technology, the main implementation methods for guiding vehicle target recognition and tracking include TOF laser radar, camera, millimeter wave radar, radio communication and GPS, etc. These vehicle following technologies each have the following defects:

1. Target recognition and tracking based on TOF LiDAR: TOF LiDAR uses geometric modeling to identify and track targets, but point cloud data becomes sparse as distance increases, and detection of long-distance targets is limited. Close-range tracking may pose a safety risk during high-speed tracking. In addition, lasers cannot penetrate dust. In off-road environments, a large amount of dust raised by the high-speed driving of the guide vehicle may cause misidentification of the TOF LiDAR, or even completely obscure the guide vehicle, rendering the target recognition function of the unmanned vehicle ineffective. See Figure 6. In Figure 6, it can be clearly seen that a large number of point clouds have appeared that interfere with laser detection. Under more severe working conditions, the point cloud representing the dust in front of the vehicle and the point cloud representing the guide vehicle are completely mixed into a large mass and cannot be segmented or detected.

2. Target recognition and tracking based on visual images: Visual target recognition requires conversion from the image coordinate system to the vehicle coordinate system. There are errors in the conversion, which is not conducive to the precise tracking control and high-speed following control of the following vehicle. Visual sensors are greatly affected by illumination, visual angle occlusion, etc., especially in off-road environments where there is a lot of dust, making it difficult to apply to high-speed and precise following systems in off-road environments. The poor working conditions of vision are also shown in Figure 6. In addition, at night, conventional cameras are unreliable, and expensive night vision cameras must be used. However, the lack of cameras also leads to the lack of stable feature recognition in the system, especially the self-recovery process after the target is lost is very unstable, so it is necessary to reasonably use visual images in a certain form. This application uses 4D millimeter wave radar combined with FMCW laser radar, and uses these two as the main detection methods. The success rate of stable tracking is already very high, so in order to save costs, no expensive night vision cameras are selected. In addition, relying on visual images is prone to false detection. For example, a car on the highway mistakenly thinks the photo of a car on a billboard is a vehicle, automatically brakes, and causes an accident. Therefore, in this application, vision is mainly used as an auxiliary detection for starting and lost recovery.

3. Target recognition and tracking based on millimeter-wave radar: Millimeter-wave radar can effectively track dynamic targets, but the resolution of millimeter-wave radar is low and cannot show the outline, and cannot accurately perceive the size and center of the guide vehicle, which seriously affects the direction control accuracy when following the vehicle at high speed. In addition, millimeter-wave radar has difficulty in detecting and distinguishing static vehicles. When the vehicle target is initialized, it is easy to cause the tracking target initialization to fail. In addition, it is difficult to effectively retrieve the target when the target is lost. See Figure 7, which is a working condition of millimeter-wave radar detection. At this time, the laser radar point cloud can still detect the rear of the front vehicle, but the millimeter wave has lost the target. The millimeter-wave radar itself cannot detect the point cloud. In this case, the algorithm built into the sensor has the probability of discarding the information of the front vehicle. The external manifestation is that the raw data of the sensor is unreliable and cannot feedback all the target information of the suspected guide vehicle.

4. Target tracking based on radio communication transmission of position information: It requires major modifications to the guide vehicle, including the installation of a positioning system and a radio communication system. This method has certain limitations and is not suitable for situations where versatility and economy are required. In addition, there are potential safety risks, such as the influence of radio transmission delays and electromagnetic interference. Due to the existence of communication delays, the position accuracy of the guide vehicle may be affected to a certain extent, thereby affecting the control accuracy of the following vehicle. Therefore, in the off-road conditions targeted by this application, radio cannot be used.

5. Target tracking based on the Global Positioning System: GPS can be used to track the guide vehicle, but the GPS signal is unstable in off-road environments. When passing through tunnels, forests, and valleys, there will be obvious signal attenuation or loss, resulting in inaccurate satellite positioning results, which makes it difficult to adapt to the requirements of high-speed and precise vehicle following in complex off-road environments. Under special conditions, the satellite positioning signal will be severely distorted and unusable. Therefore, GPS cannot be used in the off-road conditions targeted by this application. Other positioning systems, including Beidou, have the same problem.

To summarize, TOF laser radar cannot handle the dust problem caused by high-speed driving in off-road environments; cameras have difficulty determining the distance of the target object and the dust problem caused by high-speed driving in off-road environments; millimeter-wave radar has a low resolution and easily identifies objects that are close to each other as the same object; radio communication costs are too high, and there are problems with electromagnetic interference and communication delays; GPS signals are unstable in off-road environments and are prone to inaccurate positioning, which affects the accuracy of position determination.

When following a vehicle at a close distance at high speed in an off-road environment, the trajectory control accuracy of the unmanned vehicle is very high. This is because the road conditions in the off-road environment are poor. Even if the trajectory deviation is small, it may hit obstacles such as puddles and cliffs, resulting in serious consequences. Although the aforementioned sensors can be fused to a certain extent, the performance is poor even with fusion perception in harsh working conditions such as dust, rain, snow, and fog.

Summary of the invention

The purpose of the present invention is to provide a method, system and medium for off-road environment perception and tracking and guiding vehicles in response to the above-mentioned problems in the prior art, which combines cameras, 4D millimeter-wave radars and FMCW lidars to solve the problems of the prior art that the vehicle is easily lost and has low accuracy in off-road environments such as high speed, close distance and dust.

In order to solve the above technical problems, the specific technical solutions of the present invention are as follows:

In one aspect, the present invention provides a method for off-road environment perception and tracking and guiding a vehicle, comprising the following steps:

Confirm autonomous following vehicles and leading vehicles;

A sensor is provided to be installed on the autonomous following vehicle, wherein the sensor includes a camera, a 4D millimeter wave radar and a FMCW laser radar;

Acquiring detection information of the sensor;

An initial identification process, a continuous tracking process or a target retrieval process for the guide vehicle is executed according to the detection information.

As an improved solution, the initial recognition process includes: guiding the vehicle into the initial detection range, manually inputting the characteristic information of the guiding vehicle, and the perception process of the autonomous following vehicle enters the initial recognition state. If the recognition fails, the characteristic information of the guiding vehicle is re-entered or the position of the guiding vehicle is adjusted. If the recognition is successful, the continuous tracking state is entered, and the guiding vehicle is then defined as the tracking target.

As an improved solution, the continuous tracking process includes: if the target is not lost in the continuous tracking state, then the continuous tracking state is maintained; if the target is lost, then the target retrieval state is entered.

As an improved solution, the target retrieval process includes: if the target is successfully retrieved, the tracking state is maintained; if the target is not retrieved, the vehicle is stopped and the initial recognition process is returned.

As an improved solution, the camera is used to obtain image information and transmit it to a computer. The computer implements a target detection function through deep learning and using a pre-trained data set and a target detection algorithm. The computer identifies the features of the guide vehicle through the target detection function and generates and saves feature data.

The 4D millimeter wave radar is used to obtain 4D millimeter wave point cloud data to obtain the distance information, direction information, speed information, width information and height information of the target. The millimeter wave can penetrate non-metallic materials such as dust, rain, snow and fog, thereby assisting in locking the guide vehicle;

The FMCW laser radar is used to obtain FMCW laser radar point cloud data, which contains distance information, azimuth information, speed information, width information and height information of each point. FMCW is superior to ToF laser radar in many aspects such as distance measurement, speed measurement, anti-interference, power (eye safety), signal-to-noise ratio, etc. FMCW laser radar uses the Doppler effect to directly obtain radial velocity, and can obtain the velocity of each point in the million point cloud. Therefore, although FMCW laser radar cannot penetrate dust, rain, snow and fog, it can effectively distinguish between dust and guide vehicles by feeding back the velocity information of the point cloud, thereby assisting in locking the guide vehicle.

As an improved solution, performing an initial identification process according to the detection information further includes:

Processing the FMCW laser radar point cloud data and the 4D millimeter wave point cloud data, removing dust from the point cloud, and projecting them onto the image information, performing initial fusion of the image and point cloud information, and obtaining point cloud fusion data;

The manually input guide vehicle characteristic information is matched with the point cloud fusion data. If the match is successful, the position information, structure information, speed information, and direction information of the guide vehicle are obtained; if the match fails, the guide vehicle is readjusted to enter the initial detection range or the guide vehicle characteristic information is re-entered.

As an improved solution, the continuous tracking process is performed according to the detection information, further comprising:

Obtaining the direction information, speed information, and position information of the guide vehicle at the last moment, and determining the area of interest according to the direction information, speed information, and position information of the guide vehicle at the last moment;

The current position information of the guide vehicle, FMCW lidar point cloud data, and 4D millimeter wave point cloud data are obtained and matched with the area of interest. If the match is successful, the route is planned according to the current point cloud fusion data of the guide vehicle and the vehicle is continuously followed. If there is no match, the target retrieval state is entered.

As an improved solution, executing the target retrieval process according to the detection information further includes:

The position information, direction information, and speed information of the guide vehicle before it is lost are obtained, and the position of the guide vehicle at the next moment is estimated. The point cloud fusion data is filtered according to the estimated position of the guide vehicle at the next moment. The filtered point cloud fusion data is matched with the feature data. If they match, the system enters a continuous following state. If they do not match, the system stops and enters an initial recognition state.

On the other hand, the present invention also provides a system for off-road environment perception and tracking and guiding vehicles, comprising:

A sensor is provided on the autonomous following vehicle, and includes a camera, a 4D millimeter wave radar, and a FMCW laser radar;

A control unit is used to obtain detection information from the sensor and execute an initial recognition process, a continuous tracking process or a target retrieval process according to the detection information.

On the other hand, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the method for off-road environment perception and tracking and guiding a vehicle are implemented.

The beneficial effects of the technical solution of the present invention are:

The present invention uses a camera to initialize and lock the guided vehicle, and uses FMCW laser radar point cloud and 4D millimeter wave radar point cloud for verification, which greatly improves the accuracy of guided vehicle identification.

After locking the guiding vehicle, the FMCW lidar and 4D millimeter-wave radar are combined to continuously follow the vehicle, which can achieve continuous following in harsh environments such as high speed, close distance, and dust.

After the target is lost, the position of the guide vehicle is estimated using the guide vehicle’s last perception information, and the vehicle is briefly followed. The guide vehicle locked by the FMCW lidar point cloud and 4D millimeter-wave radar point cloud is projected onto the camera image to search for the guide vehicle, which has a good target recovery capability.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the specific implementation methods of the present invention or the technical solutions in the prior art, the drawings required for use in the specific implementation methods or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are some implementation methods of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

FIG1 is a schematic flow chart of a method for off-road environment perception and tracking and guiding a vehicle according to Embodiment 1 of the present invention;

FIG2 is a detailed flow diagram of initial identification in the method for off-road environment perception and tracking and guiding a vehicle according to Embodiment 1 of the present invention;

FIG3 is a detailed logic diagram of the continuous tracking process in the method for off-road environment perception and tracking and guiding a vehicle according to Embodiment 1 of the present invention;

4 is a detailed flow chart of target retrieval in the method for off-road environment perception and tracking and guiding a vehicle according to Embodiment 1 of the present invention;

5 is a schematic diagram of the system for off-road environment perception and tracking and guiding vehicles according to Embodiment 2 of the present invention;

FIG6 is a diagram showing the effect of target recognition and tracking based on TOF laser radar in the prior art;

FIG. 7 is a diagram showing the effect of target recognition and tracking based on millimeter-wave radar in the prior art.

Detailed ways

The preferred embodiments of the present invention are described in detail below in conjunction with the accompanying drawings so that the advantages and features of the present invention can be more easily understood by those skilled in the art, thereby making a clearer and more definite definition of the protection scope of the present invention.

In the description of the present invention, it should be noted that the embodiments described in the present invention are only part of the embodiments of the present invention, rather than all of the embodiments; based on the embodiments in the present invention, all other embodiments obtained by ordinary technicians in the field without making creative work are within the scope of protection of the present invention.

Embodiment 1: This embodiment provides a method for off-road environment perception and tracking and guiding a vehicle, as shown in FIG1 , comprising the following steps:

Configure perception sensors: configure cameras, 4D millimeter-wave radars, and FMCW laser radars on the unmanned vehicle. The cameras are used to obtain image information, the 4D millimeter-wave radars are used to obtain 4D millimeter-wave point cloud data, and the FMCW laser radars are used to obtain FMCW laser radar point cloud data.

Initial identification: The guided vehicle first enters the initial detection range, and then the guided vehicle features are manually input to enter the initial identification state. If the identification fails, the guided vehicle features are re-entered. If the identification is successful, it enters the continuous tracking state;

Continuous tracking: If the target is not lost in the continuous tracking state, the continuous tracking state is maintained; if the target is lost, the target retrieval state is entered;

Target retrieval: If the target is successfully retrieved, the tracking state will continue. If the target is not retrieved, the vehicle will stop and readjust the guided vehicle to enter the initial detection range, input the guided vehicle feature information, and perform initial identification.

As an embodiment of the present invention, the camera is used to provide high-resolution image information; the camera accurately identifies the guided vehicle through target detection technology. Specifically, the camera transmits the captured image information to the computer, and the computer realizes the target detection function through deep learning and using pre-trained data sets and target detection algorithms. The target detection function identifies the characteristics of the guided vehicle, such as a specific model, color, specific license plate, special markers, etc., and can accurately identify the guided vehicle and save the feature data. And the feature data can be continuously verified during the tracking process to determine whether the guided vehicle is followed. When the target tracking fails, the camera can also be combined with 4D millimeter wave radar and FMCW laser radar point cloud fusion data to recalculate the guided vehicle position and estimate the possible position of the guided vehicle. It should be noted that using camera images, based on deep learning methods, to detect vehicles and their features is already a conventional and common basic method in the field of autonomous driving, and there are many general open source detection algorithms, such as the yolo series. This application does not make in-depth creations based on camera vehicle detection, nor does it protect this. It is only a means adopted by this application. This application focuses on protecting the entire set of perception methods, namely 1. Selecting the most effective sensor; 2. The overall perception process, especially the mutual correction and fusion of the perception results of the guide vehicle between different sensors, and the processing method after the loss; 3. The specific process of each sub-process. This application is mainly devoted to how to make good use of the best sensors for detection, the most effective correlation between each other, minimize the failure rate, and achieve the highest probability of stable tracking.

As an embodiment of the present invention, 4D millimeter-wave radar can obtain more target height information than traditional millimeter-wave radar. 4D millimeter-wave radar can sense point cloud information and obtain target distance, direction, speed and height information, which can be used to distinguish roadblocks, sidewalks, low buildings, etc. The 4D millimeter-wave data height information can also help the system build a more accurate three-dimensional model of the environment and complete the autonomous navigation system's vehicle following task.

As an embodiment of the present invention, FMCW laser radar can provide very high close-range measurement accuracy, usually between a few millimeters and a few centimeters, can generate high-density point cloud data, and can obtain data for each point in the point cloud, capture the details of the target and the environment, and provide accurate spatial information, which can be used for accurate mapping and target positioning. The point cloud density of FMCW laser radar is relatively high, so it can provide high-precision resolution. However, FMCW laser radar may be affected under some harsh environmental conditions. The water vapor and particles in the haze will scatter the laser beam, reduce the detection distance and resolution of FMCW laser radar, and make obstacle detection difficult. The sand and dust particles suspended in the sandstorm will interfere with the propagation path of FMCW laser radar, causing signal attenuation and data instability. At this time, 4D millimeter wave radar is used as a supplement. The wavelength of 4D millimeter wave signal is shorter, and it can penetrate non-metallic media relatively easily without being affected too much. Combined with FMCW laser radar point cloud data, it can significantly improve the perception ability under special meteorological or environmental conditions.

As an embodiment of the present invention, as shown in Figure 2, the initial identification includes: projecting the FMCW lidar point cloud data and the 4D millimeter wave point cloud data into the image information, performing initial fusion of the image and point cloud information, and obtaining point cloud fusion data; then manually inputting the characteristic information of the guiding vehicle, matching the characteristic information with the point cloud fusion data, and if the match is successful, obtaining the position, structure, speed, and direction information of the guiding vehicle; if the match fails, readjusting the vehicle to enter the detection range or re-entering the characteristic information of the guiding vehicle.

As an embodiment of the present invention, the initial recognition further includes: firstly fusing the 4D millimeter wave radar point cloud and the FMCW laser radar point cloud, removing dust, etc., and then projecting the point cloud onto the image to further determine the target; specifically including the following steps:

1. Establish the vehicle coordinate system, and pay attention to unifying the coordinate systems of the FMCW laser radar and the 4D millimeter wave radar; the origin is located in the middle of the front of the unmanned vehicle, the right is the positive direction of the x-axis, the front is the positive direction of the y-axis, and the upward is the positive direction of the z-axis, which conforms to the right-hand rule.

2. Each time a detection is performed, the point cloud P _k-1 of the perception result of the preceding vehicle at the previous moment, as well as the complete point cloud P' _kr of the 4D millimeter-wave radar and the complete point cloud P' _kl of the FMCW lidar at the current moment are first obtained. It is particularly noted that the FMCW lidar point cloud and the 4D millimeter-wave radar can sense and obtain the radial velocity vr of each point, while the existing ordinary lidar and millimeter-wave radar cannot obtain this parameter.

For _Pk-1 , obtain the position of the vehicle at the last moment, that is, p _k-1 .x, p _k-1 .y, p _k-1 .z, perform point cloud clustering in _P'kr and _P'kl , and select the point cloud cluster near the position, including the point cloud cluster _Pkr of the 4D millimeter wave radar at this moment and the point cloud cluster _Pkl of the FMCW laser radar at this moment; Note that it is not suitable to perform laser point cloud recognition based on deep learning, such as the PointPillar method, because there is dust, etc., and the effect of the existing point cloud deep learning method is poor. Then do the following screening:

1) Use _Pkr to segment and filter the point cloud, that is, traverse _Pkl

select pi in P _kl which satisfy

max distance(pi, points in P _kl )<threshold_1

return pi

After this screening, the point cloud and penetration effect of 4D millimeter wave can be used to remove some noise points such as dust; the screened pi points are recombined into P _kl ;

2) Using each point p _k-1 of the point cloud at the last moment, calculate the radial velocity v of the guide vehicle, that is

v = average(p _k-1 .vr)

Perform point cloud segmentation and screening on P _kl , that is, traverse P _kl

select pi in P _kl which satisfy

distance(pi.vr, v)<threshold_2

return pi

After this screening, we can further remove some noise points such as dust with the help of the velocity information of the points of the FMCW laser radar, and reassemble the screened pi points into P _kl ;

3) Use P _kr to supplement the point cloud, that is, traverse P _kr

select pi in P _kr which satisfy

max distance(pi, points in P _kl )<threshold_3

return pi

After this screening, the points that have not been detected by the laser radar can be supplemented. This is because dust will block part of the laser from being emitted to the guide vehicle. These parts will not have laser point clouds, but will have millimeter wave point clouds. The selected pi points are supplemented with the points of P _kl to obtain the point cloud P _k guided at this moment.

3. In most working conditions of this application, only the fusion data of 4D millimeter wave radar and FMCW radar can accurately obtain the outline of the guide vehicle; when initially looking for the vehicle, or when verification or lost tracking is required, the camera signal will be used for further matching of the point cloud; at this time:

1) Use the intrinsic and extrinsic matrix to convert the 3D point cloud data to the 2D image coordinates in the camera coordinate system; this process includes transforming the point cloud data from the point cloud coordinate system to the camera coordinate system, and using the camera intrinsic parameters to project the 3D coordinates to the 2D image plane, which can be achieved through the projection matrix.

2) Perform deep learning detection on the image to determine whether there is a vehicle in the point cloud projection area and whether the vehicle features are consistent with the predetermined/recorded vehicle features.

As an embodiment of the present invention, as shown in FIG3 , the continuous tracking includes: determining a region of interest ROI (Region of interest) based on the direction, speed, and position information of the guiding vehicle at the previous moment, matching the guiding vehicle's current position information, FMCW lidar point cloud data, and 4D millimeter wave point cloud data with the ROI, and if the match is successful, planning a driving route based on the guiding vehicle's point cloud data, and continuously following the vehicle, and if the guiding vehicle's point cloud data does not match the ROI, entering a target retrieval state.

As an embodiment of the present invention, as shown in Figure 4, the target retrieval includes: estimating the position of the guide vehicle at the next moment based on the position, direction, and speed of the guide vehicle at the previous moment before it was lost, filtering point cloud data based on the position estimation result, matching the filtered point cloud data with the result of image detection, and entering a continuous following state if they match; if not, stopping and entering an initial recognition state.

Example 2. This example is based on the same inventive concept as the method for off-road environment perception and tracking and guiding vehicles described in Example 1, and provides a system for off-road environment perception and tracking and guiding vehicles. The system adopts a combination of cameras, FMCW laser radars, and 4D millimeter-wave radars to realize a vehicle following perception system, that is, to perceive the position and contour of the guiding vehicle with high precision in an off-road high-speed and close-range environment, and send the results to the subsequent automatic vehicle following motion control unit.

The system architecture is shown in Figure 5. The camera's image information, FMCW lidar point cloud information, and 4D millimeter wave point cloud information are combined to initialize the locking and guiding vehicle information. The 4D millimeter wave radar is used to lock the guiding vehicle and continue to follow the vehicle. The vehicle position information is transmitted to the vehicle motion control unit. If the target is lost, the possible position of the guiding vehicle is estimated based on the vehicle's last position, direction, and speed information, and the guiding vehicle is retrieved. If the vehicle is successfully retrieved, the following will continue. If the retrieval fails, the system will stop.

Embodiment 3. This embodiment provides a computer-readable storage medium, and the storage medium is used to store computer software instructions used to implement the method for off-road environment perception and tracking and guiding the vehicle described in the above-mentioned embodiment 1, which includes a program for executing the above-mentioned method for off-road environment perception and tracking and guiding the vehicle; specifically, the executable program can be built into the device for off-road environment perception and tracking and guiding the vehicle described in embodiment 2, so that the multi-mode autonomous following control device can implement the method for off-road environment perception and tracking and guiding the vehicle described in embodiment 1 by executing the built-in executable program.

The camera has high definition, but the position information is not accurate enough. It can be used to lock and guide the vehicle; the laser radar has a high resolution but the effect is poor in rain, snow, fog and dust environments; the 4D millimeter wave radar can penetrate rain, snow, fog and dust, and can be used as a supplement to follow the vehicle when the FMCW laser radar is less effective; different from the prior art, this application uses a combination of cameras, FMCW laser radars, and 4D millimeter wave radars to achieve vehicle following perception. The camera is used to initialize and lock the guiding vehicle, and the FMCW laser radar and 4D millimeter wave radar are used for continuous following. The high resolution of the camera is used to lock and guide the vehicle, and the laser radar point cloud and 4D millimeter wave radar point cloud are used for verification, which greatly improves the accuracy of vehicle identification. After locking and guiding the vehicle, the FMCW laser radar and 4D millimeter wave radar are used to continue to follow the vehicle, which can achieve continuous following in harsh environments such as high speed, close distance, and dust. After the target is lost, it can enter the target retrieval state. When the target is lost, it can estimate the position of the guide vehicle, use 4D millimeter wave data and FMCW lidar data and camera target detection results to match and retrieve the guide vehicle. It has a certain target loss retrieval capability.

Those of ordinary skill in the art will appreciate that the units and steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in the above description according to function. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered beyond the scope of this article.

Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

In the several embodiments provided herein, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or can be electrical, mechanical or other forms of connection.

The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of this article.

In addition, each functional unit in each embodiment of this invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional unit.

If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this article is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of this article. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

The above descriptions are merely embodiments of the present invention and are not intended to limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims (5)

1. A method of off-road environment awareness and tracking a lead vehicle, comprising the steps of:

the method comprises the steps that a sensor installed on an autonomous following vehicle is arranged, and the sensor comprises a camera, a 4D millimeter wave radar and an FMCW laser radar;

acquiring detection information of the sensor;

Executing an initial identification process, a continuous tracking process or a target recovery process for guiding the vehicle according to the detection information;

The initial identification process comprises the following steps: the guiding vehicle enters an initial detection range, the guiding vehicle characteristic information is manually input, the sensing flow of the autonomous following vehicle enters an initial recognition state, the guiding vehicle enters a continuous tracking state and is defined as a tracking target when the recognition is successful, and the guiding vehicle position is readjusted or the guiding vehicle characteristic information is readmitted when the recognition is failed;

The continuous tracking process comprises the following steps: if the target is not lost in the continuous tracking state, the continuous tracking state is maintained, and if the target is lost, the target recovery state is entered;

The target retrieving process comprises the following steps: if the target recovery is successful, keeping a continuous tracking state, stopping the vehicle if the target recovery is failed, and returning to the initial identification flow;

acquiring detection information of the sensor, further comprising:

The camera is used for acquiring image information and transmitting the image information to the computer, the computer realizes a target detection function by deep learning a target detection algorithm and using a pre-training data set, identifies the characteristics of the guided vehicle, and generates and stores characteristic data;

The 4D millimeter wave radar is used for acquiring 4D millimeter wave point cloud data and acquiring distance information, azimuth information, speed information, width information and height information of a tracking target; the 4D millimeter wave radar penetrates nonmetallic substances through millimeter waves so as to assist in locking and guiding the vehicle, wherein the nonmetallic substances comprise dust, rain, snow and fog;

The FMCW laser radar is used for acquiring FMCW laser radar point cloud data, wherein the FMCW laser radar point cloud data comprises distance information, azimuth information, speed information, width information and height information of each point, and guided vehicles and unguided vehicles are distinguished through feedback of the speed information of the point cloud, so that the guided vehicles are assisted to be locked;

executing an initial identification process according to the detection information, and further comprising:

Projecting the FMCW laser radar point cloud data and the 4D millimeter wave point cloud data into the image information, and carrying out initial fusion of the image and the point cloud information to obtain point cloud fusion data; in the point cloud fusion data, FMCW laser radar point cloud is combined with 4D millimeter wave point cloud to remove the influence of dust, rain, snow and fog, so as to lock and guide the vehicle and obtain a target detection result;

Matching the manually input feature information of the guided vehicle with the target detection result, and if the matching is successful, acquiring the position information, the structure information, the speed information and the direction information of the guided vehicle; if the matching fails, the guided vehicle is readjusted to enter the initial detection range or the guided vehicle characteristic information is readmitted.

2. The method of off-road environment awareness and tracking a lead vehicle of claim 1, wherein: executing a continuous tracking process according to the detection information, further comprising:

acquiring direction information, speed information and position information of a guided vehicle at the moment, and determining an interested area according to the direction information, the speed information and the position information of the guided vehicle at the moment;

And acquiring the current position information of the guided vehicle, 4D millimeter wave point cloud data and FMCW laser radar point cloud data, and matching the current position information, the 4D millimeter wave point cloud data and the FMCW laser radar point cloud data with the region of interest, planning a driving route according to the current point cloud fusion data of the guided vehicle if the matching is successful, continuously following the vehicle, and entering a target recovery state if the matching is not successful.

3. The method of off-road environment awareness and tracking a lead vehicle of claim 2, wherein: executing a target recovery process according to the detection information, further comprising:

acquiring position information, direction information and speed information of a guided vehicle at the previous moment when the guided vehicle is lost, estimating the position of the guided vehicle at the next moment, and screening point cloud fusion data according to the estimated position of the guided vehicle at the next moment; and matching the screened point cloud fusion data with the characteristic data, entering a continuous vehicle following state if the matched point cloud fusion data are matched with the characteristic data, stopping the vehicle if the matched point cloud fusion data are not matched with the characteristic data, and entering an initial identification state.

4. A system for off-road environment awareness and tracking a lead vehicle employing the method for off-road environment awareness and tracking a lead vehicle of any one of claims 1-3, the system comprising:

The sensor is arranged on the autonomous following vehicle and comprises a camera, a 4D millimeter wave radar and an FMCW laser radar;

and the control unit is used for acquiring the detection information of the sensor and executing an initial identification process, a continuous tracking process or a target recovery process according to the detection information.

5. A computer readable storage medium, characterized in that it has stored thereon a computer program which, when executed by a processor, implements the steps of the method of off-road environment awareness and tracking a lead vehicle according to any of claims 1-3.