WO2024016877A1 - Roadside sensing simulation system for vehicle-road collaboration - Google Patents

Abstract

The present invention relates to the technical field of roadside sensing simulation, and provides a roadside sensing simulation system for vehicle-road collaboration. The roadside sensing simulation system for vehicle-road collaboration comprises a simulation platform module, a simulation framework module, a middleware module, and a node module; the simulation platform module comprises a graphics engine unit and a physics engine unit, the physics engine unit is connected to the graphics engine unit, and the graphics engine unit and the physics engine unit are both connected to the simulation framework module; the simulation framework module comprises a simulation environment unit, a dynamic scene unit, a roadside sensor unit, a positioning simulation unit, a communication simulation unit, a dynamics simulation unit, etc. According to the system, sensors are deployed on a roadside to send collected road surface information to a vehicle via V2X communication, so that the vehicle has a beyond-visual-range sensing capability; and the problems of RSU configuration and sample data generation can be well solved by constructing the roadside sensing simulation system.

Description

A road test perception simulation system for vehicle-road collaboration Technical field

The invention relates to the technical field of drive test perception simulation, and specifically is a drive test perception simulation system oriented to vehicle-road collaboration.

Background technique

Intelligent transportation system can effectively improve the safety and efficiency of road traffic through artificial intelligence and information and communication technology. It has been widely recognized. It consists of two parts: "smart car" and "smart road". Vehicle-road collaboration is the development of ITS. The advanced stage is used to realize communication between vehicles and vehicles and roadside systems, so that vehicles can better perceive the surrounding environment, receive relevant information about assisted driving, and allow road supervision departments to handle traffic accidents more effectively.

Roadside sensing is an important part of the development of vehicle-road collaborative applications. By deploying sensors on the roadside, the collected road information is sent to the vehicle through V2X communication, so that the vehicle has beyond-line-of-sight sensing capabilities. In practical applications, it provides To achieve optimal roadside sensing effects, different scenarios often require different RSU configurations. The selection and installation of RSUs is a time-consuming and labor-intensive process. In addition, the identification of traffic participants is the core of roadside sensing. Based on machine The learning recognition algorithm requires a large basket of labeled data, and manual labeling is an extremely inefficient way to verify. With the continuous improvement of computer hardware performance in recent years, applying simulation technology to the field of intelligent transportation has become a necessary means for various R&D institutions to accelerate the development process.

At present, with the rapid development of the field of intelligent transportation, simulation technology plays an increasingly important role in it. In particular, there have been many simulation applications and research on autonomous driving and vehicle-road collaboration. However, simulation for roadside sensing Simulation is still rarely involved, but it is an indispensable technology for the application development of vehicle-road collaboration.

Contents of the invention

(1) Technical problems solved

In view of the shortcomings of the existing technology, the present invention provides a road test perception simulation system oriented to vehicle-road collaboration. This system deploys sensors on the roadside and communicates the collected road surface information to the vehicle through V2X, so that the vehicle has With beyond-line-of-sight sensing capability, the problem of RSU configuration and sample data generation can be well solved by building this roadside sensing simulation system.

(2) Technical solutions

In order to achieve the above objectives, the present invention is implemented through the following technical solutions: a road test perception simulation system oriented to vehicle-road collaboration, including a simulation platform module, a simulation framework module, a middleware module, and a node module. The simulation platform module includes Graphics engine unit and physics engine unit, and the physics engine unit is connected with the graphics engine unit, and the graphics engine unit and the physics engine unit are both connected with the simulation framework module, the simulation framework module includes a simulation environment unit, a dynamic scene unit, a road Side sensor unit, positioning simulation unit, communication simulation unit and dynamics simulation unit, etc., the middleware module includes ROS, YARP and other external communication units, the node module includes a vehicle control unit, data processing unit, etc., the middleware module There is a two-way communication connection between the middleware module and the simulation framework module, and there is a two-way communication connection between the middleware module and each unit in the node module.

Preferably, the drive test perception simulation system is a simulation system developed based on LGSVL and suitable for roadside perception, Among them, the custom scene function is used to develop a simulation environment suitable for roadside sensing, the custom vehicle and sensor model functions are used to create a roadside sensing unit, and custom communication content is used to realize the collection and transmission of roadside sensing data.

Preferably, the drive test perception simulation system includes a simulation scene construction, which is composed of a static environment unit, a dynamic traffic unit and a roadside unit;

Static environment units mainly include lanes used for vehicle driving, buildings in the scene, green plants, street lights, etc. in the area. These constitute the objective environment of the simulation scene and do not change with changes in other conditions during the simulation test process;

Dynamic traffic unit, which is a key component of the simulation test scenario, mainly refers to the control, traffic flow, pedestrian flow and other parts of the simulation that have dynamic characteristics, including traffic light simulation, motor vehicle simulation, pedestrian simulation, etc.;

The roadside unit is the core component of vehicle-road collaboration and is responsible for the collection, processing and transmission of vehicle-road information. It is also the key research object of the roadside perception simulation system for vehicle-road collaboration.

Preferably, the static environment unit is modeled by blender and then rendered in Unity HD to obtain the static environment of the simulation system.

Preferably, the dynamic traffic simulation scene construction method implemented by the dynamic traffic unit mainly includes construction based on real traffic case data, generalization construction based on real case data, and construction based on microscopic traffic simulation system.

Preferably, the roadside unit includes a camera, laser radar, millimeter wave radar, industrial computer, etc.

Preferably, a data collection and construction method for a vehicle-road collaboration-oriented road test perception simulation system includes the following steps:

S1. Simulate point cloud data generation

Referring to the scanning method of real lidar, simulate the emission of each real radar ray. By intersecting with all objects in the scene, if there is an intersection point within the maximum detection distance of lidar, the corresponding point cloud coordinates will be returned. Assume that the lidar is simulated. is the L line, the horizontal resolution is R, and the horizontal scanning range is 360°. The number N of rays emitted in each frame is:
N＝L×360/R

If the detection distance is D, the pseudo code generated by simulating point cloud data in the scene is:

It can be seen from the above formula and pseudo code that when the lidar frequency is high, the environment in the scene is complex and the model is sufficiently refined, the calculation amount of intersection through simulated rays is extremely large. Take the lidar as 64 lines, the horizontal resolution is 0.4, and the frequency Taking 10Hz as an example, the number of lidar rays emitted per second alone is as high as 576,000. On this basis, it is also necessary to traverse all object models in the scene except lidar for each ray. In order to achieve the effect of real-time simulation, CPU parallel or GPU computing can be used to improve computing efficiency. LGSVL uses GPU to calculate point cloud data;

S2. True value data generation and processing

After having simulated point cloud data, it is generally necessary to match the real value data to be used as a data set for model recognition training. The true value data corresponds to the manual label data in the real data. The data content includes the position, orientation, bounding box size, speed, type, etc. of the identifiable object. Different from the manual labeling process, the true value data is compared to the simulation system. It is known that only the true value data and the point cloud data need to be synchronized and output, so the efficiency of output tags can be greatly improved;

S3. Simulation data output

Since the simulated point cloud data and the true value data are collected through different sensors, in order to achieve mutual matching of each frame file, the current ROS time is used as the naming of each frame of point cloud data and true value data. For example, the current ROS time is n.ms, the point cloud data file collected at the corresponding time is saved as nm.pcd, and the true value data file is nm.txt. Import the simulated point cloud data and true value data of the same frame into Rviz for display, and the output simulation data is obtained. .

Preferably, in addition to the position coordinates, the real point cloud data in step S1 also has a key information which is the reflection intensity. The reflection intensity mainly reflects the reflectivity of different physical materials to the near-infrared light used by the lidar. Therefore, the simulated point cloud data also needs to consider the intensity value. In LGSVL, the metallicity and color values in the model material are obtained and normalized to obtain intensity values ranging from 0 to 255.

Preferably, the true value data in step S2 is generated by creating a new true value data sensor in LGSVL. In order to match the true value data with the point cloud data, the configuration parameters of the true value data sensor and the lidar sensor need to be consistent. Such as position and attitude, effective range, frequency, etc.

(3) Beneficial effects

The invention provides a road test perception simulation system oriented to vehicle-road collaboration. It has the following beneficial effects:

1. The present invention provides a road test perception simulation system oriented to vehicle-road collaboration. The system is developed and constructed based on the autonomous driving simulation software LGSVL. The development content includes a simulation environment, a roadside unit and data collection and communication, and With the help of the simulation environment, the relationship between the height of the lidar and the road point cloud coverage is analyzed, which can provide a reference for the actual installation position of the lidar. By comparing the vehicle recognition model obtained from the point cloud data output in the simulation environment with the real The mutual verification results between the models obtained from the data indicate that the simulation system's simulation of lidar and environment can restore the real situation to a high degree, thereby obtaining better simulation results.

2. The present invention provides a road test perception simulation system oriented to vehicle-road collaboration. This system deploys sensors on the roadside and sends the collected road information to the vehicle through V2X communication, so that the vehicle has beyond-line-of-sight perception. Capability, by constructing this roadside sensing simulation system, the problems of RSU configuration and sample data generation can be well solved.

Description of drawings

Figure 1 is a schematic structural diagram of the roadside sensing simulation system of the present invention;

Figure 2 is an overall plan view of the simulation scene of the present invention.

Detailed ways

The technical solutions in the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some of the embodiments of the present invention, rather than all the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts fall within the scope of protection of the present invention.

Example:

As shown in Figure 1-2, the embodiment of the present invention provides a road test perception simulation system for vehicle-road collaboration, including simulation Real platform module, simulation framework module, middleware module, node module, the simulation platform module includes a graphics engine unit and a physics engine unit, and the physics engine unit is connected to the graphics engine unit, and both the graphics engine unit and the physics engine unit Connected to the simulation framework module, which includes a simulation environment unit, a dynamic scene unit, a roadside sensor unit, a positioning simulation unit, a communication simulation unit, a dynamics simulation unit, etc., and the middleware module includes ROS, YARP, etc. External communication unit, the node module includes a vehicle control unit, a data processing unit, etc., a two-way communication connection between the middleware module and the simulation framework module, and a two-way communication connection between the middleware module and each unit in the node module. Communication connection.

The road test perception simulation system is a simulation system developed based on LGSVL and suitable for roadside perception. The custom scene function is used to develop a simulation environment suitable for roadside perception, and the custom vehicle and sensor model functions are used to create roadside perception. The unit uses customized communication content to collect and transmit roadside sensing data.

The drive test perception simulation system includes a simulation scene construction, which is composed of a static environment unit, a dynamic traffic unit and a roadside unit;

Static environment units mainly include lanes used for vehicle driving, buildings in the scene, green plants, street lights, etc. in the area. These constitute the objective environment of the simulation scene and do not change with changes in other conditions during the simulation test process;

Dynamic traffic unit, which is a key component of the simulation test scenario, mainly refers to the control, traffic flow, pedestrian flow and other parts of the simulation that have dynamic characteristics, including traffic light simulation, motor vehicle simulation, pedestrian simulation, etc.;

The roadside unit is the core component of vehicle-road collaboration and is responsible for the collection, processing and transmission of vehicle-road information. It is also the key research object of the roadside perception simulation system for vehicle-road collaboration.

The static environment unit is modeled by blender and then rendered by Unity HD to obtain the static environment of the simulation system. The dynamic traffic simulation scene construction method implemented by the dynamic traffic unit mainly includes construction based on real traffic case data, based on real case data Generalized construction, and construction based on microscopic traffic simulation system, the roadside unit includes camera, laser radar, millimeter wave radar, industrial computer, etc.

The data collection and construction method of the road test perception simulation system for vehicle-road collaboration includes the following steps:

S1. Simulate point cloud data generation

Referring to the scanning method of real lidar, simulate the emission of each real radar ray. By intersecting with all objects in the scene, if there is an intersection point within the maximum detection distance of lidar, the corresponding point cloud coordinates will be returned. Assume that the lidar is simulated. is the L line, the horizontal resolution is R, and the horizontal scanning range is 360°. The number N of rays emitted in each frame is:

N＝L×360/R

If the detection distance is D, the pseudo code generated by simulating point cloud data in the scene is:

It can be seen from the above formula and pseudo code that when the lidar frequency is high, the scene environment is complex and the model is sufficiently refined, the calculation amount of intersection through simulated rays is extremely large. Take the lidar as 64 lines, the horizontal resolution is 0.4, and the frequency Taking 10Hz as an example, the number of lidar rays emitted per second alone is as high as 576,000. On this basis, it is also necessary to traverse all object models in the scene except lidar for each ray. In order to achieve the effect of real-time simulation, CPU parallel or GPU computing can be used to improve computing efficiency. LGSVL uses GPU to calculate point cloud data;

S2. True value data generation and processing

After having simulated point cloud data, it is generally necessary to match the real value data to be used as a data set for model recognition training. The true value data corresponds to the manual label data in the real data. The data content includes the position, orientation, bounding box size, speed, type, etc. of the identifiable object. Different from the manual labeling process, the true value data is compared to the simulation system. It is known that only the true value data and the point cloud data need to be synchronized and output, so the efficiency of output tags can be greatly improved;

S3. Simulation data output

Since the simulated point cloud data and the true value data are collected through different sensors, in order to achieve mutual matching of each frame file, the current ROS time is used as the naming of each frame of point cloud data and true value data. For example, the current ROS time is n.ms, the point cloud data file collected at the corresponding time is saved as nm.pcd, and the true value data file is nm.txt. Import the simulated point cloud data and true value data of the same frame into Rviz for display, and the output simulation data is obtained. .

Experimental test

In reality, due to the high cost of lidar, the layout of lidar needs to be optimized in roadside layout so that the effective coverage area of a single lidar is utilized as much as possible. For roads with RSUs arranged on only one side, due to the large differences in the shapes of various types of vehicles, small vehicles may be blocked by large vehicles, thus posing a challenge to the over-the-horizon function provided by vehicle-road collaboration. In order to reduce this In the case of lidar blind spots caused by vehicle shielding, the simplest and most effective way is to increase the installation height of lidar. Obtaining the minimum installation height of lidar requires comprehensive testing of lidar parameters, road environment parameters, vehicle parameters and other conditions. Through Real road testing is not realistic, but it can be completed simply and intuitively with the help of the roadside perception simulation system proposed in this article.

Compared with object recognition based on two-dimensional images collected by cameras, object recognition based on LiDAR point cloud data is not affected by ambient light and has higher robustness, so it plays an important role in vehicle-road collaboration. Correspondingly, due to the huge amount of point cloud data in a single frame, recognition based on point cloud is even more difficult than image recognition using deep learning methods. Especially the process of producing label data is extremely manual. difficult. A large amount of label data can be generated quickly and accurately through a simulation system, but whether simulated data can replace real data still needs to be verified through experiments.

Design 4 groups of experiments to verify simulated data. The first group uses real data to train and test with real data. The second group uses simulated data to train and test with simulated data. The third group uses real data to train and test with simulated data. The fourth group uses simulated data to train. For real data testing, the four sets of experiments used the same training network. The data sizes of the training set and the test set were all obtained at a ratio of 4:1. The final results are shown in Table 1 below:

Table 2 Test comparison between simulated data and real data

Among them, Precision is the precision rate of recognition, relative to the samples detected in the test set, Recall is the recall rate, relative to the entire test set, F1 score is the harmonic average of precision rate and recall rate, from Table 2 It can be seen that whether real data is used to test simulated data or simulated data is used to test real data, the final results show that various evaluation indicators can be relatively close to pure real data. It can be seen that the simulated point cloud output through the simulation system The data can better restore the characteristics of real data.

Although the embodiments of the present invention have been shown and described, those of ordinary skill in the art will understand that various changes, modifications, and substitutions can be made to these embodiments without departing from the principles and spirit of the invention. and modifications, the scope of the invention is defined by the appended claims and their equivalents.

Claims (9)

A road test perception simulation system oriented to vehicle-road collaboration, which is characterized in that it includes a simulation platform module, a simulation framework module, a middleware module, and a node module. The simulation platform module includes a graphics engine unit and a physics engine unit, and the physics The engine unit is connected to the graphics engine unit, and both the graphics engine unit and the physics engine unit are connected to the simulation framework module. The simulation framework module includes a simulation environment unit, a dynamic scene unit, a roadside sensor unit, a positioning simulation unit, and a communication simulation. unit and dynamics simulation unit, etc., the middleware module includes ROS, YARP and other external communication units, the node module includes a vehicle control unit, data processing unit, etc., the middleware module and the simulation framework module have two-way communication Connection, a two-way communication connection between the middleware module and each unit in the node module. A road test perception simulation system for vehicle-road collaboration according to claim 1, characterized in that the road test perception simulation system is a simulation system developed based on LGSVL and suitable for roadside perception, wherein, using custom The scene function development is suitable for the simulation environment of roadside sensing, using customized vehicle and sensor model functions to create roadside sensing units, and using customized communication content to realize the collection and transmission of roadside sensing data. A road test perception simulation system for vehicle-road cooperation according to claim 1, characterized in that the road test perception simulation system includes a simulation scene construction, and the simulation scene construction consists of a static environment unit, a dynamic traffic unit and Roadside unit composition; Static environment units mainly include lanes used for vehicle driving, buildings in the scene, green plants, street lights, etc. in the area. These constitute the objective environment of the simulation scene and do not change with changes in other conditions during the simulation test process; Dynamic traffic unit, which is a key component of the simulation test scenario, mainly refers to the control, traffic flow, pedestrian flow and other parts of the simulation that have dynamic characteristics, including traffic light simulation, motor vehicle simulation, pedestrian simulation, etc.; The roadside unit is the core component of vehicle-road collaboration and is responsible for the collection, processing and transmission of vehicle-road information. It is also the key research object of the roadside perception simulation system for vehicle-road collaboration. A road test perception simulation system for vehicle-road collaboration according to claim 3, characterized in that the static environment unit is modeled by blender and then rendered in Unity high definition to obtain the static environment of the simulation system. A road test perception simulation system for vehicle-road collaboration according to claim 3, characterized in that the dynamic traffic simulation scene construction method implemented by the dynamic traffic unit mainly includes construction based on real traffic case data, based on real cases Generalization construction of data and construction of microscopic traffic simulation system. A road test perception simulation system for vehicle-road collaboration according to claim 3, characterized in that the roadside unit includes a camera, laser radar, millimeter wave radar, industrial computer, etc. A data collection and construction method for a road test perception simulation system oriented to vehicle-road collaboration, which is characterized by including the following steps: S1. Simulate point cloud data generation Referring to the scanning method of real lidar, simulate the emission of each real radar ray. By intersecting with all objects in the scene, if there is an intersection point within the maximum detection distance of lidar, the corresponding point cloud coordinates will be returned. Assume that the lidar is simulated. is the L line, the horizontal resolution is R, and the horizontal scanning range is 360°. The number N of rays emitted in each frame is:
N＝L×360/R If the detection distance is D, the pseudo code generated by simulating point cloud data in the scene is:
It can be seen from the above formula and pseudo code that when the lidar frequency is high, the scene environment is complex and the model is sufficiently refined, the calculation amount of intersection through simulated rays is extremely large. Take the lidar as 64 lines, the horizontal resolution is 0.4, and the frequency Taking 10Hz as an example, the number of lidar rays emitted per second alone is as high as 576,000. On this basis, it is also necessary to traverse all object models in the scene except lidar for each ray. In order to achieve the effect of real-time simulation, CPU parallel or GPU computing can be used to improve computing efficiency. LGSVL uses GPU to calculate point cloud data; S2. True value data generation and processing After having simulated point cloud data, it is generally necessary to match the real value data to be used as a data set for model recognition training. The true value data corresponds to the manual label data in the real data. The data content includes the position, orientation, bounding box size, speed, type, etc. of the identifiable object. Different from the manual labeling process, the true value data is compared to the simulation system. It is known that only the true value data and the point cloud data need to be synchronized and output, so the efficiency of output tags can be greatly improved; S3. Simulation data output Since the simulated point cloud data and the true value data are collected through different sensors, in order to achieve mutual matching of each frame file, the current ROS time is used as the naming of each frame of point cloud data and true value data. For example, the current ROS time is n.ms, the point cloud data file collected at the corresponding time is saved as nm.pcd, and the true value data file is nm.txt. Import the simulated point cloud data and true value data of the same frame into Rviz for display, and the output simulation data is obtained. . A data collection and construction method for a vehicle-road collaboration-oriented road test perception simulation system according to claim 7, characterized in that, in addition to position coordinates, the real point cloud data in step S1 also has a key information which is reflection Intensity, reflection intensity mainly reflects the reflectivity of different physical materials to the near-infrared light used by lidar. Therefore, the simulated point cloud data also needs to consider the intensity value. In LGSVL, the metallicity and color values in the model material are obtained and normalized to obtain intensity values ranging from 0 to 255. A data collection and construction method for a vehicle-road collaboration-oriented road test perception simulation system according to claim 7, characterized in that the true value data in step S2 is generated by creating a new true value data sensor in LGSVL. To match the true value data with the point cloud data, the configuration parameters of the true value data sensor and the lidar sensor need to be consistent, such as position and attitude, effective range, frequency, etc.