CN120635865A - A method and system for predicting and avoiding obstacles in autonomous driving - Google Patents
A method and system for predicting and avoiding obstacles in autonomous drivingInfo
- Publication number
- CN120635865A CN120635865A CN202511132004.XA CN202511132004A CN120635865A CN 120635865 A CN120635865 A CN 120635865A CN 202511132004 A CN202511132004 A CN 202511132004A CN 120635865 A CN120635865 A CN 120635865A
- Authority
- CN
- China
- Prior art keywords
- obstacle
- vehicle
- time
- features
- deep learning
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Granted
Links
Landscapes
- Traffic Control Systems (AREA)
Abstract
本发明涉及自动驾驶技术领域，具体涉及一种自动驾驶障碍物意图预测与避让方法及系统。方法包括实时获取图像数据和三维点云数据，分别提取静态特征和动态特征，采用注意力机制进行加权后融合形成融合特征向量，将融合特征向量输入多个独立训练好的深度学习模型中，利用模型差异加权融合输出的结果获得预测意图。基于预测意图设定运行状态和输入控制的关联约束，以路径长度和时间成本为双优化目标建立避让策略模型，在预测时域内持续更新输入控制直到优化目标值迭代至最小，输出避让策略的时间序列，结合实时运行状态生成最终执行指令。本发明能够突破传统基于静态特征和预设规则系统的瓶颈，在提升行车安全冗余度的同时显著优化道路通行效率。
The present invention relates to the field of autonomous driving technology, and in particular to a method and system for predicting and avoiding obstacles in autonomous driving. The method includes acquiring image data and three-dimensional point cloud data in real time, extracting static features and dynamic features respectively, fusing them after weighting using an attention mechanism to form a fused feature vector, inputting the fused feature vector into multiple independently trained deep learning models, and obtaining the predicted intention using the results of the weighted fusion output of the model differences. Based on the predicted intention, the associated constraints of the operating state and input control are set, and an avoidance strategy model is established with the path length and time cost as the dual optimization objectives. The input control is continuously updated in the prediction time domain until the optimization target value is iterated to the minimum, and the time series of the avoidance strategy is output, and the final execution instruction is generated in combination with the real-time operating state. The present invention can break through the bottleneck of the traditional system based on static features and preset rules, and significantly optimize the road traffic efficiency while improving the redundancy of driving safety.
Description
Technical Field
The invention relates to the technical field of automatic driving, in particular to a method and a system for predicting and avoiding intention of an automatic driving obstacle.
Background
In the automatic driving technology, the accurate identification and efficient avoidance of surrounding obstacles are key points for ensuring the driving safety and improving the road passing efficiency. Existing obstacle recognition methods mainly include, but are not limited to, those relying on data acquired by sensors such as radar, lidar, cameras, and image recognition techniques based on such data. Although these techniques enable detection and localization of obstructions to some extent, they are often limited to analysis of single sensor data or focus only on static identification of the appearance of obstructions.
For example, radar sensors are capable of accurately measuring the distance and relative velocity of an obstacle, but have limitations in identifying the specific type, shape, texture, etc. of the obstacle, while camera sensors are capable of capturing rich visual information, including the color, shape, texture, etc. of the obstacle, but their performance may be significantly affected in complex environments (e.g., night, rainy and foggy weather). In addition, although the existing image recognition technology can recognize the appearance characteristics of the obstacle, the existing image recognition technology often lacks deep understanding of the dynamic behavior of the obstacle, and it is difficult to accurately predict the future movement track and intention of the obstacle.
Disclosure of Invention
The technical problem to be solved by the embodiment of the invention is to provide an automatic driving obstacle intention prediction and avoidance method and system, so as to solve the problems that the obstacle cannot be accurately identified in an automatic driving environment and the avoidance strategy is unreasonable in the prior art.
The invention discloses an automatic driving obstacle intention prediction and avoidance method, which comprises the following steps:
Acquiring image data and three-dimensional point cloud data of a front obstacle in an automatic driving process in real time;
Extracting static features of the obstacle from the image data, and extracting dynamic features of the obstacle from the three-dimensional point cloud data;
Weighting the static features and the dynamic features by adopting an attention mechanism, and fusing the weighted static features and the dynamic features to obtain fused feature vectors;
Respectively inputting the fusion feature vectors into a plurality of independently trained deep learning models, and weighting and fusing output results of the plurality of deep learning models based on model differences of the plurality of deep learning models to obtain the prediction intention of the obstacle;
setting a vehicle running state and an input control association constraint according to the prediction intention, and establishing an avoidance strategy model for the double optimization targets according to the path length and the time cost;
Continuously updating the input control in a prediction time domain until the double optimization target value of the avoidance strategy model iterates to the minimum in the association constraint, and outputting the time sequence of the input control to form the avoidance strategy;
and acquiring the real-time running state of the vehicle in the automatic driving process, and generating a final vehicle execution instruction by combining the avoidance strategy and the real-time running state.
Optionally, the acquiring the image data and the three-dimensional point cloud data of the front obstacle in the automatic driving process includes:
Acquiring image data and three-dimensional point cloud data of a front obstacle in an automatic driving process in real time, wherein the image data comprises appearance information of the obstacle, and the three-dimensional point cloud data comprises shape and position information of the obstacle;
Preprocessing the acquired image data, including removing noise from the image data by wavelet transformation, and enhancing the contrast of the image data by histogram equalization, wherein the function expression of the image data preprocessing is as follows:
in the formula, Representing the image data after the preprocessing thereof,Representing the image data prior to preprocessing,Representing a composite transform function of noise removal and contrast enhancement,Representing residual noise.
Optionally, the extracting the static feature of the obstacle from the image data includes:
Extracting static features of the obstacle from the preprocessed image data by using an appearance flow network constructed by a plurality of convolution layers, wherein the static features comprise shape, texture and color histograms of the obstacle, and the appearance flow network extracts the function expression of the static features as follows:
in the formula, A feature map representing the output of the appearance flow network, Representing a non-linear activation function,A weight matrix representing the convolution kernel,Image data representing an input appearance stream network,Representing a convolution offset term.
Optionally, the extracting the dynamic feature of the obstacle from the three-dimensional point cloud data includes:
Extracting dynamic characteristics of the obstacle from the three-dimensional point cloud data by using a motion flow network constructed by gating circulation, wherein the dynamic characteristics comprise speed, direction and acceleration of the obstacle, and the motion flow network extracting dynamic characteristics has a function expression as follows:
in the formula, Indicating that the gate is to be reset,Indicating that the gate weight is to be reset,Indicating that the gate bias term is reset,The representation of the update gate is made,Representing the update of the gate weight,Representing an update of the gate bias term,The state of candidate concealment is indicated,Representing the weight of the hidden state of the candidate,Representing the candidate hidden state bias term, Representing a non-linear activation function,Indicating the hidden state of the previous time step,Three-dimensional point cloud data representing the current time step input,Representing the hyperbolic tangent activation function,Representing element multiplication.
Optionally, the method for predicting and avoiding the intention of the automatic driving obstacle further comprises a method for fusing the static characteristic and the dynamic characteristic, and the method comprises the following steps:
Defining a first weight matrix and a first bias term of an attention mechanism, mapping the static feature into an attention weight calculation space by using the defined first weight matrix, and obtaining the attention weight of the static feature through normalization function calculation, wherein the calculation function expression of the attention weight of the static feature is as follows:
Defining a second weight matrix and a second bias term of an attention mechanism, mapping the dynamic feature into a calculation space of the attention weight by using the defined second weight matrix, and obtaining the attention weight of the dynamic feature through normalization function calculation, wherein a calculation function expression of the attention weight of the dynamic feature is as follows:
in the formula, The attention weight representing the static feature,A first matrix of weights is represented and,Representing a static characteristic of the object,A first bias term is indicated and a second bias term is indicated,The attention weight representing the dynamic feature,A second weight matrix is represented and is used to represent,The dynamic characteristics are represented by a graph of the dynamic characteristics,A second bias term is indicated and is used,Representing a normalization function;
Splicing the weighted static features and the weighted dynamic features to obtain the fusion feature vector, wherein the function expression of splicing the static features and the dynamic features is as follows:
in the formula, The fused feature vector after the concatenation is represented,Representing element multiplication.
Optionally, the method for predicting and avoiding the intention of the automatic driving obstacle further includes a method for predicting the fused feature vector by using a plurality of deep learning models to obtain the predicted intention, including:
Acquiring fusion characteristic vectors of a plurality of known intention barriers in the history automatic driving process and taking the fusion characteristic vectors as training data;
The method comprises the steps that a plurality of deep learning models which are built by a convolutional neural network, a full-connection layer and a nonlinear activation function are adopted, and the plurality of deep learning models are independently trained by using different initialization parameters and training data;
respectively inputting the fusion feature vectors of the obstacle obtained in real time into a plurality of trained deep learning models, and calculating and obtaining the weight of the output result of each deep learning model by using a random forest algorithm;
Multiplying the output result of each deep learning model with the corresponding weight, and adding and fusing the weighted output results of all the deep learning models to obtain the final prediction intention, wherein the weighted and fused function expression of a plurality of the deep learning models is as follows:
in the formula, The final predicted intent is indicated as such,Representing the output of the i-th deep learning model,Representing the weight of the i-th deep learning model.
Optionally, setting association constraint of vehicle running state and input control according to the prediction intention, and establishing an avoidance strategy model for the double optimization targets according to path length and time cost, including:
setting the association between the running state and the input control in the automatic driving process of the vehicle according to the dynamic characteristics of the vehicle and the road constraint, wherein the function expression of the association between the running state and the input control is as follows:
in the formula, The vehicle running state at the time t is indicated,The vehicle input control at the time t is indicated,A system dynamics model representing a vehicle;
According to the predicted intention of the obstacle, setting the running state and the constraint of the input control in the automatic driving process of the vehicle, wherein the functional expression of the running state and the constraint of the input control is as follows:
in the formula, A set of constraints is represented and,Representing the obstacle prediction intention at time t;
Establishing an avoidance strategy model in a prediction time domain by taking path length and time cost as double optimization targets, wherein the function expression of the avoidance strategy model is as follows:
in the formula, The double optimized target value output by the avoidance strategy model is represented,AndRepresenting the weight coefficients, T representing the prediction horizon.
Optionally, the acquiring the real-time running state of the vehicle in the automatic driving process and generating a final vehicle execution instruction by combining the avoidance strategy and the real-time running state include:
Determining the expected running state of the vehicle according to the avoidance strategy, and acquiring the current actual running state of the vehicle in the automatic driving process;
and calculating and acquiring target acceleration of vehicle control execution according to the expected running state and the actual running state, wherein a calculation function expression of the target acceleration is as follows:
in the formula, Indicating a target acceleration at which the vehicle control is executed,Indicating that a desired speed is to be achieved,Indicating that the desired acceleration is to be achieved,Indicating the current actual speed of the vehicle,Indicating the current actual acceleration of the vehicle,Is a gain factor of a proportion of the gain,Is a differential gain coefficient;
And outputting a corresponding vehicle control instruction according to the calculated target acceleration.
The invention also discloses a prediction and avoidance system, which adopts the method for predicting and avoiding the intention of the automatic driving obstacle, and comprises the following steps:
The data acquisition module is used for acquiring image data and three-dimensional point cloud data of a front obstacle in an automatic driving process in real time;
The feature extraction module is used for extracting static features of the obstacle from the image data and extracting dynamic features of the obstacle from the three-dimensional point cloud data;
The feature fusion module is used for respectively weighting the static features and the dynamic features by adopting an attention mechanism and fusing the weighted static features and the weighted dynamic features to obtain fusion feature vectors;
the intention prediction module is used for respectively inputting the fusion feature vectors into a plurality of independently trained deep learning models, and weighting and fusing output results of the plurality of the deep learning models based on model differences of the plurality of the deep learning models to obtain the predicted intention of the obstacle;
The model building module is used for setting the association constraint of the running state of the vehicle and the input control according to the prediction intention, and building an avoidance strategy model for the double optimization targets according to the path length and the time cost;
The avoidance strategy generation module is used for continuously updating the input control in a prediction time domain until the double optimization target values of the avoidance strategy model iterate to the minimum in the association constraint, and outputting the time sequence of the input control to form an avoidance strategy;
The control instruction generation module is used for acquiring the real-time running state of the vehicle in the automatic driving process and generating a final vehicle execution instruction by combining the avoidance strategy and the real-time running state.
The invention also discloses a computer readable storage medium, on which a computer program is stored, which when executed by a processor, implements the method for predicting and avoiding the intention of the automatic driving obstacle.
Compared with the prior art, the method and the system for predicting and avoiding the intention of the automatic driving obstacle, provided by the embodiment of the invention, have the beneficial effects that:
The method comprises the steps of acquiring image data and three-dimensional point cloud data in real time, respectively extracting static features and dynamic features, weighting by adopting an attention mechanism, fusing to form fused feature vectors, inputting the fused feature vectors into a plurality of independently trained deep learning models, and obtaining a prediction intention by using a model difference weighting fusion output result. Setting an associated constraint of an operation state and an input control based on a prediction intention, establishing an avoidance strategy model by taking path length and time cost as double optimization targets, continuously updating the input control in a prediction time domain until the iteration of an optimization target value is minimum, outputting a time sequence of the avoidance strategy, and generating a final execution instruction by combining the real-time operation state, thereby solving the problems of inaccurate obstacle identification, insufficient intention prediction and unreasonable avoidance strategy in a complex environment in the prior art, realizing the effects of more accurate prediction intention, more flexible constraint setting, optimization target and more efficient iteration control, and remarkably improving the driving safety and road traffic efficiency.
Drawings
The technical scheme of the invention will be further described in detail below with reference to the accompanying drawings and examples, wherein:
FIG. 1 is a block diagram schematically illustrating the overall steps of a method for predicting and avoiding an intention of an autopilot obstacle according to an embodiment of the present invention;
Fig. 2 is a schematic block diagram of an intention prediction and avoidance flow of an automatic driving obstacle intention prediction and avoidance method provided by an embodiment of the invention.
Detailed Description
It should be noted that, without conflict, the embodiments of the present application and features of the embodiments may be combined with each other. Preferred embodiments of the present application will now be described in detail with reference to the accompanying drawings.
The invention discloses an automatic driving obstacle intention prediction and avoidance method, which is shown in fig. 1 and 2 and comprises the following steps:
s1, acquiring image data and three-dimensional point cloud data of a front obstacle in an automatic driving process in real time;
S2, extracting static features of the obstacle from the image data, and extracting dynamic features of the obstacle from the three-dimensional point cloud data;
s3, weighting the static features and the dynamic features by adopting an attention mechanism, and fusing the weighted static features and dynamic features to obtain fused feature vectors;
S4, respectively inputting the fusion feature vectors into a plurality of independently trained deep learning models, and weighting and fusing output results of the plurality of deep learning models based on model differences of the plurality of deep learning models to obtain prediction intention of the obstacle;
S5, setting a vehicle running state and input control association constraint according to the prediction intention, and establishing an avoidance strategy model for the double optimization targets according to the path length and the time cost;
s6, continuously updating the input control in a prediction time domain until the double optimization target value of the avoidance strategy model iterates to the minimum in the association constraint, and outputting a time sequence of the input control to form an avoidance strategy;
And S7, acquiring a real-time running state of the vehicle in the automatic driving process, and generating a final vehicle execution instruction by combining the avoidance strategy and the real-time running state.
Through implementation of the embodiment of the method for predicting and avoiding the intention of the automatic driving obstacle, image data and three-dimensional point cloud data are obtained in real time and processed in parallel, synchronous depth extraction of static features and dynamic features of the obstacle is achieved, and the limitation of single data source analysis is broken through. And the two types of features are weighted respectively by adopting an attention mechanism, weight distribution is dynamically adjusted according to feature importance, and then high-dimensional fusion feature vectors with complementary information are formed through fusion, so that feature expression capability is remarkably improved. The fusion feature vectors are respectively input into a plurality of independently trained deep learning models to perform multi-modal analysis, the prediction advantages of the models are integrated through a weighted fusion mechanism by utilizing the diversity output results generated by the model structure differences, single model deviation is effectively overcome, and the high-precision prediction intention of future motion states and behavior motivations of the obstacle is obtained. Based on the prediction intention, setting association constraint conditions of the vehicle running state and the input control, constructing a mathematical optimization model with path length and time cost as double optimization targets, continuously rolling and updating the input control in a prediction time domain until the double optimization targets iteratively converge to a minimum value under the condition that all association constraint conditions are met, and finally outputting an avoidance strategy formed by an optimized input control sequence. Therefore, a final execution instruction is dynamically generated by combining the actual running state of the vehicle in real time, and the problems of misjudgment of the intention of the obstacle and rigidity of the avoidance strategy caused by limitation of static feature identification and missing of dynamic behavior prediction in the prior art are solved. The embodiment of the invention realizes full-dimensional accurate sensing of the intention of the obstacle, probabilistic modeling prediction of the motion state and real-time self-adaptive optimization of avoidance decision in the complex environment through a multi-level feature fusion mechanism, a multi-model collaborative prediction mechanism and a dynamic optimization solving mechanism, so that an automatic driving system has the capacity of prospective disposal of sudden conditions, ensures the uniformity of the optimality and timeliness of driving tracks, can greatly reduce the false triggering rate of emergency avoidance, can further improve the smoothness and traffic flow fusion degree of continuous avoidance operation, breaks through the technical bottleneck of the traditional system based on static features and preset rules, and obviously optimizes the road passing efficiency while improving the driving safety redundancy.
Further, acquiring image data and three-dimensional point cloud data of a front obstacle in an automatic driving process includes:
acquiring image data and three-dimensional point cloud data of a front obstacle in an automatic driving process in real time, wherein the image data comprises appearance information of the obstacle, and the three-dimensional point cloud data comprises shape and position information of the obstacle;
Preprocessing acquired image data, including noise removal of the image data by wavelet transformation, and enhancing contrast of the image data by histogram equalization, wherein the function expression of the image data preprocessing is as follows:
in the formula, Representing the image data after the preprocessing thereof,Representing the image data prior to preprocessing,Representing a composite transform function of noise removal and contrast enhancement,Representing residual noise.
By implementing the embodiment of the method for predicting and avoiding the intention of the automatic driving obstacle, the noise immunity and the texture definition are obviously improved while the integrity of appearance information is maintained for the preprocessed image data. The noise removing operation can inhibit invalid pixel disturbance caused by environmental interference, the contrast enhancement can strengthen target edges and detail features, and the residual noise is allowed to avoid feature loss caused by excessive smoothing, so that the three-dimensional point cloud data can be transmitted in an original state to fully maintain the spatial precision and real-time performance of the shape and position information of the three-dimensional point cloud data. The preprocessed high-quality image data provides low-noise and high-distinction input basis for static feature extraction, can overcome the problem of feature distortion caused by single data quality degradation in complex scenes such as rain and fog/night, and provides high-fidelity data support for fusion feature vector construction, so that the perception robustness of an automatic driving system to essential attributes (appearance/shape/position) of obstacles in a severe environment is improved.
As described above, it is preferable that the image data and the three-dimensional point cloud data of the obstacle ahead are acquired in real time by the sensors such as the high-precision camera (resolution 4096x3072 pixels, frame rate 60 fps) and the laser radar (LiDAR, scanning frequency 20Hz, point cloud density 200 ten thousand points per second) during running of the automated driving automobile. The image data details the appearance information of the obstacle such as the pose (standing, walking, running, etc.), the clothing color (red, blue, green, etc.), the size (height, width, etc.), etc., and the three-dimensional point cloud data accurately depicts the shape (e.g., body contour, trunk shape) and the position information (distance to the vehicle, direction angle, etc.) of the obstacle. In order to improve the accuracy of subsequent processing, the acquired data is preprocessed, namely high-frequency noise in the image data is removed by utilizing wavelet transform (Daubechies 4 wavelet basis, 5-layer decomposition), and meanwhile, the contrast of the image is enhanced by adopting histogram equalization (the contrast is stretched to be in the range of 0-255).
Further, extracting static features of the obstacle from the image data includes:
Extracting static features of the obstacle from the preprocessed image data by using an appearance flow network constructed by a plurality of convolution layers, wherein the static features comprise shapes, textures and color histograms of the obstacle, and the function expression for extracting the static features by using the appearance flow network is as follows:
in the formula, A feature map representing the output of the appearance flow network, Representing a non-linear activation function,A weight matrix representing the convolution kernel,Image data representing an input appearance stream network,Representing a convolution offset term.
Through implementation of the embodiment of the method for predicting and avoiding the intention of the automatic driving obstacle, an appearance flow network is constructed, depth feature learning of spatial dimension is carried out on input image data through a convolution kernel weight matrix, the high-order nonlinear modeling capability of the geometric structure and the surface attribute of the obstacle is enhanced by combining a nonlinear activation function, and the introduction of a convolution bias term can effectively adapt to feature distribution offset of the image data under different brightness conditions. The network multi-level convolution structure is enabled to extract microscopic texture details while keeping the integrity of macroscopic shape contours, the output feature map has both local feature resolution and global structure consistency, the representation robustness of essential attributes of obstacles in complex scenes such as rain and fog shielding, night low illumination and the like is obviously enhanced, and high-quality static feature input for resisting environmental interference is provided for intention prediction.
As described above, the appearance flow network preferably employs a deep convolutional neural network, specifically a ResNet-101 model (Deep Residual Network with 101 layers,101 layer depth residual network), for feature extraction. ResNet-101 contain 101 convolutional layers, with the last fully-connected layer removed to output a signature. These feature maps contain static features such as shape, texture, color histogram, etc. of the obstacle. Wherein, the nonlinear activation function adopts a ReLU nonlinear activation function (RECTIFIED LINEAR Unit, modified linear units).
Further, extracting dynamic features of the obstacle from the three-dimensional point cloud data includes:
the dynamic characteristics of the obstacle are extracted from the three-dimensional point cloud data by utilizing a motion flow network constructed in a gating cycle, wherein the dynamic characteristics comprise the speed, the direction and the acceleration of the obstacle, and the function expression for extracting the dynamic characteristics by the motion flow network is as follows:
in the formula, Indicating that the gate is to be reset,Indicating that the gate weight is to be reset,Indicating that the gate bias term is reset,The representation of the update gate is made,Representing the update of the gate weight,Representing an update of the gate bias term,The state of candidate concealment is indicated,Representing the weight of the hidden state of the candidate,Representing the candidate hidden state bias term, Representing a non-linear activation function,Indicating the hidden state of the previous time step,Three-dimensional point cloud data representing the current time step input,Representing the hyperbolic tangent activation function,Representing element multiplication.
Through implementation of the embodiment of the method for predicting and avoiding the intention of the automatic driving obstacle, dynamic characteristics (including speed, direction and acceleration) of the obstacle are extracted from three-dimensional point cloud data by utilizing a motion flow network constructed by gating circulation, and the extraction process of the dynamic characteristics is accurately defined by a functional expression, namely resetting a gateBy weightBias termCalculating previous time step hidden stateWith current inputNon-linear combinations of (a) update gatesBy weightBias termCalculating the combination of the two, and candidate hidden statesThen based on reset gateHiding state from previous time stepWeighted result of (2) and current inputBy weightBias termThe hyperbolic tangent activation output is calculated, so that effective information of dynamic characteristics of the obstacle is adaptively screened through a double-gate control mechanism of a reset gate and an update gate, redundant interference is restrained, continuous evolution rules of the dynamic characteristics of the obstacle are accurately captured through space-time correlation modeling of candidate hidden states on real-time three-dimensional point cloud data, the time sequence behavior mode and instantaneous state change of the obstacle can be deeply fused in the dynamic characteristic extraction process, the characterization robustness of dynamic evolution of speed, direction and acceleration in complex motion tracks (such as acceleration lane and sharp turn) is remarkably enhanced, and a high-precision dynamic characteristic foundation with strong time sequence dependency is provided for intention prediction.
As described above, the motion flow network preferably employs a gating loop unit (Gated Recurrent Unit, GRU) to convert point cloud data acquired by the lidar into time series data and input into the motion flow network to extract dynamic characteristics of the obstacle. The moving states of the obstacle such as speed, direction, acceleration and the like and the change trend of the moving states with time can be captured through the moving flow network.
Further, the method for predicting and avoiding the intention of the automatic driving obstacle further comprises a method for fusing static characteristics and dynamic characteristics, and the method comprises the following steps:
Defining a first weight matrix and a first bias term of an attention mechanism, mapping the static feature into an attention weight calculation space by using the defined first weight matrix, and obtaining the attention weight of the static feature through normalization function calculation, wherein the calculation function expression of the attention weight of the static feature is as follows:
Defining a second weight matrix and a second bias term of the attention mechanism, mapping the dynamic feature into a calculation space of the attention weight by using the defined second weight matrix, and obtaining the attention weight of the dynamic feature through normalization function calculation, wherein a calculation function expression of the attention weight of the dynamic feature is as follows:
in the formula, The attention weight representing the static feature,A first matrix of weights is represented and,Representing a static characteristic of the object,A first bias term is indicated and a second bias term is indicated,The attention weight representing the dynamic feature,A second weight matrix is represented and is used to represent,The dynamic characteristics are represented by a graph of the dynamic characteristics,A second bias term is indicated and is used,Representing a normalization function;
splicing the weighted static features and dynamic features to obtain fusion feature vectors, wherein the function expression of splicing the static features and the dynamic features is as follows:
in the formula, The fused feature vector after the concatenation is represented,Representing element multiplication.
Through implementation of the embodiment of the method for predicting and avoiding the intention of the automatic driving obstacle, a dual-parameter space is constructed by utilizing a first weight matrix and a second weight matrix which are separated, so that static features and dynamic features are independently mapped in a nonlinear manner in the calculation process of the attention weights, and the attention weights which are strictly matched with the feature importance are respectively generated through normalization functions. And then performing fine fusion of feature dimension alignment on the weighted static features and dynamic features by adopting element multiplication, wherein the mechanism can break through the problem of feature expression confusion caused by weight sharing in traditional feature splicing, and remarkably enhance the collaborative characterization capability of a fusion feature vector on multi-modal attributes of an obstacle under a complex scene, namely when the reliability of the static features is weakened by environmental interference, the first weight matrix automatically reduces the weight distribution of the fuzzy texture, and the second weight matrix synchronously enhances the contribution proportion of the dynamic features, so that the information completeness of the fusion feature vector can be still maintained under the complex environment, and a robust input foundation for self-adaptive environmental change is provided for the intention prediction of the obstacle.
As described above, the normalization function preferably uses a Softmax function (Soft Maximum Function, flexible maximum function).
Further by way of example, a task of obstacle detection in front of an autonomous vehicle is handled:
the appearance flow network is responsible for extracting static features of the obstacle, such as shape, color, texture, etc., from a single image frame, these features being represented as a vector of dimension 128 The motion flow network is responsible for extracting dynamic characteristics of the obstacle, such as speed, acceleration, direction change, etc., from the continuous video frames, which are also represented as a vector with a dimension of 128。
In order to merge these two feature vectors into one merged feature vector, attention mechanisms are introduced. First, two weight matrices are definedAndThey are all 1x128 in dimension for mapping appearance features and motion features into the computation space of attention weights. At the same time, two bias terms are definedAndAll of them have dimensions 1 for adjusting the calculation of the attention weight.
Next, a static feature is calculated using the softmax functionAnd dynamic characteristicsAttention weight of (a)And;
It is assumed that the calculation is performed
= [0.6, 0.2, 0.1, ], 0.1] (Only part of the elements are shown, actually 128-dimensional vectors),
= [0.5, 0.3, 0.1, ], 0.1] (Only part of the elements are shown).
Attention weightingAndRepresenting static features during fusionAnd dynamic characteristicsThe importance of each element of (a) is occupied.
And then, splicing the weighted features to obtain a fused feature vector with 256 dimensions, wherein the fused feature vector contains fusion information of appearance features and motion features, and provides richer feature representation for subsequent obstacle classification and detection tasks.
Further, the method for predicting and avoiding the intention of the automatic driving obstacle further comprises a method for predicting the fusion feature vector by utilizing a plurality of deep learning models to obtain the prediction intention, and the method comprises the following steps:
Acquiring fusion characteristic vectors of a plurality of known intention barriers in the history automatic driving process and taking the fusion characteristic vectors as training data;
Adopting a plurality of deep learning models which are built by a convolutional neural network, a full-connection layer and a nonlinear activation function, and independently training the plurality of deep learning models by using different initialization parameters and training data;
Respectively inputting the fusion feature vectors of the obstacle obtained in real time into a plurality of trained deep learning models, and calculating and obtaining the weight of the output result of each deep learning model by using a random forest algorithm;
Multiplying the output result of each deep learning model with the corresponding weight, and adding and fusing the weighted output results of all the deep learning models to obtain the final prediction intention, wherein the weighted and fused function expression of the multiple deep learning models is as follows:
in the formula, The final predicted intent is indicated as such,Representing the output of the i-th deep learning model,Representing the weight of the i-th deep learning model.
Through implementation of the embodiment of the method for predicting and avoiding the intention of the automatic driving obstacle, the weight of the output result of each deep learning model is dynamically calculated by using a random forest algorithm, and each independently trained deep learning model forms the inherent heterogeneity of the model structure due to the differentiation of the initialization parameters and the training data, so that the prediction results of different deep learning models on the same fusion feature vector have obvious diversity. The random forest algorithm adaptively distributes the weight of each deep learning model through feature importance analysis based on the fusion feature vector training data of known intention barriers in the history automatic driving process, so that the decision contribution of the high-adaptability deep learning model is automatically enhanced when the barrier movement mode is suddenly changed. Finally, through a weighted fusion mechanism that weights are multiplied with output results of corresponding deep learning models and then are added completely, generalized bottleneck and scene dependence limit of a single deep learning model are broken through, so that prediction intention results have multi-model redundancy advantages and random forest dynamic calibration capability, and identification robustness and prediction fault tolerance of real intention of an obstacle in a complex interaction scene (such as multi-obstacle parallel lane changing and intention-blurred deceleration behaviors) are remarkably improved.
As described above, the nonlinear activation function of the deep learning model preferably employs a softmax activation function (Soft Maximum Activation Function, flexible maximum activation function). In order to improve the classification accuracy, an integrated learning method such as a random forest or gradient lifting tree is adopted to fuse the output results of a plurality of deep learning models.
Further illustrated is:
Assuming an autopilot scenario comprising 10 obstacles, each obstacle has 256 dimensions of integrated features. If 3 deep learning models (e.g., model a, model B, and model C) are trained to predict the intent of these obstacles. These models have the same network structure but use different initialization parameters and training data.
During the training process, cross-validation may be used to evaluate the performance of each model and select the optimal model parameters. Assume that model a, model B, and model C have 90%, 85%, and 88% accuracy, respectively, on the validation set.
Then, a random forest is used as an ensemble learning method to calculate the weight of each model. Random forests improve the accuracy of classification by building multiple decision trees and integrating their predicted results. In an embodiment of the invention, 50 decision trees may be constructed to calculate the weight of each model.
And obtaining weights of the model A, the model B and the model C to be 0.4, 0.3 and 0.3 respectively through calculation of random forests. These weights reflect the importance of each model in the final prediction.
And finally multiplying the output result of each model by the corresponding weight, and adding the weighted output results to obtain the final prediction intention. This predicted intent represents a probability distribution of each obstacle intent and may be used for decision and control of the autopilot system.
Further, setting association constraint of vehicle running state and input control according to the prediction intention, and establishing an avoidance strategy model for the double optimization targets according to path length and time cost, wherein the method comprises the following steps:
Setting the association between the running state and the input control in the automatic driving process of the vehicle according to the dynamics characteristic of the vehicle and the road constraint, wherein the function expression of the association between the running state and the input control is as follows:
in the formula, The vehicle running state at the time t is indicated,The vehicle input control at the time t is indicated,A system dynamics model representing a vehicle;
according to the predicted intention of the obstacle, setting the constraint of the running state and the input control in the automatic driving process of the vehicle, wherein the functional expression of the running state and the constraint of the input control is as follows:
in the formula, A set of constraints is represented and,Representing the obstacle prediction intention at time t;
establishing an avoidance strategy model in a prediction time domain by taking path length and time cost as double optimization targets, wherein the function expression of the avoidance strategy model is as follows:
in the formula, The double optimized target value output by the avoidance strategy model is represented,AndRepresenting the weight coefficients, T representing the prediction horizon.
Through implementation of the embodiment of the method for predicting and avoiding the intention of the automatic driving obstacle, the system dynamics model is utilized to accurately embed the physical characteristics (such as steering inertia and driving response) of the vehicle and the geometric constraints (such as curvature boundary and gradient limitation) of the road, so that the running state updating process strictly follows the actual mechanical dynamics law. Meanwhile, the time-varying obstacle prediction intention is used as the self-adaptive input of the constraint function, the avoidance strategy is forced to respond to the external threat evolution in real time in the prediction time domain, and the rolling joint optimization of the path length and the time cost in the prediction time domain is realized through the double optimization objective function. Wherein, the Representing the running state of the vehicle at the time t, such as the position, the speed, the acceleration and the like of the vehicle; vehicle input controls representing time t, such as throttle, brake, steering, etc.; A system dynamics model representing the vehicle taking into account the dynamics of the vehicle and road constraints, g is a set of constraints for ensuring that the avoidance strategy does not violate traffic rules, avoid collisions, keep lanes, etc., which may include speed limitations, steering angle limitations, avoid collisions with other vehicles or obstacles, etc., weight coefficients AndFor balancing priorities among different optimization targets, the cooperative adjustment breaks through the barrier that efficiency and safety cannot be considered in the traditional single-target optimization, and finally continuously updated input control is realized under the double closed-loop actions of state transition constraint and intention driving constraintThe sequence systematically converges to a dynamic track solution with minimized double optimization target values on the premise of meeting the motion boundary and obstacle avoidance safety threshold of the vehicle. Namely, the double optimization target problem is solved through iteration, namely, on the premise of meeting constraint conditions, an input control sequence which enables the avoidance strategy model J to be minimum is foundThus, a series of optimal input controls are obtained, which will constitute an avoidance strategy for guiding the automatic avoidance behavior of the vehicle.
Based on the foregoing, in detail, controlling input in the prediction time domainExecuting closed-loop rolling optimization to ensure that the double-optimization target value of the avoidance strategy model is in the system dynamics modelAnd intent constraint setIs updated iteratively continuously under the double boundary constraint of (c). Each iteration is based on the current vehicle running stateSolving constraint optimization problem again, and self-adaptively adjusting input control through gradient descent methodUntil the double optimization target value converges to the minimum value in the current prediction time domain. The mechanism breaks through the static limitation of the traditional open-loop decision, so that the time sequence of the input control of the final output strictly complies with the mechanical constraint of the steering inertia of the vehicle and the curvature of the road, dynamically couples the safety avoidance boundary of the obstacle prediction intention, and the path length and the time cost are in the weight coefficientAndThe overall optimum balance under the accurate proportioning is further generated at the output end, and the avoidance strategy which is physically realized, agile in intention response and optimal in energy consumption is generated.
Further, acquiring a real-time running state of the vehicle in the automatic driving process, and generating a final vehicle execution instruction by combining the avoidance strategy and the real-time running state, wherein the method comprises the following steps:
determining an expected running state of the vehicle according to the avoidance strategy, and acquiring the current actual running state of the vehicle in the automatic driving process;
And calculating and acquiring target acceleration of vehicle control execution according to the expected running state and the actual running state, wherein a calculation function expression of the target acceleration is as follows:
in the formula, Indicating a target acceleration at which the vehicle control is executed,Indicating that a desired speed is to be achieved,Indicating that the desired acceleration is to be achieved,Indicating the current actual speed of the vehicle,Indicating the current actual acceleration of the vehicle,Is a gain factor of a proportion of the gain,Is a differential gain coefficient;
And outputting a corresponding vehicle control instruction according to the calculated target acceleration.
Through implementation of the above-described embodiment of the automated driving obstacle intent prediction and avoidance method, a proportional gain coefficient is utilizedFor the expected speedAnd actual speedIs dynamically scaled by the difference of (2) while utilizing differential gain coefficientsFor expected accelerationAnd actual accelerationTo make a real-time correction compensation for the deviation of (c), a dual closed loop negative feedback mechanism of velocity and acceleration is formed. The feedforward-feedback composite control strategy for fusing the actual running state can break through the limitation of open loop control which only depends on the expected running state in the prior art, so that the target acceleration is achievedThe calculated result of the (2) can immediately respond to the transient error between the motion state of the vehicle and the demand of the avoidance strategy, namely when the actual acceleration isDue to abrupt road adhesion or mechanical delay lagging behind the desired accelerationWhen the derivative termWill produce a compensating incremental forced closed loop convergence when the actual speed isDeviation from the desired speedWhen the proportion termThe restorative adjustment amount is automatically generated. The cooperative action of the two ensures that the finally generated vehicle execution instruction has dynamic deviation rectifying capability and motion continuity guarantee, and completely solves the problems of acceleration setback, speed oscillation and response delay caused by state feedback loss in the traditional method while meeting the safety constraint of an avoidance strategy, thereby remarkably improving the smoothness and working condition adaptability of automatic driving control.
As described above, the avoidance strategy is transmitted to the control module of the automatic driving system, and the automatic avoidance of the vehicle is realized by controlling the executing mechanisms such as the accelerator, the brake, the steering and the like of the vehicle. When executing the avoidance strategy, the control module will be based on the target accelerationAnd calculating corresponding control instructions, such as adjusting the opening degree of an accelerator, the braking force or the steering angle and the like. The control instructions realize accurate control of the vehicle by controlling the executing mechanisms such as throttle, brake and steering of the vehicle, so that the automatic driving automobile can accurately track the expected acceleration and speed to realize safe and stable avoidance operation.
The invention also discloses a prediction and avoidance system, which adopts the method for predicting and avoiding the intention of the automatic driving obstacle, and comprises the following steps:
The data acquisition module is used for acquiring image data and three-dimensional point cloud data of a front obstacle in an automatic driving process in real time;
The feature extraction module is used for extracting static features of the obstacle from the image data and extracting dynamic features of the obstacle from the three-dimensional point cloud data;
the feature fusion module is used for respectively weighting the static features and the dynamic features by adopting an attention mechanism and fusing the weighted static features and the weighted dynamic features to obtain fusion feature vectors;
the intention prediction module is used for respectively inputting the fusion feature vectors into a plurality of independently trained deep learning models, and weighting and fusing the output results of the plurality of deep learning models based on the model differences of the plurality of deep learning models to obtain the predicted intention of the obstacle;
the model building module is used for setting the vehicle running state and the association constraint of input control according to the prediction intention, and building an avoidance strategy model for the double optimization targets according to the path length and the time cost;
the avoidance strategy generation module is used for continuously updating the input control in the prediction time domain until the double optimization target values of the avoidance strategy model iterate to the minimum in the association constraint, and the time sequence of the output input control forms the avoidance strategy;
The control instruction generation module is used for acquiring the real-time running state of the vehicle in the automatic driving process and generating a final vehicle execution instruction by combining the avoidance strategy with the real-time running state.
The invention also discloses a computer readable storage medium, on which a computer program is stored, characterized in that the computer program when executed by a processor implements the method for predicting and avoiding the intention of the automatic driving obstacle.
The invention also discloses a computer device, which comprises a memory, a processor and a computer program stored on the memory and capable of running on the processor, and is characterized in that the method for predicting and avoiding the intention of the automatic driving obstacle is realized when the processor executes the computer program.
The present invention is described in terms of flowchart illustrations and/or block diagrams of methods, apparatus (systems) and computer program products according to embodiments. It will be understood that each flowchart and/or block of the flowchart illustrations and/or block diagrams, and combinations of flowcharts and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general purpose computer, special purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory produce an article of manufacture including instruction means which implement the function specified in the flowchart flow or flows and/or block diagram block or blocks.
These computer program instructions may also be loaded onto a computer or other programmable data processing apparatus to cause a series of operational steps to be performed on the computer or other programmable apparatus to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide steps for implementing the functions specified in the flowchart flow or flows and/or block diagram block or blocks.
It should be understood that the foregoing embodiments are merely illustrative of the technical solutions of the present invention, and not limiting, and that modifications of the technical solutions described in the foregoing embodiments or equivalent substitutions of some technical features thereof may be made by those skilled in the art, and all such modifications and substitutions should be considered to fall within the scope of the present invention.
Claims (10)
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202511132004.XA CN120635865B (en) | 2025-08-13 | 2025-08-13 | Method and system for predicting and avoiding intention of automatic driving obstacle |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN202511132004.XA CN120635865B (en) | 2025-08-13 | 2025-08-13 | Method and system for predicting and avoiding intention of automatic driving obstacle |
Publications (2)
| Publication Number | Publication Date |
|---|---|
| CN120635865A true CN120635865A (en) | 2025-09-12 |
| CN120635865B CN120635865B (en) | 2025-12-09 |
Family
ID=96966409
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN202511132004.XA Active CN120635865B (en) | 2025-08-13 | 2025-08-13 | Method and system for predicting and avoiding intention of automatic driving obstacle |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN120635865B (en) |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN120922179A (en) * | 2025-10-15 | 2025-11-11 | 北京理工大学前沿技术研究院 | A method and system for automatic driving control of mining trucks |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2025019373A1 (en) * | 2023-07-14 | 2025-01-23 | Canon U.S.A., Inc. | Autonomous navigation of a continuum robot |
| CN119580226A (en) * | 2024-12-06 | 2025-03-07 | 狗熊机器人有限公司 | A method and system for autonomous driving obstacle recognition based on deep learning |
| CN119886702A (en) * | 2024-12-30 | 2025-04-25 | 广东电网有限责任公司 | Power grid safety production detection method and device, electronic equipment and storage medium |
| CN119942206A (en) * | 2025-01-08 | 2025-05-06 | 平安科技（深圳）有限公司 | Image abnormal state classification method, device, computer equipment and storage medium |
-
2025
- 2025-08-13 CN CN202511132004.XA patent/CN120635865B/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2025019373A1 (en) * | 2023-07-14 | 2025-01-23 | Canon U.S.A., Inc. | Autonomous navigation of a continuum robot |
| CN119580226A (en) * | 2024-12-06 | 2025-03-07 | 狗熊机器人有限公司 | A method and system for autonomous driving obstacle recognition based on deep learning |
| CN119886702A (en) * | 2024-12-30 | 2025-04-25 | 广东电网有限责任公司 | Power grid safety production detection method and device, electronic equipment and storage medium |
| CN119942206A (en) * | 2025-01-08 | 2025-05-06 | 平安科技（深圳）有限公司 | Image abnormal state classification method, device, computer equipment and storage medium |
Non-Patent Citations (2)
| Title |
|---|
| DAN-DAN ZHU 等: "The Path Optimization Algorithm of Car Navigation System considering Node Attributes under Time-Invariant Network", MOBILE INFORMATION SYSTEMS, 4 January 2021 (2021-01-04), pages 1 - 20 * |
| 骆仕杰 等: "融合随机森林和神经网络的教学质量分析算法", 小型微型计算机系统, 31 August 2023 (2023-08-31), pages 1686 - 1692 * |
Cited By (1)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN120922179A (en) * | 2025-10-15 | 2025-11-11 | 北京理工大学前沿技术研究院 | A method and system for automatic driving control of mining trucks |
Also Published As
| Publication number | Publication date |
|---|---|
| CN120635865B (en) | 2025-12-09 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| CN118627018B (en) | New energy automobile automatic driving fusion sensing method based on improved multi-head attention mechanism | |
| CN115880333B (en) | A 3D Single-Target Tracking Method Based on Multimodal Information Fusion | |
| CN109597087B (en) | Point cloud data-based 3D target detection method | |
| WO2021135554A1 (en) | Method and device for planning global path of unmanned vehicle | |
| CN110533695A (en) | A kind of trajectory predictions device and method based on DS evidence theory | |
| CN111967373B (en) | Self-adaptive enhanced fusion real-time instance segmentation method based on camera and laser radar | |
| CN120635865B (en) | Method and system for predicting and avoiding intention of automatic driving obstacle | |
| CN113552883B (en) | Ground unmanned vehicle autonomous driving method and system based on deep reinforcement learning | |
| CN116595871A (en) | Vehicle track prediction modeling method and device based on dynamic space-time interaction diagram | |
| CN119283848A (en) | A control method for unmanned vehicles based on deep learning | |
| CN118238832B (en) | Intelligent driving method and device based on visual perception | |
| CN120141518A (en) | A navigation method for unmanned vehicles based on multimodal fusion | |
| CN113589810B (en) | Dynamic autonomous obstacle avoidance movement method and device for intelligent body, server and storage medium | |
| CN120595851A (en) | UAV flight attitude adjustment method and system based on reinforcement learning | |
| CN117475615A (en) | A vehicle trajectory prediction method based on spatial motion relationship and long and short time memory network | |
| CN119796253B (en) | A method, system, and autonomous vehicle for lane-changing decision-making. | |
| CN119904766B (en) | Environment complexity-based end-to-end unmanned aerial vehicle autonomous control method | |
| CN118968252A (en) | Obstacle detection method, device, electronic device and storage medium | |
| CN118470061A (en) | A multi-target tracking method and system based on improved Pointpillars network | |
| CN117246319A (en) | A DQN emergency braking adaptive control method based on driver style | |
| Yin | Design of Deep Learning Based Autonomous Driving Control Algorithm | |
| Wang et al. | A Multi-sensing Input and Multi-constraint Reward Mechanism Based Deep Reinforcement Learning Method for Self-driving Policy Learning | |
| CN115576327B (en) | Autonomous learning method based on edge computing and reasoning of autonomous driving smart car | |
| Xing et al. | Research on Target Recognition Algorithm Based on Improved PointNet++ | |
| CN121143362A (en) | Robot obstacle avoidance track automatic control system based on deep learning |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| GR01 | Patent grant | ||
| GR01 | Patent grant |