TWI841069B - Method for planning routes of vehicle, electronic device and computer-readable storage medium - Google Patents

Abstract

The present application provides a method for planning routes of a vehicle, electronic device and computer-readable storage medium. The method includes: acquiring images of the vehicle while the vehicle is moving; inputting the images into a target route planning model and obtaining target route information; extracting embedded vectors from the target route information; inputting the embedded vectors into a target driving style model and obtaining a driving route corresponding to a driving style of a user. The present application can improve the driving experience of drivers.

Description

Driving route planning method, electronic device and computer-readable storage medium

The present invention relates to the field of automatic driving technology, and in particular to a driving route planning method, electronic equipment and computer-readable storage medium.

The vehicle's automated driving system (ADS) needs to set up motion trajectory planning during the vehicle's driving process. In traditional technology, a motion trajectory planning network is usually used to output the vehicle's driving route planning. This type of method usually generates a driving route based on a unified driving rule, for example: when encountering an obstacle, avoid it from the left first. However, in the actual driving process, there are many uncertainties in the vehicle and the road, which makes it impossible to accurately plan the route, thus affecting the driving experience or driving safety.

In view of the above, it is necessary to provide a driving route planning method, electronic equipment and computer-readable storage medium that can solve the technical problem that the driving route generated by a single driving rule affects the driving experience.

This application provides a driving route planning method, the method comprising: obtaining an image of a vehicle during driving; inputting the image into a target route planning model to obtain target route information; extracting an embedded vector of the target route information; inputting the embedded vector into a target driving style model to obtain a driving route corresponding to the driving style.

In some optional implementations, before obtaining the image of the vehicle during driving, the method further includes: obtaining historical driving data of the vehicle; generating training data and sample labels corresponding to the training data based on the historical driving data; and training the initial route planning model using the training data to obtain the target route planning model.

In some optional embodiments, the training data is used to train the initial route planning model to obtain the target route planning model, including: logging the training data into the initial route planning model for training to obtain first predicted route information; calculating the first loss function value of the initial route planning model according to the first predicted route information and the sample label corresponding to the training data; if the first loss function value is less than or equal to the first threshold value, determining the trained initial route planning model as the target route planning model; if the first loss function value is greater than the first threshold value, returning to execute the step of obtaining the historical driving data of the vehicle.

In some optional embodiments, after obtaining the target route planning model, the method further includes: based on the training data and the target route planning model, training the initial driving style model to obtain the target driving style model, including: logging the training data into the target route planning model to obtain second predicted route information; extracting an embedding vector of the second predicted route information; inputting the embedding vector into the initial driving style model; driving style model, obtaining the third predicted route information; calculating the second loss function value of the initial driving style model according to the sample label and the third predicted route information; if the second loss function value is less than or equal to the second valve value, determining the trained initial driving style model as the target driving style model; if the second loss function value is greater than the second valve value, returning to the step of executing the step of obtaining the historical driving data of the vehicle.

In some optional embodiments, the training data includes image information with a time sequence relationship, and the step of logging the training data into the target route planning model to obtain the second predicted route information includes: inputting the image information with a time sequence relationship into the target route planning model to obtain the second predicted route information corresponding to different time periods.

In some optional embodiments, the method further includes: encoding the second predicted route information corresponding to the different time periods to obtain the encoded feature vector; mapping the encoded feature vector to the embedding space by linear transformation; calculating the embedding vector corresponding to the encoded feature vector in the embedding space, wherein the corresponding embedding vector includes the embedding vector corresponding to the second preset route information corresponding to the different time periods.

In some optional embodiments, the embedded vectors corresponding to the second preset route information corresponding to different time periods include: the vehicle position, vehicle head direction, vehicle speed, and vehicle yaw angular velocity corresponding to each time period.

In some optional embodiments, the sample label corresponding to the training data includes: obtaining the motion trajectory synthesized based on the positioning system and the inertial sensor as the sample label according to the temporal relationship of the image information in the training data.

This application also provides an electronic device, which includes a processor and a memory, and the processor is used to implement the driving route planning method when executing a computer program stored in the memory.

This application also provides a computer-readable storage medium, on which a computer program is stored, and when the computer program is executed by a processor, the driving route planning method is implemented.

The driving route planning method provided by this application can extract target route information, and further improve the experience of extracting a driving route corresponding to the user's driving style by extracting the embedded vector of the target route information.

1: Electronic equipment

10: Communication bus

11: Storage

12: Processor

13: Shooting equipment

21~24: Steps

31~35: Steps

41~46: Steps

Figure 1 is a schematic diagram of the application scenario of the driving route planning method provided by the embodiment of this application.

Figure 2 is a flow chart of the driving route planning method provided by the embodiment of this application.

Figure 3 is a training flowchart of the target route planning model provided by the embodiment of this application.

Figure 4 is a training flow chart of the target driving style model provided in the embodiment of this application.

For ease of understanding, some explanations of concepts related to the embodiments of this application are given as examples for reference.

It should be noted that in this application, "at least one" means one or more, and "plurality" means two or more than two. "And/or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and/or B can mean: A exists alone, A and B exist at the same time, and B exists alone, where A and B can be singular or plural. The terms "first", "second", "third", "fourth", etc. (if any) in the specification, claim and drawings of this application are used to distinguish similar objects, rather than to describe a specific order or precedence.

In order to better understand the driving route planning method, electronic device and computer-readable storage medium provided by the embodiment of this application, the application scenario of the driving route planning method of this application is first described below.

FIG1 is a schematic diagram of an application scenario of the driving route planning method provided by the embodiment of the present application. The driving route planning method provided by the embodiment of the present application is applied to an electronic device 1, wherein the electronic device 1 includes, but is not limited to, a storage device 11, at least one processor 12, and a camera 13 connected to each other via a communication bus 10. The camera 13 can be a vehicle-mounted camera or an external vehicle camera, such as a camera, to capture multiple images or videos in front of the vehicle.

The schematic diagram 1 is only an example of the electronic device 1 and does not constitute a limitation on the electronic device 1. It may include more or fewer components than shown in the diagram, or combine certain components, or different components. For example, the electronic device 1 may also include input and output devices, network access devices, etc.

In the embodiment of the present application, the electronic device 1 is applied to a vehicle, for example, it can be an onboard electronic device in a vehicle, or it can be an independent electronic device and can communicate and exchange data with the onboard device, thereby realizing the control of the vehicle.

In order to solve the technical problem that a driving route generated by a single driving rule affects the driving experience, the present application embodiment provides a driving route planning method, which is applied to an electronic device 1 and can recommend a vehicle driving route in combination with a user's driving style, thereby improving the accuracy of route planning and making the route planning more in line with the user's driving style, thereby effectively improving the driving experience.

As shown in FIG. 2, it is a flow chart of the driving route planning method provided by the embodiment of the present application. The driving route planning method described in the present application is applied in an electronic device (such as the electronic device 1 of FIG. 1). According to different requirements, the order of the steps in the flow chart can be changed, and some steps can be omitted.

Step 21, obtaining images of the vehicle during driving.

In some embodiments of the present application, the image in front of the vehicle can be captured by a built-in camera device on the vehicle (e.g., a dash cam, or cameras installed at various locations on the vehicle, etc.), or the video in front of the vehicle can be obtained by the camera device, and then each frame in the video can be extracted to obtain the image of the vehicle during driving, so as to analyze the image later and obtain driving information related to the user.

Step 22, input the image into the target route planning model to obtain the target route information.

In the embodiment of the present application, the image captured by the shooting device in real time is input into the target route planning model for processing, and then the target route information output by the target route planning model is obtained to provide the driving route of the vehicle. For example, the image in front of the vehicle is captured in real time by the shooting device, and the image at time t and the image at time t+1 are input into the target route planning model, and the target route information of the vehicle at time t+1 is output, that is, the route of the vehicle from time t to time t+1.

Before inputting the image into the target route planning model, the initial route planning model can be trained to obtain a trained target route planning model, thereby improving the accuracy of the target route information output by the target route planning model. The initial route planning model can include any one or a combination of models such as Long Short-Term Memory (LSTM), Recurrent Neural Network (RNN), and Convolutional Neural Networks (CNN).

Figure 3 is a training flow chart of the target route planning model provided in the embodiment of this application. As shown in Figure 3, the specific training steps of the target route planning model are as follows:

Step 31, obtain the historical driving data of the vehicle, and generate training data and sample labels corresponding to the training data based on the historical driving data.

In some embodiments of the present application, the historical driving data of the vehicle can be obtained to establish a vehicle motion trajectory data set, which may include but is not limited to: obstacle information on the driving section, images taken by a camera, and vehicle motion trajectory information synthesized based on a global positioning system (GPS) and an inertial measurement unit (IMU), wherein GPS is used to measure the vehicle's driving speed, positioning information, and current time information, and IMU is used to obtain the vehicle's driving acceleration and angular velocity information. The historical driving data of a vehicle can be driving data of a certain time period specified by the user, for example, driving data of the vehicle from March to June, or driving data of a certain area specified by the user, for example, driving data of the vehicle between location A and location B.

After obtaining the historical driving data of the vehicle, the historical driving data is pre-processed to generate training data and sample labels corresponding to the training data, wherein the sample labels corresponding to the training data can be the real route information corresponding to the training data. For example, the training data can be images taken by a shooting device at different times, and the sample labels corresponding to the training data can be route information synthesized based on GPS and IMU.

Step 32, log the training data into the initial route planning model for training to obtain the first predicted route information.

After obtaining the training data, the training data is logged into the initial route planning model for training to obtain the first predicted route information, wherein the first predicted route information includes the vehicle position, vehicle head direction, vehicle driving speed, vehicle yaw angular velocity, etc. in multiple time segments.

Step 33, calculate the first loss function value of the initial route planning model based on the first predicted route information and the sample label corresponding to the training data.

After obtaining the first predicted route information output by the initial route planning model, in order to determine whether the initial route planning model has been trained, the first loss function value of the initial route planning model can be calculated, and whether the initial route planning model has been trained can also be judged based on the preset number of training times.

Specifically, the first error value of the sample label corresponding to the first predicted route information and the training data can be calculated, and the first error value is used as the first loss function value of the initial route planning model, wherein the first error value can be calculated using the Euclidean distance.

Step 34, determine the size of the first loss function value and the first valve value.

Step 35, if the first loss function value is less than or equal to the first threshold value, the trained initial route planning model is used as the target route planning model.

In the embodiment of this application, the first threshold value is pre-set as a standard for measuring whether the initial route planning model has been trained. If the first loss function value is less than or equal to the first threshold value, it indicates that the trained initial route planning model has been trained, and the trained initial route planning model is used as the target route planning model.

Alternatively, it can also be based on the preset number of training times. If the current number of training times reaches the preset number of training times, it indicates that the training of the target route planning model is completed.

If the first loss function value is greater than the first threshold value, return to step 31 and retrain the initial route planning model.

In the embodiment of the present application, based on the preset first valve value, if the first loss function value is greater than the first valve value, it indicates that the initial route planning model has not completed the training, and returns to execute step 31.

Alternatively, based on the preset number of training times, if the current number of training times does not reach the preset number of training times, continue to train the initial route planning model.

After the target route planning model is trained, the real-time images can be input into the target route planning model during the driving process of the vehicle, so that the target route planning model outputs the target route information.

Step 23, extract the embedding vector of the target route information.

In the embodiment of the present application, after the target route information is obtained based on the target route planning model, in order to avoid the influence of data noise and better utilize the target route planning model to obtain the target route information, an auto encoder (AE) is used to extract the embedding vector (Embedding Vector) of the target route information, wherein the auto encoder is used to learn the mapping relationship, thereby obtaining a reconstructed vector, namely the embedding vector.

Before inputting the target route information into the autoencoder, in order to improve the accuracy of embedding vector extraction, a self-encoder can be pre-trained. After the self-encoder is trained, the target route information is input into the self-encoder. After receiving the target route information, the self-encoder encodes the target route information to obtain the encoded feature vector, and the encoded feature vector is mapped to the embedding space by linear transformation. According to all the parameters, mapping matrix and bias weights contained in the convolutional network, the embedding vector corresponding to the encoded feature vector in the embedding space is calculated.

For example, multiple route information obtained by driving at the same place at different times is input into the self-encoder, and the route information at each time and location can be extracted from the self-encoder, that is, the vehicle position, vehicle head direction, vehicle driving speed, and vehicle yaw angular velocity at different times and locations.

Step 24, input the embedded vector into the target driving style model to obtain the driving route corresponding to the driving style.

After obtaining the embedding vector, the embedding vector is used as the input of the target driving style model to obtain the driving route corresponding to the user's driving style. Before the embedding vector is input into the target driving style model, the initial target driving style model can be trained so that the driving route output by the trained initial target driving style model is closer to the user's driving style. The trained initial target driving style model is the target driving style model. The initial target driving style model can include any one or a combination of models such as Long Short-Term Memory (LSTM), Recurrent Neural Network (RNN), and Convolutional Neural Networks (CNN).

In this application embodiment, in order to improve the user's driving experience, the driving route can be optimized in combination with the user's driving style to provide effective assistance to the vehicle driving. Furthermore, when the vehicle is in automatic driving mode, the user can be provided with a driving route with a driving style according to the user's own driving style, thereby effectively improving the user's driving experience.

Figure 4 is a training flow chart of the target driving style model provided in the embodiment of this application. As shown in Figure 4, the specific training steps of the target driving style model are as follows:

Step 41, obtain the training data and the sample labels corresponding to the training data, log the training data into the target route planning model, and obtain the second predicted route information.

In some embodiments of the present application, after the target route planning model is obtained through training, the training data is logged into the target route planning model, wherein the training data is image information with a time sequence relationship, and the second predicted route information corresponding to different time periods output by the target route planning model is obtained.

Step 42, extract the embedding vector of the second predicted route information.

In some embodiments of the present application, after obtaining the second predicted route information corresponding to different times, in order to remove data noise in the second predicted route information, the second predicted route information corresponding to different times is input into the self-encoder, and the second predicted route information corresponding to different times is encoded to obtain the encoded feature vector. The encoded feature vector is mapped to the embedding space by linear transformation, and the embedding vector corresponding to the encoded feature vector in the embedding space is calculated according to all parameters, mapping matrix and bias weights contained in the convolution network. The embedding vector includes the vehicle position, vehicle head direction, vehicle speed and vehicle yaw angular velocity corresponding to each time period.

Step 43, input the embedded vector into the initial driving style model to obtain the third predicted route information.

After the embedding vector is obtained, the embedding vector is used as training data for training the initial driving style model. The embedding vector is input into the initial driving style model for training to generate the third predicted route information that meets the user's driving style.

Step 44, calculate the second loss function value of the initial driving style model based on the sample label and the third predicted route information.

In order to determine whether the third preset route information meets the user's driving style, the Euclidean distance is used to calculate the second loss function value of the initial driving style model. Based on the size of the first loss function value, it is determined whether the initial driving style model has been trained. It can also be determined whether the initial driving style model has been trained based on the preset number of training times.

Step 45, determine the size of the second loss function value and the second valve value.

Step 46, if the second loss function value is less than or equal to the second threshold value, the trained initial driving style model is determined as the target driving style model.

In the embodiment of the present application, the second valve value is pre-set as a standard for measuring whether the initial driving style model has been trained. If the second loss function value is less than or equal to the second valve value, it indicates that the trained initial driving style model has been trained, and the trained initial driving style model is used as the target driving style model.

Alternatively, it can also be based on a preset number of training times. If the current number of training times reaches the preset number of training times, it indicates that the target driving style model training is completed.

If the second loss function value is greater than the second valve value, return to step 41 and retrain the initial driving style model.

In the embodiment of the present application, based on the preset second valve value, if the second loss function value is greater than the second valve value, it indicates that the initial driving style model has not completed the training, and returns to execute step 41.

Or, based on the preset number of training times, if the current number of training times does not reach the preset number of training times, continue to train the initial driving style model.

After determining that the initial driving style model training is completed, the target driving style model is obtained, and the target driving style model is used to generate a driving route with the user's driving style to improve the user's driving experience when making an appointment to enter automatic driving.

This application improves the accuracy of obtaining target route information by training the target route planning model, filters out the noise in the target route information by extracting the embedding vector of the target route information, and further trains the target driving style model to make the obtained driving route more in line with the user's driving style, thereby improving the user's automatic driving experience.

Please continue to refer to Figure 1. In this embodiment, the memory 11 can be an internal memory of the electronic device 1, that is, a memory built into the electronic device 1. In other embodiments, the memory 11 can also be an external memory of the electronic device 1, that is, a memory externally connected to the electronic device 1.

In some embodiments, the memory 11 is used to store program codes and various data, and to achieve high-speed and automatic access to programs or data during the operation of the electronic device 1.

The memory 11 may include a random access memory, and may also include a non-volatile memory, such as a hard disk, a memory (Memory), a plug-in hard disk, a smart media card (SMC), a secure digital (SD) card, a memory card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

In one embodiment, the processor 12 may be a central processing unit (CPU), or other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor, or the processor may be any other conventional processor, etc.

If the program code and various data in the memory 11 are implemented in the form of a software functional unit and sold or used as an independent product, they can be stored in a computer-readable storage medium. Based on this understanding, the present application implements all or part of the processes in the above-mentioned embodiment method, such as the driving route planning method, which can also be completed by instructing the relevant hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of the above-mentioned method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in the form of source code, object code, executable file or some intermediate form. The computer-readable medium may include: any entity or device capable of carrying the computer program code, recording medium, flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), etc.

It is understandable that the module division described above is a logical function division, and there may be other division methods in actual implementation. In addition, each functional module in each embodiment of the present application may be integrated in the same processing unit, or each module may exist physically separately, or two or more modules may be integrated in the same unit. The above-mentioned integrated module may be implemented in the form of hardware or in the form of hardware plus software functional modules.

Finally, it should be noted that the above embodiments are only used to illustrate the technical solution of this application and are not limiting. Although this application is described in detail with reference to the preferred embodiments, ordinary technicians in this field should understand that the technical solution of this application can be modified or replaced by equivalents without departing from the spirit and scope of the technical solution of this application.

21~24: Steps

Claims (10)

A driving route planning method, wherein the method includes: obtaining an image of a vehicle during driving; inputting the image into a target route planning model to obtain target route information; extracting an embedding vector of the target route information; inputting the embedding vector into a target driving style model to obtain a driving route corresponding to the driving style. The driving route planning method as described in claim 1, wherein before obtaining the image of the vehicle during driving, the method further includes: obtaining historical driving data of the vehicle; generating training data and sample labels corresponding to the training data based on the historical driving data; and training the initial route planning model using the training data to obtain the target route planning model. The driving route planning method as described in claim 2, wherein the training data is used to train the initial route planning model to obtain the target route planning model, including: logging the training data into the initial route planning model for training to obtain first predicted route information; calculating the first loss function value of the initial route planning model according to the first predicted route information and the sample label corresponding to the training data; if the first loss function value is less than or equal to the first threshold value, determining the trained initial route planning model as the target route planning model; if the first loss function value is greater than the first threshold value, returning to the step of executing the step of obtaining the historical driving data of the vehicle. The driving route planning method as described in claim 2, wherein after obtaining the target route planning model, the method further comprises: based on the training data and the target route planning model, training the initial driving style model to obtain the target driving style model, including: logging the training data into the target route planning model to obtain second predicted route information; extracting the embedding vector of the second predicted route information; inputting the embedding vector into The initial driving style model obtains the third predicted route information; the second loss function value of the initial driving style model is calculated according to the sample label and the third predicted route information; if the second loss function value is less than or equal to the second threshold value, the trained initial driving style model is determined as the target driving style model; if the second loss function value is greater than the second threshold value, the step of obtaining the historical driving data of the vehicle is returned to be executed. The driving route planning method as described in claim 4, wherein the training data includes image information with a time sequence relationship, and the step of logging the training data into the target route planning model to obtain the second predicted route information includes: inputting the image information with a time sequence relationship into the target route planning model to obtain the second predicted route information corresponding to different time periods. The driving route planning method as described in claim 5, wherein the method further comprises: encoding the second predicted route information corresponding to the different time periods to obtain the encoded feature vector; mapping the encoded feature vector to the embedding space through linear transformation; calculating the embedding vector corresponding to the encoded feature vector in the embedding space, wherein the corresponding embedding vector includes the embedding vector corresponding to the second preset route information corresponding to the different time periods. The driving route planning method as described in claim 6, wherein the embedded vectors corresponding to the second preset route information corresponding to the different time periods include: the vehicle position, vehicle head direction, vehicle speed, and vehicle yaw angular velocity corresponding to each time period. A driving route planning method as described in any one of claims 2 to 7, wherein the sample label corresponding to the training data includes: obtaining a motion trajectory synthesized based on a positioning system and an inertial sensor as the sample label according to the temporal relationship of the image information in the training data. An electronic device, wherein the electronic device includes a processor and a memory, and the processor is used to execute a computer program stored in the memory to implement the driving route planning method as described in any one of claims 1 to 8. A computer-readable storage medium, wherein the computer-readable storage medium stores at least one instruction, and when the at least one instruction is executed by a processor, the driving route planning method as described in any one of claim items 1 to 8 is implemented.

Images