[go: up one dir, main page]

CN111626097A - A method, device, electronic device and storage medium for predicting the future trajectory of an obstacle - Google Patents

A method, device, electronic device and storage medium for predicting the future trajectory of an obstacle Download PDF

Info

Publication number
CN111626097A
CN111626097A CN202010274946.2A CN202010274946A CN111626097A CN 111626097 A CN111626097 A CN 111626097A CN 202010274946 A CN202010274946 A CN 202010274946A CN 111626097 A CN111626097 A CN 111626097A
Authority
CN
China
Prior art keywords
scene map
trajectory
obstacle
area
historical
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
CN202010274946.2A
Other languages
Chinese (zh)
Inventor
薛睿
鹿朋
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Zhejiang Geely Holding Group Co Ltd
Geely Automobile Research Institute Ningbo Co Ltd
Original Assignee
Zhejiang Geely Holding Group Co Ltd
Geely Automobile Research Institute Ningbo Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Zhejiang Geely Holding Group Co Ltd, Geely Automobile Research Institute Ningbo Co Ltd filed Critical Zhejiang Geely Holding Group Co Ltd
Priority to CN202010274946.2A priority Critical patent/CN111626097A/en
Publication of CN111626097A publication Critical patent/CN111626097A/en
Pending legal-status Critical Current

Links

Images

Classifications

    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06VIMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00Scenes; Scene-specific elements
    • G06V20/50Context or environment of the image
    • G06V20/56Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle
    • G06V20/58Recognition of moving objects or obstacles, e.g. vehicles or pedestrians; Recognition of traffic objects, e.g. traffic signs, traffic lights or roads
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06FELECTRIC DIGITAL DATA PROCESSING
    • G06F16/00Information retrieval; Database structures therefor; File system structures therefor
    • G06F16/20Information retrieval; Database structures therefor; File system structures therefor of structured data, e.g. relational data
    • G06F16/29Geographical information databases
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/04Architecture, e.g. interconnection topology
    • G06N3/045Combinations of networks
    • GPHYSICS
    • G06COMPUTING OR CALCULATING; COUNTING
    • G06NCOMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00Computing arrangements based on biological models
    • G06N3/02Neural networks
    • G06N3/08Learning methods
    • G06N3/084Backpropagation, e.g. using gradient descent

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Data Mining & Analysis (AREA)
  • General Engineering & Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Biophysics (AREA)
  • Evolutionary Computation (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Databases & Information Systems (AREA)
  • Multimedia (AREA)
  • Remote Sensing (AREA)
  • Traffic Control Systems (AREA)

Abstract

The application discloses a method, a device, electronic equipment and a storage medium for predicting future trajectories of obstacles, wherein the method comprises the following steps: acquiring a scene map corresponding to an area where a vehicle is located; the scene map comprises traffic information of an area, historical tracks of obstacles in a certain range of a vehicle in the area are obtained, the historical tracks are projected to corresponding positions of the scene map to obtain a scene map to be predicted containing the historical tracks, the scene map to be predicted is input into a trained track prediction model, and future tracks of the obstacles are obtained. In this way, the accuracy of future trajectory prediction of the obstacle can be improved by introducing the scene map and the trajectory prediction model.

Description

一种障碍物未来轨迹的预测方法、装置、电子设备及存储介质A method, device, electronic device and storage medium for predicting the future trajectory of an obstacle

技术领域technical field

本申请涉及互联网技术领域,尤其涉及一种障碍物未来轨迹的预测方法、装置、电子设备及存储介质。The present application relates to the field of Internet technologies, and in particular, to a method, device, electronic device and storage medium for predicting the future trajectory of an obstacle.

背景技术Background technique

障碍物的轨迹预测指根据障碍物历史的运动路径,预测其未来的轨迹。在自动驾驶场景中,需要对无人驾驶车辆周围,可能对其造成影响的障碍物的运动轨迹进行预测,而障碍物包括:机动车辆、非机动车辆和行人等。面临的具体难点主要包括以下几个方面:The trajectory prediction of the obstacle refers to predicting the future trajectory of the obstacle based on the historical movement path of the obstacle. In an autonomous driving scenario, it is necessary to predict the trajectory of the obstacles around the unmanned vehicle that may affect it, and the obstacles include: motor vehicles, non-motor vehicles, and pedestrians. The specific difficulties faced mainly include the following aspects:

1)运动轨迹需同时满足交通规则和物理约束;1) The motion trajectory needs to satisfy both traffic rules and physical constraints;

2)障碍物运动状态,以及周围交通环境等的特征描述;2) The motion state of the obstacle and the characteristic description of the surrounding traffic environment;

3)需能够提供出多种可能的合理轨迹,以便进行后续筛选;3) It needs to be able to provide a variety of possible reasonable trajectories for subsequent screening;

4)路口场景交通路况复杂,包含多种可能性,且没有明确的车道划分范围。4) The traffic conditions in the intersection scene are complex, contain multiple possibilities, and there is no clear lane division range.

发明内容SUMMARY OF THE INVENTION

本申请实施例提供了一种障碍物未来轨迹的预测方法、装置、电子设备及存储介质,用于提高障碍物的未来轨迹预测的准确性。Embodiments of the present application provide a method, device, electronic device, and storage medium for predicting a future trajectory of an obstacle, so as to improve the accuracy of predicting the future trajectory of an obstacle.

一方面,本申请实施例提供了一种障碍物未来轨迹的预测方法,该方法包括:On the one hand, an embodiment of the present application provides a method for predicting a future trajectory of an obstacle, the method comprising:

获取车辆所在的区域对应的场景地图;其中,场景地图包括区域的交通信息;Obtain the scene map corresponding to the area where the vehicle is located; wherein, the scene map includes the traffic information of the area;

获取车辆一定范围内的障碍物在区域内的历史轨迹;Obtain the historical trajectory of obstacles within a certain range of the vehicle in the area;

将历史轨迹投影到场景地图的对应位置,得到包含历史轨迹的待预测的场景地图;Project the historical trajectory to the corresponding position of the scene map, and obtain the scene map to be predicted containing the historical trajectory;

将待预测的场景地图输入已训练好的轨迹预测模型,得到障碍物的未来轨迹。Input the scene map to be predicted into the trained trajectory prediction model to obtain the future trajectory of the obstacle.

可选的,方法还包括训练得到轨迹预测模型的步骤;训练得到轨迹预测模型包括:Optionally, the method further includes the step of obtaining a trajectory prediction model through training; the trajectory prediction model obtained through training includes:

获取样本数据集,样本数据集包括多个训练场景地图和每个训练场景地图对应的实际未来轨迹;每个训练场景地图包含训练历史轨迹;Obtain a sample data set, where the sample data set includes multiple training scene maps and the actual future trajectory corresponding to each training scene map; each training scene map includes a training historical trajectory;

构建预设机器学习模型,将预设机器学习模型确定为当前机器学习模型;Build a preset machine learning model, and determine the preset machine learning model as the current machine learning model;

基于当前机器学习模型,对训练场景地图进行轨迹预测操作,确定训练场景地图对应的预测未来轨迹;Based on the current machine learning model, the trajectory prediction operation is performed on the training scene map, and the predicted future trajectory corresponding to the training scene map is determined;

基于训练场景地图对应的预测未来轨迹和实际未来轨迹,确定损失值;Determine the loss value based on the predicted future trajectory and the actual future trajectory corresponding to the training scene map;

当损失值大于预设阈值时,基于损失值进行反向传播,对当前机器学习模型进行更新以得到更新后的机器学习模型,将更新后的机器学习模型重新确定为当前机器学习模型;重复步骤:基于当前机器学习模型,对训练场景地图进行轨迹预测操作,确定训练场景地图对应的预测未来轨迹;When the loss value is greater than the preset threshold, backpropagation is performed based on the loss value, the current machine learning model is updated to obtain the updated machine learning model, and the updated machine learning model is re-determined as the current machine learning model; repeat the steps : Based on the current machine learning model, perform trajectory prediction operation on the training scene map, and determine the predicted future trajectory corresponding to the training scene map;

当损失值小于或等于预设阈值时,将当前机器学习模型确定为轨迹预测模型。When the loss value is less than or equal to the preset threshold, the current machine learning model is determined as the trajectory prediction model.

可选的,获取车辆一定范围内的障碍物在区域内的历史轨迹,包括:Optionally, obtain the historical trajectories of obstacles within a certain range of the vehicle in the area, including:

通过传感器获取障碍物在区域内多个连续的时间点的历史位置信息;Obtain the historical position information of obstacles at multiple consecutive time points in the area through sensors;

将多个连续的时间点的历史位置信息拼接得到历史轨迹;The historical trajectory is obtained by splicing the historical location information of multiple consecutive time points;

其中,传感器包括摄像头、激光雷达和毫米波雷达中的一项或者多项。The sensors include one or more of cameras, lidars, and millimeter-wave radars.

可选的,获取车辆所在的区域对应的场景地图,包括:Optionally, obtain a scene map corresponding to the area where the vehicle is located, including:

确定车辆所在的区域;determine the area in which the vehicle is located;

基于区域对应的高精地图获取车辆预设范围内的交通信息;Obtain the traffic information within the preset range of the vehicle based on the high-precision map corresponding to the area;

将交通信息投影到俯瞰角度下的预设地图中的对应位置,得到场景地图。Project the traffic information to the corresponding position in the preset map under the overlook angle to obtain the scene map.

可选的,得到障碍物的未来轨迹之后,还包括;Optionally, after obtaining the future trajectory of the obstacle, it also includes;

将未来轨迹显示至待预测的场景地图中,并显示待预测的场景地图,以使车辆根据未来轨迹规避障碍物。Display the future trajectory in the scene map to be predicted, and display the scene map to be predicted, so that the vehicle can avoid obstacles according to the future trajectory.

另一方面提供了一种障碍物未来轨迹的预测装置,该装置包括:Another aspect provides a device for predicting the future trajectory of an obstacle, the device comprising:

第一获取模块,用于获取车辆所在的区域对应的场景地图;其中,场景地图包括区域的交通信息;The first acquisition module is used to acquire the scene map corresponding to the area where the vehicle is located; wherein, the scene map includes the traffic information of the area;

第二获取模块,用于获取车辆一定范围内的障碍物在区域内的历史轨迹;The second acquisition module is used to acquire the historical trajectory of obstacles within a certain range of the vehicle in the area;

投影模块,用于将历史轨迹投影到场景地图的对应位置,得到包含历史轨迹的待预测的场景地图;The projection module is used to project the historical trajectory to the corresponding position of the scene map, and obtain the scene map to be predicted including the historical trajectory;

预测模块,用于将待预测的场景地图输入已训练好的轨迹预测模型,得到障碍物的未来轨迹。The prediction module is used to input the scene map to be predicted into the trained trajectory prediction model to obtain the future trajectory of the obstacle.

另一方面提供了一种电子设备,该电子设备包括处理器和存储器,存储器中存储有至少一条指令或至少一段程序,该至少一条指令或至少一段程序由处理器加载并执行以实现如上述的障碍物未来轨迹的预测方法。Another aspect provides an electronic device, the electronic device includes a processor and a memory, the memory stores at least one instruction or at least a piece of program, the at least one instruction or at least a piece of program is loaded and executed by the processor to realize the above A prediction method for the future trajectory of obstacles.

另一方面提供了一种计算机可读存储介质,存储介质中存储有至少一条指令或至少一段程序,该至少一条指令或至少一段程序由处理器加载并执行以实现如上述的障碍物未来轨迹的预测方法。Another aspect provides a computer-readable storage medium, the storage medium stores at least one instruction or at least one program, and the at least one instruction or at least one program is loaded and executed by a processor to realize the above-mentioned future trajectory of the obstacle. method of prediction.

本申请实施例提供的障碍物未来轨迹的预测方法、装置、电子设备及存储介质,具有如下技术效果:The method, device, electronic device, and storage medium for predicting the future trajectory of an obstacle provided by the embodiments of the present application have the following technical effects:

获取车辆所在的区域对应的场景地图;其中,场景地图包括区域的交通信息,获取车辆一定范围内的障碍物在区域内的历史轨迹,将历史轨迹投影到场景地图的对应位置,得到包含历史轨迹的待预测的场景地图,将待预测的场景地图输入已训练好的轨迹预测模型,得到障碍物的未来轨迹。如此,可以通过引入场景地图以及轨迹预测模型提高障碍物的未来轨迹预测的准确性。Obtain the scene map corresponding to the area where the vehicle is located; wherein, the scene map includes the traffic information of the area, obtain the historical trajectory of the obstacles within a certain range of the vehicle in the area, project the historical trajectory to the corresponding position of the scene map, and obtain the historical trajectory containing the historical trajectory The scene map to be predicted is input into the trained trajectory prediction model to obtain the future trajectory of the obstacle. In this way, the accuracy of future trajectory prediction of obstacles can be improved by introducing a scene map and a trajectory prediction model.

附图说明Description of drawings

为了更清楚地说明本申请实施例或现有技术中的技术方案和优点,下面将对实施例或现有技术描述中所需要使用的附图作简单的介绍,显而易见地,下面描述中的附图仅仅是本申请的一些实施例,对于本领域普通技术人员来讲,在不付出创造性劳动的前提下,还可以根据这些附图获得其它附图。In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present application or in the prior art, the following briefly introduces the accompanying drawings used in the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description The drawings are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can also be obtained based on these drawings without any creative effort.

图1是本申请实施例提供的一种应用环境的示意图;1 is a schematic diagram of an application environment provided by an embodiment of the present application;

图2是本申请实施例提供的一种障碍物未来轨迹的预测方法的流程示意图;2 is a schematic flowchart of a method for predicting a future trajectory of an obstacle provided by an embodiment of the present application;

图3是本申请实施例提供的一种障碍物未来轨迹的预测方法的流程示意图;3 is a schematic flowchart of a method for predicting a future trajectory of an obstacle provided by an embodiment of the present application;

图4是本申请实施例提供的一种障碍物未来轨迹的预测方法的流程示意图;4 is a schematic flowchart of a method for predicting a future trajectory of an obstacle provided by an embodiment of the present application;

图5是本申请实施例提供的一种训练轨迹预测模型的流程示意图;5 is a schematic flowchart of a training trajectory prediction model provided by an embodiment of the present application;

图6是本申请实施例提供的一种障碍物未来轨迹的预测装置的结构示意图;6 is a schematic structural diagram of a device for predicting a future trajectory of an obstacle provided by an embodiment of the present application;

图7是本申请实施例提供的一种障碍物未来轨迹的预测方法的服务器的硬件结构框图。FIG. 7 is a hardware structural block diagram of a server for a method for predicting a future trajectory of an obstacle provided by an embodiment of the present application.

具体实施方式Detailed ways

下面将结合本申请实施例中的附图,对本申请实施例中的技术方案进行清楚、完整地描述,显然,所描述的实施例仅仅是本申请一部分实施例,而不是全部的实施例。基于本申请中的实施例,本领域普通技术人员在没有做出创造性劳动的前提下所获得的所有其他实施例,都属于本申请保护的范围。The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work shall fall within the protection scope of the present application.

需要说明的是,本申请的说明书和权利要求书及上述附图中的术语“第一”、“第二”等是用于区别类似的对象,而不必用于描述特定的顺序或先后次序。应该理解这样使用的数据在适当情况下可以互换,以便这里描述的本申请的实施例能够以除了在这里图示或描述的那些以外的顺序实施。此外,术语“包括”和“具有”以及他们的任何变形,意图在于覆盖不排他的包含,例如,包含了一系列步骤或单元的过程、方法、系统、产品或服务器不必限于清楚地列出的那些步骤或单元,而是可包括没有清楚地列出的或对于这些过程、方法、产品或设备固有的其它步骤或单元。It should be noted that the terms "first", "second", etc. in the description and claims of the present application and the above drawings are used to distinguish similar objects, and are not necessarily used to describe a specific sequence or sequence. It is to be understood that data so used may be interchanged under appropriate circumstances so that the embodiments of the application described herein can be practiced in sequences other than those illustrated or described herein. Furthermore, the terms "comprising" and "having" and any variations thereof, are intended to cover non-exclusive inclusion, for example, a process, method, system, product or server comprising a series of steps or units is not necessarily limited to those expressly listed Rather, those steps or units may include other steps or units not expressly listed or inherent to these processes, methods, products or devices.

请参阅图1,图1是本申请实施例提供的一种应用环境的示意图,该示意图包括车辆101和服务器102,其中,一种可选的实施方式中,该服务器102可以是设置在车辆101中的车载服务器,该车载服务器包含轨迹预测模型,并可以实时的获取想要的数据,以备后续可以得到障碍物的未来轨迹。另一种可选的实施方式中,该车辆101内可以设置有自己的车载服务器,而该车载服务器和图1中显示的服务器102并不是同一个服务器,车载服务器将获得的数据传输给服务器102后,可以由服务器完成后续的步骤,最终得到障碍物的未来轨迹。下面将第一种情况涉及的车载服务器和第二种情况涉及的服务器统一称呼为服务器。Please refer to FIG. 1. FIG. 1 is a schematic diagram of an application environment provided by an embodiment of the present application. The schematic diagram includes a vehicle 101 and a server 102. In an optional implementation manner, the server 102 may be provided in the vehicle 101. The in-vehicle server contains the trajectory prediction model, and can obtain the desired data in real time, so that the future trajectory of the obstacle can be obtained in the future. In another optional embodiment, the vehicle 101 may be provided with its own on-board server, and the on-board server and the server 102 shown in FIG. 1 are not the same server, and the on-board server transmits the obtained data to the server 102 After that, the server can complete the subsequent steps, and finally get the future trajectory of the obstacle. The in-vehicle server involved in the first situation and the server involved in the second situation are collectively referred to as servers below.

具体的,服务器102获取车辆101所在的区域对应的场景地图,其中,场景地图包括区域的交通信息,同时,服务器102可以获取车辆101一定范围内的障碍物在区域内的历史轨迹,将历史轨迹投影到场景地图的对应位置,得到包含历史轨迹的待预测的场景地图,最后,服务器102将待预测的场景地图输入已训练好的轨迹预测模型,得到障碍物的未来轨迹。Specifically, the server 102 obtains a scene map corresponding to the area where the vehicle 101 is located, wherein the scene map includes the traffic information of the area, and at the same time, the server 102 can obtain the historical trajectory of the obstacles within a certain range of the vehicle 101 in the area, and convert the historical trajectory Projecting to the corresponding position of the scene map to obtain the scene map to be predicted including the historical trajectory, and finally, the server 102 inputs the scene map to be predicted into the trained trajectory prediction model to obtain the future trajectory of the obstacle.

以下介绍本申请一种障碍物未来轨迹的预测方法的具体实施例,图2是本申请实施例提供的一种障碍物未来轨迹的预测方法的流程示意图,本说明书提供了如实施例或流程图的方法操作步骤,但基于常规或者无创造性的劳动可以包括更多或者更少的操作步骤。实施例中列举的步骤顺序仅仅为众多步骤执行顺序中的一种方式,不代表唯一的执行顺序。在实际中的系统或服务器产品执行时,可以按照实施例或者附图所示的方法顺序执行或者并行执行(例如并行处理器或者多线程处理的环境)。具体的如图2所示,该方法可以包括:A specific embodiment of a method for predicting a future trajectory of an obstacle according to the present application is described below. FIG. 2 is a schematic flowchart of a method for predicting a future trajectory of an obstacle provided by an embodiment of the present application. method operating steps, but may include more or less operating steps based on routine or non-creative labor. The sequence of steps enumerated in the embodiments is only one of the execution sequences of many steps, and does not represent the only execution sequence. When an actual system or server product is executed, it can be executed sequentially or in parallel (for example, in a parallel processor or multi-threaded processing environment) according to the embodiments or the methods shown in the accompanying drawings. Specifically, as shown in Figure 2, the method may include:

S201:获取车辆所在的区域对应的场景地图。其中,场景地图包括区域的交通信息。S201: Obtain a scene map corresponding to the area where the vehicle is located. Wherein, the scene map includes the traffic information of the area.

本申请实施例中,该车辆可以是无人驾驶车辆。In this embodiment of the present application, the vehicle may be an unmanned vehicle.

本申请实施例中,可以用图3所示的一种障碍物未来轨迹的预测方法中的部分内容具体实施步骤S201中的内容,具体方法为:In this embodiment of the present application, the content in step S201 may be specifically implemented by using part of the content in the method for predicting the future trajectory of an obstacle shown in FIG. 3 , and the specific method is as follows:

S2011:确定车辆所在的区域。S2011: Determine the area where the vehicle is located.

本申请实施例中,服务器可以根据车辆的定位系统确定车辆所在的区域。In this embodiment of the present application, the server may determine the area where the vehicle is located according to the positioning system of the vehicle.

S2013:基于区域对应的高精地图获取车辆预设范围内的交通信息。S2013: Acquire traffic information within a preset range of the vehicle based on the high-precision map corresponding to the area.

一种可选的实施方式中,交通信息可以包括车道信息、车道中心线信息、人行横道信息、交通灯信息和停止线信息等等。具体的,可以是某个路口(三岔路口,十字路口等等)的车道信息,车道中心线信息、人行横道信息、交通灯信息和停止线信息等等。In an optional embodiment, the traffic information may include lane information, lane centerline information, pedestrian crossing information, traffic light information, stop line information, and the like. Specifically, it may be lane information of a certain intersection (three forks, intersection, etc.), lane center line information, pedestrian crossing information, traffic light information, stop line information, and so on.

S2015:将交通信息投影到俯瞰角度下的预设地图中的对应位置,得到场景地图。S2015: Project the traffic information to a corresponding position in the preset map under the bird's-eye view angle to obtain a scene map.

该场景地图可以以图片的形式保存,场景地图上不同的交通信息可以以不通的颜色表示,比如,车道信息可以以深红色表示,人行横道信息可以以浅绿色表示。The scene map can be saved in the form of a picture, and different traffic information on the scene map can be expressed in different colors, for example, the lane information can be expressed in dark red, and the crosswalk information can be expressed in light green.

S203:获取车辆一定范围内的障碍物在区域内的历史轨迹。S203: Obtain the historical trajectory of obstacles within a certain range of the vehicle in the area.

本申请实施例中的障碍物可以是机动车辆、非机动车辆和行人。The obstacles in the embodiments of the present application may be motor vehicles, non-motor vehicles and pedestrians.

一种可选的实施方式中,图4可以表示本申请一种障碍物未来轨迹的预测方法的具体实施例,该种方式中,S203的步骤可以用如下的内容表示:In an optional implementation manner, FIG. 4 may represent a specific example of a method for predicting the future trajectory of an obstacle in the present application. In this manner, the step of S203 may be represented by the following content:

S2031:通过传感器获取障碍物在区域内多个连续的时间点的历史位置信息。S2031: Acquire historical position information of obstacles at multiple consecutive time points in the area through the sensor.

本申请实施例中,传感器包括摄像头、激光雷达和毫米波雷达中的一项或者多项,传感器可以设置在车辆的四周,用来获取该车辆一定范围内的障碍物的行动轨迹。In the embodiment of the present application, the sensor includes one or more of a camera, a lidar, and a millimeter-wave radar, and the sensor can be arranged around the vehicle to obtain the trajectory of obstacles within a certain range of the vehicle.

由于车辆的一定范围内的可能存在多个障碍物,每个障碍物的运动轨迹可以表示为一连串的多个连续的时间点的历史位置

Figure BDA0002444431310000071
其中,t=1...tobs。Since there may be multiple obstacles within a certain range of the vehicle, the motion trajectory of each obstacle can be represented as a series of historical positions of multiple consecutive time points
Figure BDA0002444431310000071
where t=1...t obs .

S2033:将多个连续的时间点的历史位置信息拼接得到历史轨迹。S2033: splicing the historical location information of multiple consecutive time points to obtain a historical track.

S205:将历史轨迹投影到场景地图的对应位置,得到包含历史轨迹的待预测的场景地图。S205: Project the historical track to a corresponding position of the scene map, and obtain a scene map to be predicted including the historical track.

服务器将历史轨迹投影到场景地图的对应位置,该轨迹轨迹可以以某种颜色显示在场景地图中,得到包含历史轨迹的待预测的场景地图。The server projects the historical track to the corresponding position of the scene map, and the track track can be displayed in the scene map in a certain color, so as to obtain the scene map to be predicted including the historical track.

S207:将待预测的场景地图输入已训练好的轨迹预测模型,得到障碍物的未来轨迹。S207: Input the scene map to be predicted into the trained trajectory prediction model to obtain the future trajectory of the obstacle.

其中,轨迹预测模型是一种机器学习模型,机器学习(Machine Learning,ML)是一门多领域交叉学科,涉及概率论、统计学、逼近论、凸分析、算法复杂度理论等多门学科。专门研究计算机怎样模拟或实现人类的学习行为,以获取新的知识或技能,重新组织已有的知识结构使之不断改善自身的性能。机器学习是人工智能的核心,是使计算机具有智能的根本途径,其应用遍及人工智能的各个领域。机器学习和深度学习通常包括人工神经网络、置信网络、强化学习、迁移学习、归纳学习、式教学习等技术。机器学习可以分为有监督的机器学习,无监督的机器学习和半监督的机器学习。可选的,轨迹预测模型可以使用卷积神经网络或其他具有类似功能的神经网络结构,并根据需要进行训练、验证、测试数据获取的网络模型。Among them, the trajectory prediction model is a machine learning model, and machine learning (ML) is a multi-domain interdisciplinary subject involving probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in how computers simulate or realize human learning behaviors to acquire new knowledge or skills, and to reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent, and its applications are in all fields of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, teaching learning and other technologies. Machine learning can be divided into supervised machine learning, unsupervised machine learning and semi-supervised machine learning. Optionally, the trajectory prediction model may use a convolutional neural network or other neural network structure with similar functions, and perform training, verification, and testing of the network model for data acquisition as required.

下面基于一种有监督的机器学习介绍如何训练轨迹预测模型,如图5所示,包括:The following describes how to train a trajectory prediction model based on a supervised machine learning, as shown in Figure 5, including:

S501:获取样本数据集,样本数据集包括多个训练场景地图和每个训练场景地图对应的实际未来轨迹;每个训练场景地图包含训练历史轨迹;S501: Obtain a sample data set, where the sample data set includes a plurality of training scene maps and an actual future trajectory corresponding to each training scene map; each training scene map includes a training historical trajectory;

场景地图中的训练历史轨迹由于来自不同的时间,因此,同样使用像素颜色来对历史时间做区分。可以理解为,距离当前状态的时间越接近的特征,颜色越鲜明,时间约远的状态颜色越暗淡。Since the training historical trajectories in the scene map come from different times, the pixel color is also used to distinguish the historical time. It can be understood that the closer the time to the current state, the brighter the color, and the darker the color of the state that is about farther away.

S503:构建预设机器学习模型,将预设机器学习模型确定为当前机器学习模型;S503: Build a preset machine learning model, and determine the preset machine learning model as the current machine learning model;

S505:基于当前机器学习模型,对训练场景地图进行轨迹预测操作,确定训练场景地图对应的预测未来轨迹;S505: Based on the current machine learning model, perform a trajectory prediction operation on the training scene map, and determine the predicted future trajectory corresponding to the training scene map;

S507:基于训练场景地图对应的预测未来轨迹和实际未来轨迹,确定损失值;S507: Determine the loss value based on the predicted future trajectory and the actual future trajectory corresponding to the training scene map;

S509:判断损失值是否大于预设阈值,若是,则转至步骤S511;否则,转至步骤S513;S509: determine whether the loss value is greater than the preset threshold, if so, go to step S511; otherwise, go to step S513;

S511:基于损失值进行反向传播,对当前机器学习模型进行更新以得到更新后的机器学习模型,将更新后的机器学习模型重新确定为当前机器学习模型;随后,转至步骤S505;S511: Back-propagating based on the loss value, updating the current machine learning model to obtain the updated machine learning model, and re-determining the updated machine learning model as the current machine learning model; then, go to step S505;

S513:将当前机器学习模型确定为轨迹预测模型。S513: Determine the current machine learning model as the trajectory prediction model.

其中,本申请实施例中的样本数据集可以存储在某个存储区域,该存储区域可以是一个区块链。其中,区块链是分布式数据存储、点对点传输、共识机制、加密算法等计算机技术的新型应用模式。区块链(Blockchain),本质上是一个去中心化的数据库,是一串使用密码学方法相关联产生的数据块,每一个数据块中包含了一批次网络交易的信息,用于验证其信息的有效性(防伪)和生成下一个区块。区块链可以包括区块链底层平台、平台产品服务层以及应用服务层。The sample data set in the embodiment of the present application may be stored in a certain storage area, and the storage area may be a blockchain. Among them, blockchain is a new application mode of computer technology such as distributed data storage, point-to-point transmission, consensus mechanism, and encryption algorithm. Blockchain, essentially a decentralized database, is a series of data blocks associated with cryptographic methods. Each data block contains a batch of network transaction information to verify its Validity of information (anti-counterfeiting) and generation of the next block. The blockchain can include the underlying platform of the blockchain, the platform product service layer, and the application service layer.

区块链底层平台可以包括用户管理、基础服务、智能合约以及运营监控等处理模块。其中,用户管理模块负责所有区块链参与者的身份信息管理,包括维护公私钥生成(账户管理)、密钥管理以及用户真实身份和区块链地址对应关系维护(权限管理)等,并且在授权的情况下,监管和审计某些真实身份的交易情况,提供风险控制的规则配置(风控审计);基础服务模块部署在所有区块链节点设备上,用来验证业务请求的有效性,并对有效请求完成共识后记录到存储上,对于一个新的业务请求,基础服务先对接口适配解析和鉴权处理(接口适配),然后通过共识算法将业务信息加密(共识管理),在加密之后完整一致的传输至共享账本上(网络通信),并进行记录存储;智能合约模块负责合约的注册发行以及合约触发和合约执行,开发人员可以通过某种编程语言定义合约逻辑,发布到区块链上(合约注册),根据合约条款的逻辑,调用密钥或者其它的事件触发执行,完成合约逻辑,同时还提供对合约升级注销的功能;运营监控模块主要负责产品发布过程中的部署、配置的修改、合约设置、云适配以及产品运行中的实时状态的可视化输出,例如:告警、监控网络情况、监控节点设备健康状态等。平台产品服务层提供典型应用的基本能力和实现框架,开发人员可以基于这些基本能力,叠加业务的特性,完成业务逻辑的区块链实现。应用服务层提供基于区块链方案的应用服务给业务参与方进行使用。The underlying platform of the blockchain can include processing modules such as user management, basic services, smart contracts, and operation monitoring. Among them, the user management module is responsible for the identity information management of all blockchain participants, including maintenance of public and private key generation (account management), key management, and maintenance of the corresponding relationship between the user's real identity and blockchain address (authority management), etc. When authorized, supervise and audit the transactions of some real identities, and provide rule configuration for risk control (risk control audit); the basic service module is deployed on all blockchain node devices to verify the validity of business requests, After completing the consensus on valid requests, record them in the storage. For a new business request, the basic service first adapts the interface for analysis and authentication processing (interface adaptation), and then encrypts the business information through the consensus algorithm (consensus management), After encryption, it is completely and consistently transmitted to the shared ledger (network communication), and records are stored; the smart contract module is responsible for the registration and issuance of contracts, as well as contract triggering and contract execution. Developers can define contract logic through a programming language and publish to On the blockchain (contract registration), according to the logic of the contract terms, call the key or other events to trigger execution, complete the contract logic, and also provide the function of contract upgrade and cancellation; the operation monitoring module is mainly responsible for the deployment in the product release process , configuration modification, contract settings, cloud adaptation, and visual output of real-time status in product operation, such as: alarms, monitoring network conditions, monitoring node equipment health status, etc. The platform product service layer provides the basic capabilities and implementation framework of typical applications. Based on these basic capabilities, developers can superimpose business characteristics to complete the blockchain implementation of business logic. The application service layer provides application services based on blockchain solutions for business participants to use.

可选的,轨迹预测模型是一种神经卷积网络模型CNN的一种模型,包括输入层,多个卷积层,多个池化层和全连接层和输出层。Optionally, the trajectory prediction model is a model of a neural convolutional network model CNN, including an input layer, multiple convolutional layers, multiple pooling layers, a fully connected layer, and an output layer.

一种可选的实施方式中,多个卷积层,多个池化层和全连接层串联连接,且多个卷积层和多个池化层间隔设置。In an optional implementation manner, multiple convolution layers, multiple pooling layers and fully connected layers are connected in series, and multiple convolution layers and multiple pooling layers are arranged at intervals.

另一种可选的实施例中,还可以包括多个混合层,多个混合层可以由多条支路并联组成,多个卷积层,多个池化层、多个混合层和全连接层串联连接。In another optional embodiment, it may also include multiple mixing layers, and multiple mixing layers may be composed of multiple branches in parallel, multiple convolution layers, multiple pooling layers, multiple mixing layers, and fully connected layers. Layers are connected in series.

举个例子,假设该联合模型图的结构为输入层,注意力机制层,第一卷积层、第一池化层、第二卷积层、第二池化层……全连接层和输出层,输出层包括Softmax分类模块。For example, suppose the structure of the joint model graph is input layer, attention mechanism layer, first convolutional layer, first pooling layer, second convolutional layer, second pooling layer... fully connected layer and output layer, the output layer includes the Softmax classification module.

一种可选的实施方式中,在车辆得到障碍物的未来轨迹之后,还可以将未来轨迹显示至待预测的场景地图中,并在车辆的显示屏幕上显示待预测的场景地图,以使车辆根据未来轨迹规避障碍物。In an optional embodiment, after the vehicle obtains the future trajectory of the obstacle, the future trajectory can also be displayed in the scene map to be predicted, and the scene map to be predicted can be displayed on the display screen of the vehicle, so that the vehicle can Avoid obstacles based on future trajectories.

上述的实施方案可以应用在各种交通场景下,比如,应用在交通路口中。The above-mentioned embodiments can be applied in various traffic scenarios, for example, in traffic intersections.

本申请实施例还提供了一种障碍物未来轨迹的预测装置,图6是本申请实施例提供的一种障碍物未来轨迹的预测装置的结构示意图,如图6所示,该装置包括:An embodiment of the present application also provides a device for predicting a future trajectory of an obstacle. FIG. 6 is a schematic structural diagram of a device for predicting a future trajectory of an obstacle provided by an embodiment of the present application. As shown in FIG. 6 , the device includes:

第一获取模块601用于获取车辆所在的区域对应的场景地图;其中,场景地图包括区域的交通信息;The first obtaining module 601 is used to obtain a scene map corresponding to the area where the vehicle is located; wherein, the scene map includes the traffic information of the area;

第二获取模块602用于获取车辆一定范围内的障碍物在区域内的历史轨迹;The second acquisition module 602 is used to acquire the historical trajectory of obstacles within a certain range of the vehicle in the area;

投影模块603用于将历史轨迹投影到场景地图的对应位置,得到包含历史轨迹的待预测的场景地图;The projection module 603 is used for projecting the historical track to the corresponding position of the scene map to obtain the scene map to be predicted containing the historical track;

预测模块604用于将待预测的场景地图输入已训练好的轨迹预测模型,得到障碍物的未来轨迹。The prediction module 604 is configured to input the scene map to be predicted into the trained trajectory prediction model to obtain the future trajectory of the obstacle.

在一种可选的实施方式中,该装置还包括:In an optional embodiment, the device further includes:

第二确定模块具体用于通过传感器获取障碍物在区域内多个连续的时间点的历史位置信息;将多个连续的时间点的历史位置信息拼接得到历史轨迹。The second determining module is specifically configured to obtain historical position information of obstacles at multiple consecutive time points in the area through the sensor; and splicing the historical position information of the multiple continuous time points to obtain a historical trajectory.

在一种可选的实施方式中,该装置还包括模型训练模块,用于:In an optional embodiment, the apparatus further includes a model training module for:

获取样本数据集,样本数据集包括多个训练场景地图和每个训练场景地图对应的实际未来轨迹;每个训练场景地图包含训练历史轨迹;Obtain a sample data set, where the sample data set includes multiple training scene maps and the actual future trajectory corresponding to each training scene map; each training scene map includes a training historical trajectory;

构建预设机器学习模型,将预设机器学习模型确定为当前机器学习模型;Build a preset machine learning model, and determine the preset machine learning model as the current machine learning model;

基于当前机器学习模型,对训练场景地图进行轨迹预测操作,确定训练场景地图对应的预测未来轨迹;Based on the current machine learning model, the trajectory prediction operation is performed on the training scene map, and the predicted future trajectory corresponding to the training scene map is determined;

基于训练场景地图对应的预测未来轨迹和实际未来轨迹,确定损失值;Determine the loss value based on the predicted future trajectory and the actual future trajectory corresponding to the training scene map;

当损失值大于预设阈值时,基于损失值进行反向传播,对当前机器学习模型进行更新以得到更新后的机器学习模型,将更新后的机器学习模型重新确定为当前机器学习模型;重复步骤:基于当前机器学习模型,对训练场景地图进行轨迹预测操作,确定训练场景地图对应的预测未来轨迹;When the loss value is greater than the preset threshold, backpropagation is performed based on the loss value, the current machine learning model is updated to obtain the updated machine learning model, and the updated machine learning model is re-determined as the current machine learning model; repeat the steps : Based on the current machine learning model, perform trajectory prediction operation on the training scene map, and determine the predicted future trajectory corresponding to the training scene map;

当损失值小于或等于预设阈值时,将当前机器学习模型确定为轨迹预测模型。When the loss value is less than or equal to the preset threshold, the current machine learning model is determined as the trajectory prediction model.

在一种可选的实施方式中,第一获取模块用于:In an optional implementation manner, the first acquisition module is used for:

确定车辆所在的区域;determine the area in which the vehicle is located;

基于区域对应的高精地图获取车辆预设范围内的交通信息;Obtain the traffic information within the preset range of the vehicle based on the high-precision map corresponding to the area;

将交通信息投影到俯瞰角度下的预设地图中的对应位置,得到场景地图。Project the traffic information to the corresponding position in the preset map under the overlook angle to obtain the scene map.

在一种可选的实施方式中,该装置还包括显示模块,用于:In an optional implementation manner, the device further includes a display module for:

将未来轨迹显示至待预测的场景地图中,并显示待预测的场景地图,以使车辆根据未来轨迹规避障碍物。Display the future trajectory in the scene map to be predicted, and display the scene map to be predicted, so that the vehicle can avoid obstacles according to the future trajectory.

本申请实施例中的装置与方法实施例基于同样地申请构思。The apparatus and method embodiments in the embodiments of the present application are based on the same concept of the application.

本申请实施例所提供的方法实施例可以在计算机终端、服务器或者类似的运算装置中执行。以运行在服务器上为例,图7是本申请实施例提供的一种障碍物未来轨迹的预测方法的服务器的硬件结构框图。如图7所示,该服务器700可因配置或性能不同而产生比较大的差异,可以包括一个或一个以上中央处理器(Central Processing Units,CPU)710(处理器710可以包括但不限于微处理器MCU或可编程逻辑器件FPGA等的处理装置)、用于存储数据的存储器730,一个或一个以上存储应用程序723或数据722的存储介质720(例如一个或一个以上海量存储设备)。其中,存储器730和存储介质720可以是短暂存储或持久存储。存储在存储介质720的程序可以包括一个或一个以上模块,每个模块可以包括对服务器中的一系列指令操作。更进一步地,中央处理器710可以设置为与存储介质720通信,在服务器700上执行存储介质720中的一系列指令操作。服务器700还可以包括一个或一个以上电源760,一个或一个以上有线或无线网络接口750,一个或一个以上输入输出接口740,和/或,一个或一个以上操作系统721,例如Windows ServerTM,Mac OS XTM,UnixTM,LinuxTM,FreeBSDTM等等。The method embodiments provided by the embodiments of the present application may be executed in a computer terminal, a server, or a similar computing device. Taking running on a server as an example, FIG. 7 is a hardware structural block diagram of a server of a method for predicting a future trajectory of an obstacle provided by an embodiment of the present application. As shown in FIG. 7 , the server 700 may vary greatly due to different configurations or performances, and may include one or more central processing units (Central Processing Units, CPUs) 710 (the processors 710 may include but are not limited to microprocessors) MCU or programmable logic device FPGA, etc.), memory 730 for storing data, one or more storage media 720 (eg, one or more mass storage devices) storing application programs 723 or data 722. Among them, the memory 730 and the storage medium 720 may be short-term storage or persistent storage. The program stored in the storage medium 720 may include one or more modules, and each module may include a series of instructions to operate on the server. Furthermore, the central processing unit 710 may be configured to communicate with the storage medium 720 to execute a series of instruction operations in the storage medium 720 on the server 700 . Server 700 may also include one or more power supplies 760, one or more wired or wireless network interfaces 750, one or more input and output interfaces 740, and/or, one or more operating systems 721, such as Windows Server™, Mac OS XTM, UnixTM, LinuxTM, FreeBSDTM and so on.

输入输出接口740可以用于经由一个网络接收或者发送数据。上述的网络具体实例可包括服务器700的通信供应商提供的无线网络。在一个实例中,输入输出接口740包括一个网络适配器(Network Interface Controller,NIC),其可通过基站与其他网络设备相连从而可与互联网进行通讯。在一个实例中,输入输出接口740可以为射频(RadioFrequency,RF)模块,其用于通过无线方式与互联网进行通讯。Input-output interface 740 may be used to receive or transmit data via a network. The specific example of the above-mentioned network may include a wireless network provided by the communication provider of the server 700 . In one example, the I/O interface 740 includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through the base station so as to communicate with the Internet. In one example, the input/output interface 740 may be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet in a wireless manner.

本领域普通技术人员可以理解,图7所示的结构仅为示意,其并不对上述电子装置的结构造成限定。例如,服务器700还可包括比图7中所示更多或者更少的组件,或者具有与图7所示不同的配置。Those of ordinary skill in the art can understand that the structure shown in FIG. 7 is only a schematic diagram, which does not limit the structure of the above-mentioned electronic device. For example, server 700 may also include more or fewer components than shown in FIG. 7 , or have a different configuration than that shown in FIG. 7 .

本申请的实施例还提供了一种存储介质,所述存储介质可设置于服务器之中以保存用于实现方法实施例中一种障碍物未来轨迹的预测方法相关的至少一条指令、至少一段程序、代码集或指令集,该至少一条指令、该至少一段程序、该代码集或指令集由该处理器加载并执行以实现上述障碍物未来轨迹的预测方法。Embodiments of the present application further provide a storage medium, where the storage medium can be set in a server to store at least one instruction and at least one piece of program related to implementing a method for predicting a future trajectory of an obstacle in the method embodiment , a code set or an instruction set, the at least one instruction, the at least one piece of program, the code set or the instruction set is loaded and executed by the processor to implement the above method for predicting the future trajectory of an obstacle.

可选地,在本实施例中,上述存储介质可以位于计算机网络的多个网络服务器中的至少一个网络服务器。可选地,在本实施例中,上述存储介质可以包括但不限于:U盘、只读存储器(ROM,Read-Only Memory)、随机存取存储器(RAM,Random Access Memory)、移动硬盘、磁碟或者光盘等各种可以存储程序代码的介质。Optionally, in this embodiment, the above-mentioned storage medium may be located in at least one network server among multiple network servers of a computer network. Optionally, in this embodiment, the above-mentioned storage medium may include but is not limited to: a U disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a mobile hard disk, a magnetic Various media that can store program codes, such as discs or optical discs.

由上述本申请提供的障碍物未来轨迹的预测方法、设备或存储介质的实施例可见,本申请中通过获取车辆所在的区域对应的场景地图;其中,场景地图包括区域的交通信息,获取车辆一定范围内的障碍物在区域内的历史轨迹,将历史轨迹投影到场景地图的对应位置,得到包含历史轨迹的待预测的场景地图,将待预测的场景地图输入已训练好的轨迹预测模型,得到障碍物的未来轨迹。如此,可以通过引入场景地图以及轨迹预测模型提高障碍物的未来轨迹预测的准确性。It can be seen from the above-mentioned embodiments of the method, device or storage medium for predicting the future trajectory of an obstacle provided in the present application that in the present application, the scene map corresponding to the area where the vehicle is located is obtained; The historical trajectory of the obstacles in the area, project the historical trajectory to the corresponding position of the scene map, obtain the scene map to be predicted including the historical trajectory, input the scene map to be predicted into the trained trajectory prediction model, get The future trajectory of the obstacle. In this way, the accuracy of future trajectory prediction of obstacles can be improved by introducing a scene map and a trajectory prediction model.

需要说明的是:上述本申请实施例先后顺序仅仅为了描述,不代表实施例的优劣。且上述对本说明书特定实施例进行了描述。其它实施例在所附权利要求书的范围内。在一些情况下,在权利要求书中记载的动作或步骤可以按照不同于实施例中的顺序来执行并且仍然可以实现期望的结果。另外,在附图中描绘的过程不一定要求示出的特定顺序或者连续顺序才能实现期望的结果。在某些实施方式中,多任务处理和并行处理也是可以的或者可能是有利的。It should be noted that: the above-mentioned order of the embodiments of the present application is only for description, and does not represent the advantages and disadvantages of the embodiments. And the foregoing describes specific embodiments of the present specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps recited in the claims can be performed in an order different from that in the embodiments and still achieve desirable results. Additionally, the processes depicted in the figures do not necessarily require the particular order shown, or sequential order, to achieve desirable results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

本说明书中的各个实施例均采用递进的方式描述,各个实施例之间相同相似的部分互相参见即可,每个实施例重点说明的都是与其他实施例的不同之处。尤其,对于设备实施例而言,由于其基本相似于方法实施例,所以描述的比较简单,相关之处参见方法实施例的部分说明即可。Each embodiment in this specification is described in a progressive manner, and the same and similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and reference may be made to the partial descriptions of the method embodiments for related parts.

本领域普通技术人员可以理解实现上述实施例的全部或部分步骤可以通过硬件来完成,也可以通过程序来指令相关的硬件完成,所述的程序可以存储于一种计算机可读存储介质中,上述提到的存储介质可以是只读存储器,磁盘或光盘等。Those of ordinary skill in the art can understand that all or part of the steps of implementing the above embodiments can be completed by hardware, or can be completed by instructing relevant hardware through a program, and the program can be stored in a computer-readable storage medium. The storage medium mentioned may be a read-only memory, a magnetic disk or an optical disk, etc.

以上所述仅为本申请的较佳实施例,并不用以限制本申请,凡在本申请的精神和原则之内,所作的任何修改、等同替换、改进等,均应包含在本申请的保护范围之内。The above descriptions are only preferred embodiments of the present application, and are not intended to limit the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application shall be included in the protection of the present application. within the range.

Claims (10)

1.一种障碍物未来轨迹的预测方法,其特征在于,所述方法包括:1. a prediction method of obstacle future trajectory, is characterized in that, described method comprises: 获取车辆所在的区域对应的场景地图;其中,所述场景地图包括所述区域的交通信息;Obtain a scene map corresponding to the area where the vehicle is located; wherein, the scene map includes the traffic information of the area; 获取所述车辆一定范围内的障碍物在所述区域内的历史轨迹;Obtain the historical trajectory of obstacles within a certain range of the vehicle in the area; 将所述历史轨迹投影到所述场景地图的对应位置,得到包含所述历史轨迹的待预测的场景地图;Projecting the historical track to a corresponding position of the scene map to obtain a scene map to be predicted containing the historical track; 将所述待预测的场景地图输入已训练好的轨迹预测模型,得到所述障碍物的未来轨迹。Input the scene map to be predicted into the trained trajectory prediction model to obtain the future trajectory of the obstacle. 2.根据权利要求1所述的方法,其特征在于,所述方法还包括训练得到所述轨迹预测模型的步骤;2. The method according to claim 1, wherein the method further comprises the step of obtaining the trajectory prediction model by training; 所述训练得到所述轨迹预测模型包括:The trajectory prediction model obtained by the training includes: 获取样本数据集,所述样本数据集包括多个训练场景地图和每个所述训练场景地图对应的实际未来轨迹;每个训练场景地图包含训练历史轨迹;Obtaining a sample data set, the sample data set includes a plurality of training scene maps and an actual future trajectory corresponding to each of the training scene maps; each training scene map includes a training historical trajectory; 构建预设机器学习模型,将所述预设机器学习模型确定为当前机器学习模型;constructing a preset machine learning model, and determining the preset machine learning model as the current machine learning model; 基于所述当前机器学习模型,对所述训练场景地图进行轨迹预测操作,确定所述训练场景地图对应的预测未来轨迹;Based on the current machine learning model, a trajectory prediction operation is performed on the training scene map, and a predicted future trajectory corresponding to the training scene map is determined; 基于所述训练场景地图对应的预测未来轨迹和实际未来轨迹,确定损失值;determining a loss value based on the predicted future trajectory and the actual future trajectory corresponding to the training scene map; 当所述损失值大于预设阈值时,基于所述损失值进行反向传播,对所述当前机器学习模型进行更新以得到更新后的机器学习模型,将所述更新后的机器学习模型重新确定为所述当前机器学习模型;重复步骤:基于所述当前机器学习模型,对所述训练场景地图进行轨迹预测操作,确定所述训练场景地图对应的预测未来轨迹;When the loss value is greater than a preset threshold, backpropagation is performed based on the loss value, the current machine learning model is updated to obtain an updated machine learning model, and the updated machine learning model is re-determined is the current machine learning model; repeating the steps: based on the current machine learning model, perform a trajectory prediction operation on the training scene map, and determine the predicted future trajectory corresponding to the training scene map; 当所述损失值小于或等于所述预设阈值时,将所述当前机器学习模型确定为所述轨迹预测模型。When the loss value is less than or equal to the preset threshold, the current machine learning model is determined as the trajectory prediction model. 3.根据权利要求1所述的方法,其特征在于,所述交通信息包括:车道信息、车道中心线信息、人行横道信息、交通灯信息和停止线信息。3. The method according to claim 1, wherein the traffic information comprises: lane information, lane centerline information, pedestrian crossing information, traffic light information and stop line information. 4.根据权利要求1所述的方法,其特征在于,所述获取所述车辆一定范围内的障碍物在所述区域内的历史轨迹,包括:4 . The method according to claim 1 , wherein the acquiring the historical trajectory of obstacles within a certain range of the vehicle in the area comprises: 5 . 通过传感器获取所述障碍物在所述区域内多个连续的时间点的历史位置信息;Acquiring historical position information of the obstacle at multiple consecutive time points in the area through a sensor; 将所述多个连续的时间点的历史位置信息拼接得到所述历史轨迹;The historical track is obtained by splicing the historical position information of the multiple consecutive time points; 其中,所述传感器包括摄像头、激光雷达和毫米波雷达中的一项或者多项。Wherein, the sensor includes one or more of a camera, a lidar, and a millimeter-wave radar. 5.根据权利要求1所述的方法,其特征在于,所述获取车辆所在的区域对应的场景地图,包括:5. The method according to claim 1, wherein the acquiring the scene map corresponding to the area where the vehicle is located comprises: 确定所述车辆所在的区域;determine the area in which the vehicle is located; 基于所述区域对应的高精地图获取所述车辆预设范围内的交通信息;Obtain traffic information within the preset range of the vehicle based on the high-precision map corresponding to the area; 将所述交通信息投影到俯瞰角度下的预设地图中的对应位置,得到所述场景地图。Projecting the traffic information to a corresponding position in a preset map from an overlook angle to obtain the scene map. 6.根据权利要求1所述的方法,其特征在于,所述得到所述障碍物的未来轨迹之后,还包括;6. The method according to claim 1, wherein after obtaining the future trajectory of the obstacle, further comprising: 将所述未来轨迹显示至所述待预测的场景地图中,并显示所述待预测的场景地图,以使所述车辆根据所述未来轨迹规避所述障碍物。The future trajectory is displayed in the scene map to be predicted, and the scene map to be predicted is displayed, so that the vehicle avoids the obstacle according to the future trajectory. 7.一种障碍物未来轨迹的预测装置,其特征在于,所述装置包括:7. A device for predicting a future trajectory of an obstacle, wherein the device comprises: 第一获取模块,用于获取车辆所在的区域对应的场景地图;其中,所述场景地图包括所述区域的交通信息;a first obtaining module, configured to obtain a scene map corresponding to the area where the vehicle is located; wherein, the scene map includes traffic information of the area; 第二获取模块,用于获取所述车辆一定范围内的障碍物在所述区域内的历史轨迹;a second acquisition module, configured to acquire the historical trajectory of obstacles within a certain range of the vehicle in the area; 投影模块,用于将所述历史轨迹投影到所述场景地图的对应位置,得到包含所述历史轨迹的待预测的场景地图;a projection module, configured to project the historical track to a corresponding position of the scene map to obtain a scene map to be predicted that includes the historical track; 预测模块,用于将所述待预测的场景地图输入已训练好的轨迹预测模型,得到所述障碍物的未来轨迹。The prediction module is used for inputting the scene map to be predicted into the trained trajectory prediction model to obtain the future trajectory of the obstacle. 8.根据权利要求7所述的装置,其特征在于,所述第二获取模块,具体用于:8. The device according to claim 7, wherein the second acquisition module is specifically used for: 通过传感器获取所述障碍物在所述区域内多个连续的时间点的历史位置信息;Acquiring historical position information of the obstacle at multiple consecutive time points in the area through a sensor; 将所述多个连续的时间点的历史位置信息拼接得到所述历史轨迹。The historical track is obtained by splicing the historical location information of the multiple consecutive time points. 9.一种电子设备,其特征在于,所述电子设备包括处理器和存储器,所述存储器中存储有至少一条指令或至少一段程序,所述至少一条指令或所述至少一段程序由所述处理器加载并执行如权利要求1-6任一所述的障碍物未来轨迹的预测方法。9. An electronic device, characterized in that the electronic device comprises a processor and a memory, and the memory stores at least one instruction or at least one program, and the at least one instruction or the at least one program is processed by the The controller loads and executes the method for predicting the future trajectory of an obstacle according to any one of claims 1-6. 10.一种计算机存储介质,其特征在于,所述计算机存储介质中存储有至少一条指令或至少一段程序,所述至少一条指令或至少一段程序由处理器加载并执行以实现如权利要求1-6任一所述的障碍物未来轨迹的预测方法。10. A computer storage medium, characterized in that, at least one instruction or at least one program is stored in the computer storage medium, and the at least one instruction or at least one program is loaded and executed by a processor to realize the method as claimed in claim 1- 6. Any one of the methods for predicting the future trajectory of an obstacle.
CN202010274946.2A 2020-04-09 2020-04-09 A method, device, electronic device and storage medium for predicting the future trajectory of an obstacle Pending CN111626097A (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010274946.2A CN111626097A (en) 2020-04-09 2020-04-09 A method, device, electronic device and storage medium for predicting the future trajectory of an obstacle

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010274946.2A CN111626097A (en) 2020-04-09 2020-04-09 A method, device, electronic device and storage medium for predicting the future trajectory of an obstacle

Publications (1)

Publication Number Publication Date
CN111626097A true CN111626097A (en) 2020-09-04

Family

ID=72258828

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010274946.2A Pending CN111626097A (en) 2020-04-09 2020-04-09 A method, device, electronic device and storage medium for predicting the future trajectory of an obstacle

Country Status (1)

Country Link
CN (1) CN111626097A (en)

Cited By (15)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112092809A (en) * 2020-09-15 2020-12-18 北京罗克维尔斯科技有限公司 A kind of auxiliary reversing method, device, system and vehicle
CN112651990A (en) * 2020-12-25 2021-04-13 际络科技(上海)有限公司 Motion trajectory prediction method and system, electronic device and readable storage medium
CN113033527A (en) * 2021-05-27 2021-06-25 北京三快在线科技有限公司 Scene recognition method and device, storage medium and unmanned equipment
CN113283647A (en) * 2021-05-19 2021-08-20 广州文远知行科技有限公司 Method and device for predicting obstacle track and automatic driving vehicle
CN113325855A (en) * 2021-08-02 2021-08-31 北京三快在线科技有限公司 Model training method for predicting obstacle trajectory based on migration scene
CN113641734A (en) * 2021-08-12 2021-11-12 驭势科技(北京)有限公司 Data processing method, device, equipment and medium
CN114018265A (en) * 2021-10-28 2022-02-08 山东新一代信息产业技术研究院有限公司 Patrol robot driving track generation method, equipment and medium
CN114647934A (en) * 2022-03-11 2022-06-21 中海监理有限公司 Light pollution supervision monitoring method and device, electronic equipment and storage medium
CN115148025A (en) * 2022-06-28 2022-10-04 重庆长安汽车股份有限公司 Traffic target track prediction method and readable storage medium
CN115641362A (en) * 2022-11-09 2023-01-24 上海仙途智能科技有限公司 Trajectory prediction method, apparatus and storage medium
CN115781693A (en) * 2023-01-09 2023-03-14 广东隆崎机器人有限公司 Obstacle avoidance method and device for industrial robot, industrial robot and storage medium
CN115917559A (en) * 2021-04-22 2023-04-04 深圳元戎启行科技有限公司 Trajectory prediction method, device, computer equipment and storage medium
CN115938012A (en) * 2022-12-21 2023-04-07 中国电信股份有限公司 Vehicle track studying and judging model generation method and device and nonvolatile storage medium
CN116009387A (en) * 2021-09-24 2023-04-25 中强光电股份有限公司 Track tracking control method and control system
CN116503700A (en) * 2023-03-17 2023-07-28 深圳元戎启行科技有限公司 Trajectory prediction method, device, storage medium and vehicle

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107221195A (en) * 2017-05-26 2017-09-29 重庆长安汽车股份有限公司 Automobile track Forecasting Methodology and track level map
CN109213134A (en) * 2017-07-03 2019-01-15 百度在线网络技术(北京)有限公司 The method and apparatus for generating automatic Pilot strategy
CN109927719A (en) * 2017-12-15 2019-06-25 百度在线网络技术(北京)有限公司 A kind of auxiliary driving method and system based on barrier trajectory predictions
CN110293968A (en) * 2019-06-18 2019-10-01 百度在线网络技术(北京)有限公司 Control method, device, equipment and the readable storage medium storing program for executing of automatic driving vehicle
CN110371112A (en) * 2019-07-06 2019-10-25 深圳数翔科技有限公司 A kind of intelligent barrier avoiding system and method for automatic driving vehicle
CN110936963A (en) * 2018-09-21 2020-03-31 威斯通全球技术公司 Driver assistance system and method for displaying traffic information
US20210163037A1 (en) * 2018-04-18 2021-06-03 Baidu Usa Llc Map-less and localization-less lane following method for autonomous driving of autonomous driving vehicles on highway

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN107221195A (en) * 2017-05-26 2017-09-29 重庆长安汽车股份有限公司 Automobile track Forecasting Methodology and track level map
CN109213134A (en) * 2017-07-03 2019-01-15 百度在线网络技术(北京)有限公司 The method and apparatus for generating automatic Pilot strategy
CN109927719A (en) * 2017-12-15 2019-06-25 百度在线网络技术(北京)有限公司 A kind of auxiliary driving method and system based on barrier trajectory predictions
US20210163037A1 (en) * 2018-04-18 2021-06-03 Baidu Usa Llc Map-less and localization-less lane following method for autonomous driving of autonomous driving vehicles on highway
CN110936963A (en) * 2018-09-21 2020-03-31 威斯通全球技术公司 Driver assistance system and method for displaying traffic information
CN110293968A (en) * 2019-06-18 2019-10-01 百度在线网络技术(北京)有限公司 Control method, device, equipment and the readable storage medium storing program for executing of automatic driving vehicle
CN110371112A (en) * 2019-07-06 2019-10-25 深圳数翔科技有限公司 A kind of intelligent barrier avoiding system and method for automatic driving vehicle

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
刘飞: ""基于VANET的车辆碰撞预警系统研究与仿真"", 《中国优秀硕士学位论文全文数据库 信息科技辑》, no. 05, 15 May 2016 (2016-05-15), pages 1 - 10 *

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN112092809A (en) * 2020-09-15 2020-12-18 北京罗克维尔斯科技有限公司 A kind of auxiliary reversing method, device, system and vehicle
CN112651990B (en) * 2020-12-25 2022-12-16 际络科技(上海)有限公司 Motion trajectory prediction method and system, electronic device and readable storage medium
CN112651990A (en) * 2020-12-25 2021-04-13 际络科技(上海)有限公司 Motion trajectory prediction method and system, electronic device and readable storage medium
CN115917559A (en) * 2021-04-22 2023-04-04 深圳元戎启行科技有限公司 Trajectory prediction method, device, computer equipment and storage medium
CN113283647A (en) * 2021-05-19 2021-08-20 广州文远知行科技有限公司 Method and device for predicting obstacle track and automatic driving vehicle
CN113033527A (en) * 2021-05-27 2021-06-25 北京三快在线科技有限公司 Scene recognition method and device, storage medium and unmanned equipment
CN113325855A (en) * 2021-08-02 2021-08-31 北京三快在线科技有限公司 Model training method for predicting obstacle trajectory based on migration scene
CN113325855B (en) * 2021-08-02 2021-11-30 北京三快在线科技有限公司 Model training method for predicting obstacle trajectory based on migration scene
WO2023010827A1 (en) * 2021-08-02 2023-02-09 北京三快在线科技有限公司 Migration scene-based trajectory prediction model training method and apparatus
CN113641734A (en) * 2021-08-12 2021-11-12 驭势科技(北京)有限公司 Data processing method, device, equipment and medium
CN113641734B (en) * 2021-08-12 2024-04-05 驭势科技(北京)有限公司 Data processing method, device, equipment and medium
CN116009387A (en) * 2021-09-24 2023-04-25 中强光电股份有限公司 Track tracking control method and control system
CN114018265A (en) * 2021-10-28 2022-02-08 山东新一代信息产业技术研究院有限公司 Patrol robot driving track generation method, equipment and medium
CN114018265B (en) * 2021-10-28 2024-02-02 山东新一代信息产业技术研究院有限公司 Method, equipment and medium for generating running track of inspection robot
CN114647934A (en) * 2022-03-11 2022-06-21 中海监理有限公司 Light pollution supervision monitoring method and device, electronic equipment and storage medium
CN115148025A (en) * 2022-06-28 2022-10-04 重庆长安汽车股份有限公司 Traffic target track prediction method and readable storage medium
CN115148025B (en) * 2022-06-28 2023-10-20 重庆长安汽车股份有限公司 Traffic target track prediction method and readable storage medium
CN115641362A (en) * 2022-11-09 2023-01-24 上海仙途智能科技有限公司 Trajectory prediction method, apparatus and storage medium
CN115938012A (en) * 2022-12-21 2023-04-07 中国电信股份有限公司 Vehicle track studying and judging model generation method and device and nonvolatile storage medium
CN115781693A (en) * 2023-01-09 2023-03-14 广东隆崎机器人有限公司 Obstacle avoidance method and device for industrial robot, industrial robot and storage medium
CN115781693B (en) * 2023-01-09 2023-05-26 广东隆崎机器人有限公司 Obstacle avoidance method and device for industrial robot, industrial robot and storage medium
CN116503700A (en) * 2023-03-17 2023-07-28 深圳元戎启行科技有限公司 Trajectory prediction method, device, storage medium and vehicle

Similar Documents

Publication Publication Date Title
CN111626097A (en) A method, device, electronic device and storage medium for predicting the future trajectory of an obstacle
US20250178640A1 (en) Safe and scalable model for culturally sensitive driving by automated vehicles
CN112163446B (en) Obstacle detection method and device, electronic equipment and storage medium
CN108921200B (en) Method, apparatus, apparatus and medium for classifying driving scene data
JP2022547611A (en) Simulation of various long-term future trajectories in road scenes
CN116881707A (en) Automatic driving model, training method, training device and vehicle
CN110955242A (en) Robot navigation method, system, robot and storage medium
WO2020034903A1 (en) Smart navigation method and system based on topological map
TW202020748A (en) Recursive multi-fidelity behavior prediction
CN117519206B (en) Autonomous driving model, method, device and vehicle based on generative diffusion model
CN116560377B (en) Automatic driving model for predicting position track and training method thereof
WO2025020985A1 (en) Road topology detection method, and related apparatus
US20230406361A1 (en) Structured multi-agent interactive trajectory forecasting
CN110955241A (en) Obstacle avoidance method and device for mobile robot, mobile robot and storage medium
WO2024001806A1 (en) Data valuation method based on federated learning and related device therefor
US20240300542A1 (en) Trajectory prediction by sampling sequences of discrete motion tokens
CN115837919A (en) Interactive behavior decision method and device for automatic driving vehicle and automatic driving vehicle
CN116597402A (en) A kind of scene awareness method and related equipment
CN114104005B (en) Decision-making method, device, device and readable storage medium for automatic driving equipment
CN114454899B (en) Vehicle driving method and device
CN114596552B (en) Information processing method, training method, device, equipment, vehicle and medium
Wang et al. Federated learning for vehicle trajectory prediction: Methodology and benchmark study
CN114049764B (en) Traffic simulation method and system based on convolution long-time and short-time memory neural network
CN118140159A (en) Method for automatically determining sensor placement in target environment
RU2742394C1 (en) Method of creating an intelligent system for determining areas of flight routes for an unmanned aerial vehicle in simulation complexes

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
RJ01 Rejection of invention patent application after publication
RJ01 Rejection of invention patent application after publication

Application publication date: 20200904