DE112020006532T5 - COMPUTER SYSTEM AND METHOD WITH END-TO-END MODELING FOR A SIMULATED TRAFFIC AGENT IN A SIMULATION ENVIRONMENT - Google Patents

Abstract

The present invention relates to a computer-implemented training method for a traffic agent navigating a road vehicle in a simulation environment using end-to-end modeling, as well as a corresponding training computer system and a computer system for simulating a road driving environment for one or more vehicles, the one or more Includes or consists of processors and uses the traffic agents trained according to the invention.

Description

TECHNICAL PART:

The present invention relates to a computer-implemented training method for a traffic agent navigating a road vehicle in a simulation environment using end-to-end modeling, as well as a corresponding training computer system and a computer system for simulating a road driving environment for one or more vehicles, the one or more Includes or consists of processors and uses the traffic agents trained according to the invention.

STATE OF THE ART:

Before the driving characteristics of road vehicles are tested in reality, computer simulations of certain driving situations, e.g. B. when braking performed. Since the forecast period is usually only up to 2 seconds, complex driving situations, such as those B. are required when overtaking, are not predicted by these models.

The problem of developing a system that can safely control a car in a variety of traffic situations has been widely studied and is of obvious interest for autonomous vehicle development. The main focus in this research area is on safe and efficient decisions under real-time conditions. However, the simulated safe and efficient decisions may not reflect human driving decisions in natural traffic.

Human driving decisions on the road can essentially consist of several abstract levels or phases that form a driving stack. Based on a certain road situation, a driver can decide to carry out a certain overriding maneuver, e.g. B. to overtake, formulate a corresponding movement plan (also called “trajectory”) and apply control functions to actuators (gas, brake, steering) to carry out the decision.

It is therefore becoming increasingly important to simulate human driving decisions in natural traffic. Human driving decisions in natural traffic are also influenced by many factors and can be viewed at different levels. So human drivers can make different decisions depending on their mental state in the same situation, e.g. B. overtaking, following a vehicle ahead or changing lanes.

Many existing models use a hierarchical structure in the sense that more abstract decisions (e.g. which route to take) are first calculated and then “passed” to different layers that, based on this input, evolve with increasing levels of detail of the driving process deal with The driving stack is divided into several phases, which are intended to reflect the actually relevant components of the different approaches, e.g. B. in connection with simulation environments and not with the driving behavior of an autonomous vehicle.

Such phases can be considered as follows:

Perception/map generally refers to input about the environment that is available to other components.

Traffic rules generally refer to any component that provides legal restrictions on high-level decisions.

Mission planning generally refers to a strategy of when to be where over the long term (e.g., lane-level routing).

Traffic-free reference route generally refers to planning an “optimal” reference route that ignores other road users.

Behavioral planning generally refers to the planning of a behavioral plan, i. H. when exactly actions, e.g. B. lane change, are to be carried out with the involvement of other participants.

In general, decision post-processing is about correcting the decisions of the preceding components so that they comply with the basic safety rules, if necessary.

Motion/trajectory planning generally refers to planning the exact future trajectory for a short time horizon (up to 2 seconds).

Command translation generally refers to the calculation of the final commands to be sent to a (real or simulated) vehicle, such as B. Steering instructions.

Vehicle Dynamics/Physics generally refers to the simulation of vehicle behavior resulting from the generated commands.

Position update generally refers to the calculation of the resulting new position of the vehicle in the simulation.

The use of these terms varies drastically in the literature.

It can be argued that these hierarchical models have certain limitations, such as B. the fact that they are not able to make high-level decisions that are modified or even rejected by "lower" components such as a motion planner (component that decides e.g. the time window of accelerations and lane changes). (see Junqing Wei, Jarrod M. Snider, Tianyu Gu, John Dolan and Bakhtiar Litkouhi. A behavioral planning framework for autonomous driving. Pages 458-464, 06 2014). Accordingly, the hierarchical models offer only limited realism when reproducing human driving behavior.

The power of end-to-end learning (synonym "e2e" or "E2E", [end-to-end]) with neural networks has proven itself many times over in various areas. In the autonomous driving industry, the e2e approach is popular to construct robust models for various driving controls, e.g. B. for steering, pedal control, etc., in such a way that sensory inputs (e.g. image pixels) are mapped directly to control outputs. This direct mapping eliminates the need to use extensive training data with lane markings, road boundaries, etc., and allows for the extraction of salient features based on a targeted learning approach.

Bojarsky has shown that the decision-making processes of a human driver when staying in lane can be modeled in a deep neural network (see M. Bojarski, D. Del Testa, D. Dworakowski, B. Firner, B. Flepp, P. Goyal, L. D. Jackel, M Monfort, U Muller, J Zhang et al, "End to end learning for self-driving cars", arXiv preprint arXiv:1604.07316, 2016). The authors attempt to map raw images from driving footage to car steering commands, thereby implicitly embedding the layers of the driving stack in the layers of a neural network, similar to how a human mind does. The model is taught to learn lane-keeping behavior from human drivers, but lane-changing was not modeled. The method uses only steering commands, but no information about the longitudinal movement of the vehicle (i.e. acceleration/deceleration). The model is implemented in an autonomously driving car and not as a simulated traffic agent in a simulation environment. Muller presents the same approach that was used to train on data from a remote-controlled car to automate its driving behavior (see U. Muller, J. Ben, E. Cosatto, B. Flepp, and Y. L. Cun, "Off- road obstacle avoidance through end-to-end learning,” in Advances in neural information processing systems, 2006, pp. 739-746).

Xu and Gao use end-to-end deep learning to extract raw images from numerous shots of human drivers on the road for both high-level actions such as "get on" and "go" as well as on to map steering angle commands (see H. Xu, Y. Gao, F. Yu and T. Darrell, "End-to-end learning of driving models from large-scale video datasets", in Proceedings of the IEEE conference on computer vision and pattern recognition , 2017, pp. 2174-2182). The intended behavior of the model can approximate lane following, obstacle avoidance and lane changing behavior of human drivers. The work only provides a distribution of vehicle controls, e.g. B. the steering, and therefore does not claim to be highly precise to control a vehicle in a simulation or a real driving scenario. The method does not model acceleration/deceleration commands, only high-level stop and start decisions. The model is not implemented in a simulated traffic agent in a simulation environment.

Codevilla uses the same approach to learn the image for executing control commands in the longitudinal and lateral directions (steering and acceleration) of a car using the CARLA driving simulation data (see F. Codevilla, M. Miiller, A. Lopez, V. Koltun and A. Dosovitskiy, "End-to-End driving via conditional imitation learning", in 2018 IEEE International Conference on Robotics and Automation (ICRA. 1em plus 0.5em minus 0.4em IEEE, 2018, pp. 1-9). The model is taught to learn nearly all aspects of driving behavior, ie lane following, adaptive cruise control, obstacle avoidance and lane changing. The model was evaluated in a simulation environment as a traffic agent. Chen presents a similar solution in the TORCS race car simulation environment and also claims that the end-to-end model explicitly learns to focus on interpretable perceptual elements, such as B. the distance to the lane and the road boundary, the distance to other vehicles in the area and the angular deviation from the road as part of a more interpretable solution for modeling driving behavior (see C. Chen, A. Seff, A. Kornhauser and J. Xiao, "Deep driving: Learning affordance for direct perception in autonomous driving," in Proceedings of the IEEE International Conference on Computer Vision, 2015, pp. 2722-2730). However, both solutions are trained using simulation data from a computer-controlled driver and are therefore unable to actually show human-like behavior.

In particular, the relevant prior art mentioned above either restricts the control of the vehicle to lateral steering commands only or targets a specific function related to human driving, e.g. B. following the lane, changing lanes, etc. However, this limitation does not allow to show a truly human-like driving behavior in a simulated traffic environment.

Furthermore, the previous solutions rely on implicit learning of perceptual elements such as lane boundary positions, traffic vehicle positions, etc. from the visual input ([Input], image), resulting in less accurate information about the vehicle environment.

In view of the shortcomings of the prior art, it is the object of the present invention to provide a computer system and a method for simulating a road driving environment in a driving situation for one or more vehicles, such that the decision of a traffic agent reflects human-like (naturalistic) behavior, i.e. the longitudinal and lateral position of the vehicle, preferably steering and acceleration, in a way that exhibits naturalistic driving behavior in general and in particular in high-level decisions such as lane-changing behavior, e.g. in an overtaking manoeuvre.

BRIEF DESCRIPTION OF THE INVENTION:

The aforementioned object is at least partially achieved by the claimed subject matter of the invention. Advantages (preferred embodiments) are presented in the following detailed description and/or the accompanying figures as well as in the dependent claims.

1. a. Bereitstellung von Fahrdaten zu einem oder mehreren Zeitfenstern t_i = [t₁, t₂, ... t_n] für ein oder mehrere Straßenfahrzeuge als Ego-Fahrzeuge, die jeweils von einem Menschen in einer realistischen Situation auf einer Straße gefahren werden, und Bereitstellung von Kartendaten auf der jeweiligen Straße zu den gegebenen Zeitfenstern t_i,
2. b. Verarbeitung zumindest eines Teils der Fahrdaten und der Kartendaten aus Schritt a) in einen oder mehrere entsprechende Wahrnehmungsrahmen P_i = [p₁, p₂, ... p_n] je gegebenem Zeitfenster t_i, wobei jeder Wahrnehmungsrahmen P_i entsprechende Wahrnehmungsinformationen für (i) die Verkehrssituation, (ii) Informationen über den Eigenzustand des Ego-Fahrzeugs und (iii) die Straßengeometrie enthält,
3. c. Verarbeitung zumindest eines Teils der Fahrdaten und der Kartendaten von Schritt a) in einen oder mehrere entsprechende Grundwahrheits-Fahrzeugsteuerungsrahmen C_i = [c₁, c₂, ... c_n] je gegebenem Zeitfenster t_i, wobei jeder Fahrzeugsteuerungsrahmen C_i longitudinale und laterale Positionen der jeweiligen Ego-Fahrzeuge enthält,
4. d. Trainieren eines Entscheider-Computermodells des Verkehrsagenten mit dem einen oder den mehreren Wahrnehmungsrahmen P_i je gegebenen Zeitfenster t_i von Schritt b) als Eingabe in das Modell und mit dem einen oder den mehreren Grundwahrheits-Fahrzeugsteuerungsrahmen C_i je gegebenem Zeitfenster t_i aus Schritt c) als Etikett für das Training des Modells, wobei der Entscheider ein oder mehrere neuronale Netze mit Ende-zu-Ende Modellierung verwendet und konfiguriert ist entsprechende Fahrzeugsteuerungsrahmen Ĉ_i = [c₁, c₂, ... c_n] umfassend die longitudinale und laterale Positionen des jeweiligen Ego-Fahrzeugs vorherzusagen, indem er die vorhergesagten Fahrzeugkontrollrahmen Ĉ_i mit den jeweiligen Grundwahrheits-Fahrzeugsteuerungsrahmen C_i abgleicht,

wobei i eine beliebige Zahl ist, so dass i ∈ [1,2,... n] ist und wobei n den Grenzwert der gefahrenen Rahmen darstellt.Accordingly, a first aspect of the invention relates to a computer-implemented training method of a traffic agent for navigating a road vehicle in a simulation environment. The procedure includes or consists of the following steps:

1. a. Provision of driving data for one or more time windows t _i = [t ₁ , t ₂ , ... t _n ] for one or more road vehicles as ego vehicles, each of which is driven by a human being in a realistic situation on a road, and Provision of map data on the respective street at the given time windows t _i ,
2. b. processing at least part of the driving data and the map data from step a) into one or more corresponding perceptual frames P _i = [p ₁ , p ₂ , ... p _n ] per given time window t _i , each perceptual frame P _i corresponding perceptual information for ( i) the traffic situation, (ii) information about the state of the ego vehicle and (iii) the road geometry,
3. c. Processing at least part of the driving data and the map data of step a) into one or more corresponding ground truth vehicle control frames C _i = [c ₁ , c ₂ , ... c _n ] per given time window t _i , each vehicle control frame C _i longitudinal and contains lateral positions of the respective ego vehicles,
4. i.e. training a decision maker computer model of the traffic agent with the one or more perceptual frames P _i per given time window t _i from step b) as input to the model and with the one or more ground truth vehicle control frames C _i per given time window t _i from step c ) as a label for training the model, where the decider uses one or more neural networks with end-to-end modeling and is configured corresponding vehicle control frames Ĉ _i = [c ₁ , c ₂ , ... c _n ] comprising the longitudinal and predict lateral positions of the respective ego vehicle by matching the predicted vehicle control frames Ĉ _i with the respective ground truth vehicle control frames C _i ,

where i is any number such that i ∈ [1,2,...n] and where n is the limit of frames traveled.

Process steps b) and c) of the process according to the invention according to the first aspect can be carried out simultaneously or one after the other in any order.

A second aspect of the invention relates to a computer system for training a traffic agent who navigates a road vehicle in a simulation environment, comprising or consisting of one or more processors, a memory device coupled to the one or more processors and a traffic agent as a decision maker in simulated driving situations using one or more neural networks with end-to-end modeling stored in the storage device and configured to be executed by the one or more processors, characterized in that the traffic agent is configured to computer-implemented training method according to the first inventive aspect.

A third aspect of the invention relates to a computer system for simulating a road driving environment in driving situations for one or more vehicles, comprising or consisting of one or more processors, a memory device coupled to the one or more processors, and a traffic agent having one or more uses neural networks as decision makers in simulated driving situations, using one or more neural networks with end-to-end modelling, stored in the memory device and configured to be executed by the one or more processors, characterized in that that the traffic agent has been trained, according to the computer-implemented training method according to the first aspect of the invention, to predict as an action one or more vehicle control frames Ĉ _i that contain longitudinal and lateral positions that are related to a simulated vehicle in the simulation environment g are to be applied, where i is an arbitrary number such that i ∈ [1,2,...n] and where n represents the limit of the frames driven.

The inventive aspects of the present invention as disclosed herein may comprise any (sub)combination of the preferred inventive embodiments as set out in the dependent claims or as illustrated in the following detailed description and/or the accompanying figures are disclosed, provided the resulting combination of features makes sense to those skilled in the art.

character list

• Die 1a) bis 1c) zeigen schematische Darstellungen von (Teilen) des E2E-Fahrzeugsteuerungsmodells der erfindungsgemäßen Computersysteme im Training (1a) und 1b)) bzw. im Einsatz (1c)).
• 2 zeigt eine schematische Darstellung einer Sechs-Fahrzeug-Nachbarschaftsinformation.
• 3 zeigt eine schematische Darstellung einer halbkreisförmigen Straßengeometrie und die entsprechenden Verschiebungsvektoren.
• 4 zeigt ein Verteilungsdiagramm des Fehlers/Bildes im △ Peilung gegen die DFS-Grundwahrheitsvalidierungsdaten in einem erfindungsgemäßen Fahrspurfolgemodul.
• 5 zeigt ein Verteilungsdiagramm des Fehlers/Bildes in der △ Beschleunigung gegen die DFS-Grundwahrheitsvalidierungsdaten in einem erfindungsgemäßen Fahrspurfolgemodul.
• 6 zeigt ein Verteilungsdiagramm der Spurmittenabweichung in den realen DFS-Verkehrsdaten im Modul „Fahrspur folgen“.
• 7 zeigt ein Verteilungsdiagramm der Spurmittenabweichung des Moduls Fahrspur folgen, wenn es in der Simulation ausgeführt wird.
• 8a) und 8b) zeigen die Verteilungsdiagramme der Relativgeschwindigkeit gegenüber dem relativen Abstand zum Vorderwagen für den Modellversuch in der Simulation (8a)) und die DFS-Daten am Grund (8b)).

Other features and advantages of the present invention will become apparent from the accompanying drawings, wherein

• the 1a) until 1c ) show schematic representations of (parts of) the E2E vehicle control model of the computer systems according to the invention in training ( 1a) and 1b) ) or in action ( 1c )).
• 2 12 shows a schematic representation of six-vehicle neighborhood information.
• 3 shows a schematic representation of a semi-circular road geometry and the corresponding displacement vectors.
• 4 12 shows a distribution diagram of the error/image in △ bearing versus the DFS ground truth validation data in a lane following module according to the invention.
• 5 FIG. 12 shows a distribution diagram of the error/image in the △ acceleration versus the DFS ground truth validation data in a lane following module according to the invention.
• 6 shows a distribution diagram of the lane center deviation in the real DFS traffic data in the "Follow Lane" module.
• 7 shows a distribution plot of the lane misalignment of the Lane Follow module when run in the simulation.
• 8a) and 8b) show the distribution diagrams of the relative speed versus the relative distance to the vehicle in front for the model test in the simulation ( 8a) ) and the DFS data at the bottom ( 8b) ).

DETAILED DESCRIPTION OF THE INVENTION:

As discussed in more detail below, the inventors of various aspects of the present invention have found that the computer-implemented systems and methods of the present invention enable a traffic agent navigating a road vehicle in a simulation environment to make high-level (e.g., lane changing, overtaking) and low/operational level (trajectory and motion planning) simulated driving decisions that reflect human-like (naturalistic) behavior, i.e. to control the vehicle's longitudinal and lateral position, preferably bearing and acceleration, in a way which shows a naturalistic driving behavior in every driving situation.

Thus, the present invention successfully shows the lifelike decision-making behavior from the source data in the simulation environment with regard to planning, safety processes and compliance with traffic rules.

According to the present invention, the respective naturalistic driving and map data are processed into one or more perception frames per time window, which contain corresponding perception information for (i) the traffic situation, (ii) information about the ego vehicle's own state and (iii) the road geometry. Furthermore, according to the present invention, the respective naturalistic driving and map data are processed to form one or more respective vehicle control frames per time slot, each vehicle control frame containing the longitudinal and lateral position of the respective ego vehicle. The use of three categories of perceptual framework is fundamental to enable effective generalization of the computer model of the present invention.

According to the present invention, the decision maker computer model of the simulated traffic agent is trained with the respective one or more perceptual frames as input to the model and with the one or more ground truth vehicle control frames as a label for training the model, the decision maker having one or more uses neural networks with end-to-end modeling and is configured to predict respective vehicle control frames containing longitudinal and lateral positions of the respective ego vehicle by matching the predicted vehicle control frames to the respective ground truth vehicle control frames. In other words, in matching the predicted vehicle control frames with the respective ground truth vehicle control frames, the predicted vehicle control frames are approximated with the respective ground truth vehicle control frames. The method of training the model according to the invention is based on a data-driven approach, in which the model is configured to learn implicitly from the naturalistic data of the ground truth.

In the context of the present invention, the expression "an additional or alternative preferred embodiment" or "an additional or alternative further preferred embodiment" or "an additional or alternative way of configuring this embodiment" means that the feature or combination of features, disclosed in this preferred embodiment may be combined in addition to or as an alternative to the features of the subject matter of the invention, including any preferred embodiment of any aspect of the invention, provided that the resulting combination of features makes sense to a person skilled in the art.

In the context of the present invention, the terms "comprising" or "including" are to be understood as having a broad meaning, similar to the term "including" and are to be understood as including a specific integer or a particular step or group of integers or steps, but not the exclusion of any other integer or step or group of integers or steps. This definition also applies to variants of the term “comprising” such as “include” and “includes” and variants of the term “contain” such as “contain” and “includes”.

In connection with the present invention, the term “configured” is also to be understood in connection with systems and computer program components. When a system of one or more computers is configured to perform specific operations or actions, it means that the system has software, firmware, hardware, or a combination thereof installed that, when in use, causes the system to perform operations or actions . One or more computer programs configured to perform specific operations or actions means that the one or more programs include instructions that, when executed by a computing device, cause the device to perform the operations or actions.

To achieve the inventive objects, advantages, and objectives, the present invention as disclosed in this disclosure is directed to systems and methods that utilize computer hardware and software to train a virtual traffic agent that uses algorithms and Reinforced learning techniques navigated through a simulation environment. A virtual traffic agent (also referred to as “traffic agent” within the scope of the present invention) can be a car, a truck, a bus, a bicycle or a motorcycle, for example. After a virtual traffic agent has been trained according to the present invention, which simulates human driving behavior in particular in complex lane-changing driving situations, one or more trained virtual traffic agents can be injected into a simulation environment with complex driving situations. Such an embodiment is preferred since the trained traffic agents can interact, cooperate and challenge an autonomous vehicle system controlling an autonomous vehicle under test. Another advantage is that such an embodiment is suitable for testing the limits and weaknesses of the autonomous vehicle system, particularly in complex driving situations that can be attributed to assertive or aggressive driving behavior.

Thus, the systems and methods according to the invention also have the technical effect and advantage that they represent an improvement in computing technology for autonomous vehicles, since the autonomous vehicle is trained in the simulation environment according to the invention, which reflects human-like/natural driving scenarios.

• Gemäß Schritt a) liefert das erfindungsgemäße Trainingsverfahren Fahrdaten zu einem oder mehreren Zeitfenstern t_i = [t_i, t₂, ... t_n] für ein oder mehrere Straßenfahrzeuge als Ego-Fahrzeuge, die jeweils von einem Menschen in einer realistischen Situation auf einer Straße gefahren werden, und stellt Kartendaten auf der jeweiligen Straße zu den gegebenen Zeitfenstern t_i bereit, wobei i eine beliebige Zahl ist, so dass i ∈ [1,2,...n] ist und wobei n den Grenzwert der gefahrenen Rahmen darstellt. Die Fahrdaten stellen im Allgemeinen Trajektoriendaten der jeweiligen Ego-Fahrzeuge dar.

According to the first aspect of the present invention, a computer-implemented training method of a traffic agent for navigating a road vehicle in a simulation environment, characterized in that the method comprises or consists of the following steps:

• According to step a), the training method according to the invention provides driving data for one or more time windows t _i = [t _i , t ₂ , ... t _n ] for one or more road vehicles as ego vehicles, each of which is triggered by a human being in a realistic situation a road are driven, and provides map data on the respective road at the given time windows t _i , where i is an arbitrary number such that i ∈ [1,2,...n] and where n is the limit of frames driven represents. The driving data generally represent trajectory data of the respective ego vehicles.

In an additional or alternative preferred embodiment, the driving data in step a) for each of the given road vehicles includes or consists of one or more status characteristics of the respective ego vehicles for each given time window t _i , preferably including or consisting of longitudinal speed, longitudinal acceleration and position of the respective road vehicle in X and Y coordinates for each given time window t _i .

In an additional or alternative preferred embodiment, the map data from step a) contain corresponding street information which i) the number of lanes of the respective street and ii) the position of the lanes in X, Y coordinates, optionally in X, Y and comprise or consist of Z-coordinates, each for specific periods of time t _i .

According to step b), the training method according to the invention processes at least part of the driving data and map data from step a) into one or more respective perceptual frames P _i = [p ₁ ,p ₂ ,...p _n ] per given time window t _i , each perceptual frame P _i contains corresponding perceptual information for (i) traffic situation, (ii) ego vehicle eigenstate information and (iii) road geometry, where i is any number such that i ∈ [1,2,...n] and where n represents the limit of the driven frames.

• - Das Auto vor dem Ego-Fahrzeug (auf der gleichen Spur).
• - Das Auto, das dem Ego-Fahrzeug folgt (auf derselben Fahrspur).
• - Die beiden Autos vor dem Mittelpunkt des Ego-Fahrzeugs übertragen auf die beiden benachbarten Fahrspuren.
• - Die beiden Autos im hinteren Teil des Ego-Fahrzeugs übertragen auf die beiden Nachbarspuren.

In an additional or alternative preferred embodiment, the traffic situation in step b) comprises or consists of six-vehicle neighborhood information, with each displayed vehicle of the six positions i) the relative distance of the respective vehicle to the ego vehicle and ii) the relative speed of the respective Vehicle to the speed of the ego vehicle includes or consists of. With respect to the six-vehicle neighborhood, the vehicle roles according to the present invention are defined as follows:

• - The car in front of the ego vehicle (in the same lane).
• - The car following the ego vehicle (on the same lane).
• - The two cars in front of the center of the ego vehicle transferred to the two adjacent lanes.
• - The two cars in the back of the ego vehicle transfer to the two adjacent lanes.

Each of these points may or may not be present for a particular time/ego-vehicle combination and are accounted for in the model.

wobei ${\theta }_{road}^i\text{ und }{\theta }_{ego}^i$

die Peilung der Straße und des Ego-Fahrzeugs zu jedem gegebenem Zeitfenster t_i darstellen, wobei i eine beliebige Zahl ist, so dass i ∈ [1,2,... n] ist und wobei n den Grenzwert der gefahrenen Rahmen darstellt. Um die Genauigkeit zu erhöhen, kann die Peilung der Straße durch die Peilung der Fahrspur ersetzt werden und wie folgt definiert werden $$A{d}^i={\theta }_{lane}^i-{\theta }_{ego}^i$$

wobei ${\theta }_{lane}^i\text{ und }{\theta }_{ego}^i$

die Peilung der Fahrspur und des Ego-Fahrzeugs zu jedem gegebenen Zeitfenster darstellen t_i.In an additional or alternative preferred embodiment, the inherent state information of the respective ego vehicles in step b) comprises or consists of the longitudinal speed, the longitudinal acceleration and the bearing in relation to the road direction (angular deviation Ad). In the context of the present invention, the term “bearing” of an ego vehicle stands for the alignment of the ego vehicle in relation to the global x/y axes. As an example, the angular deviation can be defined as $$A{\mathrm{i.e}}^i={\theta }_{\mathrm{right}Oa\mathrm{i.e}}^i-{\theta }_{eGO}^i$$

whereby ${\theta }_{\mathrm{right}Oa\mathrm{i.e}}^i\text{and}{\theta }_{eGO}^i$

represent the bearing of the road and the ego vehicle at any given time window t _i , where i is any number such that i ∈ [1,2,...n] and where n represents the limit of frames driven. To increase accuracy, the bearing of the road can be replaced by the bearing of the lane and defined as follows $$A{\mathrm{i.e}}^i={\theta }_{lane}^i-{\theta }_{eGO}^i$$

whereby ${\theta }_{lane}^i\text{and}{\theta }_{eGO}^i$

represent the bearing of the lane and ego vehicle at any given time window t _i .

In an additional or alternative preferred embodiment, the road geometry in step b) comprises or consists of a numeric representation of a respective roadway geometry in relation to the ego vehicle, wherein the numeric representation is preferably selected from a circular or semi-circular geometry.

wobei jeder Eintrag D_j Teil einer Folge von Verschiebungspunkten zur Position des Ego-Fahrzeugs ist, die auf der Grundlage ihrer relativen Peilwerte zur Ego-Position mit Intervallen von 1° oder mehr um den kreisförmigen oder halbkreisförmigen Bereich vor und/oder hinter dem Ego-Fahrzeug herum unterteilt sind, und wobei die Länge n des Verschiebungsvektors D_j 1 bis 360 für die kreisförmige Geometrie und 1 bis 180 für die halbkreisförmige Geometrie darstellt.The circular or semi-circular numerical representation of the respective lane geometry with two lane boundaries takes place, for example, in the form of a vector of displacements D _j at each of the two lane boundaries for any time window t _i with $$D_j=\left[{\mathrm{i.e}}_1,{\mathrm{i.e}}_2...{\mathrm{i.e}}_n\right]$$

where each entry D _j is part of a sequence of offset points to the position of the ego vehicle calculated based on their relative bearings to the ego position at intervals of 1° or more around the circular or semi-circular area in front of and/or behind the ego around the vehicle, and where the length n of the displacement vector D _j represents 1 to 360 for the circular geometry and 1 to 180 for the semi-circular geometry.

The semi-circular geometry represents the front area as 180°, while the circular geometry represents both the front and rear areas as 360°.

According to step c), the training method according to the invention processes at least part of the driving data and map data from step a) to form one or more corresponding basic truth driving vehicle control frame C _i = [c ₁ , c ₂ , ... c _n ] per given time window t _i , each vehicle control frame C _i containing longitudinal and lateral positions of respective ego vehicles, where i is any number such that i ∈ [1,2,...n] and where n represents the limit of frames driven.

In an additional or alternative preferred embodiment, the longitudinal and lateral positions of the respective ego vehicles in step c) and step d) comprise or consist of acceleration and bearing values, preferably they comprise or consist of changes in acceleration (Δacceleration) and bearing (Δbearing ) applied to the respective ego vehicles in the time window t _i . The use of changes in acceleration and bearing values is preferred because from these values the changes in steering, accelerator pedal depth and brake pedal depth can be determined directly.

Processing steps b) and c) can be carried out simultaneously or one after the other in any order.

According to step d), the inventive method trains a decision maker computer model of the traffic agent with the one or more perceptual frames P _i per given time window t _i of step b) as input for the model and with the one or more ground truth vehicle control frames C _i for given ones Time window t _i of step c) as a label for the training of the model, where the decider uses and configures one or more neural networks with end-to-end modeling, the corresponding vehicle control frames Ĉ _i = [c ₁ , c ₂ , .. c _n ] containing longitudinal and lateral positions of the respective ego vehicle by comparing the predicted vehicle control frames Ĉ _i with the respective ground truth vehicle control frames C _i , where i is any number such that i ∈ [1,2 ,...n] and where n is the limit for driven frames.

In other words, the perception P _i of the naturalistic data from step b) is used as input to the computer model and the naturalistic ground truth data of the vehicle control frames C _i in step c) is used as a tag for training purposes. In contrast, in a computer system simulating a driving environment according to the third aspect of the invention, the traffic agent trained according to the invention does not use the naturalistic vehicle control frames of step c) during operation and replaces the naturalistic perceptual frames of step b) with simulated perceptual frames. During use, according to the third aspect of the invention, the decider of the inventive simulation computer system predicts as an action one or more vehicle control frames Ĉ _i containing longitudinal and lateral positions of a simulated vehicle in the simulation environment, where i is any number such that i ∈ [1 ,2,...n] and where n is the gated frame limit.

As already discussed above in relation to the preferred embodiment of step c), the ground truth vehicle control frames C _i and the predicted vehicle control frames Ĉ _i will include or consist of acceleration and bearing values, preferably changes in acceleration (Δ acceleration) and bearing (Δ bearing ), which include or consist of the respective ego vehicles at the time window t _i .

In an additional or preferred embodiment, the decision-maker computer model of the traffic agent in step d) uses three or more neural networks, with preferably at least some or all of the neural networks being combined in a branched architecture.

In an additional or alternative preferred embodiment, at least some or all of the neural networks are deep neural networks, preferably at least some or all of the deep neural networks independently of one another comprise or consist of one, two or more layers, each layer independently of one another having a number of Has neurons in the range from 1 to 512, the number of neurons per layer preferably being different in the deep neural network.

oder „Fahrspur wechseln" ${\text{SC}}_{LC}^i=\left[{\text{SC}}_{\text{LC}}^1,{\text{SC}}_{\text{LC}}^2,\dots {\text{ SC}}_{LC}^n\right]$

$$

und wobei das Entscheider-Computermodell des Verkehrsagenten in Schritt d) i) ein neuronales Netz Fahrspur folgen, ii) ein neuronales Netz Fahrspur wechseln und iii) ein neuronales Netz Funktionsklassifikator umfasst, wobei

• - der eine oder die mehreren Wahrnehmungsrahmen P_i jeweils als Eingabe für die neuronalen Netze „Fahrspur folgen“, „Fahrspur wechseln“ und „Funktionsklassifikator" verwendet werden,
• - der eine oder die mehreren Grundwahrheits-Fahrzeugkontrollrahmen C_i jeweils als Etikett für ein unabhängiges Training der neuronalen Netze für Fahrspur folgen und Fahrspur wechseln verwendet werden, indem die vorhergesagten Fahrzeugkontrollrahmen Ĉ_i mit den jeweiligen Grundwahrheits-Fahrzeugkontrollrahmen C_i abgeglichen werden und
• - die jeweils angewandte Grundwahrheits-Situationskategorie ${\text{SC}}_{LF}^i$
  
  oder ${\text{SC}}_{LC}^i$
  
  je gegebenem Zeitfenster t_i als Etikett verwendet werden, um das neuronale Netz Funktionsklassifikator unabhängig zu trainieren, eine entsprechende Situationskategorie „Fahrspur folgen“ ${\widehat{SC}}_{LF}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1,{\widehat{SC}}_{\text{LF}}^2,\dots {\widehat{SC}}_{LF}^n\right]$
  
  oder „Fahrspur wechseln“ ${\widehat{SC}}_{LC}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">LC</figure-callout>}}^1,{\widehat{SC}}_{\text{LC}}^2,\dots {\widehat{SC}}_{LC}^n\right]$
  
  durch Abgleich der vorhergesagten Situationskategorie ${\widehat{SC}}_{LF}^i\text{ oder }{\widehat{SC}}_{LC}^i$
  
  mit der jeweiligen tatsächlichen Situationskategorie ${\text{SC}}_{LF}^i$
  
  und ${\text{SC}}_{LC}^i$
  
  vorherzusagen, wobei i eine beliebige Zahl ist, so dass i ∈ [1,2,...n] ist und wobei n den Grenzwert der gefahrenen Rahmen darstellt.

Accordingly, in one embodiment, the training method according to the invention also includes processing the driving data from step a) of the respective ego vehicles for each given time window t _i to form binary corresponding basic truth situation categories of "follow the lane". ${\text{SC}}_{Lf}^i=\left[{\text{<figure-callout id="1" label="SC LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">SC</figure-callout>}}_{\text{LF}}^{},{\text{SC}}_{\text{LF}}^2,...{\text{SC}}_{Lf}^n\right]$

or "change lane" ${\text{SC}}_{LC}^i=\left[{\text{<figure-callout id="1" label="SC LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">SC</figure-callout>}}_{\text{LC}}^{},{\text{SC}}_{\text{LC}}^2,...{\text{SC}}_{LC}^n\right]$

$$

and wherein the decision maker computer model of the traffic agent in step d) comprises i) a follow lane neural network, ii) a lane change neural network and iii) a function classifier neural network, wherein

• - the one or more perceptual frames P _i are respectively used as input for the neural networks "follow lane", "change lane" and "function classifier",
• - the one or more ground truth vehicle control frames C _i are each used as a label for an independent training of the neural networks for lane following and lane changing by comparing the predicted vehicle control frames Ĉ _i with the respective ground truth vehicle control frames C _i and
• - the applied basic truth situation category ${\text{SC}}_{Lf}^i$
  
  or ${\text{SC}}_{LC}^i$
  
  be used as a label for each given time window t _i to train the neural network function classifier independently, a corresponding situation category "follow lane" ${\widehat{SC}}_{Lf}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1,{\widehat{SC}}_{\text{LF}}^2,...{\widehat{SC}}_{Lf}^n\right]$
  
  or "change lane" ${\widehat{SC}}_{LC}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">LC</figure-callout>}}^1,{\widehat{SC}}_{\text{LC}}^2,...{\widehat{SC}}_{LC}^n\right]$
  
  by matching the predicted situation category ${\widehat{SC}}_{Lf}^i\text{or}{\widehat{SC}}_{LC}^i$
  
  with the respective actual situation category ${\text{SC}}_{Lf}^i$
  
  and ${\text{SC}}_{LC}^i$
  
  to predict, where i is any number such that i ∈ [1,2,...n] and where n represents the limit of frames driven.

oder ${\widehat{SC}}_{LC}^i$

mit den jeweiligen Grundwahrheits-Situationskategorien ${\text{SC}}_{LF}^i$

und ${\text{SC}}_{LC}^i$

werden die vorhergesagten Situationskategorien ${\widehat{SC}}_{LF}^i$

oder ${\widehat{SC}}_{LC}^i$

mit den jeweiligen Grundwahrheits-Situationskategorien ${\text{SC}}_{LF}^i$

und ${\text{SC}}_{LC}^i$

angenähert. Mit anderen Worten, ist das erfindungsgemäße Trainingscomputersystem für jedes Zeitfenster t_i und Ego-Fahrzeug konfiguriert, zu bestimmen, ob das jeweilige Ego-Fahrzeug der Fahrspur folgt oder die Fahrspur wechselt, wobei i eine beliebige Zahl ist, so dass i ∈ [1,2, ...n] ist und wobei n den Grenzwert der gefahrenen Rahmen darstellt.In other words, when matching the predicted situation categories ${\widehat{SC}}_{Lf}^i$

or ${\widehat{SC}}_{LC}^i$

with the respective basic truth situation categories ${\text{SC}}_{Lf}^i$

and ${\text{SC}}_{LC}^i$

become the predicted situation categories ${\widehat{SC}}_{Lf}^i$

or ${\widehat{SC}}_{LC}^i$

with the respective basic truth situation categories ${\text{SC}}_{Lf}^i$

and ${\text{SC}}_{LC}^i$

approximated. In other words, the training computer system according to the invention is configured for each time window t _i and ego vehicle to determine whether the respective ego vehicle is following the lane or changing lanes, where i is any number such that i ∈ [1, 2,...n] and where n represents the limit of frames driven.

All features and embodiments disclosed in relation to the first aspect of the present invention can be combined alone or in (sub)combination with the second aspect or the third aspect of the present invention, including any of the preferred embodiments thereof, provided the resulting combination of features is appropriate for a person skilled in the art.

According to the second aspect of the invention, a computer system for training a traffic agent navigating a road vehicle in a simulation environment is provided, comprising one or more processors, a memory device coupled to the one or more processors, and a traffic agent for decision-making in simulated driving situations Comprising or consisting of using one or more neural networks with end-to-end modeling stored in the storage device and configured to be executed by the one or more processors, characterized in that the traffic agent is so configured is that it executes the computer-implemented training method according to the first inventive aspect.

According to a further or alternative preferred embodiment, the training computer system of the second aspect can be configured such that the traffic agent comprises separate modules so that the respective naturalistic driving data and map data can be processed in a suitable manner. In particular, the traffic agent can comprise a module A for processing at least part of the naturalistic driving data and map data in order to calculate the respective perceptual frames P _i = [p ₁ ,p ₂ ,...p _n ] per given time window t _i , each perceptual frame P _i contains corresponding perception information for (i) the traffic situation, (ii) information about the ego vehicle's own state and (iii) the road geometry. Specific embodiments thereof have already been discussed in relation to the first aspect of the invention and also apply to the inventive training computer system of the second aspect of the invention. In addition, the traffic agent may comprise a module B for processing at least part of the naturalistic driving data and the map data to generate one or more corresponding ground truth vehicle control frames C _i = [c ₁ c ₂ ,...c _n ] per given time window _ti generate, wherein each vehicle control frame C _i contains longitudinal and lateral positions of the respective ego vehicles. In an additional or alternative preferred embodiment, the longitudinal and lateral positions of the respective ego vehicles comprise or consist of acceleration and bearing values, preferably they comprise or consist of changes in acceleration (Δacceleration) and bearing (Δbearing) that are applied to the respective ego Vehicles applied in time window t _i will. The use of changes in acceleration and bearing values is preferred because from these values the changes in steering, accelerator pedal depth and brake pedal depth can be determined directly.

Furthermore, according to the second aspect of the invention, the traffic agent can comprise the module C, also called the E2E Decision Maker (E2EDM) computer model, comprising one or more neural networks with end-to-end modeling. The outputs of modules A and B are used as input information to train module C's one or more E2E neural networks.

In an additional or preferred embodiment, the traffic agent module C uses three or more neural networks, with preferably at least part or all of the neural networks being combined in a branched architecture. The independent neural networks are preferably trained independently of one another.

In an additional or alternative preferred embodiment, at least some or all of the neural networks are deep neural networks, preferably at least some or all of the deep neural networks independently of one another comprise or consist of one, two or more layers, each layer independently of one another having a number of Having neurons in the range from 1 to 512, wherein more preferably the number of neurons per layer in the deep neural network is different.

oder „Fahrspur wechseln“ ${\text{SC}}_{LC}^i=\left[{\text{SC}}_{\text{LC}}^1,{\text{SC}}_{\text{LC}}^2,\dots {\text{ SC}}_{LC}^n\right]$

zu verarbeiten. Mit anderen Worten, für jedes Zeitfenster t_i und Ego-Fahrzeug ist das erfindungsgemäße Trainingscomputersystem konfiguriert, um zu bestimmen, ob das jeweilige Ego-Fahrzeug der Fahrspur folgt oder die Fahrspur wechselt, wobei i eine beliebige Zahl ist, so dass i ∈ [1,2,... n] ist und wobei n den Grenzwert der gefahrenen Rahmen darstellt.Accordingly, in one embodiment, module C comprises i) a lane-following neural network (module C2), ii) a lane-changing neural network (module C3), and iii) a function classification neural network (module C1). In this case, the training computer system can also include a module D, which is configured in such a way that it converts the naturalistic driving data and the map data for the respective ego vehicles into binary situation categories “follow the lane” that correspond to the basic truth in predetermined time windows t _i ${\text{SC}}_{Lf}^i=\left[{\text{<figure-callout id="1" label="SC LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">SC</figure-callout>}}_{\text{LF}}^{},{\text{SC}}_{\text{LF}}^2,...{\text{SC}}_{Lf}^n\right]$

or "change lane" ${\text{SC}}_{LC}^i=\left[{\text{<figure-callout id="1" label="SC LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">SC</figure-callout>}}_{\text{LC}}^{},{\text{SC}}_{\text{LC}}^2,...{\text{SC}}_{LC}^n\right]$

to process. In other words, for each time window t _i and ego vehicle, the training computer system according to the invention is configured to determine whether the respective ego vehicle is following the lane or changing lanes, where i is any number such that i ∈ [1 ,2,... n] and where n represents the limit of the frames driven.

• - der eine oder die mehreren Wahrnehmungsrahmen P_i werden jeweils als Eingabe für die neuronalen Netze „Fahrspur folgen“ (Modul C2), „Fahrspruch wechseln“ (Modul C3) und „Funktionsklassifikator“ (Modul C1) verwendet,
• - der eine oder die mehreren Grundwahrheits-Fahrzeugkontrollrahmen Ĉ_i jeweils als Etikett für ein unabhängiges Training der neuronalen Netze für das Folgen der Fahrspur (Modul C2) und den Fahrspurwechsel (Modul C3) verwendet werden, indem die vorhergesagten Fahrzeugkontrollrahmen Ĉ_i mit den jeweiligen Grundwahrheits-Fahrzeugkontrollrahmen Ĉ_i abgeglichen werden und
• - die jeweiligen Grundwahrheits-Situationskategorien ${\text{SC}}_{LF}^i$
  
  und ${\text{SC}}_{LC}^i$
  
  je gegebenem Zeitfenster t_i als Etikett verwendet werden, um das neuronale Netz Funktionsklassifikator (Modul C1) unabhängig zu trainieren die entsprechenden Situationskategorien „Fahrspur folgen“ ${\widehat{SC}}_{LF}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1,{\widehat{SC}}_{\text{LF}}^2,\dots {\widehat{SC}}_{LF}^n\right]$
  
  oder „Fahrspur wechseln“ ${\widehat{SC}}_{LC}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">LC</figure-callout>}}^1,{\widehat{SC}}_{\text{LC}}^2,\dots {\widehat{SC}}_{LC}^n\right]$
  
  durch Abgleich der vorhergesagten Situationskategorie ${\widehat{SC}}_{LF}^i$
  
  oder ${\widehat{SC}}_{LC}^i$
  
  mit der jeweiligen Grundwahrheits-Situationskategorie ${\text{SC}}_{LF}^i$
  
  oder ${\text{SC}}_{LC}^i$
  
  vorherzusagen.

With regard to the training method of the first inventive concept, the inventive training computer system of the second inventive aspect is configured such that

• - the one or more perceptual frames P _i are each used as input for the neural networks “follow lane” (module C2), “change driving call” (module C3) and “function classifier” (module C1),
• - the one or more ground truth vehicle control frames Ĉ _i are each used as a label for an independent training of the neural networks for lane following (module C2) and lane changing (module C3) by combining the predicted vehicle control frames Ĉ _i with the respective ground truth -Vehicle control frame Ĉ _i are matched and
• - the respective basic truth situation categories ${\text{SC}}_{Lf}^i$
  
  and ${\text{SC}}_{LC}^i$
  
  be used as a label for each given time window t _i to independently train the neural network function classifier (module C1) the corresponding situation categories "follow the lane" ${\widehat{SC}}_{Lf}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1,{\widehat{SC}}_{\text{LF}}^2,...{\widehat{SC}}_{Lf}^n\right]$
  
  or "change lane" ${\widehat{SC}}_{LC}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">LC</figure-callout>}}^1,{\widehat{SC}}_{\text{LC}}^2,...{\widehat{SC}}_{LC}^n\right]$
  
  by matching the predicted situation category ${\widehat{SC}}_{Lf}^i$
  
  or ${\widehat{SC}}_{LC}^i$
  
  with the respective basic truth situation category ${\text{SC}}_{Lf}^i$
  
  or ${\text{SC}}_{LC}^i$
  
  to predict.

The training computer system according to the invention is also configured so that the output of the function classifier (module C1), ie the respective situation category of the ego vehicle in the time window t _i either the neural network "follow lane" (module C2) or "lane change" (module C3) initiated.

An advantage of the computer system for training a traffic agent according to the invention is that the traffic agent is trained to predict both longitudinal and lateral positions of a vehicle in a simulated environment, the prediction reflecting naturalistic driving behavior.

All features and embodiments disclosed in relation to the second aspect of the present invention can be combined alone or in (sub)combination with the first aspect or the second aspect of the present invention, including any of the preferred embodiments thereof, provided the resulting combination of features is reasonable for a person skilled in the art.

According to the third aspect of the invention, there is provided a computer system for simulating a road driving environment in driving situations for one or more vehicles, comprising one or more processors, a memory device coupled to the one or more processors, and a traffic agent, the one or more neural networks for decision making used in simulated driving situations using one or more end-to-end modeling neural networks stored in the storage device and configured to be executed by the one or more processors, characterized that the traffic agent is trained according to the computer-implemented training method according to the first aspect of the invention to predict as an action one or more vehicle control frames Ĉ _i representing longitudinal and lateral positions of a simulated vehicle in the simulationum allowance per given time window t _i , where i is any number such that i ∈ [1,2,...n] and where n is the threshold for gated frames. In other words, the traffic agent used in the third aspect of the simulation computer system according to the invention was trained before being used in a simulation environment according to the training method according to the invention, the driving environment (simulation) being intended to provide environment data for an ego vehicle which (i) map information, (ii) traffic information and (iii) traffic regulations. This data is then processed and control commands are generated by the E2E decision maker trained according to the invention and returned to the environment for position updating. Such a computer system is also referred to as an integrated system.

As already mentioned above, the simulation computer system according to the invention does not use the naturalistic driving and map data that are used as input information for the training methods. Therefore, the simulation computer system according to the invention does not have to include a module B', which corresponds to the module B of the training computer system. In contrast, the simulated driving and map data of the simulated road user, which can be made available to the perception formation module A' by module S1' and/or module S2', are used as input information in the simulation computer system according to the invention in order to define the respective perception frames P _i for the respective time window t _i in module A'. In other words, module A' is configured in such a way that it generates the respective perceptual frames P _i for the respective time windows t _i based on the simulation data supplied by module S1' and/or S2'. The perceptual frames P _i for each given time window t _i are used as input information for the E2E decision-maker computer model (module C′) according to the invention. Module C' is configured to predict one or more vehicle control frames Ĉ _i containing longitudinal and lateral positions, preferably the changes in longitudinal and lateral position, more preferably the changes in acceleration and bearing per given time window t _i occurring on a simulated vehicle are to be applied in the simulation environment, where i is an arbitrary number such that i ∈ [1,2,...n] and where n represents the limit of frames driven.

In an additional or preferred embodiment, the decision maker computer model (module C') of the traffic agent uses three or more neural networks, with preferably at least part or all of the neural networks being combined in a branched architecture.

In an additional or alternative preferred embodiment, at least some or all of the neural networks are deep neural networks, preferably at least some or all of the deep neural networks independently of one another comprise or consist of one, two or more layers, each layer independently of one another having a number of Has neurons in the range from 1 to 512, the number of neurons per layer preferably being different in the deep neural network.

• - einen oder mehrere Wahrnehmungsrahmen P_i der simulierten Fahrzeuge je gegebenem Zeitfenster t_i jeweils als Eingabe für die neuronalen Netze „Fahrspur folgen“ (Modul C2'), „Fahrspur wechseln“ (Modul C3') und „Funktionsklassifikator“ (Modul C3') verwendet werden,
• - der Funktionsklassifikator (Modul C1') so konfiguriert ist, dass er die ein oder mehreren Wahrnehmungsrahmen P_i der simulierten Fahrzeuge je gegebenem Zeitfenster t_i in die Situationskategorie „Fahrspur folgen“ ${\widehat{SC}}_{LF}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1,{\widehat{SC}}_{\text{LF}}^2,\dots {\widehat{SC}}_{LF}^n\right]$
  
  oder „Fahrspurwechsel“ ${\widehat{SC}}_{LC}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">LC</figure-callout>}}^1,{\widehat{SC}}_{\text{LC}}^2,\dots {\widehat{SC}}_{LC}^n\right]$
  
  klassifiziert. Abhängig von der jeweiligen Klassifizierung je gegebenem Zeitfenster t_i, d. h. entweder der Klasse „Fahrspur folgen“ oder der Klasse „Fahrspur wechseln“, initiiert der Funktionsklassifikator (Modul C1') das neuronale Netz von entweder „Fahrspur folgen“ (Modul C2') oder „Fahrspur wechseln“ (Modul C3'). Ein Beispiel: Wenn der Funktionsklassifikator (Modul C1') einen Wahrnehmungsrahmen P₁ zum Zeitfenster t₁ mit der Situationskategorie „Fahrspur folgen“ ${\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1$
  
  klassifiziert, dann ist der Funktionsklassifikator (Modul C1') so konfiguriert, dass er das neuronale Netz „Fahrspur folgen“ initiiert, um den Fahrzeugsteuerungsrahmen Ĉ₁ vorherzusagen, der longitudinale und laterale Positionen enthält, vorzugsweise die Änderungen der longitudinalen und lateralen Position, noch bevorzugter die Änderungen der Beschleunigung und der Peilung, die auf das jeweilige simulierte Fahrzeug zum Zeitfenster t₁ anzuwenden sind. Alternativ, falls der Funktionsklassifikator (Modul C1') einen Wahrnehmungsrahmen P₂ zum Zeitfenster t₂ mit der Situationskategorie „Fahrspur wechseln“ ${\widehat{SC}}_{\text{LC}}^2$
  
  klassifiziert, dann ist der Funktionsklassifikator (Modul C1') so konfiguriert, dass er das neuronale Netz „Fahrspur wechseln“ initiiert, um den Fahrzeugsteuerungsrahmen Ĉ₂ vorherzusagen der longitudinale und laterale Positionen enthält, vorzugsweise die Änderungen der longitudinalen und lateralen Position, noch bevorzugter die Änderungen der Beschleunigung und der Peilung, die auf das jeweilige simulierte Fahrzeug zum Zeitfenster t₂ anzuwenden sind.

Accordingly, in one embodiment, the decision-maker computer model (module C') of the traffic agent comprises i) a lane-following neural network (module C2'), ii) a lane-changing neural network (module C3'), and iii) a neural network Network as a function classifier (module C1'), which are configured in such a way that

• - one or more perception frames P _i of the simulated vehicles for each given time window t _i as input for the neural networks "follow lane" (module C2'), "change lane" (module C3') and "function classifier" (module C3') be used,
• - the function classifier (module C1') is configured in such a way that it assigns the one or more perception frames P _i of the simulated vehicles to the situation category "follow lane" for each given time window t _i ${\widehat{SC}}_{Lf}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1,{\widehat{SC}}_{\text{LF}}^2,...{\widehat{SC}}_{Lf}^n\right]$
  
  or "lane change" ${\widehat{SC}}_{LC}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">LC</figure-callout>}}^1,{\widehat{SC}}_{\text{LC}}^2,...{\widehat{SC}}_{LC}^n\right]$
  
  classified. Depending on the respective classification for each given time window t _i , ie either the class "follow lane" or the class "change lane", the function classifier (module C1') initiates the neural network of either "follow lane" (module C2') or "Change lane" (module C3'). An example: If the function classifier (module C1') has a perception frame P ₁ for the time window t ₁ with the situation category "follow lane" ${\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1$
  
  classified, then the functional classifier (module C1') is configured to initiate the "follow lane" neural network to predict the vehicle control frame Ĉ ₁ containing longitudinal and lateral positions, preferably the changes in longitudinal and lateral position, more preferably the changes in acceleration and bearing to be applied to each simulated vehicle at time window t ₁ . Alternatively, if the function classifier (module C1') has a perception frame P ₂ for the time window t ₂ with the situation category "change lane" ${\widehat{SC}}_{\text{LC}}^2$
  
  classified, then the functional classifier (module C1') is configured to initiate the "change lane" neural network to predict the vehicle control frame Ĉ ₂ containing longitudinal and lateral positions, preferably the longitudinal and lateral position changes, more preferably the Acceleration and bearing changes to be applied to each simulated vehicle at time window t ₂ .

The output of module C' is provided to the simulated driving environment module (module S2') for application to the simulated traffic agent in the simulation environment. Module S2' is configured to provide Module S1' with the respective modified simulated environment data comprising driving data and map data of the simulated road user, such that Module S1' provides Module A' with a modified environment data set to generate the next perceptual frame.

The present invention is described below with reference to exemplary embodiments, which only serve as examples and are not intended to limit the scope of the present protective right.

DETAILED DESCRIPTION OF ILLUSTRATIONS

Further features and advantages of the present invention result from the following description of exemplary embodiments of the aspects according to the invention with reference to the attached figures.

All features disclosed below in relation to the exemplary embodiments and/or the accompanying figures can be combined alone or in any sub-combination with features of the two aspects of the present invention, including features of preferred embodiments thereof, provided that the resulting combination of features is clear to a person skilled in the art field of technology makes sense.

1a) shows a schematic representation of the traffic agent for decision making in simulated driving situations (also called “E2E vehicle control model”) 1 stored in the storage device and configured to include one or more neural networks with end-to-end modeling and executes the computer-implemented training method according to the invention. The computer system according to the invention for training a traffic agent who controls a road vehicle in a simulation environment also comprises or consists of one or more processors, a memory device coupled to the one or more processors, which in 1a) are not shown separately.

According to 1a) the naturalistic driving data and map data (not shown separately) are used as input information for module A (perception building) and module B (vehicle control building), which are included in 1a) are shown as a combined module 11. Alternatively, modules A and B can also be present as separate modules. The output information generated by modules A and B in module 11 is used as input information to enable the traffic agent to train decider 12 (also called “E2E decider” or module C) according to the training method according to the invention, which is described in detail herein.

The traffic agent 1 according to the invention comprises, for example, a combined module 11 with module A and module B. Module A is configured in such a way that it processes at least part of the naturalistic driving data and map data in order to obtain the respective perception frames P _i =[p ₁ ,p ₂ , . .. p _n ] per given time window t _i , each perception frame P _i containing corresponding perception information for (i) the traffic situation, (ii) information about the ego vehicle's own state and (iii) the road geometry. Specific exemplary embodiments have already been discussed in relation to the first aspect of the invention and also apply to the training computer system of the second aspect of the invention. Module B is configured to process at least a portion of the naturalistic driving data and the map data to generate one or more corresponding ground truth vehicle control frames C _i = [c ₁ , c ₂ , ... c _n ] per given time window t _i , each ground truth vehicle control frame C _i comprising longitudinal and lateral positions, preferably changes in longitudinal and lateral positions, eg changes in acceleration (Δacceleration) and bearing (Δbearing) to be applied to the respective ego vehicles per given time window t _i . The use of changes in acceleration and bearing values is preferred because from these values the changes in steering, accelerator pedal depth and brake pedal depth can be determined directly.

In addition, the combined module 11 can also have an additional module D (in 1b) not shown) configured to classify at least a portion of the perceptual frames P _i based on the naturalistic driving data and the map data into a binary situation category of either "follow lane" or "change lane" per given time window t _i .

According to an alternative or additional preferred embodiment, the module 12 (module C) of the traffic agent 1 uses three or more neural networks, with preferably at least part or all of the neural networks being combined in a branched architecture. The independent neural networks are preferably trained independently of one another. An example of such a configuration, where the E2E decision maker 12 comprises three neural networks 121 (function classifier), 122 (follow driving instruction) and 123 (change lanes) combined in a branched architecture is shown in FIG 1b) shown. The output data from module 11 is used as input information.

In an additional or alternative preferred embodiment, at least some or all of the neural networks are deep neural networks, preferably at least some or all of the deep neural networks independently of one another comprise or consist of one, two or more layers, each layer independently of one another having a number of Having neurons in the range from 1 to 512, wherein more preferably the number of neurons per layer in the deep neural network is different.

$$

oder „Fahrspur wechseln“ ${\text{SC}}_{LC}^i=\left[{\text{SC}}_{\text{LC}}^1,{\text{SC}}_{\text{LC}}^2,\dots {\text{ SC}}_{LC}^n\right]$

verarbeitet. Mit anderen Worten, ist das erfindungsgemäße Trainings-Computersystem für jedes Zeitfenster t_i und Ego-Fahrzeug so konfiguriert, das es bestimmt, ob das jeweilige Ego-Fahrzeug der Fahrspur folgt oder die Fahrspur wechselt, wobei i eine beliebige Zahl ist, so dass i ∈ [1,2,... n] ist und wobei n den Grenzwert der gefahrenen Rahmen darstellt.In 1b) is shown by way of example that the E2E decision maker 12 i) a neural network 122 (module C2) for following the lane, ii) a neural network 123 (module C3) for changing lanes and iii) a neural network 121 (module C1) for functional classification. In this case, the traffic agent 1 comprises the module D (in 1b) not shown) for the classification of situation categories, which is configured in such a way that the naturalistic driving data and map data for the respective ego vehicles in predetermined time windows t _i to binary corresponding basic truth situation categories "follow the lane" ${\text{SC}}_{Lf}^i=\left[{\text{<figure-callout id="1" label="SC LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">SC</figure-callout>}}_{\text{LF}}^{},{\text{SC}}_{\text{LF}}^2,...{\text{SC}}_{Lf}^n\right]$

$$

or "change lane" ${\text{SC}}_{LC}^i=\left[{\text{<figure-callout id="1" label="SC LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">SC</figure-callout>}}_{\text{LC}}^{},{\text{SC}}_{\text{LC}}^2,...{\text{SC}}_{LC}^n\right]$

processed. In other words, the training computer system according to the invention is configured for each time window t _i and ego vehicle in such a way that it determines whether the respective ego vehicle follows the lane or changes lanes, where i is any number such that i ∈ [1,2,... n] and where n represents the limit of the frames driven.

• - der eine oder die mehreren Wahrnehmungsrahmen P_i jeweils als Eingangsinformationen für die neuronalen Netze „Fahrspur folgen 122“ (Modul C2), „Fahrspur wechseln 123“ (Modul C3) und „Funktionsklassifikator 121“ (Modul C1) verwendet werden,
• - der eine oder die mehreren Grundwahrheits-Fahrzeugkontrollrahmen C_i jeweils als Etikett für das unabhängige Trainieren der neuronalen Netze „Fahrspur folgen 122“ (Modul C2) und „Fahrspruch wechseln 123“ (Modul C3) verwendet werden, indem die vorhergesagten Fahrzeugkontrollrahmen C_i mit den jeweiligen Grundwahrheits-Fahrzeugkontrollrahmen C_i abgeglichen werden und
• - die jeweiligen Grundwahrheits-Situationskategorien ${\text{SC}}_{LF}^i$
  
  und ${\text{SC}}_{LC}^i$
  
  je gegebenem Zeitfenster t_i als Etikett verwendet werden, um das neuronale Netz Funktionsklassifikator 121 (Modul C1) unabhängig voneinander zu trainieren, eine entsprechende Situationskategorie „Fahrspur folgen“ ${\widehat{SC}}_{LF}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1,{\widehat{SC}}_{\text{LF}}^2,\dots {\widehat{SC}}_{LF}^n\right]$
  
  oder „Fahrspur wechseln“ ${\widehat{SC}}_{LC}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">LC</figure-callout>}}^1,{\widehat{SC}}_{\text{LC}}^2,\dots {\widehat{SC}}_{LC}^n\right]$
  
  der vorhergesagten Situationskategorie ${\widehat{SC}}_{LF}^i$
  
  oder ${\widehat{SC}}_{LC}^i$
  
  mit der entsprechenden Grundwahrscheinlichkeitskategorie ${\text{SC}}_{LF}^i$
  
  oder ${\text{SC}}_{LC}^i$
  
  abzugleichen.

According to this example, the E2E decider 12 of the traffic agent 1 according to the invention is configured such that

• - the one or more perceptual frames P _i are used as input information for the neural networks “follow lane 122” (module C2), “change lane 123” (module C3) and “function classifier 121” (module C1),
• - the one or more ground truth vehicle control frames C _i are used as labels for the independent training of the neural networks "follow lane 122" (module C2) and "change driving claim 123" (module C3) by using the predicted vehicle control frames C _i with are matched to the respective ground truth vehicle control framework C _i and
• - the respective basic truth situation categories ${\text{SC}}_{Lf}^i$
  
  and ${\text{SC}}_{LC}^i$
  
  be used as a label for each given time window t _{i in} order to train the neural network function classifier 121 (module C1) independently of one another, a corresponding situation category “follow lane” ${\widehat{SC}}_{Lf}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1,{\widehat{SC}}_{\text{LF}}^2,...{\widehat{SC}}_{Lf}^n\right]$
  
  or "change lane" ${\widehat{SC}}_{LC}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">LC</figure-callout>}}^1,{\widehat{SC}}_{\text{LC}}^2,...{\widehat{SC}}_{LC}^n\right]$
  
  the predicted situation category ${\widehat{SC}}_{Lf}^i$
  
  or ${\widehat{SC}}_{LC}^i$
  
  with the corresponding basic probability category ${\text{SC}}_{Lf}^i$
  
  or ${\text{SC}}_{LC}^i$
  
  to match.

oder ${\widehat{SC}}_{LC}^i$

des Ego-Fahrzeugs zum Zeitfenster t_i entweder das neuronale Netz Fahrspur folgen 122 (Modul C2) oder Fahrspruch wechseln 123 (Modul C3) initiiert.The E2E decision-maker 12 is also configured in such a way that the output of the function classifier 121 (module C1), ie the respective situation category ${\widehat{SC}}_{Lf}^i$

or ${\widehat{SC}}_{LC}^i$

of the ego vehicle at the time window t _i either initiates the neural network follow the lane 122 (module C2) or change driving call 123 (module C3).

An advantage of the computer system for training according to the invention is that the traffic agent 1 is trained to predict both longitudinal and lateral positions, preferably changes in longitudinal and lateral positions to be applied to a vehicle in a simulated environment, the prediction reflects natural driving behavior. According to one example, the changes in longitudinal and lateral positions may be in the form of changes in acceleration and changes in bearing to be applied to the simulated vehicle in a given time window.

1c ) shows a schematic representation of an integrated simulation computer system 01' according to the invention, which uses a traffic agent 1' designed according to the invention, which uses a module 11' (module A') to form perceptions on the basis of the simulated environmental data provided by module 21' (module S1') and an E2E arbiter model 12' and one or more processors and a memory device coupled to the one or more processors (in 1c ) not shown separately). The driving environment module S2' (simulation) is intended to supply environment data for an ego vehicle in module S1', which contains (i) map information, (ii) traffic information and (iii) traffic rules. This data is then processed in module 11' to generate perceptions and control commands are generated by E2E decision making 12' and returned to environment 22' for position update.

The module 11 'is configured so that it generates for the respective simulated vehicle per time window perception frames containing information about (i) the traffic situation, (ii) information about the inherent state of the simulated vehicle and (iii) the road geometry, and the generated Perceptual framework as input information for the E2E decision module 12 '(module C') provides. The E2E decider module 12' was trained according to the training method according to the invention. The E2E decision-maker module 12' is thus configured to predict as an action one or more vehicle control frames Ĉ _i containing longitudinal and lateral positions, preferably changes in longitudinal and lateral positions, eg changes in acceleration and bearing, which are applied to the simulated vehicle are to be applied in the simulation environment.

As already mentioned above, the simulation computer system 01' according to the invention, which uses the traffic agent 1' according to the invention, does not use the naturalistic driving and map data that are used as input information for the training method. Therefore, the simulation computer system 01' according to the invention does not have to have a module B' that corresponds to the module B of the training computer system. In contrast, the simulated driving and map data of the simulated traffic agent 1', which are supplied by module 21' (module S1') to module 11' (module A'), are used as input information in the simulation computer system 01' according to the invention in order to define the respective perception frames P _i for the respective time window t _i in module 11' (module A'). In other words, module 11' (module A') is configured to generate the respective perceptual frames P _i for the respective time windows t _i based on the simulation data provided by module 21' (module S1'). The perceptual frames P _i per given time window t _i generated by module 11' are used as input information for the E2E decision-maker computer model 12' (module C') according to the invention. The module 12' (module C') is configured to predict the longitudinal and lateral position, preferably the changes in longitudinal and lateral position, most preferably the changes in acceleration and bearing, which will be incident on the simulated road user per given time window t _i are to be applied.

In an additional or preferred embodiment, the computer model of the decision maker 12' (module C') of the deployed traffic agent 1' uses three or more neural networks, preferably with at least part or all of the neural networks being combined in a branched architecture.

In an additional or alternative preferred embodiment, at least some or all of the neural networks are deep neural networks, preferably at least some or all of the deep neural networks independently of one another comprise or consist of one, two or more layers, each layer independently of one another having a number of Having neurons in the range from 1 to 512, wherein more preferably the number of neurons per layer in the deep neural network is different.

An example configuration of the E2E decision maker 12' in use includes the analogous configuration of the E2E decision maker 12 as in FIG 1b) shown. Accordingly, the corresponding details and preferred embodiments as described above also apply.

• - ein oder mehrere Wahrnehmungsrahmen P_i der simulierten Fahrzeuge je gegebenem Zeitfenster t_i jeweils als Eingabe für die neuronalen Netze "Fahrspur folgen 122''' (Modul C2'), „Fahrspur wechseln 123''' (Modul C3') und „Funktionsklassifikator 121''' (Modul C3') verwendet werden,
• - das neuronale Netz Funktionsklassifikator 121' (Modul C1') so konfiguriert ist, dass es die ein oder mehreren Wahrnehmungsrahmen P_i der simulierten Fahrzeuge je gegebenem Zeitfenster t_i in die Situationskategorie „Fahrspur folgen" ${\widehat{SC}}_{LF}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1,{\widehat{SC}}_{\text{LF}}^2,\dots {\widehat{SC}}_{LF}^n\right]$
  
  $$
  
  oder „Fahrspur wechseln" ${\widehat{SC}}_{LC}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">LC</figure-callout>}}^1,{\widehat{SC}}_{\text{LC}}^2,\dots {\widehat{SC}}_{LC}^n\right]$
  
  klassifiziert. Abhängig von der jeweiligen Klassifizierung je gegebenem Zeitfenster t_i, d.h. entweder der Klasse „Fahrspur folgen“ oder der Klasse „Fahrspur wechseln“, initiiert der Funktionsklassifikator 121' (Modul C1') das neuronale Netz von entweder Fahrspur folgen 122' (Modul C2') oder Fahrspur wechseln 123' (Modul C3'). Ein Beispiel: Wenn der Funktionsklassifikator 121' (Modul C1') einen Wahrnehmungsrahmen P₁ zum Zeitfenster t₁ mit der Situationskategorie „Fahrspur folgen“ ${\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1$
  
  klassifiziert, dann ist der Funktionsklassifikator 121' (Modul C1') so konfiguriert, dass er das neuronale Netz „Fahrspur folgen“ 122' initiiert, um den Fahrzeugsteuerungsrahmen Ĉ₁ vorherzusagen, der die longitudinalen und lateralen Positionen, vorzugsweise die Änderungen der longitudinalen und lateralen Position, noch bevorzugter die Änderungen der Beschleunigung und der Peilung enthält, die auf das jeweilige simulierte Fahrzeug zum Zeitfenster t₁ anzuwenden sind. Falls alternativ der Funktionsklassifikator 121' (Modul C1') einen Wahrnehmungsrahmen P₂ zum Zeitfenster t₂ mit der Situationskategorie „Fahrspurwechsel“ ${\widehat{SC}}_{\text{LC}}^2$
  
  klassifiziert, dann ist der Funktionsklassifikator 121' (Modul C1') so konfiguriert, dass er das neuronale Netz „Fahrspurwechsel“ 123' initiiert, um den Fahrzeugsteuerungsrahmen Ć̂₂ vorherzusagen die Änderungen der longitudinalen und lateralen Position, vorzugsweise die Änderungen der longitudinalen und lateralen Position, noch bevorzugter die Änderungen der Beschleunigung und der Peilung, die auf das simulierte Fahrzeug zum Zeitfenster anzuwenden sind t₂.

The decision-maker computer model 12' (module C') of the traffic agent 1' used includes i) a neural network 122' (module C2') for lane following, ii) a neural network 123' (module C3') for changing lanes and iii) a neural network 121' (module C1') for function classification, configured so that

• - one or more perception frames P _i of the simulated vehicles for each given time window t _i as input for the neural networks "follow lane 122"' (module C2'), "change lane 123"' (module C3') and "function classifier 121''' (module C3') can be used,
• - the neural network function classifier 121' (module C1') is configured in such a way that it includes the one or more perception frames P _i of the simulated vehicles for each given time window t _i in the situation category "follow the lane". ${\widehat{SC}}_{Lf}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1,{\widehat{SC}}_{\text{LF}}^2,...{\widehat{SC}}_{Lf}^n\right]$
  
  $$
  
  or "change lane" ${\widehat{SC}}_{LC}^i=\left[{\widehat{SC}}_{\text{<figure-callout id="1" label="LC" filenames="DE112020006532T5\_0087.png" state="{{state}}">LC</figure-callout>}}^1,{\widehat{SC}}_{\text{LC}}^2,...{\widehat{SC}}_{LC}^n\right]$
  
  classified. Depending on the respective classification for each given time window t _i , ie either the “follow lane” class or the “change lane” class, the function classifier 121′ (module C1′) initiates the neural network of either follow lane 122′ (module C2′ ) or changing lanes 123' (module C3'). An example: If the function classifier 121' (module C1') has a perception frame P ₁ for the time window t ₁ with the situation category "follow lane". ${\widehat{SC}}_{\text{<figure-callout id="1" label="LF" filenames="DE112020006532T5\_0087.png" state="{{state}}">LF</figure-callout>}}^1$
  
  classified, then the functional classifier 121' (module C1') is configured to initiate the lane-follow neural network 122' to predict the vehicle control frame Ĉ ₁ that includes the longitudinal and lateral positions, preferably the changes in the longitudinal and lateral position, more preferably the changes in acceleration and bearing to be applied to the respective simulated vehicle at time window t ₁ . If, alternatively, the function classifier 121' (module C1') has a perception frame P ₂ for the time window t ₂ with the situation category "lane change" ${\widehat{SC}}_{\text{LC}}^2$
  
  classified, then the function classifier 121' (module C1') is configured to initiate the "lane change" neural network 123' to predict the vehicle control frame - ₂ the longitudinal and lateral position changes, preferably the longitudinal and lateral position changes , more preferably the changes in acceleration and bearing to be applied to the simulated vehicle at time window t ₂ .

The output information of the module 12' (module C') is made available to the simulated driving environment 22' (module S2') in order to be applied to the simulated traffic agent 1' in the simulation environment. Module 22' (module S2') is configured in such a way that it supplies module 21' (module S1') with the respectively changed simulated environment data, which includes driving data and map data of the simulated road user 1', so that module 21' (module S1' ) provides the module 11' (module A') with the changed environmental data to generate the next perceptual frame.

experimental part

For the training according to the present invention, the inventors used commercial driving data DataFromSky (DFS, acquired from RCE systems sro, Czech Republic), which includes driving data of vehicles driven by humans for a period of six hours on a part (500 m) of the highway A9 were driven in Germany. In particular, the DFS data set included the following characteristics: time stamp (in seconds, s), longitudinal velocity (in meters/second, m/s), longitudi nal acceleration (in meters/second squared, m/s ² ) and global coordinates of the respective vehicle (road user) (in x, y coordinates).

für die aktuelle und zwei benachbarte Fahrspuren eines Subjekt/Ego-Fahrzeugs in einem Zeitintervall t_i wobei jede Fahrspur $l_i^j=\left[{\text{x}}_1,{\text{x}}_2,\dots {\text{x}}_{\text{n}}\right]$

als Satz von Koordinaten ${\text{x}}_{\text{j}}=\left[\begin{array}{c}{x}_j\\ y_j\\ \end{array}\right],$

so dass x_n der letzte Punkt auf der Fahrspur $l_i^j$

ist, der sich in einer maximalen Entfernung von 400 m zur Position des Ego/Subjektfahrzeugs bei t_i befindet.In addition, the OpenDRIVE digital map (downloaded from http://www.opendrive.org/) was used as map data in the simulation to generate lane points relative to each ego position, which were used to generate road geometry data for the model to construct to be used as input. These lane points can be described as follows: $$L_i=\left[{\mathrm{<figure-callout\; id="1"\; label="l\; i"\; filenames="DE112020006532T5\_0087.png"\; state="\{\{state\}\}">l</figure-callout>}}_i^{},l_i^2,l_i^3\right]$$

for the current and two adjacent lanes of a subject/ego vehicle in a time interval t _i where each lane $l_i^j=\left[{\text{x}}_1,{\text{x}}_2,...{\text{x}}_{\text{n}}\right]$

as a set of coordinates ${\text{x}}_{\text{j}}=\left[\begin{array}{c}{x}_j\\ y_j\\ \end{array}\right],$

so that x _n is the last point on the lane $l_i^j$

which is at a maximum distance of 400 m from the ego/subject vehicle position at t _i .

1. 1. Verkehrssituation: Die Eingabedaten (DFS und Open DRIVE) werden verarbeitet, um die Informationen über die Sechs-Fahrzeug-Nachbarschaft in Bezug auf jedes Ego-/Subjektfahrzeug zu erstellen, wobei jedes repräsentierte Fahrzeug in den sechs Positionen zwei Informationen bietet: (i) relative Entfernung d zum Ego-Fahrzeug und (ii) relative Geschwindigkeit v_r zur Geschwindigkeit des Ego-Fahrzeugs v_e. 2 zeigt eine schematische Darstellung einer Sechs-Fahrzeug-Nachbarschaftsinformation in einem bestimmten Zeitfenster.

As discussed in the detailed description above, the perceptual framework used in the present invention falls into three categories:

1. 1. Traffic situation: The input data (DFS and Open DRIVE) are processed to create the six-vehicle neighborhood information in relation to each ego/subject vehicle, each represented vehicle in the six positions offering two pieces of information: (i ) relative distance d to the ego vehicle and (ii) relative speed v _r to the speed of the ego vehicle v _e . 2 FIG. 12 shows a schematic representation of six-vehicle neighborhood information in a specific time window.

• - Der Pkw 311 vor dem ego-Fahrzeug 3 (auf derselben Fahrspur).
• - Das Auto 312 folgt dem Ego-Fahrzeug 3 von hinten (auf der gleichen Spur).
• - Die beiden Autos 321, 322 vor dem Mittelpunkt des Ego-Fahrzeugs übertragen auf die beiden benachbarten Fahrspuren.
• - Die beiden Autos 331, 332 im hinteren Teil des Ego-Fahrzeugs 3 übertragen auf die beiden Nachbarspuren.

As set forth above, the vehicle roles in a six-vehicle neighborhood according to the present invention are defined as follows:

• - The car 311 in front of the ego vehicle 3 (on the same lane).
• - The car 312 follows the ego vehicle 3 from behind (in the same lane).
• - The two cars 321, 322 in front of the center of the ego vehicle are transferred to the two adjacent lanes.
• - The two cars 331, 332 in the rear part of the ego vehicle 3 transfer to the two adjacent lanes.

According to 2 all positions for the displayed time window are available. All six vehicles 311, 312, 321, 322, 331 and 332 in the vicinity of the ego vehicle 3 have the same distance d to the ego vehicle 3. The relative speed v _r is the respective speed v _n of one of the six neighboring vehicles 311, 312 , 321, 322, 331 and 332 minus the speed v _e of the ego vehicle 3 (assuming that the vehicles are moving in the same direction).

wobei ${\theta }_{lane}^i\text{ und }{\theta }_{ego}^i$

die globale Richtung/Ausrichtung der Fahrspur und des Ego-Fahrzeugs zu jedem gegebenen Zeitfenster t_i darstellen.2. Ego state information: This includes longitudinal velocity, longitudinal acceleration, angular deviation and bearing of the ego vehicle with respect to the lane direction. As mentioned above, the angular deviation (Ad) is defined as: $$A{\mathrm{i.e}}^i={\theta }_{lane}^i-{\theta }_{eGO}^i$$

whereby ${\theta }_{lane}^i\text{and}{\theta }_{eGO}^i$

represent the global direction/orientation of the lane and the ego vehicle at any given time window t _i .

wobei jeder Eintrag D_j Teil einer Folge von Verschiebungspunkten zur Ego-Position ist, die auf der Grundlage ihrer relativen Peilwerte zur Ego-Position in Abständen von 5° um den halbkreisförmigen Bereich vor dem Ego verteilt sind. Daher ist die Länge n des Verschiebungsvektors D_i im Rahmen der Experimente dieser Arbeit auf festgelegt: $$n=180/5=36$$

3. Road geometry: According to the present example experiment, the DFS driving data and the _OpenDRIVE map data are processed to produce a semi-circular numerical representation of the road geometry in the form of a vector of displacements, Dj, to each of the two lane boundaries LB1 and LB2 at each time window t _i to get with $$D_j=\left[{\mathrm{i.e}}_1,{\mathrm{i.e}}_2...{\mathrm{i.e}}_n\right]$$

where each entry D _j is part of a sequence of offset points to the ego position distributed at 5° intervals around the semi-circular area in front of the ego based on their relative bearings to the ego position. Therefore, in the context of the experiments of this work, the length n of the displacement vector D _i is set to: $$n=180/5=36$$

Such a semicircular road geometry is in 3 shown schematically, wherein - for the sake of clarity - only part of the 36 displacement vectors D _j of the ego vehicle 3 is shown in this case.

• - Einzelnes Netzmodell, bei dem die Entscheidungsprozesse in einem einzigen neuronalen Netz mit einer Abfolge von Schichten n gelernt werden, wobei n > 3 ∈ ℤ⁺ empirisch bei Experimenten abgeleitet wird.
• - Funktional verzweigte Netze, bei denen die Entscheidungsprozesse auf der Grundlage grundlegender Fahrfunktionen, z. B. Fahrspur folgen und Fahrspur wechseln, unterteilt werden. Zur Modellierung dieses Ansatzes werden drei neuronale Netze verwendet, von denen jedes aus einer Folge von n Schichten aufgebaut ist, wobei n > 3 ∈ ℤ⁺ empirisch während der Experimente abgeleitet werden. Diese sind wie folgt:
• - Fahrspruchfolgenmodul 122, das zur Steuerung des Fahrzeugs bei allgemeinen Fahrspurfolgeszenarien verwendet wird,
• - Fahrspurwechselmodul 123, das zur Steuerung des Fahrzeugs bei allgemeinen Fahrspurfolgeszenarien verwendet wird,
• - Funktionsklassifikatormodul 123, das dazu dient, binär zu klassifizieren, ob es sich um eine Situation handelt, in der man der Fahrspur folgt oder die Fahrspur wechselt, und somit eines der beiden entsprechenden Modelle 122 oder 123 initiiert.

The present inventors examine two possible approaches to modeling the E2E decider according to the invention using neural networks in a simulation environment:

• - Single network model where the decision processes are learned in a single neural network with a sequence of layers n, where n > 3 ∈ ℤ ⁺ is derived empirically on experiments.
• - Functionally branched networks, in which the decision-making processes are based on basic driving functions, e.g. B. follow lane and change lanes, are divided. Three neural networks are used to model this approach, each built up of a sequence of n layers, where n > 3 ∈ ℤ ⁺ are derived empirically during the experiments. These are as follows:
• - lane following module 122 used to control the vehicle in general lane following scenarios,
• - lane change module 123 used to control the vehicle in general lane following scenarios,
• Function classifier module 123, used to classify in binary form whether it is a lane following situation or a lane changing situation, thus initiating one of the two corresponding models 122 or 123.

Each of these sub-modules was trained independently.

• - Adaptiver Tempomat (ACC): Steuerung der Drosselung/Beschleunigung des Fahrzeugs in Bezug auf das vordere Fahrzeug.
• - Verkehrsfreie Lenkung: Steuerung der Lenkung des Fahrzeugs, um die Fahrspur zu halten.

In the present experiment, the lane following module 122 aims at two abstract scenarios:

• - Adaptive cruise control (ACC): controls the vehicle's throttle/acceleration in relation to the vehicle in front.
• - Traffic Free Steering: Control the steering of the vehicle to stay in lane.

wobei k ∈ ℤ⁺ eine beliebige Anzahl von Datenproben ist, y_acc und ŷ_acc sind Grundwahrheits-Etikette und vorhergesagte Werte von △ Beschleunigung und y_bear und ŷ_bear sind Grundwahrheit-Etikette und vorhergesagte Werte der △ Peilung. Es war ein spezieller Mechanismus erforderlich, um das Problem der Fehlerkaskade während der Testläufe des Modells in der Simulation zu lösen. Dies bedeutete, dass sich winzige Fehler in jedem Einzelbild zu Zuständen addierten, die in den Trainingsdaten für die Fahrspur selten vorkamen, was dazu führte, dass das Modell die Lenkung nicht gut genug kontrollieren konnte, um die Fahrspur zu halten, und das Fahrzeug schließlich aus der Fahrspur geriet. Der Korrekturmechanismus bestand darin, die Trainingsdaten zu filtern, um in jeder Trainingsiteration den Anteil der Situationen zu erhöhen, in denen das Fahrzeug auf beiden Seiten der Fahrspurmitte verschoben war und in denen die △ Peilung so war, dass der Abstand zur Fahrspurmitte verringert wurde.A branched neural network architecture split into two completely separate networks, with no common layers for △ acceleration and △ bearing, was used for training with DFS driving data and OpenDRIVE map data. The network was optimized using the following loss function: $$L=\frac{1}{k}{\displaystyle {\sum }_{i=1}^k\left({\left(y_{acc}^i-{\widehat{y}}_{acc}^i\right)}^2+{\left(y_{bea\mathrm{right}}^i-{\widehat{y}}_{bea\mathrm{right}}^i\right)}^2\right)}$$

where k ∈ ℤ ⁺ is any number of data samples, y _acc and ŷ _acc are ground truth labels and predicted values of △ acceleration, and y _bear and ŷ _bear are ground truth labels and predicted values of △ bearing. A special mechanism was required to solve the error cascade problem during the test runs of the model in the simulation. This meant that tiny errors in each frame added up to conditions that were rare in the lane training data, resulting in the model not being able to control the steering well enough to stay in lane, and the vehicle eventually stalling the lane. The correction mechanism was to filter the training data to increase in each training iteration the proportion of situations where the vehicle was shifted to either side of the lane center and where the △ bearing was such that the distance to the lane center was reduced.

The test results for the lane following module 122, which was trained independently on real traffic data from the DFS, are in the 4 until 7 shown. The corresponding diagrams in 4 and 5 show the error of the lane following module 122 per frame in relation to the reference data set for △ storage and △ acceleration values.

The graph in Fig. 6 shows the distribution of the average lane center deviation of the vehicles in the real data, while the graph in Fig 7 shows the same distribution for the lane following model when run in the simulation environment. The distribution of lane center deviation appears to be higher in the DFS data, possibly due to the positional errors in recording the data, reported to be up to ≈0.5 m. However, the lane following module 122 shows relatively less deviation due to the correction mechanism during training.

The lane change module 123 targeted specific situations where the vehicle would likely change to one of the two adjacent lanes. The model should learn to predict △ bearing and △ acceleration values in such a way that the higher-level decision about the direction of the lane change was implicitly embedded in the lower-level output of △ bearing and △ acceleration values at each frame. The long-term effect of this is the smooth transition to the implicitly decided lane. This process is taken into account in the neural network itself, which is why the term "implicit" decision is used.

wobei k ∈ ℤ⁺ eine beliebige Anzahl von Datenproben ist, y_acc und ŷ_acc sind Grundwahrheits-Etikette und vorhergesagten Werte von △ Beschleunigung und y_bear und ŷ_bear sind Grundwahrheits-Etikette und vorhergesagten Werte der △ Peilung. In Experimenten wurde festgestellt, dass der absolute mittlere quadratische Fehler gemäß Gleichung 2 aufgrund der kleinen Werte der Grundwahrheit △ Peilung in den Daten der Spurwechselszenarien nicht gut konvergieren konnte und daher der mittlere absolute Fehler, wie in Gleichung 3 dargestellt, bevorzugt wurde.The same perceptual vector is used as input to the model, except that the road geometry displacement vector is computed with respect to the centers of neighboring lanes rather than the current lanes. The network in this case is the same branched architecture described in the Lane Follow model and has been optimized with the following loss function: $$L=\frac{1}{k}{\displaystyle {\sum }_{i=1}^k\left(\left|y_{acc}^i-{\widehat{y}}_{acc}^i\right|+\left|y_{bea\mathrm{right}}^i-{\widehat{y}}_{bea\mathrm{right}}^i\right|\right)}$$

where k ∈ ℤ ⁺ is any number of data samples, y _acc and ŷ _acc are ground truth labels and predicted values of △ acceleration, and y _bear and ŷ _bear are ground truth labels and predicted values of △ bearing. In experiments, it was found that due to the small values of the ground truth △ bearing in the data of the lane change scenarios, the absolute mean square error according to Equation 2 could not converge well and therefore the mean absolute error as presented in Equation 3 was preferred.

Table 1 below shows the distribution of lane change directions in the real data and the distribution that the Lane Change 123 model shows during a test run in the simulation. It can be seen that the percentages of corresponding lane changes for both sources are very similar, which can be taken as a rough indication of the similarity of the data on the behavior of the model with real human drivers. Table 1: Percentage of directions selected by the lane change module and real traffic data. Left lane change Right lane change ground truth data 56.2% 43.8% model in the simulation run 58.2% 41.7%

vorherzusagen ein Fahrspurwechsel P(X = LaneChange) zu sein oder das Szenario ${\widehat{SC}}_{LF}^i$

ein Spurwechsel 1 - P(X = LaneChange) erweitert um die Richtung des Fahrspurwechsels zu sein. Ein einzelnes, vollständig verbundenes neuronales Netz wurde verwendet, um das Modell mit den DFS-Daten zu trainieren, wobei es auf die folgende Kostenfunktion optimiert wurde, die als log loss bekannt ist: $$L=\frac{1}{k}{\displaystyle {\sum }_{i=1}^k\left(-y^i\text{log}\left({\widehat{y}}^i\right)+\left(1-y^i\right)\text{log}\left(1-{\widehat{y}}^i\right)\right)}$$

wobei k ∈ ℤ⁺ eine beliebige Anzahl von Datenproben ist und y und ŷ die Grundwahrheits- und vorhergesaten Ausgaben des Modells als Wahrscheinlichkeit, dass es sich bei dem Szenario um einen Fahrspurwechsel handelt (und dementsprechend die Wahrscheinlichkeit des Folgens der Fahrspur als 1 - ŷ ist).The functional classifier 121 is intended to serve as a moderator for the two main modules: follow lane 122 and change lane 123. A subset of perceptions is used as input to this model, in the form of (i) the traffic situation and (ii) information about the ego -Status to the single value probability of the scenario ${\widehat{SC}}_{LC}^i$

to predict a lane change P(X = LaneChange) to be or the scenario ${\widehat{SC}}_{Lf}^i$

a lane change 1 - P(X = LaneChange) extended to be the direction of the lane change. A single, fully connected neural network was used to train the model on the DFS data, optimizing it to the following cost function, known as the log loss: $$L=\frac{1}{k}{\displaystyle {\sum }_{i=1}^k\left(-y^i\text{log}\left({\widehat{y}}^i\right)+\left(1-y^i\right)\text{log}\left(1-{\widehat{y}}^i\right)\right)}$$

where k ∈ ℤ ⁺ any number of data samples and y and ŷ are the ground truth and predicted outputs of the model as the probability that the scenario is a lane change (and accordingly the probability of following the lane as 1 - ŷ ).

und nur in 8.592 Fällen fälschlicherweise die Situationskategorie „Fahrspurwechsel“ ${\widehat{SC}}_{LC}$

voraus. Für das Grundwahrheits-Etikett „Fahrspur wechseln“ sagt das Funktionsklassifikator-Mdell 121 in 35.778 Fällen korrekt die Situationskategorie „Fahrspur wechseln“ ${\widehat{SC}}_{LC}$

und nur in 7.553 Fällen fälschlicherweise die Situationskategorie „Fahrspur folgen“ ${\widehat{SC}}_{LF}$

voraus. Damit weist das Funktionsklassifikator-Mdell 121 einen Präzisionswert von 0,92, einen Recall-Wert von 0,93 und damit einen beachtlichen F1-Wert von 0,925 auf. Tabelle 2: Konfusionsmatrix für die Funktionsklassifizierung im Vergleich zu Referenzdaten Vorhersagemodell Fahrspur folgen Vorhersagemodell Fahrspur wechseln Grundwahrheit-Etikett Fahrspur folgen 96578 8592 Grundwahrheit-Etikett Fahrspur wechseln 7553 35778 Table 2 shows the confusion matrix for the classification of lane following and lane change scenarios by the functional classification model compared to the real traffic data. The numbers in the table represent the respective recorded instances (a total of 148,501 instances) in the real data. For the base truth label "follow lane", the function classifier model 121 according to the invention correctly says the situation category "follow lane" in 96,578 cases. ${\widehat{SC}}_{Lf}$

and only in 8,592 cases incorrectly the situation category "lane change" ${\widehat{SC}}_{LC}$

ahead. For the ground truth label “change lanes”, the function classifier model 121 correctly says the situation category “change lanes” in 35,778 cases ${\widehat{SC}}_{LC}$

and only in 7,553 cases incorrectly the situation category "Follow lane" ${\widehat{SC}}_{Lf}$

ahead. The function classifier model 121 thus has a precision value of 0.92, a recall value of 0.93 and thus a remarkable F1 value of 0.925. Table 2: Confusion matrix for functional classification compared to reference data Follow prediction model lane Change lane predictive model Ground Truth Label Follow Lane 96578 8592 Ground truth label Change lanes 7553 35778

It was found that the general behavior of the E2E decision maker model when run in the simulation environment bears similarity to that of the human drivers found in the real traffic data of the DFS. The diagrams in the 8a) and 8b) show the behavioral trend in maintaining the speed and the distance to the vehicle in front during the lane following scenarios both for the model tested in the simulation and for the real data. The behavioral trend of the data provided by the traffic agent trained according to the invention is similar to the behavioral trend of the natural DFS data.

The final E2E decision module, trained on real traffic data (DFS), was also evaluated in terms of safety compliance related to collisions with surrounding traffic vehicles. The E2E decision-making module was tested in the simulation environment and resulted in only 4 light collisions (low-speed frontal collisions) in a 30-minute drive in a congested environment.

Claims (14)

Computer-implemented training method of a traffic agent for navigating a road vehicle in a simulation environment, characterized in that the method comprises or consists of the following steps: a. Provision of driving data for one or more time windows t _i = [t ₁ , t ₂ , ... t _n ] for one or more road vehicles as ego vehicles, each of which is driven by a human being in a realistic situation on a road, and Provision of map data on the respective street for the given time windows t _i , b. Processing at least part of the driving data and the map data from step a) into one or more corresponding perceptual frames P _i = [p ₁ ,p ₂ ,...p _n ] per given time window t _i , each perceptual frame P _i corresponding perceptual information for ( i) the traffic situation, (ii) information about the state of the ego vehicle and (iii) the road geometry, c. Processing at least part of the driving data and map data from step a) into one or more corresponding ground truth vehicle control frames C _i = [c ₁ , c ₂ , ... c _n ] per given time window t _i , each vehicle control frame C _i longitudinal and lateral Positions of the respective Ego-Fahr contains witness, d. training a decision maker computer model of the traffic agent with the one or more perceptual frames P _i per given time window t _i from step b) as input to the model and with the one or more ground truth vehicle control frames C _i per given time window t _i from step c ) as a label for training the model, where the decider uses one or more neural networks with end-to-end modeling and is configured corresponding vehicle control frames Ĉ _i = [c ₁ , c ₂ , ... c _n ] comprising the longitudinal and predict lateral positions of the respective ego vehicle by matching the predicted vehicle control frames Ĉ _i with the respective ground truth vehicle control frames C _i , where i is any number such that i ∈ [1,2,...n] and where n represents the limit of the driven frames. Training method according to one of the Claims 1 , which also uses the driving data from step a) of the respective ego vehicles for each given time window t _i to binary corresponding basic truth situation categories of “follow the lane” ${\text{SC}}_{Lf}^i=\left[{\text{SC}}_{\text{LF}}^1,{\text{SC}}_{\text{LF}}^2,...{\text{SC}}_{Lf}^n\right]$

= $$

or "change lane" ${\text{SC}}_{LC}^i=\left[{\text{SC}}_{\text{LC}}^1,{\text{SC}}_{\text{LC}}^2,...{\text{SC}}_{LC}^n\right]$

processed and wherein the decision-maker computer model of the traffic agent in step d) comprises i) a follow lane neural network, ii) a lane change neural network and iii) a function classifier neural network, wherein - the one or more perceptual frames P _i each as input be used for the neural networks "follow lane", "change lane" and function classifier", - the one or more ground truth vehicle control frames C _i are used as labels for an independent training of the neural networks for lane follow and change lane by the predicted vehicle control frames Ĉ _i are compared with the respective basic truth vehicle control frames C _i and the respectively applied basic truth situation category ${\text{SC}}_{Lf}^i$

or ${\text{SC}}_{LC}^i$

be used as a label for each given time window t _i to train the neural network function classifier independently, a corresponding situation category "follow lane" ${\widehat{SC}}_{Lf}^i=\left[{\widehat{SC}}_{\text{LF}}^1,{\widehat{SC}}_{\text{LF}}^2,...{\widehat{SC}}_{Lf}^n\right]$

or "change lane" ${\widehat{SC}}_{LC}^i=\left[{\widehat{SC}}_{\text{LC}}^1,{\widehat{SC}}_{\text{LC}}^2,...{\widehat{SC}}_{LC}^n\right]$

by matching the predicted situation category ${\widehat{SC}}_{Lf}^i$

or ${\widehat{SC}}_{LC}^i$

with the respective actual situation category ${\text{SC}}_{Lf}^i$

and ${\text{SC}}_{LC}^i$

to predict, where i is any number such that i ∈ [1,2,...n] and where n represents the limit of frames driven. training procedure claim 1 or 2 , wherein the driving data in step a) for each of the given road vehicles include or consist of one or more status characteristics of the respective ego vehicles for each given time window t _i , preferably including or consisting of longitudinal speed, longitudinal acceleration and position of the respective road vehicle in X- or Y coordinates per given time window t _i . Training method according to one of the Claims 1 until 3 , wherein the map data from step a) contain corresponding road information which comprises or consists of i) the number of lanes of the respective road and ii) the position of the lanes in X and Y coordinates for given time windows t _i . Training method according to one of the Claims 1 until 4 , wherein the traffic situation in step b) includes or consists of six-vehicle neighborhood information, each vehicle shown in the six positions i) the relative distance of the respective vehicle to the ego vehicle and ii) the relative speed of the respective vehicle to the speed of the ego -Vehicle includes or consists of. Training method according to one of the Claims 1 until 5 , wherein the eigenstate information of the respective ego vehicles in step b) comprises or consists of the longitudinal velocity, the longitudinal acceleration and the bearing to the road direction (angular deviation Ad). training procedure claim 6 , where the angular deviation Ad is defined as $$A{\mathrm{i.e}}^i={\theta }_{\mathrm{right}Oa\mathrm{i.e}}^i-{\theta }_{eGO}^i$$

whereby ${\theta }_{\mathrm{right}Oa\mathrm{i.e}}^i\text{and}{\theta }_{eGO}^i$

represent respectively the bearing of the road and the ego vehicle at any given time window t _i , where i is any number such that i ∈ [1,2,...n] and where n is the limit of frames driven. Training method according to one of the Claims 1 until 7 , wherein the road geometry in step b) comprises or consists of a numerical representation of a respective roadway geometry in relation to the ego vehicle, wherein the numerical representation is preferably selected from a circular or semi-circular geometry. training procedure claim 8 , where the circular or semi-circular numerical representation of the respective lane geometry with two lane boundaries is in the form of a vector of displacements D _j to each of the two lane boundaries at any given time window t _i with $$D_j=\left[{\mathrm{i.e}}_1,{\mathrm{i.e}}_2...{\mathrm{i.e}}_n\right]$$

where each entry D _j is part of a sequence of displacement points to the ego vehicle position calculated on the basis of their relative bearings to the ego position at intervals of 1° or more around the circular or semi-circular area in front of and/or behind the ego vehicle and where the length n of the displacement vector D _j represents 1 to 360 for the circular geometry and 1 to 180 for the semi-circular geometry. Training method according to one of the Claims 1 until 9 , wherein the longitudinal and lateral positions of the respective ego vehicles in step c) and step d) include or consist of acceleration and bearing values, preferably include or consist of changes in acceleration and bearing values, which affect the respective ego vehicles in the time window t _i are applied. Computer system training a traffic agent who navigates a road vehicle in a simulation environment, comprising or consisting of one or more processors, a memory device coupled to the one or more processors and a traffic agent as a decision-maker in simulated driving situations using one or more neural networks with end- to-end modeling stored in the storage device and configured to be executed by the one or more processors, characterized in that the traffic agent is configured to execute the computer-implemented training method according to any one of Claims 1 until 10 executes Computer system for simulating a road driving environment in driving situations for one or more vehicles, which comprises or consists of one or more processors, a memory device coupled to the one or more processors and a traffic agent that uses one or more neural networks as decision makers in simulated driving situations, using one or more neural networks with end-to-end modeling stored in the storage device and configured to be executed by the one or more processors, characterized in that the traffic agent is trained according to the computer-implemented training method according to a the Claims 1 until 10 has been trained to predict as an action one or more vehicle control frames Ĉ _i containing longitudinal and lateral positions to be applied to a simulated vehicle in the simulation environment, where i is any number such that i ∈ [1,2,. ..n] and where n represents the limit of the frames driven. Computer system for training a traffic agent according to claim 10 or 11 or for simulating a road traffic environment in a driving situation according to claim 12 wherein three or more neural networks are used, preferably at least part or all of the neural networks are combined in a branched architecture. computer system according to Claim 13 , wherein at least some or all of the neural networks are deep neural networks, wherein preferably at least some or all of the deep neural networks independently of one another comprise or consist of one, two or more layers, each layer independently of one another having a number of neurons in the range of 1 to 512, the number of neurons per layer in the deep neural network also preferably being different.

Images