WO2025219350A1

WO2025219350A1 - Assisted or autonomous driving system, apparatus, computer-readable data carrier, computer program, and method for controlling a vehicle

Info

Publication number: WO2025219350A1
Application number: PCT/EP2025/060292
Authority: WO
Inventors: Mohamed-Khalil Bouzidi; Jörg Reichardt
Original assignee: Continental Automotive Technologies GmbH
Current assignee: Aumovio Germany GmbH
Priority date: 2024-04-19
Filing date: 2025-04-14
Publication date: 2025-10-23
Anticipated expiration: 2026-10-19
Also published as: DE102024203659A1

Abstract

Embodiments of the present disclosure relate to an assisted or autonomous driving system, an apparatus (700), a computer-readable data carrier, a computer program, and a method (100) for controlling a vehicle. The method (100) comprises obtaining (110) multiple trajectory candidates for the object and determining (120) a relevance of the trajectory candidates. Further, the method (100) comprises selecting (130), based on the relevance, one or more of the trajectory candidates for consideration in controlling the vehicle.

Description

Assisted or autonomous driving system, apparatus, computer-readable data carrier, computer program, and method for controlling a vehicle

Embodiments of the present disclosure relate to an assisted or autonomous driving system, an apparatus, a computer-readable data carrier, a computer program, and a method for controlling a vehicle. In particular, embodiments relate to a concept for predicting the motion of traffic participants in the environment and/or controlling the vehicle with respect to the predicted motion.

Motion prediction plays an increasingly important role in various fields of technology, e.g., assisted/autonomous driving, robotics, drones, and more. In some applications, motion prediction as applied for controlling and/or navigating. For this, planning algorithms considering the predicted motion may be used.

Classical planning algorithms optimize control signals to minimize a cost function of the planned trajectories. For comfortable driving in automotive applications, those control signals must not change rapidly or drastically. Planning algorithms must therefore optimize over the longest possible time horizon. To do this, it is proposed to use as input a prediction module that predicts trajectories of surrounding obstacles.

In dynamic environments such as traffic, however, predictions about the state of the environment (about traffic participants in the environment) which are required for planning are uncertain over the entire planning horizon (due to sensor noise, limited sensor range, hidden traffic participants, and unknown traffic participant’s intention). In addition, the probability distributions for a development of a traffic situation are generally multi-modal, i.e. , at the end of the planning horizon, several quite different environmental states may be similarly probable. Since existing approaches do not take this uncertainty into account and only optimize for one of these possible environmental states, they struggle in dense traffic situations.

In so-called “uncertainty-aware” planners, a planning problem is formulated as a Partially Observable Markov Decision Process (POMDP) and solved using a Monte Carlo Tree Search (MCTS). Such sampling-based approaches suffer from the fact that with increasing time horizon and action space, the problem is no longer real-time solvable. For this reason, the action space and time steps are only coarsely sampled, causing suboptimal solutions. These discrete action and state spaces are too coarse for many motion planning problems and cannot deal with highly dynamic environments Furthermore, no safety can be guaranteed (e.g. no collision constraints) with these approaches.

There are some variations of model predictive controllers (MPC), which also take into account the uncertainty. One approach is to use robust MPC methods, such as Min-Max MPC, tube-based MPC are used. These methods consider the possible worst case for optimization and ensures that collisions are avoided even in this case. However, these approaches lead to conservative, overcautious behaviors, which can lead to the so-called “Frozen Robot Problem”. To reduce conservatism, stochastic MPCs (or variants such as Scenario-based MPC) are developed to exclude very unlikely scenarios. They provide a trade-off between conservative behavior and allowing risk to achieve the control objective. For this purpose, ’’chance constraints” are defined, i.e., constraints are defined in a probabilistic way. These methods guarantee that these constraints will be met with a certain probability.

Still, this can be very conservative. The main source of conservatism is that it ignores that new information about the scenario will be gathered in the future that the autonomous vehicle can use, i.e., it does not take into account feedback in its predictions. This concept of future feedback is included in the so-called “Branch MPC/Multi-stage MPC”, which can plan with different predictions of the traffic (e.g., different scenarios for the motion of traffic participants in the environment) by allowing different inputs/strategies for different scenario evolutions.

However, they use either model-based or rule-based methods for predicting the traffic participants behavior which both may lack accuracy and flexibility due rigidity or incompleteness of the underlying model or rulesets (e.g. the proposed rule-based approaches can only take one traffic participant into account), limiting the prediction quality in scenes with complex interaction of heterogeneous agents. The rule-based approaches can only take one traffic participant into account. Furthermore, the rule-based and model-based methods for prediction cannot achieve accurate enough prediction accuracy in scenarios with complex interaction of heterogeneous autonomous dynamic agents. Here learning-based methods excel due to their ability to handle the complex and uncertain nature of other traffic participants behavior by providing multimodal predictions. In the last couple of years many learning-based predictors have been published, that clearly outperform model-based and rule-based predictors. However, in order to be able to integrate these learning-based predictors, several steps are required, for which viable solutions have not yet been proposed.

So, existing approaches may lead to uncomfortable or dangerous driving in uncertain or quickly changing traffic environments e.g., if interaction with traffic participants necessary. As well, they may exhibit too conservative behavior leading to the “frozen robot problem” and may not be able to handle complex interaction of heterogeneous autonomous dynamic agents.

Hence, there may be a demand for an improved concept for controlling the vehicle with respect to the object’s motion.

This demand may be satisfied by the subject matter of the appended independent claims. Complementary dependent claims disclose optional embodiments thereof.

Embodiments provide a method for controlling a vehicle based on the motion of an object in the environment. The method comprises obtaining multiple trajectory candidates for the object and determining a relevance of the trajectory candidates. Further, the method comprises selecting, based on the relevance, one or more of the trajectory candidates for consideration in controlling the vehicle. In this way, e.g., only relevant trajectory candidates are selected. For this, e.g., trajectory candidates are selected if the relevance exceeds a predefined threshold. This kind of selection allows to focus only on important trajectories for the object which may lead to a higher efficiency when controlling the vehicle. In particular, the proposed approach avoids that irrelevant trajectories are considered, thereby consuming extra computing resources.

At the same time, the proposed approach may ensure that realistic traffic scenarios are covered to avoid uncomfortable driving and/or undesired traffic situations.

So, the skilled person will particularly appreciate that embodiments of the proposed approach allows real-time processing and, thus, e.g., controlling the vehicle in real-time.

In practice, e.g., one or more of the trajectory candidates having the highest relevance are selected. In doing so, an adaptive or predefined number of the trajectory candidates having the highest relevance are selected, e.g., the two or three most relevant trajectory candidates. Optionally, a threshold-based selection leading to a varying number of selected trajectory candidates is applied.

In some embodiments, the relevance is indicative of a collision risk and a probability of occurrence for the trajectory candidates. In particular, the relevance may be proportional to the collision risk and the probability of occurrence. So, the relevance may be higher for a higher collision risk and/or a higher occurrence probability. In practice, the relevance may be determined by a function considering the collision risk and the probability of occurrence (also referred to herein as “occurrence probability”). In this way, uncritical and unlikely trajectory candidates may be discarded to save computing resources while critical and likely trajectory candidates are selected in order to consider them when controlling the vehicle. A skilled person having benefit from the present disclosure will appreciate that, alternatively, another relation (exponential, logarithmic, and/or the like) may be applied for the relevance and the occurrence probability and/or collision risk. Also, the occurrence probability and the collision risk may be weighted (differently). In this way, the proposed approach may be adapted to different implementations.

In some embodiments, obtaining the trajectory candidates comprises obtaining multiple optional trajectories for the object, assigning the trajectories to different topology classes based on the topology of the trajectories, and obtaining, for the topology classes, a trajectory representative of a respective topology class to obtain the trajectory candidates for different topology classes. In this way, equivalent and/or similar optional trajectories may be summarized to avoid redundancies and, thus, save computing resources.

In practice, assigning the trajectory may comprise applying uniform visibility deformation (UVD) for grouping trajectories of the same topology class. In doing so, UVD summarizes similar trajectories by mapping them onto a common representation. To this end, it is proposed to check whether two/different trajectories belong to the same group by uniformly discretizing the trajectories, e.g., selecting multiple points on the trajectories. Then, one can connect each of these points of the trajectories and check whether the lines intersect with the ego vehicle. If they intersect with the ego-vehicle they do not belong to the same group and vice versa. To discretize the trajectories uniformly the time may be used as parametrization (e.g., as third dimension) and select equidistant timesteps (see, e.g., Fig. 4, step 410).

In some embodiments, the method further comprises controlling the vehicle based on the selected trajectory candidates. In doing so, the vehicle is maneuvered with respect to the different trajectory candidates indicative of different traffic scenarios. In practice, the vehicle, e.g., is maneuvered comfortably and/or such that it does not collide with the object for the different traffic scenarios. Controlling the vehicle may comprise applying Branch model predictive control (MPC) based on the selected trajectory candidates. In doing so, different branches of the Branch MPC handle one of the selected trajectory candidates. So, e.g., each branch determines a suitable driving behavior for a respective traffic situation in order to prepare for the traffic situations which may occur according to the selected trajectory candidates.

In some embodiments, the method further comprises obtaining a probability distribution for the selected trajectory candidates, determining an overlap of their probability distributions, and adapting or setting a decision postponing time of the Branch MPC based on the overlap. For example, the decision postponing time is longer for a longer overlap of the probability distributions (along their trajectory candidates), as laid out in more detail later.

The skilled person will appreciate that the method is executed iteratively for subsequent time steps. In doing so, an appropriate ego-trajectory may be determined and/or adapted step-by-step.

In some embodiments, controlling the vehicle comprises obtaining multiple predictions for the motion of the object using a learning-based predictor for a current time step as well as respective cost function results for the multiple predictions, obtaining cost function results for optimization results of a previous time step, and selecting a prediction or optimization result based on the cost function results for warmstarting an optimization for determining the object’s motion. In this way, the warmstarting process for determining the object’s motion may be made more efficient. In particular, the proposed solution may avoid that the optimization ends up in local minima which may lead to undesired sub optimal optimization results.

The skilled person will appreciate, that the proposed approach may be applied for any object. In automotive applications, the object may be any object which may occur in traffic, e.g., any kind of traffic participant (another vehicle, a cyclist, a pedestrian, an animal, and/or the like). As well, the skilled person will appreciate that the proposed approach may be applied to any kind of vehicle. So, in context of the present disclosure, the term “vehicle” is to be understood broadly. Accordingly, the vehicle may be any kind of ground vehicle (e.g., car, truck, bus, motorcycle, robot, and/or the like) but also any kind of aircraft (e.g., a drone) and/or any kind of watercraft (e.g., a boat).

It is noted that, in embodiments of the present disclosure, various multimodal, interaction-aware motion predictors may be used for obtaining different scenarios for the object’s motion.

Also, various risk metrices and/or decision functions can be used for selecting the scenarios (trajectory candidates).

Apart from that, the skilled person will appreciate that the proposed method may be executed fully or partially by a computer or any other kind of programmable hardware. So, the proposed method may be a computer-implemented method where steps of the proposed method are executed by a computer or any other kind of programmable hardware.

Accordingly, some embodiments provide a computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out an embodiment of the proposed method.

Such computer program may be stored on any kind of computer-readable data carrier.

So, embodiments of the proposed approach may provide a computer-readable data carrier having stored thereon the proposed computer program.

As well, the proposed approach may be implemented in an apparatus. Accordingly, embodiments of the present disclosure may provide an apparatus. An exemplary apparatus comprises one or more interfaces for communication and a data processing circuit configured to execute an embodiment of the proposed method.

Such apparatus may be implemented in different applications, as mentioned above. In automotive applications, the proposed apparatus, e.g., is implemented in an assisted or autonomous driving system.

So, embodiments of the present disclosure may provide an assisted or autonomous driving system comprising an embodiment of the proposed apparatus.

Further, embodiments are now described with reference to the attached drawings. It should be noted that the embodiments illustrated by the referenced drawings show merely optional embodiments as an example and that the scope of the present disclosure is by no means limited to the embodiments presented:

Brief description of the drawings

Fig. 1 shows a flow chart schematically illustrating an embodiment of a method for controlling a vehicle based on the motion of an object in the environment;

Fig. 2 shows an exemplary traffic scenario to schematically illustrate a use case of the proposed approach;

Fig. 3 shows block diagram schematically illustrating an optional embodiment of the proposed approach;

Fig. 4 shows a block diagram schematically illustrating a selection of trajectory candidates based on a collision risk and an occurrence probability;

Fig. 5 shows an embodiment of a warmstarting process according to the proposed approach; and Fig. 6 schematically illustrates how a decision postponing time may be adapted;

Fig. 7 shows a block diagram schematically illustrating an embodiment of an apparatus according to the proposed approach.

Embodiments of the present disclosure are based on the idea to use a learning-based multimodal predictor (such as the Motion Transformer) to predict multiple possible trajectories of the traffic participants and use a Branch MPC to plan against these multiple possible behaviors of the traffic participants for safe and comfortable autonomous driving functions. The Branch MPC then plans with different predictions of the traffic by allowing different strategies which are constrained to be identical in the first couple of timesteps (so-called “decision postponing time”), i.e. for decision postponing until it is clear which strategy to take.

The disadvantage of the Branch MPC (and equivalents thereof) is that computation time increases drastically with number of scenarios (i.e., trajectory candidates, also referred to herein as “predictions” or “scenarios”) considered in a scenario tree of Branch MPC. Empirical analyses showed that only a limited number of scenarios (typically 2-3 scenarios/predictions) can be considered to still be able to run in real-time. Hence, a strategy is desired to select the most important predictions from the predictor which cover as many possible realizations of the uncertainty as possible.

To address this, the present disclosure proposes a method which provides an appropriate selection of scenarios for efficient and real-time motion planning.

Further aspects and features are now described below in more detail with reference to Fig. 1 . The examples explained below may particularly relate to automotive applications. It is noted that, even though the present disclosure may particularly refer to automotive applications, features and aspects explained in connection with automotive implementations may be also applied to various other applications. Fig. 1 shows a flow chart schematically illustrating an embodiment of a method 100 for controlling a vehicle based on the motion of an object in the environment. As mentioned above, the object may be any kind of object which may occur in traffic, e.g., any kind of traffic participant. For other applications, the object may be any other kind of movable objects.

As can be seen from the flow chart, the method 100 comprises obtaining 110 multiple trajectory candidates for the object. For this, e.g., a multimodal motion predictor is used. In practice, e.g., a Motion transformer is applied to obtain multiple options for the traffic participant’s motion. The trajectory candidates, e.g., indicate different driving behavior and/or paths of the traffic participant. One trajectory candidate, e.g., relates to a scenario where a vehicle in the environment turns right at an intersection, another trajectory candidate relates to a scenario where the vehicle crosses the intersection (without stopping), and still another trajectory candidate relates to a scenario where the vehicle stops at the intersection (e.g., because traffic lights turn red).

In practice, the multimodal motion predictor may obtain even more scenarios. However, embodiments of the present disclosure are based on the finding that some scenarios may be less relevant than others, e.g., because they are unlikely to occur and/or they are on critical (unlikely to cause a collision).

One idea of the proposed approach is, therefore, to only select relevant trajectory candidates for consideration in controlling the vehicle.

Accordingly, the method 100 comprises determining 120 a relevance of the trajectory candidates.

In automotive applications, e.g., it may be important to avoid collisions with the traffic participant. The relevance may, therefore, depend on a collision risk of a respective trajectory candidate. The collision risk may be determined from probability distributions, a predicted velocity, and/or a distance between the vehicle and the respective trajectory candidate and/or based on whether the trajectory candidate crosses a planned path of the vehicle.

Also, the relevance may depend on how likely the traffic participant follows the respective trajectory candidate, i.e. , the probability of occurrence/occurrence probability. The occurrence probability, e.g., corresponds to a prediction probability of the trajectory candidates, i.e., a confidence associated with the trajectory candidates.

Accordingly, a metric and/or function considering the collision risk and/or the occurrence probability may be used for obtaining the relevance. The skilled person will appreciate that, for this, different metrics and/or functions may be used and that the metric and/or function may be adapted to the desired use case.

In practice, the relevance may be higher for higher collision risk and/or a higher occurrence probability.

In other applications (other than automotive applications), the relevance may depend on other parameters.

Further, the method 100 comprises selecting 130, based on the relevance, one or more of the trajectory candidates for consideration in controlling the vehicle. In applications, e.g., the one or more trajectory candidates with the highest relevance are selected. In this way, only the most relevant trajectory candidates are considered in planning the motion vehicle while less relevant trajectory candidates are discarded. In doing so, the motion planning may be more efficient and may be executed in real-time. So, the proposed approach may allow consideration of multiple scenarios and applications for assisted and/or autonomous driving.

For selecting the relevant trajectory candidates, a threshold-based approach may be applied and/or predefined or adaptive number of trajectory candidate with the highest relevance may be selected. In practice, e.g., the number of selected trajectory candidates may be not higher than a number of trajectory candidates which may be processed in real-time. So, the maximum number of selected trajectory candidates may depend on available computing resources. As well, less trajectory candidates may be selected if less trajectory candidates exceed a predefined threshold for the relevance. In this way, it may be ensured that the trajectory candidates may be processed in real-time and avoided that improbable trajectory candidates consume computing resources.

The proposed approach may be applied for controlling a vehicle. So, some embodiments my further comprise controlling the vehicle based on the selected trajectory candidates. The selected trajectory candidates, e.g., then, may be used to maneuver the vehicle in consideration of different traffic scenarios. For this, e.g., Branch MPC or equivalent motion planning algorithms may be used. When using such motion planning algorithm using different so-called “branches” for different scenarios, different branches of the motion planning algorithm may handle different selected trajectory candidates.

Further aspects and details are described below with reference the use case of Fig. 2.

As can be seen, the use case relates to a scenario where an ego-vehicle 210 and a traffic participant 220 (another vehicle) approach an intersection from different directions.

A multimodal motion predictor may be applied to predict different future scenarios for the vehicle 220. According to a first future scenario 230, the vehicle 220 may turn right at the intersection and according to a second future scenario 240 crosses the intersection. As the skilled person will understand, in motion planning, the ego-vehicle 210 may need to consider both future scenarios for a comfortable driving behavior and/or to avoid a collision with the vehicle 220.

A diagram on the right-hand side of the illustration of the explained traffic situation, indicates different speed profiles the ego-vehicle 210 over time. The upper speed profile indicates how the ego-vehicle 210 may ideally behave for the first scenario, the lower speed profile indicates how the ego-vehicle 210 may ideally behave for the second scenario. However, none of the upper and lower speed profile may be appropriate for both future scenarios. Therefore, motion planning algorithms according to the present disclosure (e.g., Branch MPC) may be configured to consider multiple future traffic scenarios, here, the first and the second future scenario. Such motion planning algorithm may determine the middle speed profile providing appropriate driving behavior for both of future scenarios. The proposed selection of trajectory candidates allows to determine the most relevant future scenarios for motion planning in real-time.

As can be seen from Fig. 3, the motion planning may include multiple steps. Fig. 3 shows a block diagram schematically illustrating how the proposed approach may be implemented.

In a first step 310, a (learning-based) predictor may be applied to obtain multiple optional trajectories for the object (e.g., traffic participant), as schematically illustrated in Fig. 4. In doing so, multiple equivalent or similar optional trajectories may be obtained. For example, some of the optional trajectories may indicate a very similar behavior, e.g., multiple optional trajectories may relate to the same future scenario and may (only) slightly differ in the speed and/or path. For example, equivalent optional trajectories may similarly indicate that a traffic participant crosses the intersection but with slightly different speed and/or slightly different paths. So, equivalent optional trajectories may exhibit equivalent or similar topologies. Equivalent optional trajectories may exhibit redundancies may cause extra consumption of computing resources in motion planning. For this reason, the present disclosure suggests summarizing such equivalent trajectories in a (common) trajectory candidate representative of their topology classes (see, e.g., step 410). To this end, step 310/410 may further include assigning the trajectories to different topology classes based on the topology of the trajectories. In doing so, trajectories are assigned to the same topology class if they are geometrically similar, e.g., if (according to an arbitrary metric) their geometric deviation does not exceed a predefined threshold. To this end, e.g., uniform visibility deformation (UVD) is applied for grouping the optional trajectories of the same topology class. In this context, topology classes may be therefore also understood as “UVD classes”.

The underlying idea is that for trajectory candidates of the same UVD class, considering only one scenario is sufficient, for example, the one closest to the ego-vehicle, since the remaining trajectories can be easily maintained with a slightly higher safety margin. For trajectories of different classes, two fundamentally different behaviors of the (autonomous) vehicle may be required (see example in Fig. 1 , where the ego vehicle must brake or accelerate at the intersection depending on the behavior of the traffic participant). Thus, one cannot simply consider one of the two trajectories and incorporate the other by just adding a safety margin, as this would lead to an extremely conservative behavior (see problem of robust and stochastic MPC).

Summarizing optional trajectories, as mentioned above, allows to obtain, for the topology classes, a trajectory (also referred to herein as “joint predictions”) representative of a respective topology class to obtain trajectory candidates for different topology classes.

As well, a collision risk may be obtained per prediction/trajectory candidate (see step 420).

That combines topology-based classification of trajectories and collision risk-based ranking (see above).

Then, (if more trajectory candidates are obtained then may be processed in real-time) in another step 320/430 trajectory candidates are (ranked and) selected based on their relevance according to an embodiment of proposed method. In doing so, only relevant trajectory candidates (also referred to herein as “relevant joint predictions”) are kept while less relevant trajectory candidates may be discarded and/or ignored in motion planning. For this, it is proposed to calculate the collision risk of every trajectory candidate crossing a planned path of the ego vehicle.

If more trajectory candidates cross the planned path of the ego vehicle, the most relevant trajectory candidates are selected from them. To this end, a metric may be used to select the most relevant trajectory candidates from the trajectory candidates for the selected UVD classes. An exemplary decision function for this may be defined as follows: d(c,p) = c + A. ■ p, where c is the collision probability, p is the probability of the prediction, and is a tuning factor for a trade-off between conservatism and performance. This decision function is used to avoid excluding very likely predictions that are unlikely to collide with the autonomous vehicle. For example, let's assume we have two scenarios with a 1 % probability each and a collision risk of 60%. On the other hand, we have another scenario with a 98% probability, but the collision risk is 1 %. If one would only consider trajectories with a high collision risk and exclude the very likely trajectory, one would adopt an excessively conservative approach. Accordingly, the tuning factor may be adapted based on or considering the desired driving behavior.

The steps may be performed iteratively for subsequent time steps. So, the scenario selection approach may be performed in subsequent time steps. This can lead to the selection of new scenarios in each time step, complicating the conventional warmstart strategy of MPC. In some approaches, the optimal solution from the last time step is used as a warmstart for the current time step.

The conventional warmstart strategy using the calculated optimal trajectory of the last time step is assuming little change between the predictions in the current and the last time step. However, due to the consideration of multimodal predictions and scenario selection, it is more common for new scenarios to be introduced where there was no optimal solution in the last time step. So, embodiments of the proposed approach may suggest obtaining multiple predictions for the motion of the object using a learning-based predictor for a current time step as well as respective cost function results for the multiple predictions. Then, cost function results may be obtained for optimization results of a previous time step for selecting a prediction or optimization result based on the cost function results for warmstarting an optimization for determining the object’s motion.

For this, it is proposed to utilize a learning-aided warmstart process (see, e.g., Bouzidi et al., “Learning-Aided Warmstart of Model Predictive Control in Uncertain Fast-Changing Traffic”) and adapt it to the used motion planning algorithm, here, Branch MPC. The learning-based predictor anticipates the behavior of the ego vehicle. The prediction is further refined using a sampling-based approach, and the cost function of the predictions is determined. Subsequently, in step 330, the costs (e.g., indicative of an effort to handle the predictions, e.g., to maneuver the ego-vehicle with respect to the predictions) of predictions are compared with the optimal solutions from the last time step and the warmstart for the respective prediction with the least costs is selected. As can be seen from Fig. 5, for this, non-obstacle related costs (independent from obstacles) and so-called “collision costs” (also referred to herein as “collision avoidance costs”) are determined (see steps 510 and 520). The non-obstacle related costs may include or correspond to costs (arbitrary measure of an effort) for maneuvering the ego-vehicle if the vehicle would (need to) not interact with the object (also referred to herein as “obstacle”).

The collision costs include or correspond to costs which occur when interacting with the object to avoid a collision with it. For example, the closer the object is to the ego-vehicle, the higher are the collision costs.

It should be noted that only the collision avoidance costs may be recalculated multiple times (e.g., for subsequent time steps), as the remaining costs are independent of the prediction for the object (e.g., traffic participants). The Branch MPC optimizes over expected cost. This expected cost is a weighted average of the cost of all considered scenarios. The weights are the probability that the respective scenario occurs. In step 350, this probability is extracted from the scenario selection step where the sum of all probabilities of the respective topology/UVD class is taken.

An important parameter in motion planning, e.g., in Branch MPC, may be the decision postponing time (also referred to in the present context as “control horizon”). This indicates how long, e.g., how many time steps, trajectories of different branches are identical, i.e. , the equality constraints on the control inputs are present. Before a control horizon marking an end of the decision postponing time (in each timestep) an ego action (action of the ego-vehicle) is the same. For timesteps after the control horizon the actions of the ego-vehicle can differ (e.g. one plan or trajectory may let the ego-vehicle accelerate and another one may let the ego-vehicle brake).

The assumption of motion planning algorithms (e.g., Branch MPC) is that the ego-vehicle knows, after the control horizon/decision postponing time, the real/actual trajectory of the object and which strategy to take, i.e., it is a way of passive information gathering. This parameter may also influence how conservative the approach is. If the decision postponing time is too short, a decision for the wrong scenario may be made too early. If decided too late, there is a risk of missing an opportunity, for example, to merge into a promising gap. This decision postponing time has in literature been set as a constant tuning parameter a priori. However, embodiments of the present disclosure are based on the finding that its appropriate length may depend on the scenario/environment, i.e., when enough information is available for a decision. Therefore, a learning-aided adaptive decision postponing is proposed that is recalculated at runtime at every time step.

Further aspects and features in connection with the adaptive decision postponing is laid out in more detail below with reference to Fig. 6.

As can be seen from Fig. 6, to this end, it is determined when the selected trajectory candidates may (likely) be distinguishable. To determine when (e.g., from which time step) the selected scenarios are likely distinguishable, it is proposed to use a distance between the trajectory candidates, e.g., the standard deviation of individual waypoints of the selected trajectory candidates.

For this, embodiments of the present disclosure suggest obtaining a probability distribution for the selected trajectory candidates, determining an overlap of their probability distributions, and adapting or setting a decision postponing time of the Branch MPC based on the overlap.

The skilled person will appreciate that different distance metrics/measures and/or different probability distributions may be used for this.

An exemplary probability distribution is or includes the Gaussian distribution and the distance measure may be indicative of the Bhattacharyya Distance between the probability distributions. So, it is, proposed, e.g., to assess how much Gaussian distributions of two trajectory candidates (see Fig. 6) overlap using the Bhattacharyya Distance (for subsequent timesteps) within the prediction horizon (see step 620). If the probability distributions are distinguishable (see step 630), e.g., if a specific threshold/distance is exceeded, the decision postponing time is set to this particular point in time (where the probability distributions are distinguishable). In practice, the distance may be adapted or set such that a desired trade-off is achieved in terms of conservative and reliable driving behavior.

If more than one object is involved, it is proposed use the previously calculated collision risk to determine initially which participants are relevant for decision postponing. If there are more than two objects, is it proposed to calculate the distinguishability for each object and select the maximum decision postponing time.

The proposed adaptive decision postponing time provides reliable decisions without undesired conservative driving behavior.

In summary, an idea of the proposed approach lies in a novel Framework for Motion

Planning under Uncertainty integrating learning-based Predictor into a motion planning algorithm. The present disclosure introduces a trajectory candidate (scenario) selection strategy as interface between predictor and motion planning algorithm while combining the concept of topology classification and the usage of risk metrices. Furthermore, it is proposed to use a learning-aided warmstart strategy to warmstart changing branches in the motion planning algorithm (e.g., Branch MPC). Moreover, the present disclosure proposes an adaptive decision postponing strategy to adapt the control horizon in runtime.

• The proposed method can leverage the strengths of learning-based multimodal predictors, i.e. , consideration of interactions with multiple traffic participants and the incorporation of more precise predictions into the planning process.

• The proposed method is model agnostic, i.e. the Prediction module can be exchanged by any other predictor which is configured to output multimodal (optional) trajectories, preferably, with respective variance and is interaction-awareness.

• The proposed approach allows a time-efficient realization of Branch MPC due to scenario selection, which reduces the number of considered scenarios. This means that irrelevant scenarios that unnecessarily increase computation time are not included in the scenario tree.

• The use of the same predictor model, which is already employed for predicting traffic participants, for warmstarting saves computational time.

• By estimating the distinguishability of scenarios and determining the control horizon, the comfort and safety of driving are improved.

• The approach enhances the robustness of the planner without significantly increasing conservative behavior. Embodiments of the proposed method may be implemented in a computer program, e.g., a computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the proposed method. The computer program, e.g., is stored and may be retrieved from a computer-readable data carrier having stored thereon said computer program.

As well, the proposed approach may be implemented in an apparatus as laid out in more detail with reference to Fig. 7.

Fig. 7 shows a block diagram schematically illustrating an embodiment of such an apparatus 700. The apparatus comprises one or more interfaces 710 for communication and a data processing circuit 720 configured to execute the proposed method.

In embodiments, the one or more interfaces 710 may comprise wired and/or wireless interfaces for transmitting and/or receiving communication signals in connection with the execution of the proposed concept. In practice, the interfaces, e.g., comprise pins, wires, antennas, and/or the like. As well, the interfaces may comprise means for (analog and/or digital) signal or data processing in connection with the communication, e.g., filters, samples, analog-to-digital converters, signal acquisition and/or reconstruction means as well as signal amplifiers, compressors and/or any encryption/decryption means.

The data processing circuit 720 may correspond to or comprise any type of programable hardware. So, examples of the data processing circuit 720, e.g., comprise a memory, microcontroller, field programable gate arrays, one or more central, and/or graphical processing units. To execute the proposed method, the data processing circuit 720 may be configured to access or retrieve an appropriate computer program for the execution of the proposed method from a memory of the data processing circuit 720 or a separate memory which is communicatively coupled to the data processing circuit 720. In practice, the proposed apparatus may be installed on a vehicle. So, embodiments may also provide a vehicle comprising the proposed apparatus. In implementations, the apparatus, e.g., is part or a component of an assisted or autonomous driving system.

However, in implementations, computing resources for the vehicle may be outsourced to an external server separate from the vehicle. In such implementations, the proposed approach may be also implemented outside of the vehicle.

In the foregoing description, it can be seen that various features are grouped together in examples for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed examples require more features than are expressly recited in each claim. Rather, as the following claims reflect, subject matter may lie in less than all features of a single disclosed example. Thus, the following claims are hereby incorporated into the description, where each claim may stand on its own as a separate example. While each claim may stand on its own as a separate example, it is to be noted that, although a dependent claim may refer in the claims to a specific combination with one or more other claims, other examples may also include a combination of the dependent claim with the subject matter of each other dependent claim or a combination of each feature with other dependent or independent claims. Such combinations are proposed herein unless it is stated that a specific combination is not intended. Furthermore, it is intended to include also features of a claim to any other independent claim even if this claim is not directly made dependent to the independent claim.

Although specific embodiments have been illustrated and described herein, it will be appreciated by those of ordinary skill in the art that a variety of alternate and/or equivalent implementations may be substituted for the specific embodiments shown and described without departing from the scope of the present embodiments. This application is intended to cover any adaptations or variations of the specific embodiments discussed herein. Therefore, it is intended that the embodiments be limited only by the claims and the equivalents thereof.

Claims

Patent claims

1 . A method (100) for controlling a vehicle based on the motion of an object in the environment, the method (100) comprising: obtaining (110) multiple trajectory candidates for the object; determining (120) a relevance of the trajectory candidates; and selecting (130), based on the relevance, one or more of the trajectory candidates for consideration in controlling the vehicle.

2. The method (100) of claim 1 , wherein one or more of the trajectory candidates having the highest relevance are selected.

3. The method (100) of claim 1 or 2, wherein the relevance is indicative of a collision risk and a probability of occurrence for the trajectory candidates.

4. The method (100) of claim 3, wherein the relevance is proportional to the collision risk and the probability of occurrence.

5. The method (100) of any one of the preceding claims, wherein obtaining the trajectory candidates comprises: obtaining multiple optional trajectories for the object; assigning the trajectories to different topology classes based on the topology of the trajectories; and obtaining, for the topology classes, a trajectory representative of a respective topology class to obtain the trajectory candidates for different topology classes.

6. The method (100) of claim 5, wherein assigning the trajectory comprises applying uniform visibility deformation for grouping trajectories of the same topology class.

7. The method (100) of any one of the preceding claims, wherein the method (100) further comprises controlling the vehicle based on the selected trajectory candidates.

8. The method (100) of claim 7, wherein controlling the vehicle comprises applying Branch model predictive control, MPC, based on the selected trajectory candidates, wherein different branches of the Branch MPC handle one of the selected trajectory candidates.

9. The method (100) of claim 8, wherein the method (100) further comprises: obtaining a probability distribution for the selected trajectory candidates; determining an overlap of their probability distributions; and adapting or setting a decision postponing time of the Branch MPC based on the overlap.

10. The method (100) of any one of the preceding claims, wherein the method (100) is executed iteratively for subsequent time steps.

11. The method (100) of claims 10, wherein controlling the vehicle comprises: obtaining multiple predictions for the motion of the object using a learning-based predictor for a current time step as well as respective cost function results for the multiple predictions; obtaining cost function results for optimization results of a previous time step; and selecting a prediction or optimization result based on the cost function results for warmstarting an optimization for determining the object’s motion.

12. A computer program comprising instructions which, when the computer program is executed by a computer, cause the computer to carry out the method (100) of any one of the claims 1 to 11 .

13. A computer-readable data carrier having stored thereon the computer program of claim 12.

14. An apparatus (700) comprising: one or more interfaces (710) for communication; and a data processing circuit (720) configured to execute the method (100) of any one of the claims 1 to 11 .

15. An assisted or autonomous driving system comprising the apparatus (700) of claim 14.