DE112019000582B4 - Method, control arrangement and machine learning-based system for autonomous vehicles to act proactively with respect to situations involving an increased risk of a traffic accident - Google Patents

Abstract

A method (600) for generating an artificial intuition functionality capable of acting proactively with respect to situations involving an increased risk of a traffic accident; the method (600) comprising:
Collecting (601) vehicle environment data by means of at least one sensor (110a, 110b, 110c) in a vehicle (100a, 100b) driven by a driver;
Detecting (602) a preventive driver action or reaction that indicates an increased risk of a traffic accident;
Marking (604) the collected (601) vehicle environment data at the time the precautionary driver action or reaction is detected;
Extracting (605) collected (601) vehicle environment data to be used for learning based on the marked (604) vehicle environment data;
generating (606) an artificial intuition functionality capable of acting proactively with respect to situations involving an increased risk of a traffic accident by training a machine learning-based system (200) based on the extracted (605) collected (601) vehicle environment data; and
Implementing (607) the generated (606) artificial intuition functionality in a control arrangement (510) of an autonomous vehicle (500).

Description

technical field

This document discloses a method, a control arrangement and a machine learning based system. More specifically, a method, a control arrangement and a machine learning based system for generating an artificial intuition functionality capable of acting proactively with respect to situations involving an increased risk of a traffic accident are disclosed.

background

The technical development for partially or fully autonomous vehicles is progressing rapidly, and fully autonomous vehicles will soon have a place in our everyday lives. A key to delivering an intelligent and predictive autonomous vehicle is to use algorithms that are good at evaluating the environment in driving situations and proactively driving in a way that avoids difficult and unnecessarily complex situations.

The trend of using machine learning-based systems such as neural networks in autonomous vehicles has increased dramatically over the past decade. Neural networks can be trained to recognize and classify certain patterns or signatures in data and can be used in various types of applications. Some examples of application areas for neural networks in autonomous vehicles include the detection of drivable surfaces, pedestrians, and traffic signs.

A key to success in creating good neural networks is to train the network on large and well-labeled data sets. The training data must be labeled in a precise and accurate way so that the network can learn and be able to distinguish the unique features or patterns in the data so that it is able to achieve the ambitious goal, which may be, for example, recognizing objects, surfaces or other things of interest.

An important factor in generating good neural networks that are able to recognize and classify the desired outcome is to train the networks on large and well-labeled datasets, which for autonomous vehicles can consist of many hours of collected traffic data. Labeling the training data can be a large and tedious task that can take many hours, days, or even years depending on the size of the dataset.

If a network is to be trained to detect dangerous driving situations, the network can use the speed limit, the curvature of the road, the width of the road, and data from environmental sensors such as radar, lidar, and cameras. The dataset can consist of trips taken by many different people in different types of situations, or data collected from one or more test drivers.

Existing solutions are typically based on the concept of detecting the current surrounding traffic situation through sensors on board the vehicle, such as detecting a pedestrian walking in front of the vehicle in the lane, which then triggers an appropriate action, such as braking the vehicle, sounding the horn and/or steering the vehicle away from the pedestrian.

However, a human driver will usually adapt his driving style not only to the actual traffic situation, but also to intuitively perceived information, such as keeping an additional distance from another vehicle that is behaving strangely (or from a pedestrian on the sidewalk who is behaving strangely), even though the vehicle (or pedestrian) does not currently pose any danger. The reason for the "strange" behavior of the other road user may be dizziness, simultaneous use of a mobile phone, consumption of alcohol or drugs, etc., which may later cause an accident involving the other vehicle/pedestrian.

Another example of appropriate driving behavior may be avoiding driving in puddles of water near a pedestrian on the sidewalk. Neither the puddle nor the pedestrian pose a dangerous situation, and yet it is a desired behavior by the vehicle driver. An approaching autonomous vehicle may be surprised by the other vehicle's lateral maneuver.

It would be desirable to find a convenient way to establish a sense of intuition for the autonomous vehicle.

document US 2015 / 0 161 505 A1 discloses a method for programming a neural network computer. Several vehicles are equipped with observational neural networks that correlate driver behavior with observations from a forward-facing camera. The collected data is then combined by the neural network computer.

The document describes a "deep driving", i.e. a neural network that imitates a driver's behaviour based on the observation data collected by the forward-facing camera; i.e. when an obstacle appears in the lane, a measure to avoid an impact is activated, such as braking the vehicle and activating the horn. As explained above, it would be desirable to try to transfer human intuition to the neural network and thereby enable the neural network to be preventive in some situations that do not constitute a danger in themselves, but that can develop into a potentially dangerous situation.

document WO 2017 / 051 120 A1 presents a driver assistance system based on an analysis of sensor detections. The system is trained by recognizing and using the driver's reactions in a traffic situation, such as a change in speed or an eye movement, together with sensor data describing the traffic situation.

The document discloses a driver assistance system, not an autonomous vehicle. Furthermore, the solution is used to provide an emergency braking means to slow down the vehicle if the driver does not react to a dangerous situation. A prediction of a potentially dangerous situation is not made.

document CN 1 06 347 359 A1 discloses a solution that includes collecting driving patterns adopted by a driver when the driver intervenes in driving situations of a partially autonomous vehicle in exceptional cases, as well as driving situation information of the partially autonomous vehicle in driving situations; determining the risk levels of driving risks of the partially autonomous vehicle in the driving situations according to the driving patterns adopted by the driver; breaking down object information of all situation objects in the driving situations from the driving situation information; learning the object information and the corresponding risk levels, recognizing risk objects that are relevant to the risk levels in the situation objects, and determining the correlation relationship between the object information of the risk objects and the risk levels so that they serve as a risk assessment strategy of the autonomous vehicle; recognizing the risk level of the driving risk of the autonomous vehicle through the risk assessment strategy.

This document also seems to handle detection of risk objects in the lane and the appropriate action to be taken by the driver.

document JP 2008 - 238 831 A describes a driver assistance system where an environment external to the vehicle is analyzed by various sensors. The human driver is considered as a teacher, and the driver's operations in response to different triggers of the detected environment external to the vehicle are recorded. Later, when a similar event is detected, a warning can be issued to the driver informing him of the detected situation.

This document does not address autonomous vehicles or implementations of intuitive driving actions.

document US 2017 / 0 096 164 A1 relates to a model for controlling a vehicle, which may include a neural network. A driver's behavior during normal driving is used as training data.

The document does not concern the detection of dangerous situations.

The document US 8 780 195 B1 describes a system for remote-controlled vehicles that collects data using sensors and mimics human instincts.
The document US 2017 / 0 004 401 A1 concerns computer models with artificial intuition and automatic creation of object models based on a response to external actions.

It turns out that further development is needed to implement autonomous vehicles that have intuition and are able to act preemptively before an actually dangerous situation occurs.

Summary

It is therefore an object of this invention to solve at least some of the above problems and to provide an improved autonomous vehicle.

According to a first aspect of the invention, this object is achieved by a method for generating an artificial intuition functionality that is able to act proactively with respect to situations that pose an increased risk for a traffic accident. The method comprises collecting vehicle environment data by means of at least one sensor in a vehicle driven by a driver. The method further comprises detecting a preventive driver action or reaction that indicates an increased risk of a traffic accident. In addition, the method comprises marking the collected vehicle environment data at the time when the precautionary driver action or reaction is detected. The method further comprises extracting collected vehicle environment data to be used for learning based on the marked vehicle environment data. In addition, the method comprises generating an artificial intuition functionality that is able to act proactively with respect to situations that involve an increased risk of a traffic accident by training a machine learning-based system based on the extracted collected vehicle environment data. The method also comprises implementing the generated artificial intuition functionality in a control arrangement of an autonomous vehicle.

According to a second aspect of the invention, this object is achieved by a machine learning-based system configured to generate an artificial intuition functionality capable of acting proactively with respect to situations involving an increased risk of a traffic accident, in which the machine learning-based system is trained on the basis of the method according to the first aspect.

According to a third aspect of the invention, this object is achieved by a control arrangement in an autonomous vehicle which is adapted to implement an artificial intuition functionality capable of acting proactively with respect to situations involving an increased risk of a traffic accident, on the basis of the method according to the method according to the first aspect.

According to a fourth aspect of the invention, this object is achieved by a computer program comprising program code for carrying out a method according to the first aspect when the computer program is executed in a computer.

Thanks to the aspects described, by collecting vehicle environment data with sensors in manned vehicles, detecting preventive driver actions/reactions and linking them with vehicle environment data, it is possible to mark the vehicle environment data at times when a preventive driver action/reaction is detected. This allows the most relevant vehicle environment data to be selected with minimal human interaction.

The machine learning-based system can hereby be provided with relevant training data for developing the artificial intuition functionality. The time required to label the training data is drastically reduced. Once the machine learning-based system has developed the artificial intuition functionality, this functionality can be implemented in a control arrangement of an autonomous vehicle.

This will increase the road safety of the autonomous vehicle by imitating the intuitive preventive behavior of a human driver.

Further advantages and additional new features will become apparent from the detailed description below.

figures

• 1 ein Fahrzeug mit Sensoren gemäß einer Ausführungsform darstellt;
• 2 ein Szenario mit zwei Fahrzeugen und einem auf Maschinenlernen beruhenden System gemäß einer Ausführungsform darstellt;
• 3 ein Szenario mit einem erhöhten Risiko für eine gefährliche Situation schematisch darstellt, wobei der Fahrer des Fahrzeugs eine Positionierung auf einer Fahrspur zu zwei unterschiedlichen Zeitpunkten anpasst;
• 4 einen Fahrzeuginnenraum eines autonomen Fahrzeugs, das einen menschlichen Fahrer hat, und ein auf Maschinenlernen beruhendes System gemäß einer Ausführungsform schematisch darstellt;
• 5 einen Fahrzeuginnenraum eines autonomen Fahrzeugs und ein auf Maschinenlernen beruhendes System gemäß einer Ausführungsform schematisch darstellt;
• 6 ein Ablaufdiagramm ist, dass eine Ausführungsform des Verfahrens darstellt.

Embodiments of the invention will now be described in more detail with reference to the accompanying drawings, of which:

• 1 illustrates a vehicle with sensors according to an embodiment;
• 2 illustrates a scenario with two vehicles and a machine learning based system according to an embodiment;
• 3 schematically represents a scenario with an increased risk of a dangerous situation, where the driver of the vehicle adjusts a positioning in a lane at two different times;
• 4 schematically illustrates a vehicle interior of an autonomous vehicle having a human driver and a machine learning-based system according to an embodiment;
• 5 schematically illustrates a vehicle interior of an autonomous vehicle and a machine learning-based system according to an embodiment;
• 6 is a flow chart illustrating one embodiment of the method.

Detailed description

Embodiments of the invention described herein are embodied as a method, a control arrangement and a machine learning based System that may be practiced in the embodiments described below. However, these embodiments may be illustrated and implemented in many different ways and are not limited to the examples set forth herein; rather, these illustrative examples of embodiments are provided so that this disclosure will be thorough and complete.

Still other objects and features will become apparent from the following detailed description when considered in conjunction with the accompanying drawings. It is to be understood, however, that the drawings are for purposes of illustration only and not as a definition of the limits of the embodiments described herein, for which reference must be made to the appended claims. Furthermore, the drawings are not necessarily drawn to scale unless otherwise indicated; they are intended merely to conceptually illustrate the structures and operations described herein.

1 depicts a vehicle 100a having a human driver traveling on a road. The vehicle 100a includes a number of environmental sensors 110a, 110b, 110c for assisting the driver as he or she moves the vehicle 100a.

The sensors 110a, 110b, 110c may, in various embodiments, include, for example, a camera, a stereo camera, an infrared camera, a video camera, a radar, a lidar, an ultrasonic device, a passive infrared sensor (PIR), a time-of-flight camera, or a similar device. The vehicle 100a may include multiple sensors 110a, 110b, 110c directed in certain directions around the vehicle 100a. However, the majority of the sensors 110a, 110b, 110c may be directed in the direction of travel. The sensors 110a, 110b, 110c may be of the same type or of different types. An advantage of using different types of sensors is that different types of sensors have different capabilities; i.e., a camera may work well in daylight, while a laser sensor may be advantageous in the dark, etc.

The vehicle 100a may include any means of transportation in the broad sense, such as a truck, car, motorcycle, trailer, bus, aircraft, watercraft, spacecraft, or other similar manned conveyances moving, for example, on wheels, rails, in the air, on water, or in similar media. However, the examples of vehicles 100a provided herein are primarily illustrated by road vehicles having wheels.

In order to be able to train a machine learning based system, such as a neural network, to develop an artificial intuition functionality capable of acting proactively regarding situations involving an increased risk of a traffic accident, the behavior, actions and reactions of the driver of the manned vehicle 100a may be observed and this information may be used to tag recorded sensor data obtained from the sensors 110a, 110b, 110c of the vehicle 100a.

A practical way to label large sets of collected sensor data without much manual effort is to use collected information such as a speed profile, a lane positioning of the vehicle 100a, a driver's pulse, a driver's brain activity, a driver's stress level, a driver's eye position, etc. to understand when driving situations are difficult or complex to understand and to reproduce the intuition of average drivers in an autonomous vehicle, which is thereby enabled to act proactively in complex areas in a safe manner.

Typical patterns in data or in the human measurements for detecting the increased awareness of difficult and complex situations and the increased level of defensive driving may include any, some, or all of the following: a detected reduced speed of the manned vehicle 100a, although the allowed speed and the comfortable threshold caused by longitudinal acceleration are not exceeded; a lane positioning of the driver, such as positioning in the middle of the road to counteract difficult situations (i.e., driver actions); an increased heart rate of the human driver indicating an increased level of awareness or an increased stress level; increased brain activity; special patterns in eye movements of the driver, etc. (i.e., driver reactions).

The purpose of collecting data and monitoring the driver is not to imitate the driver's actions, but simply to mark certain traffic scenarios that are particularly interesting in order to train the machine learning based system/neural network.

The trained machine learning-based system(s) use information such as low-level detections from the sensors 110a, 110b, 110c of the vehicle 100a, drivable surfaces by the sensor fusion, camera images, objects reported by the sensor fusion, etc.

After reviewing all collected sensor data related to a driver action/reaction, the collected sensor data can be used to train an algorithm to detect similar situations in real time.

The output of the trained machine learning system/networks may be, for example: a suggested level of awareness for an autonomous vehicle, which can be used to determine the behavior of the entire vehicle; a recommended lane positioning and/or a recommended speed for the autonomous vehicle.

An advantage with the embodiments disclosed herein is that the effort required to gather training data is dramatically reduced. This allows the knowledge and intuition of a human driver to be transferred to an autonomous vehicle. The method can also be used for online and in-vehicle training of a neural network.

2 depicts a scenario in which multiple vehicles 100a, 100b with a respective driver are traveling within an area. Sensor signals are collected from sensors 110a, 110b, 110c in/on the respective vehicles 100a, 100b. These signals may be provided to a machine learning-based system 200, for example via a wireless communication interface via a receiver 205.

The optional wireless communication may be established via a wireless communication interface, such as vehicle-to-vehicle (V2V) communication or vehicle-to-infrastructure (V2I) communication. Sometimes the collective term vehicle-to-everything (V2X) is used.

In some embodiments, communication between vehicles 100a, 100b and machine learning-based system 200 may be performed via V2X communication, for example based on a dedicated short-range communication (DSRC) device. DSRC operates in some embodiments in the 5.9 GHz band with a bandwidth of 75 MHz and an approximate range of 1000 m.

The wireless communication can be performed according to any IEEE standard for wireless vehicular communication, such as a special operating mode of IEEE 802.11 for vehicular networks called Wireless Access in Vehicular Environments (WAVE). IEEE 802.11p is an extension of the 802.11 Wireless LAN medium access layer (MAC) and physical layer (PHY) specifications.

Such a wireless communication interface may include or at least be based on wireless communication technology such as Wi-Fi, Wireless Local Area Network (WLAN), Ultra Mobile Broadband (UMB), Bluetooth (BT), Near Field Communication (NFC), Radio-Frequency Identification (RFID), to name just a few possible examples of wireless communications in some embodiments.

Alternatively, communication may take place via a wireless interface that includes or is at least based on radio access technologies such as 3GPP LTE, LTE-Advanced, E-UTRAN, UMTS, GSM or the like via a wireless communication network, to name just a few possibilities.

However, the data collected in the vehicles 100a, 100b may, for example, be stored locally in the respective vehicles 100a, 100b in respective data storage devices. At a later time, the collected data may be transmitted to the machine learning-based system 200 via a wireless communication interface and/or manually, for example by storing the collected data on a recordable disc or storage device in the vehicles 100a, 100b, which is brought to the machine learning-based system 200 and manually appended.

The machine learning-based system 200 may be connected to or include a database 210 in which information obtained from the vehicles 100a, 100b may be stored.

The machine learning-based system 200 may include an artificial neural network and/or a connectionist system or based thereon. Therefore, the machine learning-based system 200 may be at least somewhat similar to biological neural networks that train animal/human brains. Such systems 200 may learn tasks by considering examples in general without task-specific programming (and thereby show progressively improved performance). For example, in image recognition, the system 200 may learn to recognize images that include a traffic situation with an increased risk of a traffic accident by analyzing example images captured by sensors 110a, 110b, 110c in/on the respective vehicles 100a, 100b that have been manually marked as "potentially dangerous" or "not dangerous" and using the results to recognize potentially dangerous situations in other traffic environments. They do this without prior knowledge of dangerous traffic situations, for example that children playing near the road pose a potential danger, since they may get lost in the game and forget the danger posed by the traffic. Instead, they develop their own set of safety-related characteristics based on the learning material they process.

The artificial neural network can be based on a collection of connected units or nodes called artificial neurons (analogous to biological neurons in an animal brain). Each connection (analogous to a synapse) between artificial neurons can transmit a signal from one to another. The artificial neuron that receives the signal can process it and then transmit signals to artificial neurons connected to it.

In some embodiments of system 200, the signal at a connection between artificial neurons may be a real number, and the output of each artificial neuron may be calculated by a nonlinear function of the sum of their inputs. Artificial neurons and connections may have a weight that adjusts as learning progresses. The weight may increase or decrease the strength of the signal at a connection. Artificial neurons may have a threshold so that only when the cumulative signals exceed the threshold is the signal sent. Artificial neurons may be organized into layers. Different layers may perform different types of transformations on their inputs. Signals move from the first (input) to the last (output) layer, possibly after passing through the layers several times. This allows system 200 to solve problems in a very similar way to how a human brain would, i.e., by studying examples and drawing conclusions.

3 shows an example of a scenario as described above, wherein a vehicle 100a is travelling in a direction of travel 101 on an urban road, wherein obstacles such as trees 320a, 320b, 320c obscure a child 300 playing with a ball 310; this may represent a potentially dangerous situation involving an increased risk of a traffic accident.

In the context of the solutions described herein, the potentially dangerous situation involving an increased risk of a traffic accident may include a situation where no sensor detections of an immediate danger are possible, for example a human or animal suddenly jumping into the lane in front of the vehicle 100a (which is detected by the vehicle sensors and handled according to other vehicle accident avoidance algorithms). Instead, the potentially dangerous situations involving a risk of a traffic accident are areas where additional caution may be appropriate even if no immediate danger is detected. Some examples may be an urban area with hedges, a road near a playground, school, sports field, etc. Some other examples may be a road junction with poor visibility; a bus stop where a bus has stopped (passengers sometimes leave the bus and cross the road in front of the bus and may not be visible to a vehicle 100a, 100b approaching from behind the bus until they are already on the road); and similar traffic situations.

In a situation such as that in 3 As shown, the sensors 110a, 110b, 110c may not detect either the child 300 or the ball 310, and the maximum allowed speed may be 50 km/h, but since there is a risk of children running out from behind the hedge, a human driver of the vehicle 100a would possibly reduce the speed and/or adjust the vehicle position since the probability of such an event is high.

In this case, between a first time t ₁ and a second time t _{2 ,} the driver may adjust the position of the vehicle 100a laterally more toward the center of the road in order to have more reaction time available in case a child/pedestrian/animal runs onto the road from behind the obstacles 320a, 320b, 320c.

Triggers or patterns that may be evaluated in the training data in such a situation may be a reduced speed, lane positioning (i.e. positioning the autonomous vehicle closer to the center of the road to have increased reaction time available) or a higher level of alert level.

As previously mentioned, the collected data regarding driver responses may be collected in the vehicle 100a and later provided to the machine learning based system 200 for analysis.

4 represents an example of a scenario that might occur in any of the 1 , 2 and/or 3 is shown as it can be perceived by the driver of the vehicle 100a.

Environmental sensing data from the sensors 110a, 110b, 110c in/on the vehicle 100a may be recorded either in a data storage device on board the vehicle 100a and later provided to the machine learning based system 200; or it may be passed to the machine learning based system 200 by wireless signal transmission and stored in a database 210 external to the vehicle.

The vehicle 100a may include a driver monitoring sensor 410 and a wireless transmitter 420 configured to transmit information to the machine learning based system via a receiver 250.

The driver monitoring sensor 410 may continuously or at regular time intervals detect a physical parameter of the driver of the vehicle 100a that indicates stressful moments of the driver, such as increased heart rate, increased breathing, increased body temperature, increased sweating, certain brain activity patterns, certain eye movements, tense facial expressions, and/or muscle tension.

Other driver monitoring sensors may determine and detect times when the vehicle speed is lower than the speed limit (possibly within a range) and/or when a vehicle position is adjusted laterally toward the center of the road. The latter may be detected by a steering wheel angle sensor, by a Global Positioning System (GPS) or by a similar spatially based radio navigation system, in combination with map data and/or sensor data, detecting the lateral road boundaries.

This allows a driver action/reaction to be detected that may indicate that the traffic situation in the surrounding area represents a potential danger, even if no immediate danger or obstacles can be detected by the vehicle sensors 110a, 110b, 110c. These driver actions/reactions may be linked to the collected vehicle environment data as detected by the sensors 110a, 110b, 110c and may be forwarded to the machine learning-based system 200.

The machine learning-based system 200 can then use the detected driver actions/reactions to filter out particularly interesting traffic situations to investigate in order to develop the artificial intuition functionality.

In some embodiments, detection and collection of both driver actions (such as a reduction in vehicle speed by more than, for example, 10 km (or another adjustable threshold) below the maximum permitted speed) and driver reactions (such as detection of an increased heart rate of the driver, etc.) may be performed. The presence of both an action and a reaction may be considered a more reliable indication that the vehicle 100a is simultaneously involved in a traffic situation with an increased risk of a traffic accident. The combination of the driver action and the driver reaction may mark the collected vehicle environment data as collected by the sensors 110a, 110b, 110c at the time the preventive driver action and reaction were detected.

The driver can be a selected, particularly experienced driver. This can avoid actions and/or reactions of inexperienced, careless and/or bold drivers.

In some embodiments, information may be collected from multiple vehicles 100a, 100b driven by multiple drivers. This may shorten the learning process because more data from a wider range of scenarios may be collected and used for the learning processes.

The collected sensor data, which have been marked with the detected driver actions/reactions, are then filtered so that traffic scenarios that are associated with the detected driver actions/ reactions are used for the learning process of the machine learning-based system 200.

5 shows an interior view of an autonomous vehicle 500. The autonomous vehicle 500, similar to the manned vehicles 100a, 100b mentioned above, can comprise any means of transport in the broad sense, such as a truck, a car, a motorcycle, a trailer, a bus, an airplane, a watercraft, a spacecraft or other manned means of transport that move, for example, on wheels, rails, in the air or in water or in similar media without an (active) driver. However, the autonomous vehicle 500 does not necessarily have to be a vehicle of the same type as the manned vehicle 100a, 100b.

The autonomous vehicle 500 may include sensors of the same or similar types as the previously described sensors of the manned vehicle 100a, 100b; i.e., a camera, video camera, time-of-flight camera, a laser, a lidar, an IR sensor, etc.

The autonomous vehicle 500 may include a control arrangement 510 configured to implement an artificial intuition functionality as developed by the machine learning-based system 200. In addition, the autonomous vehicle 500 may include a database 520 connected to the control arrangement 510, wherein various rules of the artificial intuition functionality as obtained by the machine learning-based system 200 may be stored.

The artificial intuition functionality may be provided by the system 200 in any manner (wired, wireless).

Further, the autonomous vehicle 500 may include a wireless receiver or transceiver 530. The wireless receiver 530 may be configured to receive information from the machine learning-based system 200 via the wireless communication interface, such as the artificial intuition functionality generated by the machine learning-based system 200 that is capable of acting proactively regarding situations involving an increased risk of a traffic accident.

The control arrangement 510 can hereby implement an artificial intuition functionality as generated and obtained by the machine learning-based system 200. Thus, when a traffic situation is detected by environmental sensors in the autonomous vehicle 500 that is identical to or similar to a traffic situation previously determined by the machine learning-based system 200 on the basis of the information input from the manned vehicles 100a, 100b.

In some alternative embodiments, the current time of day and/or time of year may be determined, and this information may be used to filter situations involving an increased risk of a traffic accident. For example, the urban area that is 3 shown, pose an increased risk of a traffic accident in summer and during the day, but perhaps not in winter or at night.

Furthermore or alternatively, an ambient temperature can be determined. This information can also be used to filter situations that involve an increased risk of a traffic accident. For example, the area in town that is 3 may, in some embodiments, present an increased risk of a traffic accident when the temperature is, for example, above 10 or 15 degrees. The situation may be reversed on a stretch of road near a hill where children regularly play with sledges in winter, for example when the temperature is lower than 0 degrees.

6 illustrates an example of a method 600 according to an embodiment. The flowchart in 6 shows the method 600 for generating an artificial intuition functionality capable of acting proactively regarding situations involving an increased risk of a traffic accident.

Step 601 includes collecting vehicle environment data using at least one sensor 110a, 110b, 110c in a vehicle 100a, 100b driven by a driver.

The vehicle environment data can be collected from multiple vehicles 100a, 100b driven by different drivers.

In still other embodiments, the vehicle environment data may be collected by a dedicated driver in a vehicle 100a, 100b.

Step 602 includes detecting a preventative driver action or response that indicates an increased risk of a traffic crash.

The preventive driver actions or reactions may be detected by multiple vehicles 100a, 100b in some embodiments. those driven by different drivers.

In some embodiments, the preventative driver actions or reactions may be detected by a dedicated driver in a vehicle 100a, 100b using a human machine interface (HMI) to trigger data of interest, for example, a push button.

The preventive driver action or reaction indicating an increased risk of a traffic accident may be detected by a driver observation sensor 410, such as a camera monitoring a driver, a video camera, a heart rate monitor, a blood pressure monitor, a respiration monitor, etc., or a vehicle speed monitor and/or a vehicle displacement detector.

Step 603, which may only be performed in some embodiments, includes determining a time at which the preventative driver action or response is detected 602.

In some embodiments, the artificial intuition functionality may be generated taking into account the particular point in time.

Step 604 includes marking the collected 601 vehicle environment data at the time the preventative driver action or response is detected 602.

Thanks to the labeling, the most relevant sequences of the collected data can be identified and used for learning, which dramatically speeds up the analysis of the collected data.

Step 605 includes extracting the collected 601 vehicle environment data to be used for learning based on the marked 604 vehicle environment data.

A driver who generates multiple detected 602 preventive driver actions or reactions that deviate by more than a threshold level from an average preventive driver action or reaction level may, in some embodiments, be disregarded when extracting collected 601 vehicle environment data to be used for learning.

Step 606 includes generating an artificial intuition functionality capable of acting proactively regarding situations involving an increased risk of a traffic accident by training a machine learning-based system 200 based on the extracted 605 collected 601 vehicle environment data.

The machine learning-based system 200 may be based on a neural network. The machine learning-based system 200 is configured to generate an artificial intuition functionality capable of acting proactively regarding situations involving an increased risk of a traffic accident by training the machine learning-based system 200 based on at least some of the method steps 601-606.

Step 607 includes implementing the generated 606 artificial intuition functionality in a control arrangement 510 of an autonomous vehicle 500.

Implementing the generated 606 artificial intuition functionality may include setting a level of perception of the autonomous vehicle 500.

Implementation of the generated 606 artificial intuition functionality may include recommending a lane positioning or speed of the autonomous vehicle 500.

The control arrangement 510 in the autonomous vehicle 500 is configured to implement an artificial intuition functionality capable of acting proactively with respect to situations involving an increased risk of a traffic accident based on at least some of the method steps 601-607.

The described method steps 601-607 may, in various embodiments, be performed by a computer algorithm, a machine-executable code, a non-transitory computer-readable medium, or by software instructions programmed into suitable programmable logic such as a processor in the control arrangement 510.

The above-mentioned computer program product may, for example, be provided in the form of a data carrier carrying computer program code for performing at least some of the steps 601-607 according to some embodiments when loaded into the one or more processors of the control arrangement 110. The data carrier may, for example, be a hard disk, a CD-ROM disc, a memory stick, an optical storage device, a magnetic storage device or any other suitable medium such as a disc or tape capable of carrying machine-readable data in a non-transitory manner. The computer program product may further be provided as computer program code on a server and remotely downloadable into the control arrangement 510, for example via an Internet connection or an intranet connection.

The terminology used in the description of the embodiments illustrated in the accompanying drawings is not intended to limit the described method 600, the described control arrangement 510, the described computer program and/or the described machine learning-based system 200. Various changes, substitutions and/or modifications may be made without departing from embodiments of the invention as defined in the appended claims.

The term "and/or" as used herein includes any and all combinations of one or more of the associated listed objects. The term "or" as used herein is to be understood as a mathematical OR (OR), i.e., an inclusive disjunction; not a mathematical exclusive OR (XOR), unless expressly stated otherwise. It will further be understood that the terms "includes," "comprises," "including," and/or "comprising" indicate the presence of the recited features, acts, integers, steps, operations, elements, and/or components, but do not exclude the presence or addition of one or more other features, acts, integers, steps, operations, elements, components, and/or groups thereof. A single unit, such as a processor, may perform the functions of multiple objects recited in the claims. The mere fact that certain measures are recited in mutually different dependent claims does not indicate that a combination of these measures cannot be used to advantage. A computer program may be stored/distributed on a suitable medium, such as an optical storage medium or a solid-state storage medium, together with or as part of other hardware, but it may also be distributed in other forms, such as over the Internet or a wired or wireless communications system.

Claims (11)

Method (600) for generating an artificial intuition functionality capable of acting proactively with respect to situations involving an increased risk of a traffic accident; the method (600) comprising: Collecting (601) vehicle environment data by means of at least one sensor (110a, 110b, 110c) in a vehicle (100a, 100b) driven by a driver; Detecting (602) a preventive driver action or reaction indicating an increased risk of a traffic accident; Marking (604) the collected (601) vehicle environment data at the time the preventive driver action or reaction is detected; Extracting (605) collected (601) vehicle environment data to be used for learning based on the marked (604) vehicle environment data; Generating (606) an artificial intuition functionality capable of acting proactively with respect to situations involving an increased risk of a traffic accident by training a machine learning-based system (200) based on the extracted (605) collected (601) vehicle environment data; and implementing (607) the generated (606) artificial intuition functionality in a control arrangement (510) of an autonomous vehicle (500). Procedure (600) according to claim 1 , further comprising: determining (603) a point in time at which the preventive driver action or reaction is recognized (602); and wherein the artificial intuition functionality is generated taking into account the determined (603) point in time. Procedure (600) according to claim 1 or 2 wherein implementing (607) the generated (606) artificial intuition functionality comprises determining a level of perception of the autonomous vehicle (500). Method (600) according to one of the Claims 1 until 3 wherein implementing (607) the generated (606) artificial intuition functionality comprises recommending a lane positioning or a speed of the autonomous vehicle (500). Method (600) according to one of the Claims 1 until 4 , wherein the vehicle environment data are collected (601) and the preventive driver actions or reactions of several vehicles (100a, 100b) driven by different drivers are recognized (602). Method (600) according to one of the Claims 1 until 5 , whereby the vehicle environment data are collected (601) and the preventive driver actions or reactions from an associated driver in an individual vehicle (100a, 100b) are detected (602). Method (600) according to one of the Claims 1 until 6 , wherein a driver who produces multiple detected (602) preventive driver actions or reactions that deviate by more than a threshold degree from an average degree of preventive driver action or reaction is disregarded when extracting (605) collected (601) vehicle environment data to be used for learning. Method (600) according to one of the Claims 1 until 7 , wherein the preventive driver action or reaction indicating the increased risk of a traffic accident is detected by a driver observation sensor (410). A machine learning based system (200) configured to generate an artificial intuition functionality capable of acting proactively with respect to situations involving an increased risk of a traffic accident, in which the machine learning based system (200) is configured to generate an artificial intuition functionality capable of acting proactively with respect to situations involving an increased risk of a traffic accident, in which the machine learning based system (200) is configured to generate an artificial intuition functionality based on a method (600) according to one of the Claims 1 until 8 is trained. Control arrangement (510) in an autonomous vehicle (500) which is designed to implement an artificial intuition functionality which is able to act proactively with regard to situations which involve an increased risk of a traffic accident, on the basis of a method (600) according to one of the Claims 1 until 8 . Computer program comprising program code for carrying out a method (600) according to one of the Claims 1 until 8 when the computer program is executed in a computer.

Images