[go: up one dir, main page]

WO2019147939A8 - Detection of hazardous driving using machine learning - Google Patents

Detection of hazardous driving using machine learning Download PDF

Info

Publication number
WO2019147939A8
WO2019147939A8 PCT/US2019/015154 US2019015154W WO2019147939A8 WO 2019147939 A8 WO2019147939 A8 WO 2019147939A8 US 2019015154 W US2019015154 W US 2019015154W WO 2019147939 A8 WO2019147939 A8 WO 2019147939A8
Authority
WO
WIPO (PCT)
Prior art keywords
machine learning
detection
hazardous driving
hazardous
driving system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2019/015154
Other languages
French (fr)
Other versions
WO2019147939A1 (en
Inventor
Philip Shirvani
Richard Bramley
John Montrym
Nirmal Saxena
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Nvidia Corp
Original Assignee
Nvidia Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Nvidia Corp filed Critical Nvidia Corp
Publication of WO2019147939A1 publication Critical patent/WO2019147939A1/en
Publication of WO2019147939A8 publication Critical patent/WO2019147939A8/en
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/0088—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots characterized by the autonomous decision making process, e.g. artificial intelligence, predefined behaviours
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0214—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory in accordance with safety or protection criteria, e.g. avoiding hazardous areas
    • G—PHYSICS
    • G05—CONTROLLING; REGULATING
    • G05D—SYSTEMS FOR CONTROLLING OR REGULATING NON-ELECTRIC VARIABLES
    • G05D1/00—Control of position, course, altitude or attitude of land, water, air or space vehicles, e.g. using automatic pilots
    • G05D1/02—Control of position or course in two dimensions
    • G05D1/021—Control of position or course in two dimensions specially adapted to land vehicles
    • G05D1/0212—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory
    • G05D1/0221—Control of position or course in two dimensions specially adapted to land vehicles with means for defining a desired trajectory involving a learning process
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F18/00—Pattern recognition
    • G06F18/20—Analysing
    • G06F18/24—Classification techniques
    • G06F18/241—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches
    • G06F18/2413—Classification techniques relating to the classification model, e.g. parametric or non-parametric approaches based on distances to training or reference patterns
    • G06F18/24133—Distances to prototypes
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N20/00—Machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0499—Feedforward networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/092—Reinforcement learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/094—Adversarial learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/56—Context or environment of the image exterior to a vehicle by using sensors mounted on the vehicle

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Data Mining & Analysis (AREA)
  • General Health & Medical Sciences (AREA)
  • General Engineering & Computer Science (AREA)
  • Mathematical Physics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Computational Linguistics (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • Medical Informatics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Multimedia (AREA)
  • Aviation & Aerospace Engineering (AREA)
  • Radar, Positioning & Navigation (AREA)
  • Remote Sensing (AREA)
  • Automation & Control Theory (AREA)
  • Databases & Information Systems (AREA)
  • Business, Economics & Management (AREA)
  • Game Theory and Decision Science (AREA)
  • Bioinformatics & Cheminformatics (AREA)
  • Bioinformatics & Computational Biology (AREA)
  • Evolutionary Biology (AREA)
  • Traffic Control Systems (AREA)
  • Braiding, Manufacturing Of Bobbin-Net Or Lace, And Manufacturing Of Nets By Knotting (AREA)
  • Lubricants (AREA)

Abstract

An autonomous driving system could create or exacerbate a hazardous driving situation due to incorrect machine learning, algorithm design, sensor limitations, environmental conditions or other factors. This technology presents solutions that use machine learning to detect when the autonomous driving system is in this state e.g., erratic or reckless driving and other behavior, in order to take remedial action to prevent a hazard such as a collision.
PCT/US2019/015154 2018-01-26 2019-01-25 Detection of hazardous driving using machine learning Ceased WO2019147939A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201862622538P 2018-01-26 2018-01-26
US62/622,538 2018-01-26

Publications (2)

Publication Number Publication Date
WO2019147939A1 WO2019147939A1 (en) 2019-08-01
WO2019147939A8 true WO2019147939A8 (en) 2019-11-14

Family

ID=65409529

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2019/015154 Ceased WO2019147939A1 (en) 2018-01-26 2019-01-25 Detection of hazardous driving using machine learning

Country Status (2)

Country Link
US (2) US11150663B2 (en)
WO (1) WO2019147939A1 (en)

Families Citing this family (44)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US10094308B2 (en) * 2015-09-25 2018-10-09 Cummins, Inc. System, method, and apparatus for improving the performance of an operator of a vehicle
US20190156134A1 (en) 2017-11-20 2019-05-23 Ashok Krishnan Training of vehicles to improve autonomous capabilities
US11042163B2 (en) 2018-01-07 2021-06-22 Nvidia Corporation Guiding vehicles through vehicle maneuvers using machine learning models
WO2019147939A1 (en) * 2018-01-26 2019-08-01 Nvidia Corporation Detection of hazardous driving using machine learning
WO2019152888A1 (en) 2018-02-02 2019-08-08 Nvidia Corporation Safety procedure analysis for obstacle avoidance in autonomous vehicle
CN111133447B (en) 2018-02-18 2024-03-19 辉达公司 Method and system for object detection and detection confidence suitable for autonomous driving
DE112019000122T5 (en) 2018-02-27 2020-06-25 Nvidia Corporation REAL-TIME DETECTION OF TRACKS AND LIMITATIONS BY AUTONOMOUS VEHICLES
WO2019178548A1 (en) 2018-03-15 2019-09-19 Nvidia Corporation Determining drivable free-space for autonomous vehicles
US11080590B2 (en) 2018-03-21 2021-08-03 Nvidia Corporation Stereo depth estimation using deep neural networks
US11966838B2 (en) 2018-06-19 2024-04-23 Nvidia Corporation Behavior-guided path planning in autonomous machine applications
DE102019113114A1 (en) 2018-06-19 2019-12-19 Nvidia Corporation BEHAVIOR-CONTROLLED ROUTE PLANNING IN AUTONOMOUS MACHINE APPLICATIONS
US10733510B2 (en) * 2018-08-24 2020-08-04 Ford Global Technologies, Llc Vehicle adaptive learning
US10831208B2 (en) * 2018-11-01 2020-11-10 Ford Global Technologies, Llc Vehicle neural network processing
US11610115B2 (en) 2018-11-16 2023-03-21 Nvidia Corporation Learning to generate synthetic datasets for training neural networks
US11170299B2 (en) 2018-12-28 2021-11-09 Nvidia Corporation Distance estimation to objects and free-space boundaries in autonomous machine applications
DE112019006468T5 (en) 2018-12-28 2021-10-14 Nvidia Corporation DETECTION OF DISTANCE TO OBSTACLES IN APPLICATIONS WITH AUTONOMOUS MACHINERY
WO2020140049A1 (en) 2018-12-28 2020-07-02 Nvidia Corporation Distance to obstacle detection in autonomous machine applications
US11520345B2 (en) 2019-02-05 2022-12-06 Nvidia Corporation Path perception diversity and redundancy in autonomous machine applications
US11648945B2 (en) 2019-03-11 2023-05-16 Nvidia Corporation Intersection detection and classification in autonomous machine applications
CN113906271B (en) 2019-04-12 2025-05-09 辉达公司 Neural network training using ground truth data augmented with map information for autonomous machine applications
EP3745299A1 (en) * 2019-05-31 2020-12-02 Infineon Technologies AG Neural network device and method using a neural network for sensor fusion
JP7326041B2 (en) * 2019-06-25 2023-08-15 京セラ株式会社 Image processing device, imaging device, moving body, and image processing method
US11698272B2 (en) 2019-08-31 2023-07-11 Nvidia Corporation Map creation and localization for autonomous driving applications
CN113365878B (en) * 2019-09-03 2025-06-06 北京航迹科技有限公司 Redundant structure of autonomous driving system
KR102854806B1 (en) * 2019-09-16 2025-09-05 현대자동차주식회사 Apparatus for controlling behavior of autonomous vehicle and method thereof
US20210078735A1 (en) * 2019-09-18 2021-03-18 Bae Systems Information And Electronic Systems Integration Inc. Satellite threat mitigation by application of reinforcement machine learning in physics based space simulation
JP7078909B2 (en) * 2019-12-19 2022-06-01 トヨタ自動車株式会社 Vehicle control device and computer program for vehicle control
US20220343762A1 (en) * 2019-12-27 2022-10-27 Intel Corporation Quantitative driving evaluation and vehicle safety restrictions
US11592828B2 (en) 2020-01-16 2023-02-28 Nvidia Corporation Using neural networks to perform fault detection in autonomous driving applications
DE102020202540A1 (en) * 2020-02-27 2021-09-02 Psa Automobiles Sa Method for training at least one algorithm for a control unit of a motor vehicle, computer program product and motor vehicle
US11966673B2 (en) 2020-03-13 2024-04-23 Nvidia Corporation Sensor simulation and learning sensor models with generative machine learning methods
CN111553188B (en) * 2020-03-24 2023-06-09 深圳市齐远兴电子有限公司 End-to-end automatic driving vehicle steering control system based on deep learning
FR3110043B1 (en) * 2020-05-07 2022-12-09 Thales Sa Electronic decision support device, method and computer program
US12077190B2 (en) 2020-05-18 2024-09-03 Nvidia Corporation Efficient safety aware path selection and planning for autonomous machine applications
US11594040B2 (en) * 2020-08-05 2023-02-28 Fca Us Llc Multiple resolution deep neural networks for vehicle autonomous driving systems
KR20220031846A (en) * 2020-09-04 2022-03-14 현대자동차주식회사 Production factory unmanned transfer system and method
US11978266B2 (en) 2020-10-21 2024-05-07 Nvidia Corporation Occupant attentiveness and cognitive load monitoring for autonomous and semi-autonomous driving applications
US11731657B2 (en) * 2021-02-02 2023-08-22 Tusimple, Inc. Malicious event detection for autonomous vehicles
US12332614B2 (en) * 2021-03-19 2025-06-17 Nvidia Corporation Combining rule-based and learned sensor fusion for autonomous systems and applications
JP7589642B2 (en) * 2021-05-21 2024-11-26 トヨタ自動車株式会社 Manager, system, control method, control program, and vehicle
US12162510B2 (en) 2021-08-10 2024-12-10 Here Global B.V. Location intelligence for building empathetic driving behavior to enable L5 cars
US12229672B2 (en) * 2021-10-21 2025-02-18 EMC IP Holding Company LLC Detecting domain changes with domain classifiers in autonomous vehicles
DE102022203045A1 (en) * 2022-03-29 2023-10-05 Zf Friedrichshafen Ag Confidence analysis during training of an ANN
US12291198B2 (en) * 2023-03-14 2025-05-06 GM Global Technology Operations LLC Optimal engagement of automated features to assist incapacited drivers

Family Cites Families (26)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US8041483B2 (en) * 1994-05-23 2011-10-18 Automotive Technologies International, Inc. Exterior airbag deployment techniques
US11022982B2 (en) * 2014-03-18 2021-06-01 Transforation Ip Holdings, Llc Optical route examination system and method
DE102004062496A1 (en) * 2004-12-24 2006-07-06 Daimlerchrysler Ag A method of operating a collision avoidance or collision sequence mitigation system of a vehicle and collision avoidance or collision mitigation system
US8428828B2 (en) * 2009-03-05 2013-04-23 GM Global Technology Operations LLC Adaptive control system for automated vehicle applications
DE102012103134A1 (en) * 2011-04-12 2012-10-18 Conti Temic Microelectronic Gmbh A method of warning a driver of a vehicle of backward traffic
US9327693B2 (en) * 2013-04-10 2016-05-03 Magna Electronics Inc. Rear collision avoidance system for vehicle
US9460382B2 (en) 2013-12-23 2016-10-04 Qualcomm Incorporated Neural watchdog
US20160196754A1 (en) * 2015-01-06 2016-07-07 Honeywell International Inc. Airport surface monitoring system with wireless network interface to aircraft surface navigation system
US10397019B2 (en) * 2015-11-16 2019-08-27 Polysync Technologies, Inc. Autonomous vehicle platform and safety architecture
US10962972B2 (en) * 2016-01-05 2021-03-30 Carnegie Mellon University Safety architecture for autonomous vehicles
US9896096B2 (en) * 2016-04-11 2018-02-20 David E. Newman Systems and methods for hazard mitigation
US10789840B2 (en) * 2016-05-09 2020-09-29 Coban Technologies, Inc. Systems, apparatuses and methods for detecting driving behavior and triggering actions based on detected driving behavior
US11468290B2 (en) * 2016-06-30 2022-10-11 Canon Kabushiki Kaisha Information processing apparatus, information processing method, and non-transitory computer-readable storage medium
US12038765B2 (en) * 2016-08-02 2024-07-16 Transportation Ip Holdings, Llc Vehicle control system and method
US20180096601A1 (en) * 2016-10-04 2018-04-05 Toyota Motor Engineering & Manufacturing North America, Inc. Collision alert system
HK1224513A2 (en) * 2016-10-14 2017-08-18 智能3D有限公司 Method for improving the quality of 2d-to-3d automatic conversion by using machine learning
US10241684B2 (en) * 2017-01-12 2019-03-26 Samsung Electronics Co., Ltd System and method for higher order long short-term memory (LSTM) network
US10338594B2 (en) * 2017-03-13 2019-07-02 Nio Usa, Inc. Navigation of autonomous vehicles to enhance safety under one or more fault conditions
US11061402B2 (en) * 2017-11-15 2021-07-13 Uatc, Llc Sparse convolutional neural networks
WO2019147939A1 (en) * 2018-01-26 2019-08-01 Nvidia Corporation Detection of hazardous driving using machine learning
JP7110635B2 (en) * 2018-03-19 2022-08-02 株式会社デンソー Control device
US20190315345A1 (en) * 2018-04-16 2019-10-17 David E. Newman Blind spot potential-hazard avoidance system
US11351988B2 (en) * 2018-07-26 2022-06-07 Byton North America Corporation Use of sound with assisted or autonomous driving
US11059479B2 (en) * 2018-10-08 2021-07-13 Mando Corporation Collision avoidance apparatus and collision avoidance method
US11807227B2 (en) * 2018-11-02 2023-11-07 Intel Corporation Methods and apparatus to generate vehicle warnings
US20220343762A1 (en) * 2019-12-27 2022-10-27 Intel Corporation Quantitative driving evaluation and vehicle safety restrictions

Also Published As

Publication number Publication date
WO2019147939A1 (en) 2019-08-01
US11150663B2 (en) 2021-10-19
US20190235515A1 (en) 2019-08-01
US20220083068A1 (en) 2022-03-17

Similar Documents

Publication Publication Date Title
WO2019147939A8 (en) Detection of hazardous driving using machine learning
EP4610137A3 (en) System and method for providing driving assistance to safely overtake a vehicle
EP4529881A3 (en) Detecting uncontrolled movement
MX2017007899A (en) Vehicle exterior surface object detection.
MX2016013792A (en) Perception-based speed limit estimation and learning.
WO2018047016A3 (en) Event-driven region of interest management
EP4309964A3 (en) Vehicle cleaner system and vehicle cleaner control device
MY172922A (en) Vehicle control apparatus
EP4223200A3 (en) Cleaning robot and controlling method thereof
MX2015005041A (en) Detecting radar blockage based on drive history.
CA2930450C (en) Protective cover for welding display
MX340614B (en) Autonomous vehicle handling annd performance adjustment.
MX384507B (en) CONTROL METHOD AND CONTROL DEVICE FOR AUTOMATIC DRIVING VEHICLE.
EP4035962A3 (en) Automatic driving control device and automatic driving control method
WO2015148140A3 (en) Surgical stapling instrument system
EP2913247A3 (en) Vehicle steering device, failure determination method for vehicle steering device, and control method for steering motor
MX2015016464A (en) Vehicle speed adjustment.
MX2017010725A (en) Post-impact control system.
MX2015004818A (en) Parking assistance for a vehicle.
MX2015014414A (en) Road emergency activation.
MY200524A (en) Travel control method and travel control apparatus
NO20171734A1 (en) Systems and methods for sensing engagement in hazardous rated environments
MX2016003375A (en) Safety pedal obstruction and command intention detection.
IN2014DN06552A (en)
WO2015193038A3 (en) Determining a state of a vehicle and assisting a driver in driving the vehicle

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 19704955

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 19704955

Country of ref document: EP

Kind code of ref document: A1