[go: up one dir, main page]

WO2020161610A3 - Adaptive monitoring of a vehicle using a camera - Google Patents

Adaptive monitoring of a vehicle using a camera Download PDF

Info

Publication number
WO2020161610A3
WO2020161610A3 PCT/IB2020/050862 IB2020050862W WO2020161610A3 WO 2020161610 A3 WO2020161610 A3 WO 2020161610A3 IB 2020050862 W IB2020050862 W IB 2020050862W WO 2020161610 A3 WO2020161610 A3 WO 2020161610A3
Authority
WO
WIPO (PCT)
Prior art keywords
vehicle
camera
monitoring
adaptive monitoring
algorithms
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2020/050862
Other languages
French (fr)
Other versions
WO2020161610A2 (en
Inventor
Ophir Herbst
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Jungo Connectivity Ltd
Original Assignee
Jungo Connectivity Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Jungo Connectivity Ltd filed Critical Jungo Connectivity Ltd
Priority to US17/427,924 priority Critical patent/US11783600B2/en
Publication of WO2020161610A2 publication Critical patent/WO2020161610A2/en
Publication of WO2020161610A3 publication Critical patent/WO2020161610A3/en
Anticipated expiration legal-status Critical
Priority to US18/377,855 priority patent/US12330563B2/en
Priority to US19/211,324 priority patent/US20250276645A1/en
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/20—Analysis of motion
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/70—Determining position or orientation of objects or cameras
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/10—Image acquisition
    • G06V10/12—Details of acquisition arrangements; Constructional details thereof
    • G06V10/14—Optical characteristics of the device performing the acquisition or on the illumination arrangements
    • G06V10/141—Control of illumination
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W40/00—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models
    • B60W40/08—Estimation or calculation of non-directly measurable driving parameters for road vehicle drive control systems not related to the control of a particular sub unit, e.g. by using mathematical models related to drivers or passengers
    • B60W2040/0818—Inactivity or incapacity of driver
    • B60W2040/0827—Inactivity or incapacity of driver due to sleepiness
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/143—Alarm means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W50/00—Details of control systems for road vehicle drive control not related to the control of a particular sub-unit, e.g. process diagnostic or vehicle driver interfaces
    • B60W50/08—Interaction between the driver and the control system
    • B60W50/14—Means for informing the driver, warning the driver or prompting a driver intervention
    • B60W2050/146—Display means
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60W—CONJOINT CONTROL OF VEHICLE SUB-UNITS OF DIFFERENT TYPE OR DIFFERENT FUNCTION; CONTROL SYSTEMS SPECIALLY ADAPTED FOR HYBRID VEHICLES; ROAD VEHICLE DRIVE CONTROL SYSTEMS FOR PURPOSES NOT RELATED TO THE CONTROL OF A PARTICULAR SUB-UNIT
    • B60W2540/00—Input parameters relating to occupants
    • B60W2540/229—Attention level, e.g. attentive to driving, reading or sleeping
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10048—Infrared image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30196—Human being; Person
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30248—Vehicle exterior or interior
    • G06T2207/30268—Vehicle interior

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Automation & Control Theory (AREA)
  • Human Computer Interaction (AREA)
  • Transportation (AREA)
  • General Health & Medical Sciences (AREA)
  • Mechanical Engineering (AREA)
  • Health & Medical Sciences (AREA)
  • Computing Systems (AREA)
  • Databases & Information Systems (AREA)
  • Evolutionary Computation (AREA)
  • Medical Informatics (AREA)
  • Software Systems (AREA)
  • Social Psychology (AREA)
  • Artificial Intelligence (AREA)
  • Psychiatry (AREA)
  • Mathematical Physics (AREA)
  • Traffic Control Systems (AREA)
  • Image Analysis (AREA)
  • Fittings On The Vehicle Exterior For Carrying Loads, And Devices For Holding Or Mounting Articles (AREA)

Abstract

Monitoring of a full cabin of a vehicle (e.g., including driver and passengers) is provided using a single 2D camera and adaptively applying monitoring algorithms based on different signals, based on a person's state as determined from images of the cabin and/or based on a signal from a non-image sensor. Together with logic to decide which algorithms to run based on available compute resources, embodiments of the invention provide efficient monitoring of the vehicle.
PCT/IB2020/050862 2019-02-04 2020-02-04 Adaptive monitoring of a vehicle using a camera Ceased WO2020161610A2 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
US17/427,924 US11783600B2 (en) 2019-02-04 2020-02-04 Adaptive monitoring of a vehicle using a camera
US18/377,855 US12330563B2 (en) 2019-02-04 2023-10-09 Adaptive monitoring of a vehicle using a camera
US19/211,324 US20250276645A1 (en) 2019-02-04 2025-05-19 Adaptive monitoring of a vehicle using a camera

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US201962800574P 2019-02-04 2019-02-04
US62/800,574 2019-02-04

Related Child Applications (2)

Application Number Title Priority Date Filing Date
US17/427,924 A-371-Of-International US11783600B2 (en) 2019-02-04 2020-02-04 Adaptive monitoring of a vehicle using a camera
US18/377,855 Continuation-In-Part US12330563B2 (en) 2019-02-04 2023-10-09 Adaptive monitoring of a vehicle using a camera

Publications (2)

Publication Number Publication Date
WO2020161610A2 WO2020161610A2 (en) 2020-08-13
WO2020161610A3 true WO2020161610A3 (en) 2020-09-24

Family

ID=71948079

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2020/050862 Ceased WO2020161610A2 (en) 2019-02-04 2020-02-04 Adaptive monitoring of a vehicle using a camera

Country Status (2)

Country Link
US (1) US11783600B2 (en)
WO (1) WO2020161610A2 (en)

Families Citing this family (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
DE102018215513A1 (en) * 2018-09-12 2020-03-12 Zf Friedrichshafen Ag Arrangement of TOF sensors for detecting a passenger room of a people mover, evaluation device for perceiving a passenger room of a people mover and perception system for detecting a blocking of a passenger door of a people mover, a number of passengers in the people mover and of positions, body poses and activities of the passengers
US12330563B2 (en) * 2019-02-04 2025-06-17 Jungo Connectivity Ltd. Adaptive monitoring of a vehicle using a camera
US11524690B2 (en) * 2020-02-06 2022-12-13 Micron Technology, Inc. Black box operations controlled by driver mental state evaluation
CN112690794B (en) * 2020-12-30 2022-08-30 北京理工大学深圳汽车研究院(电动车辆国家工程实验室深圳研究院) Driver state detection method, system and device
CN113411502A (en) * 2021-06-29 2021-09-17 展讯半导体(成都)有限公司 Trunk monitoring method and related device
US11872957B2 (en) * 2021-12-21 2024-01-16 Veoneer Us Llc Child presence detection warning suppression
DE102022113271B4 (en) 2022-05-25 2023-12-28 Cariad Se Method for operating a vehicle interior monitoring system, control device, vehicle interior monitoring system and vehicle
US20240005520A1 (en) * 2022-06-30 2024-01-04 Lenovo Global Technology (United States) Inc. Systems and Methods for Predicting Object Location Within Images and for Analyzing the Images in the Predicted Location for Object Tracking
US12136346B2 (en) * 2022-09-08 2024-11-05 Bendix Commercial Vehicle Systems Llc Systems and method for integrating driver facing imaging device with a vehicle blind spot detection system
US20250022313A1 (en) * 2023-07-13 2025-01-16 Torc Robotics, Inc. Systems and methods for autonomous horn activation and kidnapping detection
CN117409191B (en) * 2023-12-12 2024-03-08 优备科技股份有限公司 Fire inspection early warning method based on unmanned aerial vehicle and improved YOLOv8 target detection algorithm

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110037595A1 (en) * 2008-01-02 2011-02-17 Anuj Kapuria Driver state monitoring system
US20140276090A1 (en) * 2011-03-14 2014-09-18 American Vehcular Sciences Llc Driver health and fatigue monitoring system and method using optics
US20150379362A1 (en) * 2013-02-21 2015-12-31 Iee International Electronics & Engineering S.A. Imaging device based occupant monitoring system supporting multiple functions
US9868446B1 (en) * 2012-09-27 2018-01-16 Waymo Llc Cross-validating sensors of an autonomous vehicle
US20180357498A1 (en) * 2017-06-11 2018-12-13 Jungo Connectivity Ltd. System and method for remote monitoring of a human

Family Cites Families (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090011994A1 (en) 2007-07-06 2009-01-08 Bristol-Myers Squibb Company Non-basic melanin concentrating hormone receptor-1 antagonists and methods
KR20160104354A (en) * 2015-02-26 2016-09-05 엘지이노텍 주식회사 Apparatus for Safe Starting of Vehicle
US10357195B2 (en) * 2017-08-01 2019-07-23 Panasonic Intellectual Property Management Co., Ltd. Pupillometry and sensor fusion for monitoring and predicting a vehicle operator's condition

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20110037595A1 (en) * 2008-01-02 2011-02-17 Anuj Kapuria Driver state monitoring system
US20140276090A1 (en) * 2011-03-14 2014-09-18 American Vehcular Sciences Llc Driver health and fatigue monitoring system and method using optics
US9868446B1 (en) * 2012-09-27 2018-01-16 Waymo Llc Cross-validating sensors of an autonomous vehicle
US20150379362A1 (en) * 2013-02-21 2015-12-31 Iee International Electronics & Engineering S.A. Imaging device based occupant monitoring system supporting multiple functions
US20180357498A1 (en) * 2017-06-11 2018-12-13 Jungo Connectivity Ltd. System and method for remote monitoring of a human

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
JIANG ET AL.: "Driver Inattention Monitoring System Based on Machine Vision", INTERNATIONAL INFORMATION INSTITUTE ( TOKYO ). INFORMATION ; KOGANEI, vol. 15, no. 12, December 2012 (2012-12-01), Retrieved from the Internet <URL:https://search.proquest.com/docview/1238180664/fulltext/52D3BAE337A94E04PQ/1?accountid=142944> [retrieved on 20200701] *

Also Published As

Publication number Publication date
US20220130155A1 (en) 2022-04-28
WO2020161610A2 (en) 2020-08-13
US11783600B2 (en) 2023-10-10

Similar Documents

Publication Publication Date Title
WO2020161610A3 (en) Adaptive monitoring of a vehicle using a camera
CN107076850B (en) Obstacle detection device for vehicle
WO2008090451A3 (en) Anti-drowsing device and anti-drowsing method
EP4570608A3 (en) Comfort responsibility sensitivity safety model
ITTO970071A1 (en) INTERNAL SIGNALING DEVICE FOR A VEHICLE.
WO2017079236A3 (en) Internal safety systems for autonomous driven vehicles
WO2015134840A3 (en) Vehicular visual information system and method
MX2016014391A (en) Predicting vehicle movements based on driver body language.
WO2017058961A3 (en) Autonomous vehicle with independent auxiliary control units
WO2007127409A3 (en) Imaging via blood vessels
US20150042797A1 (en) Motor vehicle rear side view display system
TWI557003B (en) Image based intelligent security system for vehicle combined with sensor
WO2021032597A3 (en) Roof module for forming a vehicle roof with an environment sensor
US9682709B2 (en) Method and device for detecting a critical snaking motion of a trailer of a vehicle combination
EP3915248A4 (en) Systems and methods for generating composite depth images based on signals from an inertial sensor
AU2018425454A8 (en) Systems and methods for autonomously backing a vehicle to a trailer
EP4223584A3 (en) Vehicle seat
US20170297489A1 (en) Method and System for Representing Vehicle Surroundings of a Motor Vehicle on a Display Device Located in the Motor Vehicle
SG11201908357RA (en) Vehicle-mounted device, calculation device, and program
EP4485920A3 (en) Imaging system and method
WO2007011976A3 (en) Crash prediction network with visual input for vehicle
US9868422B2 (en) Control apparatus of brake system and method of controlling the same
EP3576073A3 (en) Alarm system for vehicle
CN106080599A (en) A kind of speed limiting system
US20220390606A1 (en) Vehicle-mounted measurement device unit and integrated data generation method in vehicle-mounted measurement device unit

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 20753108

Country of ref document: EP

Kind code of ref document: A2

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 20753108

Country of ref document: EP

Kind code of ref document: A2