[go: up one dir, main page]

WO2017182841A1 - Facial movement and gesture sensing side-view mirror - Google Patents

Facial movement and gesture sensing side-view mirror Download PDF

Info

Publication number
WO2017182841A1
WO2017182841A1 PCT/IB2016/052229 IB2016052229W WO2017182841A1 WO 2017182841 A1 WO2017182841 A1 WO 2017182841A1 IB 2016052229 W IB2016052229 W IB 2016052229W WO 2017182841 A1 WO2017182841 A1 WO 2017182841A1
Authority
WO
WIPO (PCT)
Prior art keywords
driver
view mirror
data
facial
hand gesture
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/IB2016/052229
Other languages
French (fr)
Other versions
WO2017182841A8 (en
Inventor
Teik Seng CHONG
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Continental Automotive GmbH
Original Assignee
Continental Automotive GmbH
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Continental Automotive GmbH filed Critical Continental Automotive GmbH
Priority to EP16899319.4A priority Critical patent/EP3445612A4/en
Priority to PCT/IB2016/052229 priority patent/WO2017182841A1/en
Priority to CN201680084622.4A priority patent/CN109153354A/en
Publication of WO2017182841A1 publication Critical patent/WO2017182841A1/en
Anticipated expiration legal-status Critical
Publication of WO2017182841A8 publication Critical patent/WO2017182841A8/en
Ceased legal-status Critical Current

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B60—VEHICLES IN GENERAL
    • B60R—VEHICLES, VEHICLE FITTINGS, OR VEHICLE PARTS, NOT OTHERWISE PROVIDED FOR
    • B60R1/00—Optical viewing arrangements; Real-time viewing arrangements for drivers or passengers using optical image capturing systems, e.g. cameras or video systems specially adapted for use in or on vehicles
    • B60R1/02—Rear-view mirror arrangements
    • B60R1/06—Rear-view mirror arrangements mounted on vehicle exterior
    • B60R1/062—Rear-view mirror arrangements mounted on vehicle exterior with remote control for adjusting position
    • B60R1/07—Rear-view mirror arrangements mounted on vehicle exterior with remote control for adjusting position by electrically powered actuators
    • B60R1/072—Rear-view mirror arrangements mounted on vehicle exterior with remote control for adjusting position by electrically powered actuators for adjusting the mirror relative to its housing
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/011—Arrangements for interaction with the human body, e.g. for user immersion in virtual reality
    • G06F3/012—Head tracking input arrangements
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06F—ELECTRIC DIGITAL DATA PROCESSING
    • G06F3/00—Input arrangements for transferring data to be processed into a form capable of being handled by the computer; Output arrangements for transferring data from processing unit to output unit, e.g. interface arrangements
    • G06F3/01—Input arrangements or combined input and output arrangements for interaction between user and computer
    • G06F3/017—Gesture based interaction, e.g. based on a set of recognized hand gestures
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V20/00—Scenes; Scene-specific elements
    • G06V20/50—Context or environment of the image
    • G06V20/59—Context or environment of the image inside of a vehicle, e.g. relating to seat occupancy, driver state or inner lighting conditions
    • G06V20/597—Recognising the driver's state or behaviour, e.g. attention or drowsiness
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/161—Detection; Localisation; Normalisation
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/20—Movements or behaviour, e.g. gesture recognition
    • G06V40/28—Recognition of hand or arm movements, e.g. recognition of deaf sign language

Definitions

  • the present application relates to an improved vehicle.
  • the application relates to an improved side-view mirror for a vehicle.
  • Motor vehicles often include side-view mirrors in order for drivers of the vehicles to see areas that are behind and to the sides of the vehicle, as well as to see areas that are outside of a peripheral vision of the drivers.
  • the side-view mirror is often placed on an exterior of the vehicle and it often includes an adjustment unit, which allows the driver to remotely change the position of the side-view mirror in the vertical and horizontal direction.
  • the adjustment unit can include cables for manual adjustment or include geared motors for electric adjustment of the side-view mirror.
  • US 5781354A discloses a positioning device, which is intended for use in a housing of a car side-view mirror, in order to turn the housing to a viewing position where the driver is allowed to have a rear view, to a front parking position where the rear side of the housing is generally parallel with the car body, and to a rear parking position where the mirror side of the housing is generally parallel with the car body.
  • the application provides an improved controller unit for one or more side-view mirrors of a vehicle.
  • the vehicle is used for transporting people and goods.
  • the side-view mirror is often placed on the outside of a door the vehicle for allowing its driver to see other vehicles that are behind or that are passing.
  • the controller unit includes a driver facial directional detector module, a driver hand gesture detector module, and a computing processor.
  • the driver facial directional detector module includes a driver facial recognition camera port and a driver facial directional detector.
  • the driver facial recognition camera port is intended for receiving data of one or more images of a driver face from a camera that is directed at a predetermined driver face location.
  • the camera is directed at a predetermined location for taking or capturing one or more images of the driver face.
  • the camera then sends data of the driver face images to the driver facial recognition camera port.
  • the driver facial directional detector determines data of a viewing direction of the driver according to the driver facial image data. In short, the driver facial directional detector determines which direction that the driver is looking or facing based on said image data. As an example, the driver facial directional detector determines whether the driver is looking at the side-view mirror.
  • the driver hand gesture detector module includes a driver hand recognition camera port and a driver hand gesture detector .
  • the driver hand recognition camera port receives data of one or more images a driver hand from a camera is that is directed at a predetermined driver hand location. In other words, the camera is directed at a predetermined location for taking one or more images of the driver hand. The camera later sends data of the driver hand images to the driver hand recognition camera port .
  • the driver hand gesture detector later determines data of a hand gesture of the driver according to the driver hand image data. Put differently, the driver hand gesture detector determines or recognises which gesture that the driver hand is providing based on the driver hand image data.
  • the computing processor then generates an activation signal for an actuator of the vehicle side-view mirror according to the driver face directional data and according to the driver hand gesture data.
  • the computing processor then sends the generated activation signal to the side-view mirror actuator for moving the side-view mirror.
  • the controller unit acts to enhance driving pleasure and to improve driving safety by a providing intuitive interface control for adjusting the vehicle side-view mirror.
  • This side-view mirror is ad justed intuitively by driver hand gesture and driver facial movement, thereby allowing the driver to focus on the road condition, even changing lane, which then leads to reduced ca accidents .
  • the application also provides a further controller unit for a side-view mirror of a vehicle.
  • the side-view mirror controller unit includes a driver facial recognition camera, a driver facial directional detector, a driver hand recognition camera, a driver hand gesture detector, a computing processor, and a side-view mirror actuator.
  • the driver facial recognition camera is used for taking or capturing one or more driver facial images .
  • the driver facial directional then determines a driver face directional data according to data of the at least one driver facial image.
  • the driver hand recognition camera is used for taking one or more driver hand images.
  • the driver hand gesture detector later determines a driver hand gesture data according to data of the driver hand images .
  • the computing processor then generates side-view mirror actuator activation signal according to the driver face directional data and according to the driver hand gesture data. After this, the side-view mirror actuator actuates one or two side- view mirrors according to the side-view mirror actuator activation signal.
  • the computing processor can be further adapted such that, if the computing processor detects that the driver facial image data indicates that the driver is directed at the side-view mirror, the computing processor then generates a side-view mirror actuator activation signal according to the driver hand gesture data. This provides one method for the controller unit to move the side-view mirror.
  • One integrated unit can provide the driver facial directional detector, the driver hand gesture detector, and the computing processor for reducing cost.
  • one integrated unit can provide two parts of a group, which comprises the driver facial directional detector, the driver hand gesture detector, and the computing processor,
  • the application aIso provides a vehicle, which includes one or more side-view mirrors and one of the above a side-view mirror controller unit for actuating or moving the side-view mirror.
  • the application also provides a method of adjusting one or more side-view mirrors of a vehicle.
  • the method includes a step of taking or capturing one or more driver facial images. After this, a driver face directional data is determined according to data of the driver facial images .
  • driver hand images are taken or are captured.
  • a driver hand gesture data is then determined according to data of the driver hand images.
  • a side-view mirror actuator activation signal is then generated according to the driver hand gesture data.
  • the respective side-view mirror is actuated later according to the side-view mirror actuator activation signal
  • Fig. 1 illustrates an improved side-view mirror controller for a vehicle
  • Fig. 2 illustrates a driver facial recognition camera of the side-view mirror controller of Fig. 1,
  • Fig. 3 illustrates a driver hand recognition camera of the side-view mirror controller of Fig. 1 with a first hand gesture
  • Fig. 4 illustrates the driver hand recognition camera of
  • Fig. 5 illustrates the driver hand recognition camera of
  • Fig. 3 with a third hand gesture
  • Fig. 6 illustrates the driver hand recognition camera of Fig. 3 with a fourth gesture.
  • Fig. 1 shows a side-view mirror controller 10 for a vehicle.
  • the side-view mirror controller 10 includes a driver facial directional detector module 13, a driver hand gesture detector module 16, a computing processor 19, and a side-view mirror actuator 22.
  • the facial directional detector module 13, the driver hand gesture detector module 16, and the side-view mirror actuator 22 are electrically connected to the processor 19.
  • the driver facial directional detector module 13 includes a driver facial recognition camera 25 and a driver facial directional detector 27, which is electrically connected to the driver facial recognition camera 25.
  • the driver facial recognition camera 25 is placed inside the vehicle and is directed at a predetermined driver face direction.
  • the driver hand gesture detector module 16 includes a driver hand recognition camera 30 and a driver hand gesture detector 33, which is electrically connected to the driver hand recognition camera 30.
  • the driver hand recognition camera 30 is positioned inside the vehicle and is directed at a predetermined driver hand position.
  • the side-view mirror actuator 22 is connected to side-view mirrors 35 of the vehicle.
  • one integrated computing processor unit acts as the driver facial directional detector module 13, the driver hand gesture detector module 16, and the computing processor 19.
  • one integrated computing processor unit serves as two parts of a group of parts that comprises the driver facial directional detector module 13, the driver hand gesture detector module 16, and the computing processor 19.
  • one integrated computing processor unit can serves as the driver facial directional detector module 13 and the driver hand gesture detector module 16, and the computing processor 19, or the driver hand gesture detector module 16 and the computing processor 19, or the driver facial directional detector module 13 and the computing processor 19.
  • a driver sits in the vehicle to operate the vehicle for transporting people or goods.
  • the face of the driver is placed in the predetermined driver face position .
  • the driver facial recognition camera 25 is used for taking digital images of a face of the driver and for sending data the driver facial images to the driver facial directional de tector 27, as illustrated in Fig. 2.
  • the driver facial directional detector 27 determines a direc tion of the driver face according to the received driver fac al image data.
  • the driver facial direction refers to a direc tion that the driver is viewing or looking.
  • the driver facia directional detector 27 determines a direction, which eyes o the driver of viewing, according to the received driver faci image data .
  • the driver facial directional detector 27 then sends data of the determined driver face direction to the processor 19.
  • the driver hand recognition camera 30 is used for taking digital images of a hand of the driver and for sending data of the driver hand images to the driver hand gesture de- tector 33.
  • the driver hand gesture detector 33 determines a hand gesture of the driver according to the received driver hand image data.
  • the gesture relates to a movement of the hand while the hand is placed in a specific state or configuration.
  • the hand state is also called a hand shape. As an example, in one hand state, one finger is extended while the other four fingers are bent. In another hand state, two fingers are extended while the other three fingers are bent.
  • the different hand states and different hand movement relates to or defines different gestures .
  • the driver hand gesture detector 33 then sends data of the determined driver hand gesture to the processor 19.
  • the processor 19 later generates an electrical side-view mir- ror activation signal according to the driver face directional data and to the driver hand gesture data. After this, the pro- cessor 19 transmits the side-view mirror activation signal to the side-view mirror actuator 22.
  • the side-view mirror actuator 22 then moves or actuates the respective side-view mirror 35 according to the received side- view mirror activation signal.
  • the driver extends all fingers of his hand.
  • the driver then waves the extended fingers for moving the side-view mirror 35 in order for the side-view mirror 35 to some desired display objects.
  • the waving refers to the driver positioning his extended fingers in front of the driver hand recognition camera 30 in a first position, without essentially moving the extended fingers for about 1 second, in order to for the driver hand gesture detector 33 to take notice of the extended fingers. After this, the driver then moves the extended fingers to a second position to complete the gesture. rs to
  • the driver waves the extended fingers to the right of the driver for moving the side-view mirror 35 in order for the side-view mirror 35 to display objects, which are placed towards the right, as illustrated in Fig. 4.
  • the driver waves the extended fingers upwards for moving the side-view mirror 35 in order for the side-view mirror 35 to display objects, which are placed upwards, as illustrated in Fig. 5.
  • the driver waves the extended fingers downwards for moving the side-view mirror 35 in order for the side-view mirror 35 to display objects, which are placed downwards, as illustrated in Fig. 6.
  • a method of operating the side-view mirror controller 10 is described below.
  • the driver looks at a desired side-view mirror 35 of the vehi cle. In other words, the face of the driver is directed at th side-view mirror 35.
  • the driver facial recognition camera 25 takes digital images of the face of the driver and later sends data of the driver facial images to the driver facial directional detector 27.
  • the driver facial directional detector 27 determines a direction of the driver face according to the received driver facial image data.
  • the driver facial directional detector 27 then sends data of the determined driver face direction to the processor 19.
  • the driver later places his hand at the predetermined driver hand position and makes a gesture with his hand.
  • the driver hand recognition camera 30 then takes digital imag es of the driver hand and sends data of the driver hand image to the driver hand gesture detector 33.
  • the driver hand gesture detector 33 later determines a driver hand gesture according to the received driver hand image data.
  • the driver hand gesture detector 33 afterward sends data of the determined driver hand gesture to the processor 19.
  • the processor 19 detects that the determined driver face directional data indicates that the driver is looking at the respective side-view mirror 35, the processor 19 later generates an electrical side-view mirror activation signal according the driver face directional data and to the driver hand gesture data. In other words, the processor 19 generates the electrical side-view mirror activation signal for the side- view mirror 35, when the side-view mirror 35 is being looked at by the driver.
  • the processor 19 afterward transmits the side-view mirror activation signal to the side-view mirror actuator 22.
  • the side-view mirror actuator 22 moves the re spective side-view mirror 35 according to the received side- view mirror activation signal.
  • a camera module to capture facial movements of the driver for detecting drowsiness or sleepiness of the driver can be adapted to act as the driver facial directional detector module 13.
  • a hand gesture module for electronic control purposes of the vehicle can be adapted to act the driver hand gesture detector module 16.
  • One example of the electronic con trol is an entertainment system with a hand gesture human interface unit .
  • this side-view mirror controller 10 of Fig. 1 can be adapted from features of the vehicle with some or little additional material cost.
  • the adaption includes development of software programs for th processor 19 of the side-view mirror controller 10 of Fig. 1.
  • This side-view mirror controller 10 provides several benefits It enhances driving pleasure and also improves driving safety by a providing intuitive interface control for adjusting the side-view mirror 35 of the vehicle.
  • This side-view mirror 35 can be adjusted intuitively by hand gesture and facial movement, thereby allowing the driver to focus on the road condition, even changing lane, which then leads to reduced car accidents .
  • a controller unit for at least one side-view mirror of a vehicle comprising
  • a driver facial directional detector module that comprises
  • a driver hand gesture detector module that comprises a driver hand recognition camera port for receiving driver hand image data
  • a driver hand gesture detector for determining a driver hand gesture data according to the driver hand image data
  • a computing processor being adapted for generating an activation signal for side-view mirror actuator according to the driver face directional data and to the driver hand gesture data and
  • a controller unit for a side-view mirror of a vehicle comprising
  • a driver facial recognition camera for taking at least one driver facial image
  • a driver facial directional detector for determining a driver face directional data according to data of the at least one driver facial image
  • a driver hand recognition camera for taking at least one driver hand image
  • a driver hand gesture detector for determining a driver hand gesture data according to data of the at least one driver hand image
  • a computing processor being adapted for generating a side-view mirror actuator activation signal according to the driver face directional data and to the driver hand gesture data
  • a side-view mirror actuator for actuating at least one side-view mirror according to the side-view mirror actuator activation signal.
  • the controller unit according to item 1 or 2 , wherein the computing processor is adapted, if the computing processor detects that the driver facial image data indicates that the driver is directed at the side-view mirror, for generating a side-view mirror actuator activation signal according to the driver hand gesture data.
  • the computing processor is adapted, if the computing processor detects that the driver facial image data indicates that the driver is directed at the side-view mirror, for generating a side-view mirror actuator activation signal according to the driver hand gesture data.
  • the driver facial directional detector, the driver hand gesture detector, and the computing processor are provid ed by one integrated unit .
  • a vehicle comprising
  • controller unit according to one of the above- mentioned items for actuating the at least one side-view mirror .
  • a method of adjusting at least one side-view mirror of a vehicle comprising

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Physics & Mathematics (AREA)
  • General Engineering & Computer Science (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Health & Medical Sciences (AREA)
  • General Health & Medical Sciences (AREA)
  • Mechanical Engineering (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Psychiatry (AREA)
  • Social Psychology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Image Analysis (AREA)
  • User Interface Of Digital Computer (AREA)
  • Image Processing (AREA)

Abstract

The application provides a controller unit for a vehicle side view mirror. The controller unit includes a driver facial directional detector module, a driver hand gesture detector module, and a computing processor. The facial directional detector module includes a driver facial recognition camera port for receiving image data of a drive face, and a driver facial directional detector for determining directional data of the face according to the facial image data. The gesture detector module includes a driver hand recognition camera port for receiving image data of a driver hand, and a driver hand gesture detector for determining driver hand gesture data according t the hand image data. The computing processor is adapted for generating an activation signal for a vehicle side-view mirror actuator according to the driver face directional data and ac cording to the hand gesture data. The computing processor the sends the actuator activation signal to the mirror actuator.

Description

FACIAL MOVEMENT AND GESTURE SENSING SIDE-VIEW MIRROR
The present application relates to an improved vehicle. In particular, the application relates to an improved side-view mirror for a vehicle.
Motor vehicles often include side-view mirrors in order for drivers of the vehicles to see areas that are behind and to the sides of the vehicle, as well as to see areas that are outside of a peripheral vision of the drivers. The side-view mirror is often placed on an exterior of the vehicle and it often includes an adjustment unit, which allows the driver to remotely change the position of the side-view mirror in the vertical and horizontal direction. The adjustment unit can include cables for manual adjustment or include geared motors for electric adjustment of the side-view mirror.
US 5781354A discloses a positioning device, which is intended for use in a housing of a car side-view mirror, in order to turn the housing to a viewing position where the driver is allowed to have a rear view, to a front parking position where the rear side of the housing is generally parallel with the car body, and to a rear parking position where the mirror side of the housing is generally parallel with the car body.
It is an object of this application to provide an improved vehicle .
The application provides an improved controller unit for one or more side-view mirrors of a vehicle.
The vehicle is used for transporting people and goods. The side-view mirror is often placed on the outside of a door the vehicle for allowing its driver to see other vehicles that are behind or that are passing.
The controller unit includes a driver facial directional detector module, a driver hand gesture detector module, and a computing processor.
In detail, the driver facial directional detector module includes a driver facial recognition camera port and a driver facial directional detector.
The driver facial recognition camera port is intended for receiving data of one or more images of a driver face from a camera that is directed at a predetermined driver face location. In other words, the camera is directed at a predetermined location for taking or capturing one or more images of the driver face. The camera then sends data of the driver face images to the driver facial recognition camera port.
The driver facial directional detector then determines data of a viewing direction of the driver according to the driver facial image data. In short, the driver facial directional detector determines which direction that the driver is looking or facing based on said image data. As an example, the driver facial directional detector determines whether the driver is looking at the side-view mirror.
Similarly, the driver hand gesture detector module includes a driver hand recognition camera port and a driver hand gesture detector .
The driver hand recognition camera port receives data of one or more images a driver hand from a camera is that is directed at a predetermined driver hand location. In other words, the camera is directed at a predetermined location for taking one or more images of the driver hand. The camera later sends data of the driver hand images to the driver hand recognition camera port .
The driver hand gesture detector later determines data of a hand gesture of the driver according to the driver hand image data. Put differently, the driver hand gesture detector determines or recognises which gesture that the driver hand is providing based on the driver hand image data.
The computing processor then generates an activation signal for an actuator of the vehicle side-view mirror according to the driver face directional data and according to the driver hand gesture data. The computing processor then sends the generated activation signal to the side-view mirror actuator for moving the side-view mirror.
This application provides several benefits . The controller unit acts to enhance driving pleasure and to improve driving safety by a providing intuitive interface control for adjusting the vehicle side-view mirror. This side-view mirror is ad justed intuitively by driver hand gesture and driver facial movement, thereby allowing the driver to focus on the road condition, even changing lane, which then leads to reduced ca accidents .
The application also provides a further controller unit for a side-view mirror of a vehicle.
The side-view mirror controller unit includes a driver facial recognition camera, a driver facial directional detector, a driver hand recognition camera, a driver hand gesture detector, a computing processor, and a side-view mirror actuator. The driver facial recognition camera is used for taking or capturing one or more driver facial images . The driver facial directional then determines a driver face directional data according to data of the at least one driver facial image. In similar manner, the driver hand recognition camera is used for taking one or more driver hand images. The driver hand gesture detector later determines a driver hand gesture data according to data of the driver hand images .
The computing processor then generates side-view mirror actuator activation signal according to the driver face directional data and according to the driver hand gesture data. After this, the side-view mirror actuator actuates one or two side- view mirrors according to the side-view mirror actuator activation signal.
The computing processor can be further adapted such that, if the computing processor detects that the driver facial image data indicates that the driver is directed at the side-view mirror, the computing processor then generates a side-view mirror actuator activation signal according to the driver hand gesture data. This provides one method for the controller unit to move the side-view mirror.
One integrated unit can provide the driver facial directional detector, the driver hand gesture detector, and the computing processor for reducing cost.
Alternatively, one integrated unit can provide two parts of a group, which comprises the driver facial directional detector, the driver hand gesture detector, and the computing processor, The application aIso provides a vehicle, which includes one or more side-view mirrors and one of the above a side-view mirror controller unit for actuating or moving the side-view mirror.
The application also provides a method of adjusting one or more side-view mirrors of a vehicle.
The method includes a step of taking or capturing one or more driver facial images. After this, a driver face directional data is determined according to data of the driver facial images .
Similarly, one or more driver hand images are taken or are captured. A driver hand gesture data is then determined according to data of the driver hand images.
If the driver face directional data indicates that a face of a driver is viewing a side-view mirror, a side-view mirror actuator activation signal is then generated according to the driver hand gesture data. The respective side-view mirror is actuated later according to the side-view mirror actuator activation signal
Fig. 1 illustrates an improved side-view mirror controller for a vehicle,
Fig. 2 illustrates a driver facial recognition camera of the side-view mirror controller of Fig. 1,
Fig. 3 illustrates a driver hand recognition camera of the side-view mirror controller of Fig. 1 with a first hand gesture,
Fig. 4 illustrates the driver hand recognition camera of
Fig. 3 with a second hand gesture,
Fig. 5 illustrates the driver hand recognition camera of
Fig. 3 with a third hand gesture, and Fig. 6 illustrates the driver hand recognition camera of Fig. 3 with a fourth gesture.
In the following description, details are provided to describe embodiments of the application. It shall be apparent to one skilled in the art, however, that the embodiments may be practiced without such details .
Some parts of the embodiment have similar parts. The similar parts may have the same names or similar part numbers. The description of one similar part also applies by reference to another similar parts, where appropriate, thereby reducing repetition of text without limiting the disclosure.
Fig. 1 shows a side-view mirror controller 10 for a vehicle.
The side-view mirror controller 10 includes a driver facial directional detector module 13, a driver hand gesture detector module 16, a computing processor 19, and a side-view mirror actuator 22. The facial directional detector module 13, the driver hand gesture detector module 16, and the side-view mirror actuator 22 are electrically connected to the processor 19.
The driver facial directional detector module 13 includes a driver facial recognition camera 25 and a driver facial directional detector 27, which is electrically connected to the driver facial recognition camera 25. The driver facial recognition camera 25 is placed inside the vehicle and is directed at a predetermined driver face direction.
Similarly, the driver hand gesture detector module 16 includes a driver hand recognition camera 30 and a driver hand gesture detector 33, which is electrically connected to the driver hand recognition camera 30. The driver hand recognition camera 30 is positioned inside the vehicle and is directed at a predetermined driver hand position.
The side-view mirror actuator 22 is connected to side-view mirrors 35 of the vehicle.
In one implementation, one integrated computing processor unit acts as the driver facial directional detector module 13, the driver hand gesture detector module 16, and the computing processor 19.
In another implementation, one integrated computing processor unit serves as two parts of a group of parts that comprises the driver facial directional detector module 13, the driver hand gesture detector module 16, and the computing processor 19. In other words, one integrated computing processor unit can serves as the driver facial directional detector module 13 and the driver hand gesture detector module 16, and the computing processor 19, or the driver hand gesture detector module 16 and the computing processor 19, or the driver facial directional detector module 13 and the computing processor 19.
In use, a driver sits in the vehicle to operate the vehicle for transporting people or goods. In the seated position, the face of the driver is placed in the predetermined driver face position .
The driver facial recognition camera 25 is used for taking digital images of a face of the driver and for sending data the driver facial images to the driver facial directional de tector 27, as illustrated in Fig. 2. The driver facial directional detector 27 determines a direc tion of the driver face according to the received driver fac al image data. The driver facial direction refers to a direc tion that the driver is viewing or looking. The driver facia directional detector 27 determines a direction, which eyes o the driver of viewing, according to the received driver faci image data .
The driver facial directional detector 27 then sends data of the determined driver face direction to the processor 19.
Similarly, the driver hand recognition camera 30 is used for taking digital images of a hand of the driver and for sending data of the driver hand images to the driver hand gesture de- tector 33.
The driver hand gesture detector 33 determines a hand gesture of the driver according to the received driver hand image data. The gesture relates to a movement of the hand while the hand is placed in a specific state or configuration. The hand state is also called a hand shape. As an example, in one hand state, one finger is extended while the other four fingers are bent. In another hand state, two fingers are extended while the other three fingers are bent. The different hand states and different hand movement relates to or defines different gestures .
The driver hand gesture detector 33 then sends data of the determined driver hand gesture to the processor 19.
The processor 19 later generates an electrical side-view mir- ror activation signal according to the driver face directional data and to the driver hand gesture data. After this, the pro- cessor 19 transmits the side-view mirror activation signal to the side-view mirror actuator 22.
The side-view mirror actuator 22 then moves or actuates the respective side-view mirror 35 according to the received side- view mirror activation signal.
In one example, the driver extends all fingers of his hand. The driver then waves the extended fingers for moving the side-view mirror 35 in order for the side-view mirror 35 to some desired display objects. The waving refers to the driver positioning his extended fingers in front of the driver hand recognition camera 30 in a first position, without essentially moving the extended fingers for about 1 second, in order to for the driver hand gesture detector 33 to take notice of the extended fingers. After this, the driver then moves the extended fingers to a second position to complete the gesture. rs to
35 in ch ar placed to the left, as illustrated in Fig. 3
In another gesture, the driver waves the extended fingers to the right of the driver for moving the side-view mirror 35 in order for the side-view mirror 35 to display objects, which are placed towards the right, as illustrated in Fig. 4.
In a further gesture, the driver waves the extended fingers upwards for moving the side-view mirror 35 in order for the side-view mirror 35 to display objects, which are placed upwards, as illustrated in Fig. 5. In another gesture, the driver waves the extended fingers downwards for moving the side-view mirror 35 in order for the side-view mirror 35 to display objects, which are placed downwards, as illustrated in Fig. 6.
A method of operating the side-view mirror controller 10 is described below.
The driver looks at a desired side-view mirror 35 of the vehi cle. In other words, the face of the driver is directed at th side-view mirror 35.
In the meantime, the driver facial recognition camera 25 takes digital images of the face of the driver and later sends data of the driver facial images to the driver facial directional detector 27. After this, the driver facial directional detector 27 determines a direction of the driver face according to the received driver facial image data. The driver facial directional detector 27 then sends data of the determined driver face direction to the processor 19.
The driver later places his hand at the predetermined driver hand position and makes a gesture with his hand.
The driver hand recognition camera 30 then takes digital imag es of the driver hand and sends data of the driver hand image to the driver hand gesture detector 33. The driver hand gesture detector 33 later determines a driver hand gesture according to the received driver hand image data. The driver hand gesture detector 33 afterward sends data of the determined driver hand gesture to the processor 19.
If the processor 19 detects that the determined driver face directional data indicates that the driver is looking at the respective side-view mirror 35, the processor 19 later generates an electrical side-view mirror activation signal according the driver face directional data and to the driver hand gesture data. In other words, the processor 19 generates the electrical side-view mirror activation signal for the side- view mirror 35, when the side-view mirror 35 is being looked at by the driver.
The processor 19 afterward transmits the side-view mirror activation signal to the side-view mirror actuator 22.
Following this, the side-view mirror actuator 22 moves the re spective side-view mirror 35 according to the received side- view mirror activation signal.
In a general sense, a camera module to capture facial movements of the driver for detecting drowsiness or sleepiness of the driver can be adapted to act as the driver facial directional detector module 13.
Similarly, a hand gesture module for electronic control purposes of the vehicle can be adapted to act the driver hand gesture detector module 16. One example of the electronic con trol is an entertainment system with a hand gesture human interface unit .
In effect, this side-view mirror controller 10 of Fig. 1 can be adapted from features of the vehicle with some or little additional material cost.
The adaption includes development of software programs for th processor 19 of the side-view mirror controller 10 of Fig. 1. This side-view mirror controller 10 provides several benefits It enhances driving pleasure and also improves driving safety by a providing intuitive interface control for adjusting the side-view mirror 35 of the vehicle. This side-view mirror 35 can be adjusted intuitively by hand gesture and facial movement, thereby allowing the driver to focus on the road condition, even changing lane, which then leads to reduced car accidents .
The embodiments can also be described with the following lists of features or elements being organized into an item list. The respective combinations of features, which are disclosed in the item list, are regarded as independent subject matter, respectively, that can also be combined with other features of the application.
ITEMS
A controller unit for at least one side-view mirror of a vehicle, the controller unit comprising
a driver facial directional detector module that comprises
a driver facial recognition camera por
ceiving driver facial image data, and
a driver facial directional detector f
mining a driver face directional data
to the driver facial image data, a driver hand gesture detector module that comprises a driver hand recognition camera port for receiving driver hand image data, and
a driver hand gesture detector for determining a driver hand gesture data according to the driver hand image data,
a computing processor being adapted for generating an activation signal for side-view mirror actuator according to the driver face directional data and to the driver hand gesture data and
sending the side-view mirror actuator activation signal to a side-view mirror actuator.
A controller unit for a side-view mirror of a vehicle, the controller unit comprising
a driver facial recognition camera for taking at least one driver facial image,
a driver facial directional detector for determining a driver face directional data according to data of the at least one driver facial image,
a driver hand recognition camera for taking at least one driver hand image,
a driver hand gesture detector for determining a driver hand gesture data according to data of the at least one driver hand image,
a computing processor being adapted for generating a side-view mirror actuator activation signal according to the driver face directional data and to the driver hand gesture data, and
a side-view mirror actuator for actuating at least one side-view mirror according to the side-view mirror actuator activation signal.
The controller unit according to item 1 or 2 , wherein the computing processor is adapted, if the computing processor detects that the driver facial image data indicates that the driver is directed at the side-view mirror, for generating a side-view mirror actuator activation signal according to the driver hand gesture data. The controller unit according to one the above-mentioned items, wherein
the driver facial directional detector, the driver hand gesture detector, and the computing processor are provid ed by one integrated unit .
The controller unit according to one of items 1 to 3, wherein
two of a group, which comprises the driver facial direc tional detector, the driver hand gesture detector, and the computing processor, are provided by one integrated unit . A vehicle comprising
at least one side-view mirror and
a controller unit according to one of the above- mentioned items for actuating the at least one side-view mirror .
A method of adjusting at least one side-view mirror of a vehicle, the method comprising
taking at least one driver facial image,
determining a driver face directional data according to data of the at least one driver facial image,
taking at least one driver hand image,
determining a driver hand gesture data according to data of the at least one driver hand image,
when the driver face directional data indicates that a driver is viewing a side-view mirror, generating a side-view mirror actuator activation signal according to the driver hand gesture data, and
actuating at least one side-view mirror according to the side-view mirror actuator activation signal. Although the above description contains much specificity, thi should not be construed as limiting the scope of the embodiments but merely providing illustration of the foreseeable em bodiments . The above stated advantages of the embodiments should not be construed especially as limiting the scope of the embodiments but merely to explain possible achievements i the described embodiments are put into practice. Thus, the scope of the embodiments should be determined by the claims and their equivalents, rather than by the examples given.
REFERENCE NUMBERS
10 side-view mirror controller
13 driver facial directional detector module
16 driver hand gesture detector module
19 processor
22 side-view mirror actuator
25 driver facial recognition camera
27 driver facial directional detector
30 driver hand recognition camera
33 driver hand gesture detector
35 side-view mirror

Claims

1. A controller unit for at least one side-view mirror of a vehicle, the controller unit comprising
a driver facial directional detector module that comprises
a driver facial recognition camera port for receiving driver facial image data, and
a driver facial directional detector for determining a driver face directional data according to the driver facial image data, a driver hand gesture detector module that comprises a driver hand recognition camera port for receiving driver hand image data, and
a driver hand gesture detector for determining a driver hand gesture data according to the driver hand image data,
a computing processor being adapted for
generating an activation signal for side-view mirror actuator according to the driver face directional data and to the driver hand gesture data and
sending the side-view mirror actuator activation signal to a side-view mirror actuator.
The controller unit according to claim 1, wherein the computing processor is adapted, if the computing processor detects that the driver facial image data indicates that the driver is directed at the side-view mirror, for generating a side-view mirror actuator activation signal according to the driver hand gesture data.
3. The controller unit according to claim 1, wherein the driver facial directional detector, the driver hand gesture detector, and the computing processor are provid ed by one integrated unit .
The controller unit according to claim 1, wherein two of a group, which comprises the driver facial direc- tional detector, the driver hand gesture detector, and the computing processor, are provided by one integrated unit .
A vehicle comprising
at least one side-view mirror and
a controller unit according to claim 1 for actuatin' the at least one side-view mirror.
6. A controller unit for a side-view mirror of a vehicle, the controller unit comprising
a driver facial recognition camera for taking at least one driver facial image,
a driver facial directional detector for determining a driver face directional data according to data of the at least one driver facial image,
a driver hand recognition camera for taking at least one driver hand image,
a driver hand gesture detector for determining a driver hand gesture data according to data of the at least one driver hand image,
a computing processor being adapted for generating a side-view mirror actuator activation signal according to the driver face directional data and to the driver hand gesture data, and
a side-view mirror actuator for actuating at least one side-view mirror according to the side-view mirror actuator activation signal. The controller unit according to claim 6, wherein the computing processor is adapted, if the computing processor detects that the driver facial image data indicates that the driver is directed at the side-view mirror, for generating a side-view mirror actuator activation signal according to the driver hand gesture data.
The controller unit according to claim 6, wherein the driver facial directional detector, the driver hand gesture detector, and the computing processor are provided by one integrated unit .
The controller unit according to claim 6, wherein two of a group, which comprises the driver facial directional detector, the driver hand gesture detector, and the computing processor, are provided by one integrated unit .
A vehicle comprising
at least one side-view mirror and
a controller unit according to claim 6 for actuating the at least one side-view mirror.
A method of adjusting at least one side-view mirror of a vehicle, the method comprising
taking at least one driver facial image,
determining a driver face directional data according to data of the at least one driver facial image,
taking at least one driver hand image,
determining a driver hand gesture data according to data of the at least one driver hand image,
when the driver face directional data indicates that a driver is viewing a side-view mirror, generating a side-view mirror actuator activation signal according to the driver hand gesture data, and
actuating at least one side-view mirror according to the side-view mirror actuator activation signal.
PCT/IB2016/052229 2016-04-20 2016-04-20 Facial movement and gesture sensing side-view mirror Ceased WO2017182841A1 (en)

Priority Applications (3)

Application Number Priority Date Filing Date Title
EP16899319.4A EP3445612A4 (en) 2016-04-20 2016-04-20 SIDE VIEW MIRROR WITH DETECTION OF FACE MOVEMENT AND GESTURES
PCT/IB2016/052229 WO2017182841A1 (en) 2016-04-20 2016-04-20 Facial movement and gesture sensing side-view mirror
CN201680084622.4A CN109153354A (en) 2016-04-20 2016-04-20 Sense the side-view mirror of facial movement and gesture

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/IB2016/052229 WO2017182841A1 (en) 2016-04-20 2016-04-20 Facial movement and gesture sensing side-view mirror

Publications (2)

Publication Number Publication Date
WO2017182841A1 true WO2017182841A1 (en) 2017-10-26
WO2017182841A8 WO2017182841A8 (en) 2018-12-27

Family

ID=60115728

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2016/052229 Ceased WO2017182841A1 (en) 2016-04-20 2016-04-20 Facial movement and gesture sensing side-view mirror

Country Status (3)

Country Link
EP (1) EP3445612A4 (en)
CN (1) CN109153354A (en)
WO (1) WO2017182841A1 (en)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20230211731A1 (en) * 2022-01-05 2023-07-06 GM Global Technology Operations LLC Vehicle mirror selection based on head pose and gaze direction
CN119018052B (en) * 2024-09-04 2025-10-10 中国第一汽车股份有限公司 Vehicle rearview mirror adjustment method, device, vehicle and storage medium

Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5781354A (en) 1995-09-22 1998-07-14 Ichikoh Industries, Ltd. Positioning device for car sideview mirror
US20100080416A1 (en) * 2008-10-01 2010-04-01 Gm Global Technology Operations, Inc. Eye detection system using a single camera
US20130261871A1 (en) * 2012-04-02 2013-10-03 Google Inc. Gesture-Based Automotive Controls
EP2743133A2 (en) * 2012-12-13 2014-06-18 Nissan Motor Manufacturing (UK) Ltd. Camera system
US20150015710A1 (en) 2013-07-09 2015-01-15 Magna Electronics Inc. Vehicle vision system
US20150022664A1 (en) * 2012-01-20 2015-01-22 Magna Electronics Inc. Vehicle vision system with positionable virtual viewpoint
US20150032331A1 (en) * 2013-03-15 2015-01-29 Tarek A. El Dokor System and Method for Controlling a Vehicle User Interface Based on Gesture Angle

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH10509933A (en) * 1994-11-25 1998-09-29 アー． ブランディン，バーティル Method and apparatus for automating rearview mirror adjustment
DE102012215407A1 (en) * 2012-08-30 2014-05-28 Bayerische Motoren Werke Aktiengesellschaft Providing an input for a control
DE102012018742B3 (en) * 2012-09-21 2014-03-13 Audi Ag Method and device for adjusting a mirror for a motor vehicle
CN103303224B (en) * 2013-06-18 2015-04-15 桂林电子科技大学 Vehicle-mounted equipment gesture control system and usage method thereof
WO2015062751A1 (en) * 2013-10-28 2015-05-07 Johnson Controls Gmbh Method for operating a device for the contactless detection of objects and/or persons and their gestures and/or of control operations in a vehicle interior
DE102014019708A1 (en) * 2014-05-19 2015-11-19 Elmos Semiconductor Aktiengesellschaft Method for operating a mechanical device
CN205131102U (en) * 2015-10-13 2016-04-06 惠州市品旭实业有限公司 Intelligence driving -mirror

Patent Citations (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5781354A (en) 1995-09-22 1998-07-14 Ichikoh Industries, Ltd. Positioning device for car sideview mirror
US20100080416A1 (en) * 2008-10-01 2010-04-01 Gm Global Technology Operations, Inc. Eye detection system using a single camera
US20150022664A1 (en) * 2012-01-20 2015-01-22 Magna Electronics Inc. Vehicle vision system with positionable virtual viewpoint
US20130261871A1 (en) * 2012-04-02 2013-10-03 Google Inc. Gesture-Based Automotive Controls
EP2743133A2 (en) * 2012-12-13 2014-06-18 Nissan Motor Manufacturing (UK) Ltd. Camera system
US20150032331A1 (en) * 2013-03-15 2015-01-29 Tarek A. El Dokor System and Method for Controlling a Vehicle User Interface Based on Gesture Angle
US20150015710A1 (en) 2013-07-09 2015-01-15 Magna Electronics Inc. Vehicle vision system

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
BASTIAN PFLEGING ET AL.: "Multimodal interaction in the car", PROCEEDINGS OF THE 4TH INTERNATIONAL CONFERENCE ON AUTOMOTIVE USER INTERFACES AND INTERACTIVE VEHICULAR APPLICATIONS, AUTOMOTIVEUI, vol. 12, 1 January 2012 (2012-01-01), pages 155, XP055114585, ISBN: ISBN : 978-1-45-031751-1, DOI: 10.1145/2390256.2390282
See also references of EP3445612A4

Also Published As

Publication number Publication date
WO2017182841A8 (en) 2018-12-27
EP3445612A1 (en) 2019-02-27
EP3445612A4 (en) 2020-01-01
CN109153354A (en) 2019-01-04

Similar Documents

Publication Publication Date Title
US11124118B2 (en) Vehicular display system with user input display
US9942522B2 (en) In-vehicle camera system
US10086762B2 (en) Vehicle display device and vehicle comprising same
US11132534B2 (en) Monitoring system
US9656690B2 (en) System and method for using gestures in autonomous parking
US9701258B2 (en) Vehicle vision system
KR101809924B1 (en) Display apparatus for vehicle and Vehicle including the same
US20150375741A1 (en) Vehicle apparatus
CN109153353B (en) Vehicle display with selective image data display
CN113994312A (en) Method for operating a mobile terminal by means of a gesture recognition and control device, motor vehicle and head-mounted output device
CN104881117B (en) A kind of apparatus and method that speech control module is activated by gesture identification
KR101924059B1 (en) Display apparatus for vehicle and Vehicle including the same
GB2501575A (en) Interacting with vehicle controls through gesture recognition
US20190286123A1 (en) Remote vehicle control device and remote vehicle control method
JP4577387B2 (en) Vehicle operation input device
CN111989726B (en) Method, device and mobile tool for automatic activation of mobile tool at light signaling device
JP7138175B2 (en) Method of operating head-mounted electronic display device for displaying virtual content and display system for displaying virtual content
US20190171296A1 (en) Gesture determination apparatus and program
US20190286118A1 (en) Remote vehicle control device and remote vehicle control method
WO2018179786A1 (en) Information display device
CN105323539B (en) Vehicle safety system and operation method thereof
US20240331328A1 (en) Interactive Vision-Based Child Seat Mounting Assistance
JP7658361B2 (en) IMAGE PROCESSING APPARATUS, DISPLAY SYSTEM, IMAGE PROCESSING METHOD, AND PROGRAM
WO2017182841A1 (en) Facial movement and gesture sensing side-view mirror
JP2013180634A (en) Vehicle-mounted electric equipment and vehicle with the same

Legal Events

Date Code Title Description
NENP Non-entry into the national phase

Ref country code: DE

WWE Wipo information: entry into national phase

Ref document number: 2016899319

Country of ref document: EP

ENP Entry into the national phase

Ref document number: 2016899319

Country of ref document: EP

Effective date: 20181120

121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 16899319

Country of ref document: EP

Kind code of ref document: A1