RU2518404C2 - Vehicle control device - Google Patents

Abstract

FIELD: transport.

SUBSTANCE: vehicle control device contains steering wheel, optical emitter and optically linked with it radiation receivers connected to dedicated computer. Optical receivers represent digital television cameras the field of view of which is optical emitter surface. In the first version, optical emitter is made as strip attached to inner surface of steering wheel. In the second version, the device contains the second optical receiver, herewith the receivers represent light emitting diodes and provide illumination of objects falling into field of view of the first and the second optical receiver. The zone where vehicle operator gestures can be recognised is intersection of fields of view the first and the second television cameras on the surface of optical emitter. Appearance of operator's finger in this zone causes emitted light flux intersection and consequently appearance of shaded segments on images generated by the first and the second television cameras, and according to coordinates of these segments the dedicated computer calculates a number of spatial positions of operator's finger at successive time points and determines motion trajectory from them.

EFFECT: vehicle control device functionality enhancement.

5 cl, 9 dwg

Description

The present invention relates to vehicles and is intended to facilitate the management of vehicles (car, plane, sea vessel). This is achieved by imparting to the existing controls - the car steering wheel, steering wheel and other control devices, the gesture recognition function of the operator (driver, pilot, etc.) of the vehicle. This allows the operator, without interrupting the control process, to submit additional commands in the form of predetermined simple gestures, formed, for example, by the thumbs of the left and / or right hand in the space limited by the structural elements of the car steering wheel or helm. The commands issued by the proposed device, in turn, can be used to control electronic equipment installed on the vehicle (radio, navigator, DVR, wipers and windows), as well as to control various equipment (air conditioning, fans, lighting, warning and control systems etc.).

A number of devices are known, used, for example, in automobiles, and simplifying the management of electronic equipment. The most common of these is the infrared remote control, which allows generating control signals for the car radio, the disadvantage of which is the need for the vehicle operator to pick up this remote control, i.e. break away from the management process. The functions of this remote control can be performed by buttons located on the steering wheel, but in this case a visual or tactile search of the button required for pressing is also required, which also distracts the operator from the process of driving the vehicle.

There are systems designed to transmit information from the steering wheel area to the electronic components of the car, for example, proposed in patents by Aciers et Outillage Peugeot (France) [1], Toyoda Gosei Co., Ltd. (Japan) [2]. Using the technical solutions described there, it is possible to control automotive equipment from the surface of the steering wheel, but they all have a marked drawback - they distract the driver from his main function - traffic control.

The patent of International Business Machines Corporation (USA) [3] proposes a system for controlling automotive equipment based on the use of gesture recognition and voice commands. For this, among other things, it is proposed to install the so-called Touchpad device (widely used in laptops) on the surface of the steering wheel. Further, as proposed by the authors of this invention, the driver’s finger is used to move along the surface of the Touchpad, and the computing device that is part of the system recognizes a predefined set of commands that can be compared with certain finger movements. The disadvantage of this system is the presence of Touchpad, with the active zone occupying the surface of the steering wheel.

The closest technical solution to the proposed (prototype) is a device patented by Robert Bosh GmbH (Germany) [4]. In this device, using the optically coupled fiber optic line of a photodetector and an optical emitter, the steering force is recorded by the hand of the driver of the vehicle. In this case, as stated in the patent, the force with which the driver compresses the optical fiber is determined by measuring the degree of polarization of the radiation transmitted through this optical fiber, wound spirally around the steering wheel. Using this system, the driver can transmit commands by changing the compression ratio of the steering wheel of the car.

The disadvantages of this system is the limited set of commands that the driver can form by adjusting the steering compression force and the lack of information content of these commands.

The purpose of the claimed device is to expand the functionality of existing vehicle control devices by using the internal space bounded by the steering wheel of the car or the helm of an aircraft or sea vessel.

Figure 1 presents the functional diagram of the proposed device, consisting of a steering wheel 1, a light emitter 2, made in the form of a strip placed on the inner surface of the steering wheel, the first and second television cameras 3 and 4, optically paired with a light emitter 2, and a specialized computer 5 , the inputs of which receive signals from the outputs of television cameras. At the output of a specialized computer 5, control signals of the vehicle’s equipment are generated — radio, navigator, video recorder, wipers and windows, air conditioning, lighting and sound equipment, etc.

As can be seen from the optical scheme shown in figure 2, the working area W of this device is formed by the intersection of the viewing angles of the first - and second - Ψ ₂ television cameras 3, 4, with the surface S of the light emitter 2. The working area W has a narrow width h in the direction of the axis Y, and in fact is a two-dimensional plane, which is a spatial analogue of a glass touch-panel [5]. In the absence of objects in the working area, images of the light emitter 2 — strips β ₁ and β _2, respectively, are formed on the photodetector arrays 6 and 7, respectively. The appearance in the working area of the object R (the finger of the operator of the vehicle) crossing the working area in the Y direction causes the appearance of shaded sections α ₁ and α ₂ on the photodetector arrays 6 and 7, which are part of the television cameras 3 and 4. Their midpoints make it possible to determine the spatial position of the point M located in the working area W and belonging to the object R. The determination of the coordinates of the point M is illustrated in Fig. 3 and is based on the analysis, for example, proposed in [3], of stereo shooting from two television cameras with parallel optical axes. Considering that the optical centers of the O and O ₂ lenses lie in the W, Y plane - the coordinate of the object R is always zero, the calculation formulas of the X and Z coordinates have the form [6]

$$\mathrm{one})X=b\frac{\left(X{\text{'}\text{'}}_u+X\text{'}\text{'}{\text{'}\text{'}}_u\right)}{2\left(X{\text{'}\text{'}}_u-X\text{'}\text{'}{\text{'}\text{'}}_u\right)},$$

$$Z=bf\text{'}\text{'}\frac{\mathrm{one}}{\left(X{\text{'}\text{'}}_u\right)-X\text{'}\text{'}{\text{'}\text{'}}_u}.$$

Figure 4 shows simplified images of 6 - and gestures {A, B, C, D, E, F}, which can be formed by the operator of the vehicle for control.

Obviously, each of the gestures i∈ {A, B, C, D, E, F] can be represented in the form of trajectories [M _t ] ⁱ , t∈ {t _in ... t _out }, digitized by a specialized calculator with a cycle σ _t for the time interval between the entrance of the finger into the working area (t _in ) and the exit from it (t _out ). This set of coordinates can also be represented as a set of signals r _i (t).

If the reference gestures stored in the memory of a specialized calculator are represented in the form of trajectories [N _t ] ⁱ , t∈ {t _in ... t _our }, or as a set of signals s _i (t), we can determine the correlation receiver, consisting of i correlators that convert the input signal (path) r (t) to the i-outputs of the correlators Zi; (t), each of which is described by the convolution equation

; $$2)Zi\left(t\right)=Si\left(t\right)*r\left(t\right)$$

;

or

$$3)Zi\left(t\right)={\int }_0^{t}r\left(t\right)Si\left(t\right)dt\left[7\right]$$

Choosing i corresponding to the maximum value of z _i (t), it is possible to determine which gesture corresponds to the trajectory described by the finger of the vehicle operator.

The detection of the shaded areas α ₁ and α ₂ in a specialized computer 5 is carried out as a result of threshold processing, given that the brightness of the light emitter 2 exceeds the level of spurious illumination. In Fig.2, the image of the light emitter 2 on the photodetector arrays 6 and 7 is designated β ₁ and β _2, respectively. Moreover, the magnitude of the video signal representing the images of the zones β ₁ and β ₂ exceeds the level of spurious illumination in the regions α ₁ and α ₂ .

Figure 5 shows one of the possible options for the optical scheme of the light emitter 2, where:

8 - diffuser, 9 - a set of light emitting diodes, for example, infrared.

As a diffuser, a diffuser screen made of K01 optical ceramics can be used (in the case of using light emitting diodes 9 operating in the near infrared region [8, p. 217].

The light emitter can also be made on the basis of the side-glow optical fiber [9], as shown in Fig.6, where: 10 is the side-glow optical fiber, 11 is an LED, for example, IR radiation.

Figure 7 shows the second variant of the functional diagram of the device, consisting of a steering wheel 1, the first and second television cameras 3 and 4, with viewing angles Ψ ₁ and Ψ ₂ , the first and second emitter 13 and 14 with radiation fluxes F ₁ and F ₂ , light-absorbing strips 12 and computing device 5.

In this scheme, the highlighting of objects located in the working area of the device W, due to the action of the first and second emitters 13 and 14. Their power should provide the brightness of the sections α ₁ and α ₂ , (figure 2), exceeding the brightness of spurious illumination. To prevent glare from the inner surface of the steering wheel 1, it is possible to place a light absorber on its surface, made, for example, in the form of a strip 12. It prevents the appearance of stray light on photodetector arrays 6 and 7 arising from the reflection of light fluxes Ф ₁ and Ф ₂ . The light-absorbing strip 12 can be made by a method known in optical technology, for example, from glass with a reflection coefficient ρ≤0.016 and with a complete absence of light transmission [8, p. 354]. Its role can also be played by a hood made in the form of a groove 15 extending along the inner surface of the steering wheel 1, as shown in Fig. 8. Detection of 3 and 4 portions corresponding to α ₁ and α ₂ in the video signal of the cameras is carried out by a specialized computer 5, which takes into account the excess of a certain threshold level by the video signal.

The emitters 13 and 14 are infrared LEDs with a radiation angle sufficient to cover the inner surface of the steering wheel, and a power that allows reliable detection of radiation reflected from the driver’s finger against spurious illumination.

Figure 9 shows the optical layout of the elements of the control device, made in the form of a helm.

Specialized computer 5 can be performed on the basis of processors with a high-speed communication channel with digital television cameras, for example, DM8168 DaVinci or TMS320 OMAR ^TM from Texas Instruments. Their software should implement:

- threshold detection and fixation of the coordinates α ₁ and α ₂ on the photodetector matrices 6 and 7;

- calculation of the spatial coordinates of the point belonging to R, according to formula 1, possibly modified taking into account the shift and convergence of the optical axes of television cameras 3 and 4;

- making a decision on the driver making a control gesture based on the data of the correlation receiver described by formulas 2 or 3.

As television cameras can be used miniature digital sensors, for example OVM7690 company OmniVision.

Literature

1. US Patent No. 4,678,906 dated July 7, 1987.

2. US patent No. 4772799 from 09/20/1988

3. US patent No. 7295904 from 11/13/2007

4. US Patent No. 6114949 dated September 5, 2000.

5. Vikhrev L. With a flick of the wrist ... Components and technologies. No. 5, 2005, St. Petersburg, Finestreet.

6. Korotaev V.V., Krasnyashchikh A.V. Television measuring systems. Tutorial. St. Petersburg, ITMO, 2008 (p. 42).

7. Sklyar Bernard. Digital communication. Theoretical foundations and practical application. Second edition. Moscow, St. Petersburg, Kiev, 2003 (p. 206).

8. Reference designer of optical-mechanical devices. Under the total. ed. V.A. Panova. Leningrad, Engineering, 1980

9. http://www.fiberopticproducts.com/Sideglow.htm

Claims (5)

1. A device for driving a vehicle, comprising a steering wheel, an optical emitter and an optically coupled radiation receiver connected to a specialized calculator, further comprises a second radiation receiver optically coupled to the emitter and connected to an additional input of a specialized calculator, the first and second optical the receivers are digital television cameras, the field of view of which is the surface of the optical emitter, while optical the emitter is made in the form of a strip fixed on the inner surface of the steering wheel, and the zone in which gestures can be recognized by the operator of the vehicle is the intersection of the viewing sectors of the first and second television cameras on the surface of the optical emitter, while the appearance of the operator’s finger in this area causes the intersection of the light flux generated by the emitter, and accordingly the appearance of shadow segments in the images formed by the first and second television cameras, m, based on the coordinates of these segments, a specialized calculator calculates a number of spatial positions of the operator’s finger at successive times and determines the motion path from them, which is then compared with the set of gestures defined in advance in the memory of a specialized calculator, and a decision is made on whether this gesture belongs to one of them and is worked out the vehicle equipment control command corresponding to this gesture. 2. The device for driving a vehicle according to claim 1, characterized in that the optical emitter is made in the form of a light-diffusing screen diffusely transmitting light, the outer surface of which is directed towards the radiation receivers, and the inner one - towards the light emitting diodes located behind it. 3. The device for driving a vehicle according to claim 1, characterized in that the optical emitter is a piece of lateral glow fiber, the radiation of which is directed towards the radiation receivers, and at least one light-emitting optically coupled to it is placed at the end of this fiber diode. 4. A device for controlling a vehicle, comprising a steering wheel, an optical emitter mounted thereon and a radiation receiver optically coupled thereto, connected to a specialized calculator, further comprises an optically paired second optical receiver and a second optical emitter, the first and second optical receivers being digital television cameras, and the first and second optical emitters are LEDs, for example, infrared and provide illumination objects falling into the field of view of the first and second optical receiver, and the area in which gestures can be recognized by the operator of the vehicle is the intersection of the viewing sectors of the first and second television cameras on the inner surface of the steering wheel, while the appearance of a finger crossing the light streams of the first and second emitter, causes its illumination and the appearance of the corresponding segments in the images formed by the first and second television cameras, and further, by Inatam of these segments, a specialized calculator calculates a number of spatial positions of the finger at consecutive moments of time and determines the trajectory of its movement, which is then compared with a set of gestures defined in advance in the memory of a specialized calculator, and a decision is made on whether this gesture belongs to one of them and is developed the vehicle equipment control command corresponding to this gesture. 5. The device for driving a vehicle for driving a vehicle according to claim 4, characterized in that the inner surface of the steering wheel contains a light absorber facing the first and second emitters, preventing light from being reflected from it towards optical receivers.

Images