WO2018082161A1 - Virtual reality apparatus, and control method and device for display screen of same - Google Patents

Abstract

A virtual reality apparatus, and a control method and device for a display screen (U4) of same. The virtual reality apparatus comprises a first eyeball tracking module (U1), a second eyeball tracking module (U2), a processor (U3), and a display screen (U4); the first eyeball tracking module (U1) is configured to acquire a left eyeball state of a user; the second eyeball tracking module (U2) is configured to acquire a right eyeball state of the user; the processor (U3) is configured to control an on/off state of the display screen (U4) according to the left eyeball state and the right eyeball state. By tracking a left eyeball state and a right eyeball state of a user, controlling an on/off state of the display screen (U4) according to the left eyeball state and the right eyeball state of the user, and controlling the display screen (U4) to be turned off when the user closes the eyes, the present invention can effectively reduce power consumption of a virtual reality apparatus without increasing hardware costs.

Description

Virtual reality device and control method and device thereof Technical field

The present invention relates to the field of virtual reality device technologies, and in particular, to a virtual reality device and a control method and apparatus thereof.

Background technique

Recently, with the rise of smart wearable devices, virtual reality products with distance sensor and camera functions have gradually entered the market. Infrared cameras on virtual reality devices are mainly used to track the movement of the user's eyeballs or head movements, and the distance. The sensor can recognize the action of the user wearing or removing the virtual reality device.

The main function of the virtual reality device is display, and the power consumption of the virtual reality device is mainly consumed on the display related module. For example, the general virtual reality device may be out of power for four hours of continuous display.

Summary of the invention

According to a first aspect of the present invention, a virtual reality device is provided, including a first eye tracking module, a second eye tracking module, a processor, and a display screen, the first eye tracking module being configured to collect a user's left eyeball a state, the second eye tracking module is configured to acquire a right eye state of the user, the processor being configured to control a switching state of the display screen according to the left eye state and the right eye state.

According to a second aspect of the present invention, a method for controlling a display screen of a virtual reality device according to the first aspect of the present invention, comprising:

Controlling the first eyeball tracking module to collect the left eye state of the user, and controlling the second eyeball tracking module to collect the state of the right eyeball;

Determining whether the user is in a closed eye state according to the left eye state and the right eye state, and if so, then:

Controlling the display to turn off.

According to a third aspect of the present invention, there is provided a control device for a display screen of a virtual reality device according to the first aspect of the present invention, comprising:

An acquisition control module, configured to control the first eyeball tracking module to collect the left eye state of the user, and control the second eyeball tracking module to collect the state of the right eyeball;

a determining module, configured to determine, according to the left eye state and the right eye state, whether the user is in a closed eye state;

And a display control module, configured to control the display screen to be turned off if the determination result of the determination module is YES.

According to a fourth aspect of the invention, there is provided a computer readable storage medium storing program code for performing the control method according to the second aspect of the invention.

Other features and advantages of the present invention will become apparent from the Detailed Description of the <RTIgt;

DRAWINGS

The accompanying drawings, which are incorporated in FIG

The accompanying drawings, which are incorporated in FIG

1 is a block schematic diagram of an implementation structure of a virtual reality device according to the present invention;

2 is a block schematic diagram showing another implementation structure of a virtual reality device according to the present invention;

3 is a flow chart of an embodiment of a method for controlling a display screen of a virtual reality device according to the present invention;

4 is a flow chart showing another embodiment of a method for controlling a display screen of a virtual reality device according to the present invention;

5 is a block schematic diagram of an implementation structure of an apparatus for displaying a display screen of a virtual reality device according to the present invention;

Fig. 6 is a block schematic diagram showing another embodiment of a control device for a display screen of a virtual reality device according to the present invention.

Description of the reference signs:

U1-first eye tracking module; U2-second eye tracking module;

U3-processor; U4-display;

U5-power module; U6-wear detection module;

S1-switch.

detailed description

Various exemplary embodiments of the present invention will now be described in detail with reference to the drawings. It should be noted that the relative arrangement of the components and steps, numerical expressions and numerical values set forth in the embodiments are not intended to limit the scope of the invention unless otherwise specified.

The following description of the at least one exemplary embodiment is merely illustrative and is in no way

Techniques, methods and apparatus known to those of ordinary skill in the relevant art may not be discussed in detail, but the techniques, methods and apparatus should be considered as part of the specification, where appropriate.

In all of the examples shown and discussed herein, any specific values are to be construed as illustrative only and not as a limitation. Thus, other examples of the exemplary embodiments may have different values.

It should be noted that similar reference numerals and letters indicate similar items in the following figures, and therefore, once an item is defined in one figure, it is not required to be further discussed in the subsequent figures.

In order to solve the problem that the display power consumption of the virtual reality device existing in the prior art is large and the battery life is poor, a virtual reality device is provided. As shown in FIG. 1 , the first eye tracking module U1 is included. a second eye tracking module U2, a processor U3, and a display screen U4, the first eye tracking module U1 is configured to collect a left eye state of the user, and the second eye tracking module U2 is configured to collect a right eye state of the user; U3 is set to control the display state of the display screen U4 according to the left eye state and the right eye state.

Specifically, since the content displayed by the display U4 cannot be viewed when the left eye and the right eye of the user are in the closed state, the display U4 is in a waste state, and therefore, the processor U3 is in the left eye state and the right eye state. When the eyes are closed, you can control the display U4 to close, so that you can It can reduce the power consumption of the display U4 when the user does not look at the display U4, thereby extending the battery life.

Further, since the virtual reality device is, for example, a head-mounted virtual reality device, the occlusion of the eye is equivalent to a nighttime state during the wearing of the user, so the eyeball tracking module with strong nighttime image capturing capability is needed to track the movement and position of the eyeball. Because the infrared camera has strong night image capturing capability, the first eye tracking module U1 and the second eye tracking module U2 can both be provided by the infrared camera, that is, the first infrared camera is used to collect the left eye state of the user, and the second The infrared camera is used to collect the right eye state of the user. Since both infrared cameras are used to track the user's eyeball, the pixel resolution requirements of the two infrared cameras are relatively low, and the resolution can be, for example, 320*240. .

Infrared camera is only sensitive to black and white or shades of color, the output data is also two-dimensional, and the human eye structure is generally dark in the middle, the surrounding is light color, therefore, use the infrared camera to track the eyeball, when the user opens When looking at the front of the eye, if you map the two-dimensional data of the infrared camera to a square form, it should be the dark infrared camera data in the middle, surrounded by the infrared camera data representing the light color. When the eyeball rotates The dark data on the form will also move in the same direction or in the opposite direction; when the user closes the eye, the eyeball is occluded at this time, there will be no more depth, the corresponding infrared camera's two-dimensional data mapping It is a single-color table, from which it can be judged whether the state of the left and right eyes of the user is blinking or closed.

In a specific embodiment of the present invention, the virtual reality device further includes a power module U5, and the power module U5 is configured to supply power to the first eye tracking module U1, the second eye tracking module U2, the processor U3, and the display U4. The power module U5 may include, for example, a battery and a DCDC, and may also include a battery and a low dropout linear regulator (ie, an LDO), and may also include a battery and a power management integrated circuit (ie, a PMIC).

Further, the virtual reality device further includes a switch S1 connected to the power supply circuit of the display screen U4, and the processor U3 is configured to control the conduction state of the switch S1 according to the left eye state and the right eye state, specifically on the left When both the eyeball state and the right eyeball state are closed eyes, the processor U3 outputs, for example, the low level control switch S1 is turned off, so that the power module U5 stops supplying power to the display screen U4, and the display screen U4 is turned off; otherwise, the processor U3 will The output is, for example, a high level control switch S1 turned on, so that the power module U5 supplies power to the display U4, and the display U4 is turned on.

On this basis, the switch S1 can be provided by a MOS transistor, for example, an N-channel MOS transistor, so that the processor U3 can output a high level control switch S1 to be turned on, and output a low level control switch S1 to be turned off; The switch S1 can also be a P-channel MOS transistor, so that the processor U3 can output a low level control switch S1 to be turned on, and the output high level control switch S1 is turned off.

In a specific embodiment of the present invention, the virtual reality device further includes a wearing detection module U6, and the wearing detection module U6 is configured to detect whether the user wears the virtual reality device, and in the case that the user wears the virtual reality device, the first The eyeball tracking module U1 and the second eyeball tracking module U2 collect the eyeball state of the user; if the user does not wear the virtual reality device, the processor U3 controls the display screen U4 to be turned off, thereby further reducing the battery power consumed by the display. Further improving the battery life.

Based on this, the wearing detection module U6 can be provided by a distance sensor. The distance sensor can detect whether there is an obstacle in front by emitting infrared rays. If there is no obstacle, there will be no light returning, otherwise the light will return, and the distance between the distance sensor and the obstacle can be determined by returning the infrared time. The distance detected by the distance sensor can be used to determine whether the user wears the virtual reality device.

The present invention also provides a display control method, and FIG. 3 is a flow chart of an embodiment of a control method of a display screen of a virtual reality device according to the present invention.

According to FIG. 3, the control method comprises the following steps:

Step S310, controlling the first eyeball tracking module to collect the left eye state of the user, and controlling the second eyeball tracking module to collect the state of the right eyeball.

In a specific embodiment of the present invention, as shown in FIG. 4, the control method further includes:

In step S410, it is determined whether the user wears the virtual reality device. If yes, step S310 is performed. If no, the step S410 is continued.

In step S320, it is determined whether the user is in the closed eye state according to the left eye state and the right eye state. If yes, step S330 is performed, and if no, step S310 is continued.

In a specific embodiment of the present invention, the two-dimensional data output by the first eye tracking module and the second eye tracking module may be respectively mapped to corresponding tables, and in the case where the data color of each table is single, The result of the determination is yes.

In step S330, the display screen is turned off.

In a specific embodiment of the present invention, as shown in FIG. 4, the control method further includes:

In step S420, it is detected whether it is in a blink state, and if so, step S310 is continued, and if no, step S420 is performed.

Since blinking is the normal physiological need of the human eye under normal conditions, in order to avoid the interference of the user's normal blinking on the detection result of the eyeball state, for example, in the case where the closed eye time in the normal blinking is 100 ms, the setting detects that the closed eye time exceeds At 500ms, the current eye state is considered to be closed, so that the interference of normal blinking motion can be eliminated.

In this way, when the left eye state and the right eye state are closed eyes, by controlling the display screen to be turned off, the power consumption of the display when the user does not look at the display screen can be reduced, thereby prolonging the battery life time.

Corresponding to the above control method, the present invention also provides a control device for a display screen, and FIG. 5 is a block schematic diagram of an implementation structure of a control device for a display screen of a virtual reality device according to the present invention.

According to FIG. 5, the display control device 500 includes an acquisition control module 510, a determination module 520, and a display control module 530.

The acquisition control module 510 is configured to control the first eyeball tracking module to collect the left eye state of the user, and control the second eyeball tracking module to collect the state of the right eyeball.

The determining module 520 is configured to determine whether the user is in the closed eye state according to the left eye state and the right eye state.

The display control module 530 is configured to control the display screen to be turned off if the determination result of the determination module is YES.

In a specific embodiment of the present invention, the determining module 520 is specifically configured to: map the two-dimensional data output by the first eye tracking module and the second eye tracking module to the corresponding table, and the data on each table. When the colors are all single, the determination result is yes.

Further, in a specific embodiment of the present invention, as shown in FIG. 6, the control device 500 further includes a wearing detection module 610, configured to determine whether the user wears the virtual reality device, and if so, controls the acquisition control module 510 to control the first The eyeball tracking module collects the left eye state of the user, and controls the second eyeball tracking module to collect the state of the right eyeball.

Based on this, the control device 500 further includes a blink detection module 620 for detecting Whether the user is in a blinking state, if not, the control display control module 530 controls the display to be turned off.

The above embodiments mainly focus on the differences from the other embodiments, but it should be apparent to those skilled in the art that the above embodiments may be used alone or in combination with each other as needed.

The various embodiments in the present specification are described in a progressive manner, and the same or similar parts between the various embodiments may be referred to each other, and each embodiment focuses on the differences from other embodiments, but the field It should be clear to the skilled person that the above embodiments can be used individually or in combination with each other as needed. In addition, for the device embodiment, since it corresponds to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the description of the corresponding part of the method embodiment. The system embodiments described above are merely illustrative, and the modules illustrated as separate components may or may not be physically separate.

The invention can be an apparatus, method and/or computer program product. The computer program product can comprise a computer readable storage medium having computer readable program instructions embodied thereon for causing a processor to implement various aspects of the present invention.

The computer readable storage medium can be a tangible device that can hold and store the instructions used by the instruction execution device. The computer readable storage medium can be, for example, but not limited to, an electrical storage device, a magnetic storage device, an optical storage device, an electromagnetic storage device, a semiconductor storage device, or any suitable combination of the foregoing. More specific examples (non-exhaustive list) of computer readable storage media include: portable computer disks, hard disks, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM) Or flash memory), static random access memory (SRAM), portable compact disk read only memory (CD-ROM), digital versatile disk (DVD), memory stick, floppy disk, mechanical encoding device, for example, with instructions stored thereon A raised structure in the hole card or groove, and any suitable combination of the above. A computer readable storage medium as used herein is not to be interpreted as a transient signal itself, such as a radio wave or other freely propagating electromagnetic wave, an electromagnetic wave propagating through a waveguide or other transmission medium (eg, a light pulse through a fiber optic cable), or through a wire The electrical signal transmitted.

The computer readable program instructions described herein can be downloaded from a computer readable storage medium to various computing/processing devices or downloaded to an external computer or external storage device over a network, such as the Internet, a local area network, a wide area network, and/or a wireless network. The network can include a copper transmission cable, Fiber optic transmission, wireless transmission, routers, firewalls, switches, gateway computers, and/or edge servers. A network adapter card or network interface in each computing/processing device receives computer readable program instructions from the network and forwards the computer readable program instructions for storage in a computer readable storage medium in each computing/processing device .

Computer program instructions for performing the operations of the present invention may be assembly instructions, instruction set architecture (ISA) instructions, machine instructions, machine related instructions, microcode, firmware instructions, state setting data, or in one or more programming languages. Source code or object code written in any combination, including object oriented programming languages such as Smalltalk, C++, etc., as well as conventional procedural programming languages such as the "C" language or similar programming languages. The computer readable program instructions can execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer, partly on the remote computer, or entirely on the remote computer or server. carried out. In the case of a remote computer, the remote computer can be connected to the user's computer through any kind of network, including a local area network (LAN) or wide area network (WAN), or can be connected to an external computer (eg, using an Internet service provider to access the Internet) connection). In some embodiments, the customized electronic circuit, such as a programmable logic circuit, a field programmable gate array (FPGA), or a programmable logic array (PLA), can be customized by utilizing state information of computer readable program instructions. Computer readable program instructions are executed to implement various aspects of the present invention.

Aspects of the present invention are described herein with reference to flowchart illustrations and/or block diagrams of methods, apparatus, and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and/or block diagrams, and combinations of blocks in the flowcharts and/or block diagrams can be implemented by computer readable program instructions.

The computer readable program instructions can be provided to a general purpose computer, a special purpose computer, or a processor of other programmable data processing apparatus to produce a machine such that when executed by a processor of a computer or other programmable data processing apparatus Means for implementing the functions/acts specified in one or more of the blocks of the flowcharts and/or block diagrams. The computer readable program instructions can also be stored in a computer readable storage medium that causes the computer, programmable data processing device, and/or other device to operate in a particular manner, such that the computer readable medium storing the instructions includes An article of manufacture comprising one of the implementation flowcharts and/or block diagrams Or instructions for various aspects of the functions/actions specified in the various blocks.

The computer readable program instructions can also be loaded onto a computer, other programmable data processing device, or other device to perform a series of operational steps on a computer, other programmable data processing device or other device to produce a computer-implemented process. Thus, instructions executed on a computer, other programmable data processing apparatus, or other device implement the functions/acts recited in one or more of the flowcharts and/or block diagrams.

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the invention. In this regard, each block in the flowchart or block diagram can represent a module, a program segment, or a portion of an instruction that includes one or more components for implementing the specified logical functions. Executable instructions. In some alternative implementations, the functions noted in the blocks may also occur in a different order than those illustrated in the drawings. For example, two consecutive blocks may be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending upon the functionality involved. It is also noted that each block of the block diagrams and/or flowcharts, and combinations of blocks in the block diagrams and/or flowcharts, can be implemented in a dedicated hardware-based system that performs the specified function or function. Or it can be implemented by a combination of dedicated hardware and computer instructions. It is well known to those skilled in the art that implementation by hardware, implementation by software, and implementation by a combination of software and hardware are equivalent.

The embodiments of the present invention have been described above, and the foregoing description is illustrative, not limiting, and not limited to the disclosed embodiments. Numerous modifications and changes will be apparent to those skilled in the art without departing from the scope of the invention. The choice of terms used herein is intended to best explain the principles, practical applications, or technical improvements of the techniques in the <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt; </ RTI> <RTIgt; The scope of the invention is defined by the appended claims.

Claims (11)

A virtual reality device, comprising: a first eye tracking module, a second eye tracking module, a processor, and a display screen, wherein the first eye tracking module is configured to collect a left eye state of the user, the second An eye tracking module is configured to acquire a right eye state of the user, the processor being configured to control a switching state of the display screen in accordance with the left eye state and the right eye state. The virtual reality device according to claim 1, wherein the virtual reality device further comprises a wear detecting module, the wear detecting module being configured to detect whether the user wears the virtual reality device. A method for controlling a display screen of a virtual reality device according to claim 1 or 2, comprising: Controlling the first eyeball tracking module to collect the left eye state of the user, and controlling the second eyeball tracking module to collect the state of the right eyeball; Determining whether the user is in a closed eye state according to the left eye state and the right eye state, and if so, then: Controlling the display to turn off. The control method according to claim 3, wherein the determining whether the user is in a closed eye state according to the left eye state and the right eye state is: The two-dimensional data output by the first eye tracking module and the second eye tracking module are respectively mapped onto corresponding tables, and if the data colors on all the tables are single, the determination result is yes. The control method according to claim 3 or 4, wherein the control method further comprises: Detecting whether the user is in a blink state, and if not, controlling the display to be turned off. The control method according to any one of claims 3-5, wherein the control method further comprises: Determining whether the user wears the virtual reality device, and if so, controlling the first eye The ball tracking module collects the left eye state of the user, and controls the second eye tracking module to collect the state for the right eye. A control device for a display screen of a virtual reality device according to claim 1 or 2, comprising: An acquisition control module, configured to control the first eyeball tracking module to collect the left eye state of the user, and control the second eyeball tracking module to collect the state of the right eyeball; a determining module, configured to determine, according to the left eye state and the right eye state, whether the user is in a closed eye state; And a display control module, configured to control the display screen to be turned off if the determination result of the determination module is YES. The control device according to claim 7, wherein the determining module is specifically configured to: The two-dimensional data output by the first eye tracking module and the second eye tracking module are respectively mapped onto corresponding tables, and if the data colors on all the tables are single, the determination result is yes. The control device according to claim 7 or 8, wherein the control device further comprises: The blink detection module is configured to detect whether the user is in a blink state, and if not, control the display control module to control the display to be turned off. The control device according to any one of claims 7 to 9, wherein the control device further comprises: The wearing detection module is configured to determine whether the user wears the virtual reality device, and if so, control the acquisition control module to control the first eye tracking module to collect the left eye state of the user, and control the second eye tracking The module collects the state for the right eye. A computer readable storage medium characterized by storing program code for performing the control method according to any one of claims 3-6.

Images