WO2017104525A1 - Input device, electronic device, and head-mounted display - Google Patents

Abstract

An input device, an electronic device, and a head-mounted display are provided such that a complex input is enabled by means of a combination of gesture operations while having a simple configuration. A control device is capable of selectively setting: a single mode in which, when a single gesture operation is detected by a detection device, an input content corresponding to the single gesture operation is determined; or a combination mode in which, when multiple gesture operations are continuously detected by the detection device, a different input content from when the single gesture operation is detected is determined.

Description

Input device, electronic device and head mounted display

The present invention relates to an input device, an electronic device, and a head mounted display.

In general computers, various input operations are realized using a user interface such as a mouse or a keyboard. On the other hand, mobile terminals such as smart phones that have been rapidly developed in recent years generally have a touch panel screen that serves both as an image display and a user interface. Thus, an operation such as displaying a desired image or inputting information can be performed. However, there are cases where you want to operate the mobile device without touching the screen, such as when the user's hand is wet or dirty, but how to perform input at that time is an issue Yes. However, connecting and using a conventional mouse or keyboard impairs the great merit of carrying around a portable terminal, and it is inconvenient.

On the other hand, Patent Document 1 discloses a computing device including an infrared LED and an infrared proximity sensor. According to the technique of Patent Document 1, the infrared light emitted from the infrared LED is reflected by the hand approaching the computing device and reflected, and the reflected light can be detected by the infrared proximity sensor to detect the movement of the hand. The user can perform an input operation desired by the user such as swipe without contact by the movement (gesture) of the hand. Therefore, even if the user's hand is dirty, there is no risk of contaminating the mobile computing device.

Special table 2013-534209 JP-T-2014-527666

However, in general, it is difficult for the user to accurately perform detailed operations of the hand. Even if the detailed operations are associated with the input operation as shown in Patent Document 1, the computing device is erroneous. There is a high probability of detection. Therefore, it can be said that it is necessary to limit to simple hand movements that are easily detected by the computing device in order to prevent erroneous detection. However, when the hand movement is limited in this way, there is a problem that fine input cannot be performed.

On the other hand, Patent Document 2 discloses a technique for promoting or supporting one or more operations or techniques for gesture detection based on signals from a plurality of ambient environment sensors. However, providing a plurality of ambient environment sensors in this manner increases the cost and may increase the size of the mobile terminal.

The present invention has been made in view of the above circumstances, and provides an input device, an electronic device, and a head-mounted display that enable complex input while having a simple configuration by combining gesture operations. With the goal.

In order to achieve at least one of the above-described objects, an input device reflecting one aspect of the present invention is:
In the detection region, a detection device that detects a movement or shape change of the indicator moved by the user as a gesture operation, and generates an output corresponding to the gesture operation;
A control device for determining input contents according to the output of the detection device, and an input device comprising:
In response to the detection device detecting the single gesture operation, the control device determines a single input mode corresponding to the single gesture operation, and the detection device continuously detects a plurality of the gesture operations. Accordingly, it is possible to selectively set a combination mode for determining input contents different from the case where a single gesture operation is detected.

According to the present invention, it is possible to provide an input device, an electronic device, and a head-mounted display that enable complex input with a simple configuration by combining gesture operations.

It is a perspective view of the head mounted display (HMD) concerning this embodiment. It is the figure which looked at HMD from the front. It is the figure which looked at HMD from the upper part. It is a schematic sectional drawing which shows the structure of a display unit. It is an enlarged view of a proximity sensor. It is a block diagram of the main circuit of HMD. It is a front view when a user wears HMD. It is a side view when a user wears HMD. It is a top view when a user wears an HMD. It is a figure which shows the example of the signal waveform which a some light reception area | region outputs. It is a figure which shows the state which moved hand HD with respect to the proximity sensor 105 from the right to the left. It is a figure which shows the light reception state of the proximity sensor 105 when moving hand HD from right to left. It is a figure which shows the other example of the signal waveform which a some light reception area | region outputs. It is a figure which shows the other example of the signal waveform which a light reception area | region outputs. It is a flowchart figure which shows the sequence of a single mode. It is a flowchart which shows the sequence of combination mode. It is a flowchart figure which shows a part of modification of the sequence of FIG. It is a figure which shows the data of the telephone directory memorize | stored in RAM126. It is a figure which shows the example of the event list | wrist and combination list | wrist used by a call application corresponding to the data of a telephone directory. It is the figure which looked at the image display part 104B of the display unit 104 from the user US side. It is a figure which shows the example of an event list and a combination list. (A) is a figure which shows the input screen which the user directly input using external PC etc., (b) is a schematic diagram which shows converting the data into QR Code (trademark). (A) is a figure which shows the example of an event which inputs a content with the combination of gesture operation for every application, (b) is a figure which shows the example of a combination list | wrist.

Embodiments of the present invention will be described below with reference to the drawings. FIG. 1 is a perspective view of an HMD 100 that is an electronic apparatus according to the present embodiment. FIG. 2 is a front view of the head mounted display (hereinafter referred to as HMD) 100 according to the present embodiment. FIG. 3 is a view of the HMD 100 according to the present embodiment as viewed from above. Hereinafter, the right side and the left side of the HMD 100 refer to the right side and the left side for the user wearing the HMD 100.

As shown in FIGS. 1 to 3, the HMD 100 of this embodiment has a frame 101 as a support member. A frame 101 that is U-shaped when viewed from above has a front part 101a to which two spectacle lenses 102 are attached, and side parts 101b and 101c extending rearward from both ends of the front part 101a. The two spectacle lenses 102 attached to the frame 101 may or may not have refractive power.

A cylindrical main body 103 as a support member is fixed to the front portion 101a of the frame 101 on the upper side of the spectacle lens 102 on the right side (which may be on the left side depending on the user's dominant eye). The main body 103 is provided with a display unit 104. A display control unit 104DR (see FIG. 6 to be described later) that controls display control of the display unit 104 based on an instruction from a processor 121 described later is disposed in the main body 103. If necessary, a display unit may be arranged in front of both eyes.

FIG. 4 is a schematic cross-sectional view showing the configuration of the display unit 104. The display unit 104 as a display device that displays the detection result of the gesture operation includes an image forming unit 104A and an image display unit 104B. The image forming unit 104A is incorporated in the main body unit 103, and includes a light source 104a, a unidirectional diffuser 104b, a condenser lens 104c, and a display element 104d. On the other hand, the image display unit 104B, which is a so-called see-through type display member, is disposed on the entire plate so as to extend downward from the main body unit 103 and extend in parallel to one eyeglass lens 102 (see FIG. 1). The eyepiece prism 104f, the deflecting prism 104g, and the hologram optical element 104h.

The light source 104a has a function of illuminating the display element 104d. For example, 462 ± 12 nm (B light), 525 ± 17 nm (G light), 635 ± 11 nm (R It is composed of RGB-integrated LEDs that emit light in three wavelength bands. Thus, the light source 104a emits light having a predetermined wavelength width, whereby the image light obtained by illuminating the display element 104d can have a predetermined wavelength width, and the hologram optical element 104h transmits the image light. When diffracted, the image can be observed by the user over the entire observation angle of view at the position of the pupil B. Further, the peak wavelength for each color of the light source 104a is set in the vicinity of the peak wavelength of the diffraction efficiency of the hologram optical element 104h, so that the light use efficiency is improved.

Further, since the light source 104a is composed of LEDs that emit RGB light, the cost of the light source 104a can be reduced, and a color image is displayed on the display element 104d when the display element 104d is illuminated. The color image can be visually recognized by the user. In addition, since each of the RGB LED elements has a narrow emission wavelength width, the use of a plurality of such LED elements enables high color reproducibility and bright image display.

The display element 104d displays an image by modulating the light emitted from the light source 104a in accordance with image data, and is configured by a transmissive liquid crystal display element having pixels that serve as light transmitting regions in a matrix. Has been. Note that the display element 104d may be of a reflective type.

The eyepiece prism 104f totally reflects the image light from the display element 104d incident through the base end face PL1 by the opposed parallel inner side face PL2 and outer side face PL3, and passes through the hologram optical element 104h to the user's pupil. On the other hand, external light is transmitted and guided to the user's pupil, and is composed of, for example, an acrylic resin together with the deflecting prism 104g. The eyepiece prism 104f and the deflection prism 104g are joined by an adhesive with the hologram optical element 104h sandwiched between inclined surfaces PL4 and PL5 inclined with respect to the inner surface PL2 and the outer surface PL3.

The deflection prism 104g is joined to the eyepiece prism 104f, and becomes a substantially parallel flat plate integrated with the eyepiece prism 104f. By joining the deflecting prism 104g to the eyepiece prism 104f, it is possible to prevent distortion in the external image observed by the user through the display unit 104.

That is, for example, when the deflecting prism 104g is not joined to the eyepiece prism 104f, external light is refracted when passing through the inclined surface PL4 of the eyepiece prism 104f, so that an external image observed through the eyepiece prism 104f is distorted. . However, by joining the deflecting prism 104g having the inclined surface PL5 complementary to the eyepiece prism 104f to form an integral substantially parallel plate, external light is transmitted through the inclined surfaces PL4 and PL5 (hologram optical element 104h). The refraction at that time can be canceled by the deflecting prism 104g. As a result, it is possible to prevent distortion in the external image observed through the see-through. In addition, if the spectacle lens 102 (refer FIG. 1) is mounted | worn between the display unit 104 and a user's pupil, it will be possible for the user who normally uses spectacles to observe an image without a problem.

The hologram optical element 104h diffracts and reflects the image light (light having a wavelength corresponding to the three primary colors) emitted from the display element 104d, guides it to the pupil B, enlarges the image displayed on the display element 104d, and enlarges the user's pupil. It is a volume phase type reflection hologram guided as a virtual image. The hologram optical element 104h has, for example, three wavelength ranges of 465 ± 5 nm (B light), 521 ± 5 nm (G light), and 634 ± 5 nm (R light) with a peak wavelength of diffraction efficiency and a wavelength width of half the diffraction efficiency. The light is diffracted (reflected). Here, the peak wavelength of diffraction efficiency is the wavelength at which the diffraction efficiency reaches a peak, and the wavelength width at half maximum of the diffraction efficiency is the wavelength width at which the diffraction efficiency is at half maximum of the diffraction efficiency peak. is there.

The reflection-type hologram optical element 104h has high wavelength selectivity, and only diffracts and reflects light having a wavelength in the above-mentioned wavelength range (near the exposure wavelength). The hologram optical element 104h is transmitted, and a high external light transmittance can be realized.

Next, the operation of the display unit 104 will be described. The light emitted from the light source 104a is diffused by the unidirectional diffusion plate 104b, condensed by the condenser lens 104c, and enters the display element 104d. The light incident on the display element 104d is modulated for each pixel based on the image data input from the display control unit 104DR, and is emitted as image light. Thereby, a color image is displayed on the display element 104d.

Image light from the display element 104d enters the eyepiece prism 104f from its base end face PL1, is totally reflected a plurality of times by the inner side face PL2 and the outer side face PL3, and enters the hologram optical element 104h. The light incident on the hologram optical element 104h is reflected there, passes through the inner side surface PL2, and reaches the pupil B. At the position of the pupil B, the user can observe an enlarged virtual image of the image displayed on the display element 104d, and can visually recognize it as a screen formed on the image display unit 104B. In this case, it can be considered that the hologram optical element 104h constitutes a screen, or it can be considered that a screen is formed on the inner surface PL2. In the present specification, “screen” may refer to an image to be displayed.

On the other hand, since the eyepiece prism 104f, the deflecting prism 104g, and the hologram optical element 104h transmit almost all of the external light, the user can observe an external field image (real image) through them. Therefore, the virtual image of the image displayed on the display element 104d is observed so as to overlap with a part of the external image. In this manner, the user of the HMD 100 can simultaneously observe the image provided from the display element 104d and the external image via the hologram optical element 104h. Note that when the display unit 104 is in the non-display state, the image display unit 104B is transparent, and only the external image can be observed. In this example, a display unit is configured by combining a light source, a liquid crystal display element, and an optical system. However, instead of a combination of a light source and a liquid crystal display element, a self-luminous display element (for example, an organic EL display) is used. Element) may be used. Further, instead of a combination of a light source, a liquid crystal display element, and an optical system, a transmissive organic EL display panel having transparency in a non-light emitting state may be used. In any case, when the screen is arranged so as to fall within the visual field of the user's eye facing the image display unit 104B, and preferably at least partially overlaps the effective visual field, the user can easily visually recognize the image. Can do.

Further, in FIGS. 1 and 2, a proximity sensor 105 disposed near the center of the frame 101 and a lens 106 a of a camera 106 disposed near the side are provided in front of the main body 103 so as to face the front. It has been.

In this specification, a “proximity sensor” which is an example of a detection device refers to a proximity sensor in order to detect that an object, for example, a part of a human body (such as a hand or a finger) is in proximity to the user's eyes. This means that a signal is output by detecting whether or not it exists in a detection region in the proximity range in front of the detection surface. The proximity range may be set as appropriate according to the operator's characteristics and preferences. For example, the proximity range from the detection surface of the proximity sensor may be within a range of 200 mm. If the distance from the proximity sensor is 200mm or less, the user can put the palm and fingers into and out of the user's field of view with the arm bent, so the user can easily operate with gestures using the hands and fingers. In addition, the risk of erroneous detection of a human body or furniture other than the user is reduced. Here, the control circuit determines that the object exists in the proximity range based on a signal output from the proximity sensor when the object enters the detection area in the proximity range in front of the proximity sensor. When an object enters the detection area, a valid signal may be output from the proximity sensor to the control circuit.

There are two types of proximity sensors: passive and active. A passive proximity sensor has a detection unit that detects invisible light and electromagnetic waves emitted from an object when the object approaches. As a passive proximity sensor, there are a pyroelectric sensor that detects invisible light such as infrared rays emitted from an approaching human body, and a capacitance sensor that detects a change in electrostatic capacitance between the approaching human body. The active proximity sensor includes an invisible light or sound wave projection unit, and a detection unit that receives the invisible light or sound wave reflected and returned from the object. Active proximity sensors include infrared sensors that project infrared rays and receive infrared rays reflected by objects, laser sensors that project laser beams and receive laser beams reflected by objects, and project ultrasonic waves. Then, there is an ultrasonic sensor that receives ultrasonic waves reflected by an object. Note that a passive proximity sensor does not need to project energy toward an object, and thus has excellent low power consumption. Active proximity sensors can improve detection reliability. For example, even if the user wears gloves that do not transmit detection light emitted from the human body such as infrared light, the movement of the user's hand Can be detected. A plurality of types of proximity sensors may be used in combination.

近 接 Proximity sensors are generally smaller and cheaper than cameras, and consume less power. In addition, it is difficult for the proximity sensor to perform complicated detection such as detection of the shape of the object, but it can determine the approach and separation of the object from the detection area, so that the user can pass the hand or finger or touch the palm. By holding it over, the HMD can be operated, and complicated image processing required for gesture recognition by analysis of the image captured by the camera is also unnecessary.

FIG. 5 is an enlarged view of the proximity sensor 105 used in the present embodiment as viewed from the front. In this embodiment, an example in which a pyroelectric sensor is used as the proximity sensor 105 will be described. In FIG. 5, the proximity sensor 105 includes a light receiving unit 105 a that receives invisible light such as infrared light emitted from a human body as detection light. The light receiving unit 105a forms light receiving areas RA to RD arranged in 2 rows and 2 columns, and when receiving invisible light, signals corresponding to the light receiving areas RA to RD are individually output from the light receiving areas RA to RD. It has become. The output of each of the light receiving areas RA to RD changes in intensity according to the distance from the light receiving unit 105a to the object, and the intensity increases as the distance decreases. If it is a pyroelectric sensor that detects infrared light emitted from the human body, it is difficult to falsely detect objects other than the exposed human body. For example, when working in a confined space, false detection is effectively prevented. There is an advantage that you can.

1 and 2, the right sub-body portion 107 is attached to the right side portion 101b of the frame 101, and the left sub-body portion 108 is attached to the left side portion 101c of the frame 101. The right sub-main body portion 107 and the left sub-main body portion 108 have an elongated plate shape, and have elongated protrusions 107a and 108a on the inner side, respectively. By engaging the elongated protrusion 107 a with the elongated hole 101 d of the side portion 101 b of the frame 101, the right sub-body portion 107 is attached to the frame 101 in a positioned state, and the elongated protrusion 108 a is attached to the side of the frame 101. By engaging with the long hole 101e of the portion 101c, the left sub-main body portion 108 is attached to the frame 101 in a positioned state.

In the right sub main body 107, there are a geomagnetic sensor 109 (see FIG. 6 described later) for detecting geomagnetism, and an angular velocity sensor 110B and an acceleration sensor 110A (see FIG. 6 described later) that generate an output corresponding to the posture. The left sub-main body 108 is provided with a speaker 111A, an earphone 111C, and a microphone 111B (see FIG. 6 to be described later). The main main body 103 and the right sub main body 107 are connected so as to be able to transmit signals through a wiring HS, and the main main body 103 and the left sub main body 108 are connected so as to be able to transmit signals through a wiring (not shown). Yes. As schematically illustrated in FIG. 3, the right sub-main body 107 is connected to the control unit CTU via a cord CD extending from the rear end. A 6-axis sensor in which an angular velocity sensor and an acceleration sensor are integrated may be used. Further, the HMD can be operated by sound based on an output signal generated from the microphone 111B according to the input sound. Further, the main main body 103 and the left sub main body 108 may be configured to be wirelessly connected. Although not shown in the figure, a power circuit 113 (see FIG. 6 described later) for converting the voltage applied from the battery control unit 128 of the control unit CTU into an appropriate voltage for each unit is provided in the right sub-main unit 107. Yes.

FIG. 6 is a block diagram of main circuits of the HMD 100. The control unit CTU receives the radio waves from the processor 121, the operation unit 122, and the GPS satellite as a control device for controlling each functional device by generating a control signal for the display unit 104 and other functional devices. GPS receiver 123, communication unit 124 as an interface unit for exchanging data with the outside, ROM 125 for storing programs, information on image data, single list and combination list (details will be described later), telephone directory A RAM 126 serving as a storage unit for storing data and a voice dictionary, a battery 127 and a battery control unit 128, and a power supply circuit 130 for converting the voltage applied from the battery control unit 128 into an appropriate voltage for each unit; Storage devices such as SSD and flash memory And a scan 129. The processor 121 can use an application processor used in a smartphone or the like, but the type of the processor 121 is not limited. For example, if an application processor includes hardware necessary for image processing such as GPU or Codec as a standard, it can be said that the processor is suitable for a small HMD.

Furthermore, when the light receiving unit 105a detects invisible light as detection light emitted from the human body, the processor 121 receives a signal from the proximity sensor 105. Further, the processor 121 controls image display of the display unit 104 via the display control unit 104DR. The processor 121 and the proximity sensor 105 constitute an input device.

The processor 121 receives power from the power supply circuit 130, operates according to a program stored in at least one of the ROM 124 and the storage device 129, and receives image data from the camera 106 according to an operation input such as power-on from the operation unit 122. The data can be input and stored in the RAM 126, and can be communicated with the outside via the communication unit 124 as necessary. Further, as will be described later, the processor 121 executes image control according to the output from the proximity sensor 105, so that the user can perform screen control of the HMD 100 including call and video reproduction by gesture operation using hands and fingers. Can be executed.

FIG. 7 is a front view when the user US wears the HMD 100 of the present embodiment. FIG. 8 is a side view when the user US wears the HMD 100, and FIG. 9 is a top view thereof, which is shown together with the user's hand. Here, the gesture operation is an operation in which at least the hand HD or finger of the user US passes through the detection area of the proximity sensor 105 and can be detected by the processor 121 of the HMD 100 via the proximity sensor 105.

Next, the basic principle of gesture operation detection will be described. If there is nothing in front of the user US when the proximity sensor 105 is operating, the light receiving unit 105a does not receive invisible light as detection light, so the processor 121 determines that no gesture operation is performed. To do. On the other hand, as shown in FIG. 8, when the user US's own hand HD is approached in front of the user US, the light receiving unit 105a detects the invisible light emitted from the hand HD, and the proximity sensor 105 based on the invisible light is detected. In response to the output signal, the processor 121 determines that a gesture operation has been performed. In the following description, it is assumed that the gesture operation is performed by the user's hand HD. However, the gesture operation may be performed by the user using a pointing device made of a material that can emit invisible light. May be performed. An indicator that is moved by the user's intention, such as a finger or other part of the human body or the indicator. An imaging device (camera) can be used as the detection device instead of the proximity sensor. In such a case, the moving indicator is imaged by the imaging device, and the moving direction is determined by the image processing device (part of the control device) based on the image data output from the imaging device.

As described above, the light receiving unit 105a has the light receiving regions RA to RD arranged in 2 rows and 2 columns (see FIG. 5). Therefore, when the user US moves the hand HD closer to the front of the HMD 100 from either the left, right, up, or down directions, the output timings of signals detected in the light receiving areas RA to RD are different.

10 and 11 are examples of signal waveforms of the light receiving areas RA to RD, where the vertical axis represents the signal intensity of the light receiving areas RA to RD and the horizontal axis represents the time. For example, when the user US moves the hand HD from the right to the left in front of the HMD 100 with reference to FIGS. 8 and 9, invisible light emitted from the hand HD enters the light receiving unit 105a. In this case, the light receiving areas RA and RC first receive invisible light. Therefore, as shown in FIG. 10, first, the signals in the light receiving areas RA and RC rise, the signals in the light receiving areas RB and RD rise later, and the signals in the light receiving areas RA and RC fall, and then the light receiving areas RB, The RD signal falls. The processor 121 detects the timing of this signal, and determines that the user US has performed the gesture operation by moving the hand HD from the right to the left.

The light receiving state of the proximity sensor 105 when the hand HD is moved from the right to the left when viewed from the user side as shown in FIG. 11 is shown corresponding to FIG. 12 (the light receiving area of the proximity sensor 105 is shown by a circle). The light receiving area detected here is indicated by hatching. In FIG. 12A, none of the light receiving areas detects the hand HD, but in FIG. 12B, only the light receiving areas RA and RC detect the hand HD, and the hand HD is on the right side of the detection area. You can see that you have entered. Further, in FIG. 12C, since all the light receiving areas detect the hand HD, it can be seen that the hand HD has come in front of the proximity sensor 105. In FIG. 12D, it can be seen that only the light receiving areas RB and RD have detected the hand HD, and the hand HD has moved to the left side of the detection area. In FIG. It can be understood that the hand HD has been withdrawn from the detection area by detecting the hand HD also in the light receiving area. That is, the processor 121 can recognize that the hand HD has passed through the detection area from right to left by receiving a series of output signals from the proximity sensor 105.

On the other hand, in the example of FIG. 13, the signals in the light receiving areas RA and RB rise first, the signals in the light receiving areas RC and RD rise later, and the signals in the light receiving areas RA and RB fall, and then the light receiving areas RC and RD. The signal is falling. The processor 121 detects the timing of this signal, and determines that the user US has performed the gesture operation by moving the hand HD from the top to the bottom.

Note that, based on the output from the proximity sensor 105, the processor 121 can also detect that the hand HD has moved in the front-rear direction. FIG. 14 shows an example of a signal waveform in which the vertical axis represents the average signal intensity of the light receiving regions RA to RD and the horizontal axis represents time. The waveform indicated by the solid line is high in intensity immediately after output and decreases with time. Therefore, when such a waveform is output from the proximity sensor 105, the processor 121 determines that the gesture operation is performed by moving the hand HD away from the vicinity of the face. On the other hand, the waveform indicated by the dotted line is low in intensity immediately after output and increases with time. Therefore, when such a waveform is output from the proximity sensor 105, the processor 121 determines that the gesture operation is performed by moving the hand HD so as to approach the face from a distance.

According to the present embodiment, the presence and movement of the hand HD can be reliably detected by using the proximity sensor and positioning the detection region within the visual field of the user's eye facing the image display unit. Accordingly, the user US can view the hand operation and the operation in conjunction with each other, and can realize an intuitive operation. Furthermore, since the proximity sensor is small and consumes less power than the camera, the continuous operation time of the HMD 100 can be extended.

By the way, when a gesture operation is detected using a detection device having a simple configuration such as a proximity sensor, if it is a complicated gesture operation, there is a high risk of erroneous detection, and the usability becomes poor for the user. On the other hand, for example, by limiting the gesture operation to a simple operation such as moving the hand up and down, left and right, diagonally, near and far, or stationary, the risk of erroneous detection can be reduced. However, such a simple operation cannot cope with fine input contents. Therefore, it is conceivable to combine a plurality of gesture operations to correspond to fine input contents. However, when combining a plurality of gesture operations into one input operation, there arises a problem of how to separate it from a single gesture operation.

In order to solve such a problem, in the present embodiment, a single mode for inputting the input content desired by the user by a single gesture operation and a plurality of gesture operations are combined to obtain the input content desired by the user. The combination mode to be input can be selectively set, and the mode is switched by a specific gesture operation.

Here, as a single gesture operation that can be detected in the detection region, there are the following operations. The processor 121 recognizes the pattern in advance as an effective gesture operation.
Move from top to bottom of hand, move from bottom to top of hand, move from left to right of hand, move from right to left of hand, move from bottom left to right of hand, from bottom right of hand Move to the upper left, move from the upper left to the lower right of the hand, move from the upper right to the lower left of the hand, rotate around the hand, click operation (moves the hand quickly back and forth), rest of the hand (short), rest of the hand (Long), moving your hand from side to side at high speed.

In the case of the single mode, the input content may be determined individually corresponding to the above gesture operations. On the other hand, in the combination mode, it is necessary to use a combination of the above gesture operations. Therefore, it is necessary to switch modes so as not to confuse gesture operations.

When switching modes, a gesture operation for switching modes is set in advance, and by detecting the gesture operation, the processor 121 can switch between the single mode and the combination mode. Note that the switching from the single mode to the combination mode and the switching from the combination mode to the single mode may be performed by either the same gesture operation or different gesture operations. Here, the mode switching gesture operation (referred to as mode switching gesture operation) is common, and as shown in FIGS. 8 and 9, the hand HD is held still for 2 seconds or more in the center of the detection area. However, for example, the mode switching may be performed by detecting a voice such as “mode switching” issued by the user US using the voice recognition function of the processor 121 via the microphone 111B. In addition, when the combination mode is set, a gesture operation (referred to as a cancel gesture operation) for canceling the content of the previous gesture operation (referred to as input information) moves the hand HD left and right at high speed within the detection area. . However, when the cancel gesture operation is performed in a state where no input information is stored, the single mode may be switched. Further, when the combination mode is set, in order to confirm the selected input content, for example, a gesture operation (decision gesture operation) that moves the hand HD back and forth in the center of the detection area of the proximity sensor 105 may be set in advance. good.

In addition, a single list in which a single gesture operation in the single mode is associated with the input content and a combination list in which a combination of a plurality of gesture operations in the combination mode is associated with the input content are stored in the RAM 126 in advance. It is assumed that the processor 121 can read out as necessary. At the time of setting the combination mode, the processor 121 sequentially stores input information by successive gesture operations and collates with the combination list. The combination list includes a mode switching gesture operation, a cancel gesture operation, and a determination gesture operation in addition to an effective gesture operation.

In the following example, it is assumed that the single mode is set by default, and when the mode switching gesture operation is detected after the transition to the combination mode, or when input by a plurality of gesture operations is completed. , And transition to single mode. However, the combination mode may be continued unless the user performs a mode switching gesture operation.

FIG. 15 is a flowchart showing a single mode sequence. When the single mode is set, when the proximity sensor 105 outputs a signal in response to the gesture operation in step S101 of FIG. 15, the processor 121 is detected via the proximity sensor 105 in step S102. It is determined from the gesture operation pattern whether or not the gesture operation is valid. If the gesture operation is not valid, the sequence is returned to step S101 to wait for the next gesture operation.

On the other hand, if it is determined that the gesture operation is valid, in step S103, the processor 121 determines whether the gesture operation is a mode switching gesture operation. If the processor 121 determines that the gesture operation is not a mode switching gesture operation, step S104 is performed. Thus, the processor 121 determines the input content corresponding to the gesture operation detected through the proximity sensor 105 based on the single list, and executes it. On the other hand, when the processor 121 determines that the operation is a mode switching gesture operation, the single mode is interrupted in step S105, and the mode is shifted to the combination mode. Hereinafter, the processor 121 determines the input content on the assumption that a plurality of gesture operations are continued.

FIG. 16 is a flowchart showing the sequence of the combination mode. After the transition to the combination mode, when the proximity sensor 105 outputs a signal in response to the gesture operation in step S201, the processor 121 determines from the gesture operation pattern detected via the proximity sensor 105 in step S202. It is determined whether or not it is a mode switching gesture operation.

When determining that the detected gesture operation is a mode switching gesture operation, the processor 121 deletes the stored input information in step S210, interrupts the combination mode in step S211, and shifts to the single mode. .

On the other hand, if it is determined that the operation is not a mode switching gesture operation, in the subsequent step S203, the processor 121 determines whether or not the detected gesture operation exists in the combination list. If the detected gesture operation does not exist, in step S208, the processor 121 displays the input gesture operation content as an option on the display unit 104, and then returns the sequence to step S201.

On the other hand, if the processor 121 determines that the detected gesture operation is present in the combination list, the processor 121 shifts the flow to step S204 and further determines whether the detected gesture operation is a cancel gesture operation. . If it is determined that the operation is a canceling gesture operation, in step S205, the processor 121 deletes input information corresponding to the previous gesture operation, and in step S208, it is combined with the gesture operation performed at the current time. The possible input contents are listed and displayed on the display unit 104, and then the sequence is returned to step S201. Thereby, an invalid gesture operation is ignored, and a gesture operation that can be input can be notified to the user US.

On the other hand, if it is determined that the detected gesture operation is not a cancel gesture operation, the processor 121 stores the input information in step S206, and determines whether there are other candidates in step S207. If it is determined that there are other candidates, the processor 121 displays, in step S208, possible input contents in combination with the gesture operation performed at the present time as options on the display unit 104, and then the sequence proceeds to step S201. return.

On the other hand, if it is determined that there is no other combination candidate in the combination list, the processor 121 determines the input content in step S209 and executes an event corresponding to the content. However, even when it is determined that there are no other combination candidates in the combination list, only when a gesture operation for repeatedly moving the hand HD in the detection area is detected as an input confirmation gesture operation, for example, May be confirmed. Thereafter, the processor 121 deletes the input information stored in step S210, interrupts the combination mode in step S211, and shifts to the single mode. In the embodiment described above, the combination list includes the mode switching gesture operation, the cancel gesture operation, and the determination gesture operation. However, the processor 121 includes one or more of these in the event list or another list. You may make it refer.

FIG. 17 is a flowchart showing a part of a modification of the sequence of FIG. 16, which can be executed in place of step S203 of FIG. Here, only a different part from FIG. 16 is demonstrated. In step S301 following step S202 of FIG. 16, the processor 121 checks whether or not the gesture operation detected via the proximity sensor 105 is an effective gesture operation after collating with the combination list. If the gesture operation is not valid (ie, an invalid gesture operation is detected), the processor 121 increments the value of the built-in counter in step S302 (if invalid judgment is first, 1 is continuously 2). If it is the second time,...) Advances the sequence to step S304. On the other hand, if a valid gesture operation is detected, the processor 121 resets the value of the built-in counter in step S303 (invalidity determination is interrupted), and advances the sequence to step S204 in FIG. Run the sequence.

On the other hand, in step S304, the processor 121 determines whether or not the value of the built-in counter is greater than or equal to a threshold value N. If it is greater than or equal to N, it can be said that the number of invalid gesture operations performed by the user has exceeded a specified number. It is assumed that there is a high possibility that the user does not recognize the gestures that can be operated, and a list display flag of combinations of gesture operations that can be input is set. On the other hand, if the value of the built-in counter is less than the threshold value N, the processor 121 sets a flag for not displaying a list of combinations of gesture operations that can be input in step S306 in order not to confuse the user. After that, the sequence proceeds to step S206 in FIG. 16. However, if the list display flag is set in step S208 in FIG. 16, the processor 121 sets a combination of gesture operations that can be input to the display unit 104. The list is displayed to assist the user in inputting, but the list is not displayed when the flag for hiding the list is set.

According to the present embodiment, by setting the combination mode, it is possible to determine different input content from when a single gesture operation is detected in response to detecting a plurality of gesture operations continuously. , The degree of freedom in setting the input content increases.

Next, the operation of this embodiment will be described with a specific example. Here, while the processor 121 is executing the call application, the user makes an input using a gesture operation, so that the other party who wants to call can be selected from the phone book registration list stored in the RAM 126. Select and call the selected partner.

FIG. 18 shows the phone book data stored in the RAM 126, and FIG. 19 shows an example of an event list and a combination list corresponding to the phone book data and used in the call application. In the phone book data, the first name is classified as a group belonging to the “a” line, a group belonging to the “ka” line, a group belonging to the “sa” line, and so on. Each is associated with a corresponding phone number. The event list in FIG. 19 (a) records destinations that can be input by a combination (combination) of two gesture operations, and the event list in FIG. 19 (b) has a combination of two or three gesture operations. The other party that can be input in (Combination) is recorded in advance. An arrow in the figure means a gesture operation that moves a hand in the direction of the arrow within the detection region. In addition, you may set the other party who can input by the combination of four or more gesture operation.

Here, taking the case where the user wishes to make a call to “Technology Department Toshiyuki Komine” recorded in the phone book as an event, the call destination is stored as an event list. If a call to a specific destination is to be made in a single mode, a phone book as shown in FIG. 18 is displayed on the display unit 104 based on the phone book data, and then, for example, the hand is moved from left to right. After switching to the display of the destination group belonging to the “ka” line by the gesture operation, for example, the screen is scrolled by a single gesture operation to move the hand from top to bottom, and “ You can select Toshiyuki Komine, Engineering Department. However, when a large number of destinations are recorded in the phone book, it may be necessary to scroll the screen by repeating the same gesture operation many times until the destination that is desired to be placed is displayed. ,It takes time and effort. On the other hand, according to the present embodiment, by switching to the combination mode, it is possible to make a call to a desired destination only by a plurality of gesture operations.

Specifically, the procedure according to this embodiment will be described. Here, as shown in FIG. 19, the user US stores an opponent with a high call frequency (including “Technology Department Toshiyuki Komine”) as an event list, and associates it with a plurality of gesture operations. It is assumed that a combination list is stored. First, when the user US performs a mode switching gesture operation by holding the hand HD in the center of the detection area of the proximity sensor 105 and standing still for 2 seconds or more, the processor 121 performs a combination according to the output signal from the proximity sensor 105. Enter mode.

Thereafter, the user US moves the hand HD from the left to the right within the detection area of the proximity sensor 105 according to the gesture operation combination of “Technology Department Toshiyuki Komine” shown in FIG. When the hand HD is moved from the bottom to the top, the processor 121 determines that the user US has input the content (event) that calls to “Technology Department Toshiyuki Komine”, and the stored “Technology Department Small” Identify the phone number of Toshiyuki Tsuji and immediately execute the call operation. As a result, the user US can talk to “Technology Department Toshiyuki Komine” with a simple operation. In this embodiment, the sequence returns to the single mode simultaneously with the call execution. However, only when executing a special application such as a call, the combination mode is maintained even after the call is executed. May be.

However, there is also a risk that the user US will make a mistake. For example, the user US who wants to call “Technology Department Toshiyuki Komine” moves his hand HD from right to left in the detection area of the proximity sensor 105 by the first gesture operation after the transition to the combination mode. In this case, the processor 121 recognizes the gesture operation for cancellation by moving the hand HD further to the left and right at a high speed, so that the previous gesture operation can be invalidated and the first gesture operation can be performed again. .

According to the present embodiment, it is possible to realize the input desired by the user with a simple and small number of gesture operations. In particular, in the above example, it is easier to make a call by recording a destination with a high call frequency in the event list in advance and recording it in the combination list in association with a small number of gesture operations. It becomes possible.

Furthermore, when the number of destinations to be called is large, it can be dealt with by increasing the number of combinations of gesture operations to 3 or more accordingly. In such a case, it becomes a problem to separate the number of combinations of gesture operations between two and three. FIG. 19B shows an example of a combination list and event list in which the number of combinations of gesture operations is two and three. In this example, the hand HD is moved from top to bottom within the detection area of the proximity sensor 105. If the hand is further moved from top to bottom after the movement, three candidates will remain in the combination list. In this case, the processor 121 does not immediately call and waits for the next instruction from the user US.

The user US who intended to make a call to “Planning Department Kyoko Yokoi” recognizes that there is another candidate by not making a call, so the gesture operation for decision that has been set in advance , The processor 121 determines that the user US has entered the information that the call is made to “Planning Department Yokoi Kyoko”, identifies the stored telephone number of “Planning Department Kyoko Yokoi”, and immediately A call operation can be performed.

If the determination gesture operation is not performed, the processor 121 assumes that the user US intends to select another candidate, and the gesture operation that moves the hand HD from the bottom to the top (to “Planning Department Yumi Wakai” Call) or a gesture operation to move the hand HD from right to left (call to “Planning Department Hisashi Wakui”). If any gesture operation is performed, the input is confirmed, so that the call operation can be immediately executed to the confirmed partner. When any other gesture operation is performed, the processor 121 can sound a warning sound indicating that the gesture operation is invalid.

Also, in order to assist input, in addition to detecting gesture operations via proximity sensors, if other sensors are used in combination, the degree of freedom of the input content can be expanded and the user can easily realize the desired operation. Become. For example, in addition to the gesture operation, a combination list that combines the tilt of the neck of the user US and a specific voice can be stored in the RAM 126. In such a case, in addition to the proximity gesture 105 detecting the predetermined gesture operation, the processor 121 detects the tilt of the neck of the user US with the acceleration sensor 110A or collects the voice of the user US with the microphone 111B. The user can confirm the input contents desired by the user according to the combination list after performing speech recognition based on the analysis or the speech dictionary.

By the way, since it is difficult to store all the combination lists associated with the phone book data as described above, for example, when the user US makes a call to a destination with a low call frequency, the user US needs to If the combination list can be called from the RAM 126 and confirmed, the gesture operation can be supported, so that it can be said to be more user friendly. However, since some combination lists have a huge amount, it may not be appropriate to display all of them on the display unit 104. In the following embodiment, such a problem is solved by performing navigation display on the display unit 104.

Here, an example of executing a camera application is shown. FIG. 20 is a diagram of the image display unit 104B of the display unit 104 as viewed from the user US side. In FIG. 20, the icon IC1 displayed at the upper left indicates that the processor 121 is executing the camera application by a predetermined operation. It should be noted that images (including arrow icons and the like) that display input history and gesture operation combination candidates described below are stored in advance in the RAM 126 and the like, and are called from the processor 121 as necessary, and the proximity sensor 105 The corresponding image is displayed by collating with the output signal.

Since the default setting for input is single mode, immediately after starting the camera application, only the icon IC1 is displayed as shown in FIG. A user who sees such a screen display knows that the single mode is set.

As described above, when the user performs a mode switching gesture operation, the processor 121 interrupts the single mode and starts the combination mode based on the output signal from the proximity sensor 105. Then, as shown in FIG. 20B, three frames FR representing a history of gesture operations are displayed here at the lower right of the screen of the image display unit 104B, and a gesture is displayed near the lower left of the screen. Information CAN indicating a candidate for the combination of operations is displayed. In the frame FR, display is performed from the left (here, arrows) in the order in which the gesture operation is recognized.

Here, the information CAN indicates a combination of arrows indicating the direction of the gesture operation and the corresponding input content. It is difficult for the screen of the image display unit 104B to display a lot of information because the user observes the outside world through this screen. Therefore, in the present embodiment, only two combinations of gesture operations are displayed in the information CAN, and the rest of the triangles TR displayed above or below the displayed information CAN are displayed by, for example, clicking operations (for example, hand operations). The remaining combinations can be displayed two by two by moving the HD back and forth within the detection area of the proximity sensor 105. By viewing such a display, the user can perform the operation accurately even if the user has forgotten the combination of gesture operations for performing a desired input. The frame FR and the information CAN are referred to as navigation display.

In FIG. 20B, since it is immediately after the combination mode is started, as combinations of gesture operations, combinations of all gesture operations in which data is stored in advance in the RAM 126 or the like and possible during execution of the camera application are possible. Can be displayed as information CAN.

Here, when the user moves the hand HD from the left to the right within the detection area of the proximity sensor 105, the processor 121 determines that the user moves the hand HD from the left to the right based on the output signal from the proximity sensor 105. As shown in FIG. 20C, an arrow heading from left to right is displayed in the leftmost frame FR in the screen of the image display unit 104B. The user who sees this shows that the processor 121 has correctly recognized the gesture operation performed by the user. If it is found from the display in the frame FR that the gesture operation is not as intended by the user, the user can invalidate the previous gesture operation by the cancel gesture operation as described above.

At the same time, the processor 121 narrows down the combinations in which the gesture operation is first performed by moving the hand HD from left to right, and displays the information CAN as shown in FIG. In the updated information CAN, only gesture operations that can be performed thereafter are displayed, so that the user who sees this can input “stop video recording” by moving the hand HD from the bottom to the top, for example. It can be seen that

However, a user who does not wish to input “stop video recording” performs another gesture operation. It is assumed that this user desires input of “moving image reproduction”. In such a case, when the user subsequently moves the hand HD from the left to the right, the screen of the image display unit 104B is switched, as shown in FIG. Then, two arrows from left to right are displayed side by side in the two left frames FR in the screen of the image display unit 104B. Note that gesture operations that are not included in the combinations of gesture operations may be invalidated without being displayed on the screen.

At the same time, when the processor 121 narrows down the hand movement from left to right to the combination of gesture operations that is repeated twice, the number of combinations of gesture operations is narrowed down to one, FIG. ), Information CAN indicating a combination of one gesture operation is displayed. Here, the user performs the gesture operation for moving the hand HD from the upper left to the lower right, or the input operation is confirmed by performing the above-described gesture operation for confirming the input, and the moving image reproduction desired by the user is performed. Can do.

If the setting is made so that the combination mode is not terminated unless the mode switching gesture operation is performed, the mode switching gesture operation is performed at this point to shift to the single mode. The screen changes to the one shown in FIG. 20A, but if a gesture operation other than the mode switching gesture operation is performed, the screen of the image display unit 104B changes to the one shown in FIG. Information CAN indicating a combination candidate following the gesture operation is displayed. On the other hand, in the case of setting to return to the single mode for each operation in which one input content is confirmed, the screen of the image display unit 104B is switched to the one shown in FIG. .

In the present embodiment, it is desirable that the event list and the combination list data are stored in advance in the RAM 126 or the like in association with each other, and the processor 121 reads them as necessary. FIG. 21 is a diagram illustrating an example of an event list and a combination list.

In the example shown in FIG. 21, an event list and a combination list are used for the call application. Here, in the event list (a list of combinations of a plurality of gesture operations associated with inputable contents), each event corresponds to the input contents. Specifically, “event1” is “Sales Department Ai Aoki "Event2" is a content to input an instruction to call "Sales Department Ito Kaoru", and "event3" is to "Planning Department Takumi Aizawa" The content for inputting an instruction to call is “eventX”, and the content for inputting an instruction to end the call. The execution program corresponding to the event is determined with reference to the event list.

On the other hand, according to the combination list, “event1” can be instructed to execute by inputting the gesture operation “action3”, “event2” can be instructed to execute by inputting the gesture operation “action2 + action2”, and “event3” Execution can be instructed by inputting the operation “action1 + action4 + action5”, and execution of “eventX” can be instructed by inputting the gesture operation “action5 + action5”.

Here, “action1” is a hand movement from bottom to top, “action2” is a hand movement from top to bottom, “action3” is a hand movement from right to left, and “action4” is from left to right If the action is set in advance such that "action5" is the back-and-forth movement of the hand, the processor 121 checks the event list against the combination list when setting the combination mode. It is possible to call the other party.

Creating an event list and a combination list (here, consisting of action settings and combination settings) is generally more time-consuming than the HMD100 because it requires a lot of data and the amount of data entry. It is desirable to read out the data via the interface unit and store it in the RAM 126 or the like. FIG. 22A shows an input screen directly input by a user using an external PC or the like. Here, an example used for a camera application and a call application is shown. In the combination list in FIG. 22A, “action” defines a gesture operation pattern (action setting). Specifically, “action 1” indicates movement of the hand from the bottom to the top, and “action 2”. "Indicates the movement of the hand from top to bottom," action 3 "indicates the movement of the hand from right to left, and" action 4 "indicates the movement of the hand from left to right. In this example, “action 5” indicates a click operation by moving the hand back and forth. “Mode” indicates an application name, and “event” indicates input contents.

On the other hand, as shown in FIG. 22A, in the combination setting used in the camera application, “event1” can be instructed to be executed by inputting the gesture operation “action1”, and “event2” is the gesture operation “action2 + action3”. Execution can be instructed by input, and “event3” indicates that execution can be instructed by inputting a gesture operation “action4 + action4 + action5”.

On the other hand, as shown in FIG. 22A, in the combination setting used in the call application, “event1” can be instructed to be executed by inputting the gesture operation “action3”, and “event2” is the gesture operation “ Execution can be instructed by inputting “action2 + action2”, and “event3” indicates that execution can be instructed by inputting the gesture operation “action1 + action4 + action5”. Thus, the combination of gesture operations can be arbitrarily changed for each application.

The above event list and combination list are created on a PC or the like, and then converted into a QR code (registered trademark) (see FIG. 22B, but the QR code (registered trademark) shown is different from the actual one) The QR code (registered trademark) is imaged by the camera 106 mounted on the HMD 100 as the interface unit, and the processor 121 reads the list, so that the list can be loaded and stored in the RAM 126 or the like. Alternatively, although not shown, the event list and the combination list are created on the WEB screen and stored in a specific server, and the communication unit 124 accesses the specific server based on the URL and downloads the data to the HMD 100. You can also

∙ Explain about the function that can change the program to be executed by the combination of gesture operation for each application. FIG. 23A is a diagram illustrating an example of an event in which contents are input by a combination of gesture operations for each application. FIG. 23B shows an example of a combination list. Here, to simplify the explanation, it is assumed that all applications refer to a combination list having the same setting.

For example, when the home screen is displayed (application not executed) and the gesture operation “action2 + action4” is performed when the combination mode is set, the processor 121 starts the “QR code (registered trademark) -read application corresponding to“ event1 ”. Is determined to have been input. On the other hand, when the same gesture operation is performed during execution of the call application, the processor 121 determines that an instruction “call to Mr. XX” is input. Thus, even if the same gesture operation is performed by a running application, different contents are input. As described with reference to FIG. 22, these can be easily set by an event list and a combination list. These lists are stored in the RAM 126, and the processor 121 corresponds to each application being executed. The list can be read. Both the combination list and the event list can be directly input or externally input via a QR code (registered trademark) or the net.

In the embodiment described above, the gesture operation mainly based on the moving direction of the indicator has been described as an example. However, for example, a change in the shape of the hand such as opening and closing the hand may be added to the gesture operation to be detected. . When such a gesture operation is displayed on the display unit 104 (see FIG. 20), it can be expressed by an animation movie or the like that repeats the operation of opening and closing the hand.

The present invention is not limited to the embodiments described in the specification, and other embodiments and modifications are included for those skilled in the art from the embodiments and technical ideas described in the present specification. it is obvious. The description and the embodiments are for illustrative purposes only, and the scope of the present invention is indicated by the following claims.

100 HMD
101 Frame 101a Front part 101b Side part 101c Side part 101d Long hole 101e Long hole 102 Eyeglass lens 103 Main body part 104 Display 104A Image forming part 104B Image display part 104DR Display control part 104a Light source 104b Unidirectional diffuser 104c Condensing lens 104d Display element 104f Eyepiece prism 104g Deflection prism 104h Hologram optical element 104i Screen 105 Proximity sensor 105a Light receiving part 106 Camera 106a Lens 107 Right sub-main part 107a Protrusion 108 Left sub-main part 108a Protrusion 109 Geomagnetic sensor 110A Acceleration sensor 110B Angular velocity sensor 111A Speaker 111B Microphone 111C Earphone 113 Power supply circuit 121 Processor 122 Operation unit 123 GPS reception unit 124 Communication unit 125 ROM
126 RAM
127 Battery 128 Battery control unit 129 Storage device 130 Power supply circuit CD Code CTU Control unit HD Hand HS Wiring PL1 Base end surface PL2 Inner surface PL3 Outer surface PL4 Inclined surface PL5 Inclined surface RA-RD Light receiving area SA Detection area US User

Claims (11)

A detection device that detects a movement and / or shape change of the indicator moved by the user as a gesture operation within the detection region, and generates an output according to the gesture operation;
A control device for determining input contents according to the output of the detection device, and an input device comprising:
In response to the detection device detecting the single gesture operation, the control device determines a single input mode corresponding to the single gesture operation, and the detection device continuously detects a plurality of the gesture operations. Accordingly, an input device capable of selectively setting a combination mode for determining input contents different from the case where a single gesture operation is detected. The input device according to claim 1, wherein the control device switches between the single mode and the combination mode according to a signal output from the detection device that has detected a gesture operation for mode switching. 3. The control device according to claim 1, wherein when the combination mode is set, the control device invalidates a gesture operation performed immediately before according to a signal output from the detection device that has detected a cancel gesture operation. Input device. The input device according to any one of claims 1 to 3, wherein the control device is capable of executing a plurality of applications, and a combination of a plurality of gesture operations corresponding to contents that can be input in each application is different. 5. The input device according to claim 1, further comprising a display device that displays a detection result of the gesture operation. The input device according to claim 5, wherein the control device displays a history of the gesture operation detected by the detection device on the display device when the combination mode is set. A storage unit that stores a list of combinations of a plurality of gesture operations in association with inputable contents, and the control device, based on the list stored in the storage unit, when the combination mode is set, The input device according to claim 5, wherein gesture operation candidates that can be further combined with the gesture operation detected by the detection device are displayed on the display device. The control device collates with the list every time the detection device detects the gesture operation to determine whether the detected gesture operation is invalid or not, and the number of times it is determined to be invalid is equal to or more than a specified number The input device according to claim 7, wherein a gesture operation candidate that can be combined is displayed on the display device. The input device according to claim 7 or 8, further comprising an interface unit that receives data from outside in order to store the list in the storage unit. An electronic device equipped with the input device according to any one of claims 1 to 9. 10. A head mounted display on which the input device according to claim 5 is mounted, comprising a support member for attaching the display device and the detection device to the head of the user, the support member Is a head mounted display that supports the display device so that it is positioned in front of the user's eyes and supports the detection device so that a detection area of the detection device is in front of the user's eyes.

Images