WO2024203458A1 - Image display device, head-mounted display, projector, and image display method - Google Patents

Abstract

High resolution and high frame rate are required in an image display device, but resolution and frame rate are related generally as a tradeoff. The present invention was developed in view of this problem, and the main purpose of the present invention is to provide an image display device and method capable of improving a frame rate without lowering resolution.　In this image display device, a scanning range (11) of a region smaller than the maximum scannable range (10) in which scanning is possible is set, the scanning range (11) is scanned, and video is drawn in a drawing range (12) provided in the scanning range (11).

Description

Image display device, head-mounted display, projector, and image display method

This technology relates to an image display device, a head-mounted display, a projector, and an image display method. More specifically, this technology relates to an image display device, a head-mounted display, a projector, and an image display method that set a scanning range for an area smaller than the maximum scannable range.

　Technologies have been known for some time now to improve the resolution of images output by image display devices, and to enhance the sense of realism by producing clearer images.

For example, the following Patent Document 1 discloses a technology for presenting a high-resolution image at the wearer's gaze point using an image display device equipped with a gaze detection device, a wide-field-of-view image creation device that creates a wide-field-of-view image, and a narrow-field-of-view image creation device that creates a high-resolution narrow-field-of-view image with a narrower viewing angle than the wide-field-of-view image.

Japanese Patent Application Publication No. 8-313843

However, image display devices require high resolution and high frame rates, and there is generally a trade-off between resolution and frame rate.

This technology was developed in light of these circumstances, and its main objective is to provide an image display device and method that can improve frame rate without reducing resolution.

As a result of extensive research, the inventors have discovered that it is useful for an image display device to set a scanning range that is smaller than the maximum scannable range, scan that scanning range, and display an image.

In other words, the present technology provides an image display device that includes a drawing unit that scans an area in which an image is to be drawn based on input image information and draws the image, and a scanning range setting unit that sets a scanning range of an area smaller than the maximum scannable range in which the scanning is possible, and the drawing unit draws the image in a drawing range set within the scanning range.
In the image display device according to the present technology, a scanning time of the scanning range may be less than 100% while a scanning time of the maximum scannable range is 100%.
In the image display device according to the present technology, the drawing range may be smaller than the scanning range, and the drawing section may have a drawing position selection mechanism that selects a drawing position to be performed within the drawing range.
In the image display device relating to the present technology, the drawing unit may be provided with a scanning mirror, and the scanning may be performed by the scanning mirror, and the scanning may be performed by a combination of two axes, a non-resonant axis direction and a resonant axis direction, and scanning of the scanning range may be performed by adjusting the movement in the non-resonant axis direction.
In the image display device according to the present technology, the set scanning range may be scanned, and the setting of the next scanning range may be repeatedly performed. Also, the set scanning range may be scanned, and a scanning limit range that is smaller than the maximum scannable range and wider than the scanning range at the position of the scanning range may be set, and the setting of the next scanning range may be repeatedly performed in the scanning limit range. The scanning limit range may be set as a region that is separated by a maximum value of the movement amount of the gaze point due to the user's saccade per unit time in two directions of the center line of the scanning range.
In the image display device according to the present technology, the scanning may be performed with a horizontal direction as a resonance axis.
In the image display device according to the present technology, the drawing section may include a laser unit, and the drawing may be performed by the laser unit, or the laser unit may be provided in plurality.
The image display device according to the present technology may further include a wide-angle rendering unit that renders a wide-angle image having a wider angle of view than the image rendered by the rendering unit.
In the image display device according to the present technology, the scanning range setting unit may set the scanning range through an external input.
The image display device according to the present technology may further include a gaze detection unit that detects a gaze of a user, and may set the scanning range in accordance with eyeball information detected by the gaze detection unit.
In the image display device according to the present technology, the image drawn by the drawing unit may be output to a display unit worn by the user, and the output to the display unit may be performed by switching between a plurality of image lights based on eyeball information detected by the gaze detection unit. Furthermore, the output to the display unit may be performed by a retinal direct drawing method.

Next, the present technology provides a head mounted display including an image display device according to the present technology.
The present technology provides a projector including an image display device according to the present technology.

The present technology further provides an image display method having a scanning process of scanning an area in which an image is to be drawn based on input image information, and a drawing process of drawing the image, in which the scanning process sets a scanning range of an area smaller than a maximum scannable range in which the scanning is possible, and the drawing process draws the image in a drawing range set within the scanning range.
In the image display method according to the present technology, the scanning range may be scanned for less than 100% of the scanning time of the maximum scannable range, which is 100%.

FIG. 1 is an image diagram of scanning when drawing an image using a laser beam scanning system. 1 is an image diagram showing scanning and drawing in an image display device according to the present technology, and a graph showing drawing time. 1A and 1B are conceptual diagrams showing scanning and drawing in an image display device according to a conventional technique, and a graph showing drawing time. 13A and 13B are conceptual diagrams illustrating modified examples of scanning and drawing in the image display device according to the present technology. 1 is a schematic diagram showing a configuration example of an image display device according to the present technology. 1A and 1B are schematic diagrams illustrating an example of control of a scanning mirror in a case where the image display device according to the present technology includes a scanning mirror. 1A and 1B are schematic diagrams showing examples of a method for multiplexing laser light when an image display device according to the present technology includes a laser unit. 1 is a schematic diagram showing an example of scanning during drawing in an image display device according to the present technology having a plurality of laser units. FIG. 1 is a schematic diagram showing an example of a configuration in which an image display device according to the present technology is provided with a wide-angle rendering unit that renders a wide-angle image having a wider angle of view than an image rendered by a rendering unit. 11 is a schematic diagram showing a modified example of an image display device according to the present technology including a wide-angle rendering unit that renders a wide-angle image having a wider angle of view than an image rendered by the rendering unit. FIG. 1 is a schematic diagram showing an example of a form in which a scanning range is set by an external input in an image display device according to the present technology; 1 is a schematic diagram showing an example of a form in which a scanning range is set by an external input using a line-of-sight detection unit in an image display device according to the present technology; 1 is a schematic diagram showing an example of a form in which an image display device according to the present technology includes a gaze detection unit that detects the gaze of a user and sets a scanning range in accordance with eyeball information detected by the gaze detection unit. 11 is a schematic diagram showing the maximum time required for one frame in the image display device according to the present technology when the saccade of the user is not taken into consideration. FIG. 11 is a schematic diagram showing the maximum time required for one frame when taking into account a user's saccade in an image display device according to the present technology. FIG. 11 is a schematic diagram showing how the image display device according to the present technology repeatedly performs scanning of a set scanning range and setting of a next scanning range in a scanning limit range. FIG. This is a schematic diagram showing an example of a configuration in which an image display device relating to the present technology is equipped with a gaze detection unit that detects the user's gaze and sets a scanning range according to eyeball information detected by the gaze detection unit, and further equipped with a wide-angle drawing unit. This is a schematic diagram showing an example of a case where an image display device according to the present technology is provided with a gaze detection unit that detects the gaze of a user and sets a scanning range according to eyeball information detected by the gaze detection unit, and the image display device is provided with a wide-angle drawing unit and a light guide plate. This is a schematic diagram showing a modified example of an image display device according to the present technology that includes a gaze detection unit that detects the user's gaze and sets a scanning range according to eyeball information detected by the gaze detection unit, and further includes a wide-angle drawing unit. This is a schematic diagram showing a modified example of a configuration in which an image display device according to the present technology is provided with a gaze detection unit that detects the gaze of a user and sets a scanning range according to eyeball information detected by the gaze detection unit, the modified example being provided with a wide-angle drawing unit and equipped with a light guide plate. FIG. 11 is a schematic diagram showing an example in which an image display device according to the present technology is provided with a gaze detection unit that detects the gaze of a user, and switches between multiple image lights and outputs them to a display unit based on eyeball information detected by the gaze detection unit. FIG. 13 is a schematic diagram showing an example in which an image display device according to the present technology includes a gaze detection unit that detects the gaze of a user, and a light guide plate that switches between multiple image lights and outputs them to a display unit based on eyeball information detected by the gaze detection unit. 1 is a conceptual diagram showing an example of how a head-mounted display including an image display device according to the present technology is worn; 1 is a conceptual diagram showing a projector including an image display device according to the present technology.

Below, we will explain the preferred embodiment of the present technology. However, the embodiment shown below is an example of a representative embodiment of the present technology, and the present technology is not limited to only the preferred embodiment below, and can be freely modified within the scope of the present technology.

The image display device according to this technology includes a drawing unit that scans an area in which an image is to be drawn based on input image information and draws the image, and a scanning range setting unit that sets a scanning range for an area smaller than the maximum scannable range in which the scanning can be performed.

The image display device according to this technology is expected to reduce the time required for scanning compared to the time required to scan the maximum scannable range by setting a scanning range that is smaller than the maximum scannable range that can be scanned when the rendering unit scans the area where an image is rendered. This makes it possible to improve the frame rate without reducing the resolution of the scanning range. Here, the "maximum scannable range" can be determined by either the maximum range that the image display device can physically scan or the maximum range that the image display device can control as a scanning range.

Here, "frame rate" is a frequency value indicating the number of frames (also called "frames") (number of still images) processed per unit time in a video. It is generally expressed in units of fps (frames per second), which is a number indicating how many images one second of video is made up of. The higher the frame rate, the smoother and clearer the subject movement will be in the video.

<Drawing section>
The drawing section included in the image display device according to the present technology is not particularly limited as long as it can scan an area in which an image is to be drawn and draw the image, and for example, a laser beam scanning system (LBS) or the like can be suitably used.

When the drawing section of the image display device according to the present technology uses a laser beam scanning system, the drawing section includes an LDD (Laser Diode Driver) and a laser unit, and can perform the drawing using the laser unit. In this case, a single laser unit or multiple laser units may be used. When multiple laser units are used, it is expected that the resolution and frame rate can be further improved.

The image of scanning when drawing an image using a laser beam scanning system, which is an example of a drawing unit, is explained using Figure 1.

When drawing an image using a laser beam scanning system, scanning is performed by controlling the position at which the laser light output from the laser unit is guided using a combination of two axes, for example the non-resonant axis direction and the resonant axis direction, to scan the area where the image is to be drawn. This makes it possible to scan a two-dimensional image. In Figure 1, Figure 1A shows the movement in the non-resonant axis direction, and Figure 1B shows the movement in the resonant axis direction.

The laser light output from the laser unit can be guided by, for example, a scanning mirror. The scanning mirror may be a single scanning mirror capable of scanning in two axial directions, or a combination of two scanning mirrors capable of scanning in one axial direction.

As an example of a scanning mirror capable of scanning in two axial directions, for example, a MEMS (Micro Electro Mechanical Systems) mirror can be suitably used. Also, as an example of a scanning mirror capable of scanning in one axial direction, for example, a MEMS mirror, a galvanometer mirror, a polygon mirror, etc. can be mentioned.

When using a scanning mirror in a laser beam scanning system, efficient amplitude due to resonance can be obtained by passing a current with the same period as the resonant frequency of the scanning mirror. The direction in which this amplitude due to resonance occurs is called the resonant axis direction, and the direction in which non-resonant driving while suppressing resonance is called the non-resonant axis direction.

In this case, the driving unit of the scanning mirror is not particularly limited, and any driving unit such as electromagnetic, electrostatic, or piezoelectric can be used. In addition, the scanning mirror can be controlled using an open loop control method or FB (feedback) control.

The scanning mirror guides the laser light output from the laser unit in accordance with resonance in the resonance axis direction as shown in Figure 1B, while at the same time adding movement in the non-resonant axis direction as shown in Figure 1A, making it possible to scan the entire area in which an image is drawn with the laser light output from the laser unit. In the example shown in Figure 1B, resonance occurs in the horizontal direction, resulting in horizontal scan lines.

Figure 1C shows an image of how the laser light output from the laser unit is guided by a MEMS mirror to scan the area on which the image is to be drawn, thereby drawing the image. Since MEMS mirrors can scan in one or two axial directions, the scanning mirror can be made smaller by using a MEMS mirror that can scan in two axial directions.

In the drawing section provided in the image display device according to the present technology, when scanning the area in which an image is drawn, the vertical direction of the image to be drawn may be the non-resonant axis direction and the horizontal direction may be the resonant axis direction as in the example shown in FIG. 1B, or conversely, the vertical direction may be the resonant axis direction and the horizontal direction may be the non-resonant axis direction. Note that since the human gaze moves horizontally more frequently than vertically, it is expected that image quality that feels more natural to humans can be achieved by using the horizontal direction as the resonant axis.

<Scanning range setting unit>
The image display device according to the present technology includes a scanning range determination unit, and thereby the drawing unit included in the image display device can draw the image in a drawing range set in a scanning range that is set smaller than the maximum scannable range in which scanning can be performed. The drawing range may be smaller than the scanning range or may be the same as the scanning range.

The drawing section provided in the image display device according to the present technology may have a drawing position selection mechanism that selects the drawing position within the drawing range. The drawing position selection mechanism can draw a desired image at a desired position by, for example, controlling the emission position of the laser light output from the laser unit, thereby reducing the memory used to draw the image without reducing the resolution, and outputting an image that looks less unnatural.

Specifically, when the drawing unit of the image display device according to the present technology controls the emission position of the laser light by a laser beam scanning system and outputs the laser light by combining the LDD and the laser unit, a high frame rate of 90 fps, preferably 120 fps, and more preferably 240 fps or more can be realized while maintaining a resolution of, for example, 48 PPD or more, calculated from the characteristics of the MEMS mirror that can be used at the time of filing the present technology. Note that with the present technology, a frame rate according to the intended use can be realized by adjusting the resolution and the scanning range. Furthermore, even higher resolutions and frame rates can be realized with the present technology, depending on the performance of the scanning mirror.

The drawing section of the image display device according to the present technology may employ a method of shifting the drawing timing for each frame, such as interlacing or interleaving. By shifting the drawing timing for each frame, it is possible to expect the effect of filling the gap between the left and right scanning lines and achieving even higher resolution.

Here, interlacing is a method of handling pixels or scan lines at regular intervals when scanning an area where an image is to be drawn or when drawing said image, rather than handling them in order from the top, left, or other end, thereby increasing the number of scans per unit time and allowing for smoother movement. In addition, interleaving is a method of filling in the gaps between the left and right scanning ends by switching the scanning direction for each frame when scanning an area where an image is to be drawn or when drawing said image, thereby reducing degradation of vertical resolution at the edges of the image.

There are no particular limitations on the drawing position selection mechanism, so long as it can select the drawing position within the drawing range, but for example, a laser beam scanning system that can control the emission position of laser light can be suitably used.

Next, using Figure 2 (an image showing scanning and drawing in an image display device relating to the present technology and a graph showing drawing time) and Figure 3 (an image showing scanning and drawing in an image display device relating to the conventional technology and a graph showing drawing time), we will explain the characteristics of scanning and drawing when a scanning range setting unit in an image display device relating to the present technology sets a scanning range of an area smaller than the maximum scannable range in which the scanning is possible.

In the image display device according to the present technology, a scanning range setting unit sets a scanning range that is smaller than the maximum scannable range that can be scanned, and scans the area in which the image is to be drawn. Specifically, in the first frame shown in FIG. 2A, a scanning range 11 that is smaller than the maximum scannable range 10 is set for the maximum scannable range 10, and the image is drawn in a drawing range 12 provided within the scanning range.

For example, in an image display device designed with a total angle of view of 60°, if the total angle of view, which corresponds to the maximum scannable range, is set to 60° and the scanning range is set to 15°, the time used for scanning will be approximately 1/4 of the original time.

Similarly, in the second frame shown in FIG. 2B and the third frame shown in FIG. 2C, a scanning range 11 that is smaller than the maximum scannable range 10 can be set anywhere within the maximum scannable range, and the image can be suitably drawn within the drawing range 12 set within that scanning range.

On the other hand, in an image display device according to the prior art that does not have a scanning range setting unit, the entire area in which the image is drawn is scanned in all frames from the first frame to the third frame shown in Figures 3A to 3C, so the maximum scannable range 10 and the scanning range 11 are the same. Also, because drawing is performed in the scanning range 11, the scanning range 10 and the drawing range 12 are the same. In other words, in an image display device according to the prior art, the maximum scannable range 10, the scanning range 11, and the drawing range 12 are the same.

As a result, the image display device according to the present technology scans an area with a scanning range that is set by the scanning range setting unit to be smaller than the maximum scannable range, and therefore the scanning time can be shortened compared to the conventional technology. Figure 2D is a graph showing the drawing time of the image display device according to the present technology, and Figure 3D is a graph showing the drawing time of the image display device according to the conventional technology. By comparing the length of the horizontal axis indicating the time axis of both graphs, it can be confirmed that the present technology is able to shorten the scanning time compared to the conventional technology.

In the image display device according to the present technology, the set scanning range is not particularly limited as long as it is smaller than the maximum scannable range, but for example, the scanning range can be adjusted to any range less than 100%, such as 90% or less, 75% or less, 50% or less, 25% or less, with respect to a maximum scannable range of 100%. The smaller the ratio of the scanning range to the maximum scannable range, the shorter the scanning time per frame can be expected to be. This can improve the frame rate.

In the image display device according to the present technology, by setting a scanning range that is smaller than the maximum scannable range, the scanning time of the scanning range can be adjusted to less than 100%, for example 90% or less, 75% or less, 50% or less, 25% or less, etc., relative to the 100% scanning time of the maximum scannable range. In addition, the scanning range can be adjusted to any range according to the needs of the user.

The image display device according to this technology can continuously output image information by repeatedly scanning a set scanning range and setting the next scanning range, which is the scanning range to be scanned next. Here, the next scanning range is set before scanning of the scanning range is completed. This enables the image display device to increase the continuity of outputting image information. Here, "until scanning of the scanning range is completed" includes not only the period from when scanning of the scanning range begins until scanning is completed (during scanning), but also the case where the next scanning range is set before scanning of the scanning range begins.

Furthermore, when the image display device according to the present technology repeatedly scans the set scanning range and sets the next scanning range, it can also repeatedly set a scanning limit range at the position of the scanning range that is smaller than the maximum scannable range and wider than the scanning range, and set the next scanning range in the scanning limit range.

Compared to setting the next scanning range within the maximum scanning range, by setting the next scanning range in an area smaller than the maximum scannable range and within the scanning limit range, which is an area close to the scanning range, the maximum time required for one frame can be set shorter. By repeatedly scanning the scanning range, setting the scanning limit range, and setting the next scanning range, the frame rate can be further improved. Here, the maximum time required for one frame refers to the time difference between the previous frame and the current frame, and more specifically, the time required to draw one frame. Additionally, "close to the scanning range" refers to the set of all points within an arbitrary distance from an arbitrary point within the scanning range as the center.

The size of the scanning limit range is not limited as long as it is smaller than the maximum scannable range at the position of the scanning range and is a wider area than the scanning range, but can be adjusted to a suitable range, for example, depending on the movement speed of the input unit that inputs the scanning range described below (for example, saccade in the case of input by gaze, or finger movement speed in the case of input by finger movement).

In the image display device according to the present technology, when scanning is performed by combining two axes, a non-resonant axis direction and a resonant axis direction, it is possible to set a scanning range that is smaller than the maximum scannable range that allows suitable scanning, for example, by adjusting the movement in the non-resonant axis direction using a scanning range setting unit.

FIG. 4 is an image diagram of a modified example of scanning and drawing in an image display device according to the present technology, showing an example in which scanning and drawing of an image are performed with the vertical direction being the resonant axis direction (scanning line is vertical) and the horizontal direction being the non-resonant axis direction. Even in this case, in the first frame shown in FIG. 4A, the second frame shown in FIG. 4B, and the third frame shown in FIG. 4C, a scanning range 11 of an area smaller than the maximum scannable range 10 can be set anywhere within the maximum scannable range, and the image can be suitably drawn within the drawing range 12 set within that scanning range.

In the image display device according to the present technology, the setting of the scanning range by the scanning range setting unit is performed by an input unit that inputs the scanning range. The input unit may be an input unit provided outside the image display device (hereinafter, "external input"), or an input unit provided in the image display device (hereinafter, "internal input").

The above-mentioned method of external input is not particularly limited, but for example, the scanning range can be suitably set by inputting the scanning range to the image display device according to the present technology from an electronic device such as a smartphone, tablet, PC, cloud system, or remote control.

The above-mentioned method of internal input is not particularly limited, but may suitably be adopted, for example, a method in which the image display device according to the present technology further includes a gaze detection unit that detects the user's gaze and sets the scanning range according to the eyeball information detected by the gaze detection unit, or a method in which the image display device includes an electroencephalogram detection unit that detects the user's brain waves and sets the operation range based on the information detected by the electroencephalogram detection unit. Here, "eyeball information" refers to information related to the user's eyeball associated with the user's gaze, and examples include information related to eyeball movement (gaze), pupil position, and gaze point position.

In the image display device according to the present technology, the scanning of the scanning range set by the scanning range setting unit is not particularly limited, but for example, when the image display device according to the present technology uses a scanning mirror or the like, the path of movement in the non-resonant axis direction can be adjusted. In this case, the path of movement in the non-resonant axis direction may be a path determined in advance, or an arbitrarily determined path may be used when scanning.

<Image display device>
Next, components of the image display device according to the present technology will be described with reference to the schematic diagram of Fig. 5 showing a configuration example of the image display device according to the present technology. However, the components shown below are examples of components that may be included in the image display device according to the present technology, and the image display device according to the present technology may include components other than the components shown below.

[Image input section]
The image input unit 21 inputs image information to the image display device 20 according to the present technology. Specifically, the image input unit 21 inputs the generated, stored, or received image information to the image display device 20 by an electrical signal or the like. Note that the image light generation unit 21 may input information obtained by converting the angle of view information into spatial (positional) information to the image display device 20. By inputting the information converted into spatial (positional) information, it can also be used for realizing extended reality (XR) including, for example, augmented reality (AR), virtual reality (VR), and mixed reality (MR).

The image input unit 21 is not particularly limited as long as it can input image information to the image display device according to the present technology, but for example, a smartphone, a tablet, a personal computer, a cloud system, a remote control, or other electronic device can be suitably used. In addition, although the image input unit 21 is shown in FIG. 5 as a part independent of the image display device 20, it may be provided as a part included in the image display device.

[Image information receiving section]
In the image display device 20 according to the present technology, the image information receiving unit 22 receives image information input from the image input unit 21 and transmits the image information to a control unit 23 described later. The control unit to which the image information receiving unit transmits may be a single control unit or multiple control units. When there are multiple control units, the image information input from the image input unit can be suitably reproduced as an output image by connecting the respective control units via the image information receiving unit. In addition, at the time of the transmission, the image information receiving unit 22 may transmit the image information input from the image input unit 21 to the control unit as is, or may transmit the image information after processing according to the application.

The image information receiving unit 22 is not particularly limited as long as it can receive image information and transmit the received image information to a control unit or the like, but it is preferable to use hardware such as a CPU (Central Processing Unit) or a chipset, for example.

[Control unit (video processor)]
The control unit 23 is a main control unit that comprehensively controls the components of the image display device 20. Specifically, the control unit 23 transmits image information input from the image input unit to the drawing position selection mechanism 25 and the scanning range setting unit 27, and causes them to suitably reproduce the image as an output image.

The control unit 23 can suitably use hardware such as a CPU (Central Processing Unit) or a chipset.

[Drawing position selection mechanism]
As described above, the drawing position selection mechanism (LD control unit) 25 draws a desired image at a desired position by controlling the emission position of the laser light output from the laser unit 24. The drawing position selection mechanism is not particularly limited as long as it can select the drawing position to be performed within the drawing range, but Fig. 5 shows an example in which a laser beam scanning system is used.

[LDD and laser unit]
In the configuration example of the partial image display device shown in Figure 5, the emission position of the laser light is controlled by a laser beam scanning system, and the energy amount and wavelength of the laser output by the laser unit 24 are controlled by an LDD to draw an image, and the image information input from the image input unit can be suitably reproduced as an output image.

The laser light source of the laser unit 24 may be, for example, a semiconductor laser such as an EEL (edge-emitting laser) or a SEL (surface-emitting laser). Note that, although FIG. 5 shows an example in which the image display device 20 has one laser unit 24, it may have multiple laser units 24. Also, if there is a means capable of drawing an image by controlling the amount of energy and wavelength output by a control unit such as an LDD, that means may be used instead of the laser unit.

[Scan range setting section]
The scanning range setting unit (mirror control unit) 27 transmits information on the angle of the MEMS mirror 26, which is a scanning mirror, to a mirror driver 37, and the mirror driver 37 controls the movement of the MEMS mirror 26, thereby controlling the position to which the laser light output from the laser unit 24 is guided. This makes it possible to set a scanning range in the image display device 20 that is smaller than the maximum scannable range that can be scanned. As the scanning range setting unit 27, for example, hardware such as a CPU or a chipset can be suitably used.

[Mirror driver and scanning mirror]
As described above, the mirror driver 37 controls the movement of the MEMS mirror 26 by a combination of two axes, the non-resonant axis direction and the resonant axis direction, to control the position to which the laser light output from the laser unit 24 is guided. This allows the laser light output from the laser unit 24 to suitably scan the area where an image is to be drawn, and to draw the image.

In Figure 5, an example is shown in which a MEMS mirror is used, which is a scanning mirror capable of scanning in two axial directions, but as mentioned above, it is also possible to use a combination of two scanning mirrors capable of scanning in one axial direction.

There are no particular limitations on the driving unit of the MEMS mirror 26, and any driving unit such as electromagnetic, electrostatic, or piezoelectric can be used. In addition, since the MEMS mirror can be scanned in two axial directions, using this makes it possible to miniaturize the scanning mirror.

The image display device 20 is expected to reduce the ratio of the scanning range to the maximum scannable range by controlling the movement of the MEMS driver using the scanning range setting unit 27, thereby shortening the scanning time per frame. This may lead to an improvement in the frame rate of the image display device 20.

[Input section]
The input unit 28 inputs the scanning range to the image input unit 21 and/or the control unit 23. Thereafter, the scanning range setting unit 27 sets the scanning range under the control of the control unit 23. As described above, the input unit 28 may be an external input or an internal input.

Next, an example of scanning mirror control in a case where the image display device according to the present technology is equipped with a scanning mirror will be described using the schematic diagram of FB (feedback) control shown in FIG. 6.

The angle of the scanning mirror 26 is compared with the target angle, for example, as measured by an angle sensor, and under the control of the control unit 23, the scanning range setting unit 27 equipped with an FB control unit 48 (Feedback controller) repeatedly corrects the error in the actual angle of the scanning mirror 26 relative to the target angle, and the movement of the scanning mirror 26 by the mirror driver 37 is controlled to approach the target angle. By adjusting the angle of the scanning mirror 26 through FB control, fine control of the drawing position in the scanning range is possible. In particular, when the image display device according to the present technology further includes a gaze detection unit that detects the user's gaze and appropriately sets the scanning range according to the eyeball information detected by the gaze detection unit, it is possible to realize drawing with little delay (latency) relative to the tracking speed of the detected gaze.

[Relay optics]
In addition, although not shown in the schematic diagram showing the configuration example of the image display device according to the present technology in Fig. 5, the image display device according to the present technology can design the light to be guided and the light guide path by combining, as elements other than the elements described above, for example, a branching optical element having a function of a so-called beam splitter that can branch and guide the image light on the light guide path of the image light related to the video information, a reflective optical element having a function of reflecting and guiding the image light, a magnifying lens that magnifies the image to be output by magnifying the image light, a light shielding part that blocks light of unnecessary wavelengths, a condensing lens that condenses the image light to an arbitrary range, a light guide plate that guides the incident image light, an optical member such as a diffraction grating that adjusts the path of the incident image light, etc. In addition, each of these may be used alone or in combination of a plurality of them.

Note that while Figure 6 shows an example of controlling the scanning mirror using FB (feedback) control, the scanning mirror can also be controlled using open loop control, which is easier to control, depending on the application.

Next, an example of a laser light combining method when the image display device according to the present technology is equipped with a laser unit will be described with reference to FIG. 7. By combining RGB laser light, a full-color image can be displayed.

FIG. 7A is a schematic diagram of a case where RGB laser light is combined by a beam splitter (BS) to output white laser light. The optical paths of the RGB laser light output from a red laser output device 31, a green laser output device 32, and a blue laser output device 33 are adjusted by combining optical elements 34 such as a reflecting mirror or a half mirror (dichroic mirror) to combine the light into white light.

FIG. 7B is a schematic diagram of a case where RGB laser light is combined using a planar lightwave circuit (PLC) to output white laser light. The RGB laser light output from the red laser output device 31, the green laser output device 32, and the blue laser output device 33 is made incident on an optical waveguide 35 made of an optical fiber or the like, and the RGB laser light is focused along the optical waveguide to combine into white light.

If the image display device according to the present technology is equipped with a laser unit, the laser unit can, for example, suitably render a full-color image by using white light combined in the above-described manner.

FIG. 8 is a schematic diagram of an embodiment in which an image display device according to the present technology includes multiple laser units and scans an area in which an image is to be drawn using laser light output from the multiple laser units. Scanning using multiple laser light output from multiple laser units (multi-beam scan) is expected to reduce scanning time or improve scanning density per unit area, thereby improving resolution. In addition, the use of multiple laser light is expected to improve drawing speed, leading to an improvement in frame rate.

FIG. 9 is a schematic diagram showing an image display device according to the present technology that includes a wide-angle rendering unit that renders a wide-angle image with a wider angle of view than the image rendered by the rendering unit.

[Wide-angle rendering section]
In this embodiment, the image display device 20 includes, in addition to the drawing unit described above, a wide-angle drawing unit 41 that draws a wide-angle image with a wider angle of view than the image drawn by the drawing unit. As a result, as shown in Fig. 9B, a scanning range of an area smaller than the maximum scannable range that can be scanned is set, and an image is drawn in a drawing range set in the scanning range, thereby making it possible to improve the frame rate without reducing the resolution, and by combining a wide-angle drawing range 47 that is an image with a wide viewing angle, it is expected to improve the sense of realism.

In particular, when the image display device according to the present technology further includes a gaze detection unit that detects the user's gaze and the scanning range is suitably set according to the eyeball information detected by the gaze detection unit, only the central portion of the user's gaze can be rendered at a high resolution and a high frame rate, and the other portions can be rendered with a wide viewing angle. Therefore, for example, by rendering an image with a wide viewing angle that covers the maximum human field of vision, the device can be suitably used in applications that realize extended reality, including augmented reality, virtual reality, and mixed reality.

The wide-angle drawing unit 41 can be any drawing unit. For example, it can be composed of LBS (Laser Beam Scanning), LCOS (Liquid crystal on silicon), M-OLED (Micro Organic Light Emitting Diode), M-LED (Micro Light-Emitting Diode), etc., and these can be used alone or in combination.

FIG. 10 is a schematic diagram showing a modified embodiment of an image display device according to the present technology that includes a wide-angle rendering unit that renders a wide-angle image with a wider angle of view than the image rendered by the rendering unit.

In an embodiment relating to this modified example, the image display device 20 includes a branching optical element 42 that branches and guides laser light relating to a wide-angle image from the image information guided from the scanning mirror (MEMS mirror) 26, and a reflecting optical element 43 that reflects and guides laser light relating to an image to be drawn by the drawing unit in a drawing range provided in the scanning range that is other than the image information that has been branched and guided from the image information. Since the drawing unit and the wide-angle drawing unit can be achieved by the same output mechanism, the image display device can be designed to be compact.

By having the above configuration, the embodiment of this modified example also sets a scanning range that is smaller than the maximum scannable range, as shown in FIG. 10B, in the same way as the example of the embodiment equipped with the wide-angle drawing section, and draws an image in the drawing range set within the scanning range using laser light reflected and guided by the reflective optical element 43, it is possible to improve the frame rate without reducing the resolution, and it is expected that the sense of realism will be improved by combining a wide-angle image drawn using laser light branched and guided by the branching optical element 42.

Here, the branching optical element 42 is not particularly limited as long as it is an optical element that has the function of a so-called beam splitter, which can branch and guide the laser light related to the wide-angle image among the image information guided from the scanning mirror 26, but for example, a half mirror or the like can be suitably used.

The reflective optical element 43 is not particularly limited as long as it is an optical element that has the function of reflecting and guiding laser light, but for example, a concave mirror or the like can be suitably used.

In addition, in an embodiment related to this modified example, one or more of the following may be provided on the light guide path of the laser light for the wide-angle image: a magnifying lens 44 for magnifying the wide-angle image; a light shielding section 45 for blocking light of a wavelength related to the laser light when the laser light for the image drawn by the drawing section is mixed in the light guide path in a drawing range provided in the scanning range; a focusing lens 46 for focusing the wide-angle image to an arbitrary range; etc.

In this embodiment of the modified example, the drawing section and the wide-angle drawing section are achieved by the same output mechanism, so the laser light output from the laser unit 24 also includes laser light related to the wide-angle image. For this reason, in this embodiment, it is preferable from the viewpoint of realizing wide-angle drawing to set a larger scanning range compared to other embodiments. For example, with respect to a maximum scannable range of 100%, the scanning range can be adjusted to any range less than 100%, for example, 90% or less, 75% or less, 50% or less, 25% or less, etc. Furthermore, in the image display device related to the present technology, the scanning time of the scanning range relative to the scanning time of the maximum scannable range of 100% can be adjusted in the same ratio.

Furthermore, in this modified embodiment, a magnifying lens is used to magnify the wide-angle image on the light guide path of the laser light related to the wide-angle image, thereby achieving more preferable wide-angle drawing.

FIG. 11 is a schematic diagram showing a form in which the scanning range is set by external input in an image display device according to this technology.

In this embodiment, the external input unit 51 inputs the scanning range to the image input unit 21 and/or the control unit 23. Thereafter, the scanning range setting unit 27 sets the scanning range under the control of the control unit 23. By setting the scanning range through external input, it is possible to guide the viewer's gaze to the image displayed by the image display device, and this is expected to broaden the range of applications.

FIG. 11 shows an example of external input using a touch panel on a tablet, but the external input unit 51 is not limited to a tablet and may be, for example, a smartphone, a personal computer, a cloud system, a remote control, or other electronic device.

Furthermore, the input method into the electronic device that performs the above-mentioned external input is not limited as long as it is a method that can input information for changing the position of the scanning range relative to the maximum scannable range displayed on the electronic device. In addition to the input method using a touch panel as shown in FIG. 11, an input method using a gaze detection unit 52 (also called an "eye track system") that detects the user's gaze as shown in FIG. 12, an input unit using a mouse, track point, etc., or any other input unit can be used alone or in combination.

When using a gaze detection unit, as shown in FIG. 12, eye information detected by the gaze detection unit 52 is transmitted to the external input unit 52 by the external input unit detection unit 58, and the external input unit 51 can set the scanning range based on the transmitted eye information. Note that while FIG. 12 shows an example in which the gaze detection unit 52 and the external input unit detection unit 58 are installed in eyewear 59, it is not necessary for the gaze detection unit 52 and the external input unit detection unit 58 to be installed in eyewear, and any form can be adopted as long as it is a means that can set the scanning range by external input based on the eye information detected by the gaze detection unit 52.

FIG. 13 is a schematic diagram showing an embodiment in which an image display device according to the present technology includes a gaze detection unit that detects the gaze of a user, and sets a scanning range according to eyeball information detected by the gaze detection unit.

In this embodiment, the gaze detection unit 52 detects eye information such as the gaze, which is the direction of the user's eye, and based on the detection result, inputs the scanning range to the image input unit 21 and/or the control unit 23. After that, the scanning range setting unit 27 sets the scanning range under the control of the control unit 23.

If the gaze point is defined based on the speed of human eye movement, the actual visual resolution of humans drops exponentially from the gaze point, and it is said that text can only be read at about 2°. For this reason, as long as there is an acceptable range of viewing angle that can follow the user's line of sight, it is believed that eyewear that can be used for VR/AR purposes can be used even if the actual rendering angle is narrow.

In this embodiment, the scanning range is set by the gaze detection unit, and the image is drawn only in the high sensitivity area near the gaze point. The drawing range is dynamically changed for each frame in response to changes in the gaze position detected by the gaze detection unit, which is expected to achieve a higher frame rate than general image presentation methods without reducing resolution.

<Gaze detection unit (eye track system)>
The gaze detection unit 52 includes, for example, a light receiving and emitting unit and a signal processing unit that processes an output signal of the light receiving and emitting unit.

The light-receiving and light-emitting unit has a light-emitting element that irradiates invisible light (e.g., infrared light) onto the user's eyeball, and a light-receiving element that receives the light emitted from the light-emitting element and reflected by the eyeball. The signal processing unit processes the output signal of the light-receiving element and calculates the direction of the line of sight.

The light receiving element can be, for example, a photodiode, a split photodiode with multiple light receiving areas, an image sensor, an event type sensor (a sensor for ice sensing), etc.

FIG. 13B shows an image of the case where the gaze detection unit 52 draws an image only in the high sensitivity area near the gaze point. In the first frame shown in FIG. 13B, a scanning range 11 smaller than the maximum scannable range 10 is set based on the detection results of eyeball information such as the gaze of the user detected by the gaze detection unit 52, and the image is drawn in a drawing range 12 set in the vicinity of the gaze point within the scanning range.

Similarly, in the second and third frames shown in FIG. 13B, a scanning range 11 smaller than the maximum scannable range 10 is set in accordance with the detection results of the user's eyeball information detected by the gaze detection unit 52, and the image can be suitably drawn in a drawing range 12 set near the gaze point of the scanning range. By repeating this, a frame rate higher than that of a typical image presentation method can be achieved without reducing the resolution near the gaze point by following the user's gaze.

When the image display device according to the present technology scans a set scanning range, sets the aforementioned scanning limit range, and repeatedly sets the next scanning range within the scanning limit range, the scanning limit range can be set as an area between two positions that are separated by the maximum amount of movement of the gaze point due to saccades per unit time in two directions outward from the ends of the scanning range.

Here, in this embodiment, "center" refers to the center of the scanning range. The scanning limit range can be set as an area between two positions separated by the length from the center of the scanning range to the end of the scanning range and the maximum amount of movement of the gaze point due to saccade per unit time in two directions from the center of the scanning range. In addition, the user's eye movement per unit time occurs in the area between two positions separated by the maximum amount of movement of the gaze point due to saccade per unit time in two directions from the center of the scanning range. Therefore, by setting the next scanning range in the above-mentioned scanning limit range rather than setting it in the maximum scannable range, it is possible to optimize the mirror scanning path and shorten the blanking time in which no drawing is performed within the frame, while ensuring the output of an image that does not feel strange to the user by taking into account the eye rotation speed of human saccades, etc., and to set the maximum time required for one frame to be short. This makes it possible to further improve the frame rate while ensuring the output of an image that does not feel strange to the user.

The scanning limit range can be set, for example, as an area between two positions that are separated from both ends of the scanning range in a direction outward from the range by the maximum amount of movement of the gaze point due to saccades per unit time.

The following is a detailed explanation using figures. Figure 14 is a schematic diagram of the maximum time required for one frame when the user's saccades are not taken into account, and Figure 15 is a schematic diagram of the maximum time required for one frame when the user's saccades are taken into account.

If the user's saccades are not taken into consideration, the scanning range 11 of the second frame, which is the next scanning range following the scanning range 11 of the first frame shown in FIG. 14A, is set at any location within the maximum scannable range 10. Therefore, a mirror scanning path such as that shown in FIG. 14A can be taken.

On the other hand, when the user's saccades are taken into consideration, the scanning range 11 of the second frame, which is the next scanning range for the scanning range 11 of the first frame, is set in the scanning limit range 38, which is an area that is away from the center 38 of the scanning range 11 by the maximum amount of movement of the gaze point due to the user's saccades per unit time, as shown in FIG. 15C. Since the scanning limit range 38 is the range of the eye rotation speed of human saccades, etc., setting the next scanning range 11 within this range will not cause discomfort to the user. Furthermore, since the mirror scanning path can be optimized within the range of the eye rotation speed of human saccades, etc., the maximum time required for one frame can be set shorter than when the user's saccades are not taken into consideration, as shown in FIG. 15B.

FIG. 16 is a schematic diagram showing how the image display device of this embodiment repeatedly scans a set scanning range and sets the next scanning range in the scanning limit range. By setting a scanning limit range 38, which is an area that is an amount equal to the maximum amount of movement of the gaze point due to the user's saccade per unit time from the center 38 of the scanning range 11, and repeatedly setting the next scanning range 11 within that range, it can be confirmed that the mirror scanning path can be optimized and the blanking time during which no drawing is performed within the frame can be shortened. This makes it possible to further improve the frame rate while still ensuring that the image output is natural to the user.

When the image display device according to this embodiment is equipped with a gaze detection unit that detects the user's gaze and sets the scanning range according to the eyeball information detected by the gaze detection unit, the scanning limit range can also be set as an area separated in two directions from the center line of the scanning range by the amount of movement of the gaze point due to the user's saccade per unit time. This can make it possible to more efficiently set the maximum time required for one frame to be shorter.

Specifically, for example, if the imageable angle of view corresponding to the maximum scannable range is set to 50° or more, taking into account the range of line of sight movement due to eye rotation, the scanning range is set to 15° or more with the gaze point being set to a point where the number of cone cells on the retina is about 1/10 of the fovea centralis, and the mirror diameter is set to 0.8 mm to obtain a resolution equivalent to a visual acuity of 0.8, calculations based on the characteristics of the MEMS mirror that can be used at the time of filing this technology indicate that it may be possible to achieve image rendering at 90 fps or more.

Specifically, assuming that the saccade is 500°/sec. at its fastest, the maximum inter-frame movement amount is 5.6° at 90 fps. As in the above, for example, the imageable angle of view corresponding to the maximum scannable range is 50° or more, taking into account the range of line of sight movement due to eye rotation, the gaze point is set to a scanning range of 15° or more, with the number of cone cells on the retina being approximately 1/10 of the fovea, and the mirror diameter is set to 0.8 mm to obtain a resolution equivalent to 0.8 visual acuity. Assuming saccades as described above and taking into account the mirror scanning path, it can be assumed that the angle to which the image should be returned during blanking time when no drawing is performed within the frame is sufficient to be the image presentation angle + 5.6°. Therefore, by considering the saccade speed, the angle to which the image should be returned during blanking time when no drawing is performed within the frame can be efficiently determined while maintaining drawing that does not cause discomfort to the user, and the frame rate can be further improved.

When the image display device according to the present technology is equipped with a gaze detection unit that detects the user's gaze and sets the scanning range according to the eyeball information detected by the gaze detection unit, the maximum angle of change in scanning position, which corresponds to the scanning limit range, is adjusted, for example, to within ±5.0°, ±5.6°, or ±6.0° of the image presentation angle in two directions of the center line of the scanning range, relative to the drawable field angle of 50°, which corresponds to the maximum scannable range. This makes it possible to efficiently adjust the angle to be returned to during blanking time when no drawing is performed within the frame, and to shorten the maximum time required for one frame, thereby further improving the frame rate.

When the image display device according to the present technology is equipped with a gaze detection unit that detects the gaze of a user and sets a scanning range according to the eyeball information detected by the gaze detection unit, it is expected that the image displayed by the rendering unit can be output to a display unit worn by the user, making it suitable for use in extended reality (XR) applications including augmented reality (AR), virtual reality (VR), and mixed reality (MR).

In the above case, the method of outputting the image to the display unit worn by the user is not limited, and it can be suitably applied to a transparent drawing method in which an image is projected onto an optical element such as a transparent half mirror, and the image is recognized by looking at a virtual screen beyond the mirror, and it can also be suitably applied to a direct retinal drawing method in which an image is projected directly onto the user's retina.

In particular, when an image display device related to this technology is applied to a direct retinal imaging method, this technology can achieve a high frame rate without reducing resolution, so it is expected that the image will be closer to the human field of vision and provide clear images that are not affected by the user's focus adjustment function.

If the image display device or image display method according to the present technology is equipped with the above-mentioned eye track system, the accuracy of detection of the user's eye information by the gaze detection unit can be improved by performing periodic calibration.

The calibration method is not particularly limited, but a suitable method is, for example, to draw an image of an arbitrary object that can attract the user's attention on a display unit worn by the user, and guide the user's gaze to an arbitrary position by moving the drawing position of the object, thereby calculating the relative positional relationship between the gaze point and the drawing position of the object, and correcting the position of the user's gaze point guided by the gaze detection unit.

FIG. 17 is a schematic diagram showing an example in which the embodiment shown in FIG. 13 further includes a wide-angle rendering section.

In this embodiment, the scanning range is set by the gaze detection unit, and the image is drawn only in the high sensitivity area near the gaze point. The drawing range is dynamically changed for each frame in response to changes in the gaze position detected by the gaze detection unit. This is expected to achieve a higher frame rate than general image presentation methods without reducing resolution, and furthermore, by combining a wide-angle image drawn by the wide-angle drawing unit, it is expected to improve the sense of realism. Note that, although the embodiment shown in FIG. 17 describes a form in which the wide-angle image drawn by the wide-angle drawing unit is projected to any location by using the condensing lens 46, a design that does not use a condensing lens is also possible.

Here, FIG. 17B is a schematic diagram showing the user's visual field. The image display method of this technology renders a high-resolution image in the central visual field 53 (gazing point) of the user's line of sight detected by the line of sight detection unit, achieving a frame rate higher than that of general image presentation methods, while also rendering a wide-angle image by the wide-angle rendering unit in the peripheral visual field 54 of the user's line of sight.

FIG. 18 is a schematic diagram showing an example in which the embodiment shown in FIG. 17 further includes a light guide plate.

This embodiment includes a light guide plate 55 that guides image light related to an image drawn by the drawing unit in a drawing range provided in the scanning range as shown in FIG. 18, and image light related to a wide-angle image drawn by the wide-angle drawing unit.

In this embodiment, the image light incident on the light guide plate 55 has its direction of travel changed within the light guide plate 55 by the incident side optical member 56, and is propagated through the light guide plate 55 and focused on the user's eyeball 63 by the output side optical member 57.

Here, in FIG. 18, an example is shown in which a diffraction grating is used as the optical element for the input side optical element and the output side optical element, but these optical elements are not limited to diffraction gratings, and as long as they can change the direction of light travel, for example, a reflector, optical lens, prism, etc. can be suitably used. These optical elements can be processed into any shape, such as a reflective type or a transmissive type, depending on the design of the optical device that propagates the light.

When a diffraction grating is used as an optical component on the light guide path of a light guide plate, it can be molded integrally with the light guide plate by injection molding, which is preferable in terms of manufacturing costs. In addition, a prism can also be molded integrally with the light guide plate by injection molding as an optical component on the light guide path of the light guide plate. In particular, when a prism is used as an optical component on the incident surface side, it may be possible to reduce the loss of light when image information is input to the light guide plate.

When the optical device according to the present technology uses a diffraction grating as an optical component, the material of the diffraction grating can be the same as the material that forms the light guide path. Using the same material is preferable because it makes it easier to manufacture the light guide plate by injection molding.

When the optical device according to the present technology uses a diffraction grating as an optical component, the shape of the diffraction grating can be a one-axis type diffraction grating that has a periodic structure only in the X-axis direction, or a two-axis type diffraction grating that has a periodic structure in both the X-axis and Y-axis directions.

The shape of the diffraction grating can be designed in any shape using known methods. Furthermore, the diffraction grating can be arranged along the light guide path, but it can also be arranged at an angle to the light guide path. By adjusting the shape and arrangement of the diffraction grating, the diffraction angle and diffraction efficiency can be optimized, and light of a specific wavelength can be selectively reflected or transmitted.

The transmissive diffraction grating is preferably placed on the side of the light guide where image information is incident, and the reflective diffraction grating is preferably placed on the side of the light guide opposite the side where image information is incident. In addition, by forming a thin metal film on the side of the reflective diffraction grating opposite the light guide plate, it is possible to increase the reflectivity and reduce loss of the amount of input light.

The diffraction grating of the light guide plate can be designed to optimize the diffraction angle, diffraction efficiency, etc., to match the light of the desired wavelength propagating through the light guide path of the light guide plate. By optimizing the diffraction grating to match the light of the desired wavelength, it is expected that the angle of view of the display image output from the light guide plate can also be maximized.

Even in this embodiment that includes a light guide plate, both the transmissive imaging method and the direct retinal imaging method described above can be suitably applied.

Note that FIG. 18B is a schematic diagram showing the user's field of vision. In this embodiment as well, the image display method of the present technology can achieve the same effect as that described above in FIG. 17B.

FIG. 19 is a schematic diagram showing an example in which the embodiment shown in FIG. 13 further includes the wide-angle rendering unit shown in FIG. 10.

In this embodiment, the image light related to the image information guided from the scanning mirror (MEMS mirror) 26 is guided using the branching optical element 42 and the reflecting optical element 43 as explained in FIG. 10, so that the drawing of an image in a high sensitivity area near the gaze point and the drawing of a wide-angle image can be achieved by the same output mechanism.

FIG. 19B is a schematic diagram showing the user's field of vision. In this embodiment, the image display method of the present technology can achieve the same effect as that described above in FIG. 17B.

FIG. 20 is a schematic diagram showing an example in which the embodiment shown in FIG. 19 further includes a light guide plate.

Similar to the embodiment shown in FIG. 18, this embodiment includes a light guide plate 55 that guides image light related to an image drawn by the drawing unit in a drawing range provided in the scanning range as shown in FIG. 20, and image light related to a wide-angle image drawn by the wide-angle drawing unit. As a result, the image light incident on the light guide plate 55 has its direction of travel changed within the light guide plate 55 by the incident-side optical member 56, and is propagated within the light guide plate 55 and focused on the user's eyeball 63 by the output-side optical member 57.

In the embodiment of FIG. 20 as well, the optical members described in the embodiment of FIG. 18 can be suitably used for the incident side optical member and the output side optical member. Also, in this embodiment that includes a light guide plate, either the transmission type drawing method or the retinal direct drawing method described above can be suitably applied.

Next, using FIG. 21, an example of an embodiment will be described in which an image display device according to the present technology includes a gaze detection unit that detects the gaze of a user, and switches between multiple image lights and outputs them to a display unit based on eyeball information detected by the gaze detection unit.

In this embodiment, the laser unit 24 switches between and outputs two or more image lights L1 and L2 based on the detection results of eye information, such as the user's line of sight, detected by the line of sight detection unit 52.

In this case, for example, multiple pieces of image information with different image ranges of the image to be viewed by the user are prepared as the multiple image lights, and based on the detection results of the user's eyeball information detected by the gaze detection unit 52, the laser unit 24 switches between and outputs the multiple image lights L1, L2, thereby making it possible to suitably change the angle of view viewed by the user. Therefore, even if the angle of view of the image information represented by each of the multiple image lights becomes smaller, the user can still suitably view the image, and it is expected that the image display device related to this technology can be designed to be compact.

The laser unit 24 can switch between multiple image lights by any means, for example, by mechanical means or electrical means.

Next, FIG. 22 is a schematic diagram showing an example in which the embodiment shown in FIG. 21 further includes a light guide plate.

In this embodiment, similar to the embodiment shown in FIG. 21, the laser unit 24 switches between and outputs two or more image lights L1, L2 based on the detection results of the user's eyeball information detected by the gaze detection unit 52 as shown in FIG. 22, and is equipped with a light guide plate 55 that guides these image lights.

In this embodiment as well, the laser unit 24 switches between and outputs the multiple image lights L1 and L2 based on the detection results of the user's eyeball information, thereby making it possible to suitably change the angle of view that is viewed by the user. Therefore, even if the angle of view of the image information represented by each of the multiple image lights becomes small, the user can still view the image favorably, and it is expected that the image display device related to this technology can be designed to be compact. Furthermore, in this embodiment that includes a light guide plate, either the transmissive drawing method or the direct retinal drawing method described above can be suitably applied.

<Head-mounted display>
Next, an example of an embodiment of a head mounted display including an image display device according to the present technology will be described with reference to FIG.

As shown in FIG. 23, the head mounted display 60 includes an image display device 20 and a drawing optical element 61 as a display unit, and the image display device 20 includes a scanning mirror (MEMS mirror) 26 and a laser unit 24. Note that FIG. 23 shows an example in which RGB laser light is combined with optical elements 34 such as a reflecting mirror or half mirror by a beam splitter (BS) to adjust the optical path, thereby outputting combined white laser light.

By guiding the combined white laser light by the MEMS mirror 26, a scanning range smaller than the maximum scannable range is set in the drawing optical element 61 as the area in which to draw an image, and the image is drawn in the drawing range set in the scanning range. In this case, the laser light guided by the MEMS mirror 26 may reach the drawing optical element 61 via a relay lens 62 as shown in FIG. 23. By passing through the relay lens 62, the laser light can be focused on any area of the drawing optical element 61.

Here, the drawing optical element 61 may be a transmissive half mirror or the like, and a transmissive drawing method may be used in which an image is projected onto the half mirror and the user is able to see the image by looking at a virtual screen beyond the mirror; the drawing optical element 61 may be a diffraction grating (hologram element), and a direct retinal drawing method may be used in which the diffraction grating diffracts laser light toward the eyeball 63, and the laser light is focused near the pupil and reaches the retina, projecting the image directly onto the user's retina.

FIG. 23 shows an example in which a diffraction grating is used as the drawing optical element 61. In this case, the diffraction grating 61 is preferably placed in front of the user's eyeball 63 as shown in FIG. 23, and may be placed in a portion that corresponds to the lens of glasses, for example. By placing it in this manner, the laser light can reach the retina without being refracted by the crystalline lens of the eyeball 63.

By applying the image display device according to this technology to the direct retinal imaging method as in this embodiment, this technology can achieve a high frame rate without reducing the resolution, so it is expected that the image drawn will be closer to the human field of vision, and a clear image will be provided that is not affected by the user's focus adjustment function.

Further, an embodiment of a head mounted display including an image display device according to the present technology includes:
By providing the gaze detection unit 52 that detects the eyeball information of the user, and by setting the scanning range according to the eyeball information detected by the gaze detection unit, as described above, it is possible to realize a frame rate higher than that of a general image presentation method without reducing the resolution near the gaze point by following the user's gaze. As a result, it is expected that the head mounted display will be suitable for use in extended reality (XR) applications including augmented reality (AR), virtual reality (VR), and mixed reality (MR).

The gaze detection unit 52 may be positioned so as to detect information related to the gaze from the eyeball 63, and may be positioned, for example, in a portion corresponding to the rim, lens, or end piece of glasses related to the head mounted display 60.

The set of components described above is provided to perform image presentation and gaze detection for the other eye of the user as well, as shown in FIG. 23. Note that the image display device 20 may be configured to display an image for both eyes as shown in FIG. 23, or may be configured to display an image for one eye.

<Projector>
Next, an example of an embodiment of a projector including an image display device according to the present technology will be described with reference to FIG.

As shown in FIG. 24, the projector 70 includes an LBS system 29. The projector 70 may further include a screen 71 as a display unit. Here, the LBS system 29 includes an image input unit, an image information receiving unit, a control unit, an LD control unit, a mirror control unit, an input unit, an LDD, and a mirror driver, and may further include a scanning mirror 26 and a laser unit 24.

In this embodiment, the laser light output from the laser unit 24 is guided by the scanning mirror 26 to set a narrow field of view scanning range 72 as a scanning range on the screen 71 that is smaller than the maximum scannable range in which scanning can be performed, and the image is drawn in the drawing range provided in the narrow field of view scanning range 72. In this case, the screen 71 may be provided with a wide-angle drawing unit that draws a wide-angle image with a wider angle of view than the image drawn in the narrow field of view scanning range 72. In this case, by combining the wide-angle image drawn by the wide-angle drawing unit on the screen 71 with the image drawn in the narrow field of view scanning range 72, it is possible to improve the frame rate without reducing the resolution, and it is expected that the sense of realism will be improved.

In this case, any drawing unit can be used as the wide-angle drawing unit, and it may be composed of, for example, LBS, LCOS, M-OLED, M-LED, etc., and these can be used alone or in combination.

In addition, in the above embodiment, the scanning range is set by the scanning range setting unit through an input unit that inputs the scanning range, and the input unit can suitably adopt the above-mentioned external input or internal input unit according to the application.

In addition, the present technology can have the following configurations.
(1) An image display device comprising: a drawing unit that scans an area in which an image is to be drawn based on input image information and draws the image; and a scanning range setting unit that sets a scanning range of an area smaller than the maximum scannable range in which the scanning is possible, wherein the drawing unit draws the image within a drawing range set within the scanning range.
(2) The image display device according to (1), wherein the scanning time of the scanning range is less than 100% with respect to 100% of the scanning time of the maximum scannable range.
(3) The image display device according to (1) or (2), in which the drawing range is smaller than the scanning range.
(4) The image display device according to (3), wherein the drawing unit has a drawing position selection mechanism that selects a drawing position to be performed within the drawing range.
(5) The image display device according to any one of (1) to (4), wherein the drawing unit includes a scanning mirror and performs the scanning by using the scanning mirror.
(6) An image display device described in any one of (1) to (5), wherein the scanning is performed by a combination of two axes, a non-resonant axis direction and a resonant axis direction, and the scanning range is scanned by adjusting the movement in the non-resonant axis direction.
(7) The image display device according to any one of (1) to (6), wherein the scanning is performed with a horizontal direction as a resonance axis.
(8) The image display device according to any one of (1) to (7), further comprising: scanning the set scanning range and repeatedly setting a next scanning range.
(9) Scanning the set scanning range, and setting a scanning limit range that is smaller than the maximum scannable range and wider than the scanning range at the position of the scanning range;
The image display device according to (8), wherein the setting of the next scanning range is repeatedly performed within the scanning limit range.
(10) The image display device according to (9), wherein the scanning limit range is set in two directions from the center of the scanning range as an area between two positions separated by a length from the center to an end of the scanning range and a maximum amount of movement of the gaze point due to a saccade per unit time. (11) The image display device according to (9), wherein the scanning limit range is set in two directions from the center of the scanning range as an area between two positions separated by a length from the center to an end of the scanning range and a maximum amount of movement of the gaze point due to a saccade per unit time. (12) The image display device according to any one of (1) to (11), wherein the drawing unit includes a laser unit and performs the drawing by the laser unit.
(13) The image display device according to (12), wherein the laser unit is provided in plurality.
(14) The image display device according to any one of (1) to (13), further comprising a wide-angle drawing unit that draws a wide-angle image having a wider angle of view than the image drawn by the drawing unit.
(15) The image display device according to any one of (1) to (14), wherein the scanning range setting unit sets the scanning range through an external input.
(16) An image display device described in any one of (1) to (15), further comprising a gaze detection unit that detects the user's gaze, and sets the scanning range according to eyeball information detected by the gaze detection unit.
(17) The image display device according to (16), wherein the image drawn by the drawing unit is output to a display unit worn by the user.
(18) The image display device according to (17), wherein output to the display unit is performed by switching between a plurality of image lights based on eyeball information detected by the gaze detection unit.
(19) The image display device according to any one of (16) to (18), wherein output to the display unit is performed by a retinal direct imaging method.
(20) A head-mounted display comprising the image display device according to any one of (1) to (19).
(21) A projector comprising the image display device according to any one of (1) to (19).
(22) An image display method comprising a scanning step of scanning an area in which an image is to be drawn based on input image information, and a drawing step of drawing the image, wherein the scanning step sets a scanning range of an area smaller than a maximum scannable range in which the scanning is possible, and the drawing step draws the image in a drawing range set within the scanning range.
(23) The image display method according to (22), wherein the scanning range is scanned for less than 100% of the scanning time of the maximum scannable range, which is 100%.

10 Maximum scan range

11 Scanning range 12 Drawing range 20 Image display device 21 Image input unit 22 Image information receiving unit 23 Control unit (video processor)
24 Laser unit 25 Drawing position selection mechanism (LD control unit)
26 Scanning mirror (MEMS mirror)
27 Scanning range setting unit (mirror control unit)
28 Input section 29 LBS system 31 Red laser output device 32 Green laser output device 33 Blue laser output device 34 Optical element 35 Optical waveguide 36 LDD (Laser Diode Driver)
37 Mirror driver 38 Center of scanning range 39 Scanning limit range 41 Wide-angle drawing section 42 Branching optical element (half mirror)
43 Reflective optical element (concave mirror)
44 Magnifying lens 45 Light blocking section 46 Condenser lens 47 Wide-angle drawing range 48 FB control section 51 External input section 52 Line-of-sight detection section 53 Central field of view (gazing point) (drawing range) 54 Peripheral field of view 55 Light guide plate 56 Incident-side optical member (incident-side diffraction grating)
57 Output side optical member (output side diffraction grating)
58 External input unit Detection unit 59 Eyewear 60 Head mounted display 61 Drawing optical element (diffraction grating/hologram element)
62 Relay lens 63 Eyeball 70 Projector 71 Screen 72 Narrow field scanning range

Claims (20)

a drawing unit that scans an area in which an image is to be drawn based on input image information and draws the image;
a scanning range setting unit that sets a scanning range that is smaller than the maximum scannable range,
The drawing unit draws the image in a drawing range provided in the scanning range. The image display device according to claim 1, wherein the scanning time of the scanning range is less than 100% compared to 100% of the scanning time of the maximum scannable range. The image display device according to claim 1, wherein the drawing range is smaller than the scanning range. The image display device according to claim 3, wherein the drawing unit has a drawing position selection mechanism that selects a drawing position within the drawing range. The image display device according to claim 1, wherein the drawing unit includes a scanning mirror and performs the scanning by using the scanning mirror. The image display device according to claim 1, wherein the scanning is performed by combining two axes, a non-resonant axis direction and a resonant axis direction, and the scanning range is scanned by adjusting the movement in the non-resonant axis direction. The image display device according to claim 1, in which the scanning is performed with the horizontal direction as the resonant axis. The image display device according to claim 1, which scans the set scanning range and repeatedly sets the next scanning range. Scanning the set scanning range, and setting a scanning limit range at the position of the scanning range that is smaller than the maximum scannable range and wider than the scanning range;
The image display device according to claim 8 , wherein the next scanning range is repeatedly set within the scanning limit range. The image display device according to claim 9, wherein the scanning limit range is set as an area between two positions separated by the length from the center of the scanning range to the end of the scanning range in two directions from the center of the scanning range and the maximum amount of movement of the gaze point due to saccades per unit time. The image display device according to claim 1, wherein the drawing unit includes a laser unit and performs the drawing. The image display device according to claim 11, wherein the laser unit is multiple. The image display device according to claim 1, further comprising a wide-angle drawing unit that draws a wide-angle image having a wider angle of view than the image drawn by the drawing unit. The image display device according to claim 1, wherein the scanning range setting unit sets the scanning range through an external input. The image display device according to claim 1, further comprising a gaze detection unit that detects the gaze of a user, and sets the scanning range according to eye information detected by the gaze detection unit. The image display device according to claim 11, wherein the image drawn by the drawing unit is output to a display unit worn by the user. The image display device according to claim 15, wherein the output to the display unit is performed by a direct retinal imaging method. A head-mounted display comprising the image display device according to claim 15. A projector equipped with the image display device according to claim 1. a scanning step of scanning an area in which an image is to be drawn based on input image information;
a drawing step of drawing the image,
The scanning step sets a scanning range that is smaller than a maximum scannable range that the scanning can be performed,
The drawing step is an image display method in which the image is drawn in a drawing range provided in the scanning range.