WO2012032842A1 - Display system and detection method - Google Patents

Abstract

A display system (1) is provided with a processor; a sensor array (30) arranged in a row in a first direction; and a memory which stores data indicative of the correspondence relationship between an output voltage and the distance in each distance measuring sensor. A direction normal to a display area (810) is perpendicular to the first direction. Each of a plurality of distance measuring sensors emits light in a second direction perpendicular to the first direction and the direction normal to the display area (810) and in a direction approaching the display area (810). Each of the plurality of distance measuring sensors receives, of the light emitted, the light reflected from an object and outputs a voltage value based on the distance to the object. The processor detects the position of the object in a two dimensional image on the basis of each voltage value that each of the plurality of distance measuring sensors outputted and the data stored in the memory.

Description

Display system and detection method

The present invention relates to a display system and a detection method. The present invention particularly relates to a display system that displays a two-dimensional image in the air and a method for detecting the position of an object in the two-dimensional image.

Conventionally, a display system that displays a two-dimensional image in the air is known.
Japanese Patent Laying-Open No. 2005-141102 (Patent Document 1) discloses a stereoscopic two-dimensional image display device as the display system. The stereoscopic two-dimensional image display device includes a display unit, a microlens array, a position detection sensor, and a control unit.

The display unit includes an image display surface for displaying a two-dimensional image. The microlens array displays a two-dimensional image on a stereoscopic image display surface in a pseudo-stereoscopic manner by forming light emitted from the image display surface on a stereoscopic image display surface separated from the image display surface. The position detection sensor is arranged in association with the stereoscopic image display surface, and outputs a signal corresponding to the position subjected to physical action from the outside. The control unit changes the image in the stereoscopic image display surface according to the output signal from the position detection sensor.

JP-A-9-55152 (Patent Document 2) discloses a display device having a touchless panel switch as the display system. In the touchless panel switch, when the finger enters a predetermined area where the finger is to be detected, the light projection from the light projecting element is reflected on the finger and incident on the light receiving element. At least one reflection type optical sensor composed of an element and the light receiving element is installed in the space around the gradient index lens element for each predetermined region.

Also, conventionally, display systems that display 2D images and 3D images in the air are known.

International Publication No. 2007/116639 (Patent Document 3) discloses a display device including an imaging element as the display system. The display device forms an object to be projected, which is a two-dimensional or three-dimensional object, on the opposite side of the imaging element as a two-dimensional image or a real image of a three-dimensional image. More detailed description is as follows.

The imaging element is an optical element that causes a light beam to bend when light passes through an element surface constituting one plane. The imaging element is configured by arranging a plurality of unit optical elements that reflect light by one or more mirror surfaces arranged at an angle perpendicular to or close to the element surface. The imaging element is a real image in a space where there is no physical entity on the other side of the element surface by reflecting the light emitted from the projection object arranged on one side of the element surface to the mirror surface when passing through the element surface. As an image.

Japanese Patent Laid-Open No. 6-18258 (Patent Document 4) discloses an optical distance measuring sensor including a long-distance light receiving unit and a short-distance light receiving unit.

JP 2005-141102 A JP-A-9-55152 International Publication No. 2007/116639 Japanese Patent Laid-Open No. 6-18258

However, in Patent Document 1, it is necessary to arrange a position detection sensor so as to surround the periphery of the two-dimensional image displayed in the air. Therefore, in Patent Document 1, a frame is required around the two-dimensional image displayed in the air. Therefore, it is difficult for the user to feel the difference between the image displayed by the stereoscopic two-dimensional image display device of Patent Document 1 and the image displayed by a general display that displays an image on the display panel.

In Patent Document 2, it is detected by one sensor that an object such as a finger is located at a predetermined position in a two-dimensional image in the air. For this reason, in order to detect about the display area which displays a two-dimensional image, many sensors are needed. Furthermore, it is very difficult to determine the installation position of each sensor.

In Patent Document 3, it cannot be detected at which position in the real image of the formed two-dimensional image or three-dimensional image.

The present invention has been made in view of the above problems, and its purpose is to detect the position of an object in a two-dimensional image with a simple configuration without surrounding the two-dimensional image displayed in the air with a frame. It is to provide a possible display system and a detection method in the display system.

According to an aspect of the present invention, a display system includes a display, an optical element that displays a two-dimensional image in an aerial display area based on an image displayed on the display, a processor, and a plurality of distance measuring sensors. A first sensor array arranged in a row in one direction, and a memory storing first data indicating a correspondence relationship between an output voltage and a distance in each distance measuring sensor. The normal direction of the display area is a direction perpendicular to the first direction. Each of the plurality of distance measuring sensors emits light in a second direction perpendicular to the first direction and the normal direction and in a direction approaching the display area. Each of the plurality of distance measuring sensors receives light reflected by the object among the emitted light, and outputs a voltage value based on the distance to the object. The processor detects the position of the object in the two-dimensional image based on each voltage value output from each of the plurality of distance measuring sensors and the first data.

Preferably, the first sensor array is disposed at a position intersecting with a plane including the display area.

Preferably, the light emitted from each of the plurality of distance measuring sensors passes through the display area.
Preferably, the display system further includes a first moving mechanism that moves the display in the third direction and a second moving mechanism that moves the first sensor array in the fourth direction. The memory further stores second data indicating the correspondence between the position of the display and the position of the first sensor array. The display area moves based on the movement of the display. The processor determines the position of the first sensor array based on the position of the display and the second data. The processor moves the first sensor array to the determined position using the second moving mechanism.

Preferably, the third direction is a direction perpendicular to the display surface of the display. The display area moves in the third direction based on the movement of the display in the third direction. The fourth direction is the same direction as the third direction.

Preferably, the third direction is a direction perpendicular to the display surface of the display. The direction of the normal of the display area is different from the third direction. The display area moves in the normal direction based on the movement of the display in the third direction. The fourth direction is a direction having a component of the normal direction. The processor detects the position of the object in the two-dimensional image based on each voltage value output from each of the plurality of ranging sensors, the first data, and the position of the first sensor array or the position of the display. .

Preferably, the third direction is a direction in which the incident angle of the light with respect to the optical element based on the image displayed on the display is changed. The fourth direction is a direction in which the emission angle of each light emitted from the plurality of distance measuring sensors is changed.

Preferably, the display system further includes a second sensor array in which a plurality of distance measuring sensors are arranged in a row in the first direction. The second sensor array is arranged in parallel with the first sensor array and emits light in the same direction as the first sensor array. The processor displays at least one object in the display area. The processor detects the position of the object based on the voltage value output from each of the plurality of distance measuring sensors included in the second sensor array and the first data. The processor determines whether or not a position on a plane including a display area corresponding to the detected position of the object is included in an area displaying at least one object in the display area. The processor changes the display mode of the object in the display area from the first display mode to the second display mode based on the determination that the processor is included in the area for displaying the object.

Preferably, the processor displays at least one object in the display area. The distance measuring sensor in the first sensor array is provided at a position corresponding to at least one object.

Preferably, each of the plurality of distance measuring sensors in the first sensor array includes one light emitting element and two light receiving elements each receiving reflected light of light emitted from the light emitting element.

Preferably, the processor displays at least one object in the display area. The processor executes processing associated with the object based on the fact that the position of the object detected by the first sensor array is included in the area displaying at least one object.

According to another aspect of the present invention, the detection method is a detection method in a display system that detects the position of an object in a two-dimensional image displayed in an aerial display area. The display system includes a processor, a sensor array in which a plurality of distance measuring sensors are arranged in a row in a first direction, and a memory that stores data indicating a correspondence relationship between an output voltage and a distance in each distance measuring sensor. Including. The normal direction of the display area is a direction perpendicular to the first direction. The detection method includes a step in which each of the plurality of distance measuring sensors emits light in a second direction perpendicular to the first direction and the normal direction and approaches the display area; Each of the distance sensors receives light reflected by the object out of the emitted light, and outputs a voltage value based on the distance to the object, and the processor outputs by each of the plurality of distance sensors Detecting the position of the object in the two-dimensional image based on each of the voltage values and the data.

According to the present invention, it is possible to detect the position of an object in a two-dimensional image with a simple configuration without surrounding the two-dimensional image displayed in the air with a frame.

It is the figure which showed the external appearance and use condition of the display system. FIG. 2 is a cross-sectional view taken along line II-II in FIG. It is the block diagram which showed a part of hardware constitutions of the display system. It is a figure for demonstrating the structure of a sensor array. It is the figure which showed the external appearance of the ranging sensor. It is a figure for demonstrating the irradiation range of the light which a sensor array radiate | emits. It is a 1st figure for demonstrating the measurement principle by a ranging sensor. It is a 2nd figure for demonstrating the measurement principle by a ranging sensor. It is the figure which showed the example of a measurement by a sensor array. It is the flowchart which showed the flow of the process in a display system. It is the figure which showed the external appearance and use condition of the other display system. FIG. 12 is a cross-sectional view taken along line XII-XII in FIG. 11. It is the block diagram which showed a part of hardware constitutions of the display system. It is the figure which showed the external appearance and use condition of other display systems. FIG. 15 is a cross-sectional view taken along line XV-XV in FIG. 14. It is the block diagram which showed a part of hardware constitutions of the display system. It is a figure for demonstrating a 1st display mode and a 2nd display mode. It is the flowchart which showed the flow of the change process of the display mode in a display system. It is a figure for demonstrating the structure of the ranging sensor of another display system. It is the figure which showed the output characteristic of the sensor of a display system. It is a figure for demonstrating the sensor array of another display system. It is the figure which showed the external appearance and use condition of other display systems. FIG. 23 is a cross-sectional view taken along line XXIII-XXIII in FIG. It is the block diagram which showed a part of hardware constitutions of the display system. It is the figure which showed the correspondence of each ranging sensor and a display area. It is the figure which showed the external appearance and use condition of other display systems. FIG. 27 is a cross-sectional view taken along line XXVII-XXVII in FIG. 26. It is the block diagram which showed a part of hardware constitutions of the display system. It is the figure which showed the external appearance and use condition of other display systems. FIG. 30 is a cross-sectional view taken along line XXX-XXX in FIG. 29. It is the block diagram which showed a part of hardware constitutions of the display system.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. In the following description, the same parts are denoted by the same reference numerals. Their names and functions are also the same. Therefore, detailed description thereof will not be repeated.

In the following, “direction” represents two different directions. “Two different directions” represents, for example, two directions opposite to each other. For example, the direction of the X axis represents the positive direction of the X axis and the negative direction of the X axis. In addition, “two different directions” include, for example, a direction in which the two directions do not differ by 180 degrees, such as a clockwise direction and a counterclockwise direction.

[Embodiment 1]
FIG. 1 is a diagram illustrating an appearance and a usage state of the display system 1. With reference to FIG. 1, the display system 1 includes a housing 10, an opening 20, a sensor array 30, and a microlens array (optical element) 40.

The microlens array 40 transmits light emitted from the display (see FIG. 2) in the housing 10 and displays a two-dimensional image (hereinafter also referred to as “aerial image”) in a rectangular display area 810 in the air. . The display area 810 is an area surrounded by four sides 810a, 810b, 810c, and 810d. Note that the side 810a and the side 810b are parallel, and the side 810c and the side 810d are parallel. The direction of the normal line of the display area 810 is the Z direction.

The opening 20 has a rectangular shape. The opening 20 is formed below the display area 810 and along the side 810 b of the display area 810.

The sensor array 30 has a plurality of distance measuring sensors arranged in a row in the X direction. The sensor array 30 is disposed along the opening 20 in the housing 10. Specifically, the sensor array 30 is installed such that the sensing surface of each distance measuring sensor faces the display area 810. Details of the arrangement of the sensor array will be described later.

In the display system 1, the user touches the aerial image displayed in the display area 810 with the user's finger 910, for example. Specifically, the user touches the object with the finger 910 in order to select the object included in the aerial image. An object is, for example, an icon image. Note that since the aerial image is not a physical object, even if the two-dimensional image is touched, there is no physical contact with the two-dimensional image.

FIG. 2 is a cross-sectional view taken along line II-II in FIG. With reference to FIG. 2, the display system 1 includes a sensor array 30, a microlens array 40, and a display 50 in a housing 10.

The microlens array 40 is a light wave control device in which microlenses are arranged in a lattice pattern.

Display 50 displays an image in the direction of microlens array 40. As described above, the image displayed on the display 50 is displayed in the display area 810 as an aerial image by the microlens array 40.

Each of the distance measuring sensors of the sensor array 30 is in the Y direction perpendicular to the direction in which the distance measuring sensors are arranged (X direction) and the direction of the normal of the display area 810 (Z direction), and in a direction approaching the display area 810. Emits light. More specifically, the sensor array 30 is arranged at a position that intersects a plane including the display area 810. That is, the sensor array 30 is arranged at a position parallel to the sides 810a and 810b of the display area 810 (see FIG. 1).

The sensor array 30 may be arranged so that the light emitted from the distance measuring sensor passes through the display area 810 (that is, the light overlaps the display area 810), or the light travels along the display area 810. It may be arranged to move forward. Hereinafter, a case where light emitted from the distance measuring sensor is arranged to pass through the display area 810 will be described as an example.

FIG. 3 is a block diagram showing a part of the hardware configuration of the display system 1. Referring to FIG. 3, the display system 1 includes a sensor array 30, a display 50, a CPU (Central Processing Unit) 60, a memory 70, a display driving device 80, and an A / D (Analog / Digital) converter 90. With.

Also, the display system 1 includes a device (speaker or the like) that generates sound. The same applies to other display systems described later.

Sensor array 30 outputs an analog voltage value as a sensing result to A / D converter 90. The A / D converter 90 converts an analog voltage value into a digital voltage value. The A / D converter 90 sends the converted digital voltage value to the CPU 60.

The memory 70 includes, for example, a ROM, a RAM, and a flash memory. The memory 70 stores various data such as a program executed by the display system 1 and association data 71. The association data 71 will be described later.

CPU 60 executes a program stored in memory 70 in advance. In addition, the CPU 60 refers to the voltage value acquired from the A / D converter 90 and the association data 71 and executes processing to be described later.

The display driving device 80 receives a command from the CPU 60 and drives the display 50.

FIG. 4 is a diagram for explaining the configuration of the sensor array 30. In the sensor array 30, eight distance measuring sensors 31 to 38 are arranged in a row. Each of the distance measuring sensors 31 to 38 emits light in the Y direction. Each of the distance measuring sensors 31 to 38 receives the light reflected by the object among the emitted light, and outputs a voltage value based on the distance to the object to the A / D converter 90.

The distance measuring sensors 31 to 38 are arranged at equal intervals. Note that the distances from the distance measuring sensors 31 to 38 to the display area 810 are the same for the distance measuring sensors 31 to 38. The distance measuring sensors 31 to 38 have the same configuration except that the positions in the sensor array 30 are different. Although the number of distance measuring sensors in the sensor array 30 is eight, the number is not limited to this.

FIG. 5 is a diagram showing the external appearance of the distance measuring sensor 31. Referring to FIG. 5, the distance measuring sensor 31 includes a light emitting device 311 and a light receiving device 312. The light emitting device 311 emits light to the outside. The light receiving device 312 receives reflected light of the light emitted from the light emitting device 311. The light receiving device 312 also receives light other than reflected light (for example, indoor light or external light).

FIG. 6 is a diagram for explaining the irradiation range of the light emitted from the sensor array 30. FIG. 6A is a diagram showing an irradiation range of light emitted from the light emitting device 311 of the distance measuring sensor 31 constituting the sensor array 30. With reference to Fig.6 (a), the light-emitting device 311 radiate | emits the light which an irradiation range spreads according to the distance d. For example, when the distance measuring sensor 31 emits infrared rays, the irradiation range has a spread of ± 1.5 ° with respect to the optical axis.

FIG. 6B is a diagram showing the irradiation range of the light emitted from the distance measuring sensors 31 to 38. With reference to FIG. 6B, light can be irradiated to the entire range of the display area 810 by eight distance measuring sensors.

Thus, in the display system 1, the display area 810 and the detection area (hereinafter also referred to as “sensing area”) by the sensor array 30 overlap. The display system 1 does not perform sensing in the Z direction.

FIG. 7 is a first diagram for explaining the principle of measurement by the distance measuring sensor 31. Referring to FIG. 7, the light emitting device 311 includes an infrared LED (Light Emitting Diode) 311a which is a light emitting element, and a lens 311b. The light receiving device 312 includes an infrared light receiving element 312a and a lens 312b.

The infrared LED 311a emits infrared rays to the lens 311b. The infrared rays emitted from the lens 311b are reflected by the object 950 that is a sensing target. The reflected infrared light passes through the lens 312b and enters the infrared light receiving element 312a.

When there is an object 960 instead of the object 950, the infrared light emitted from the infrared LED 311a is reflected by the object 960. Infrared light reflected by the object 960 passes through the lens 312b and enters the infrared light receiving element 312a as described above.

The light emitted from the distance measuring sensor 31 is not limited to infrared rays.
FIG. 8 is a second diagram for explaining the principle of measurement by the distance measuring sensor 31. Referring to FIG. 8, when the distance d is greater than d1, the distance measuring sensor 31 has a characteristic that the output voltage decreases as the distance to the sensing target (objects 950, 960, finger 910) increases.

The distance range that can be detected by the distance measuring sensor 31 is a distance from the distance d1 to the distance d2 where the output voltage does not become a certain value or less. The sensor array 30 includes distance measuring sensors 31 to 38 that are included in the distance range in which the display area 810 can be detected.

The association data 71 stored in the memory 70 is data indicating the characteristics of FIG. Specifically, the association data 71 is data indicating the correspondence between the output voltage and the distance in each of the distance measuring sensors 31 to 38. In the present embodiment, since eight distance measuring sensors having the same configuration are provided, the association data 71 may be any data indicating the correspondence between the output voltage and the distance in one distance measuring sensor. .

FIG. 9 is a diagram showing an example of measurement by the sensor array 30. FIG. 9A is a diagram for explaining output voltages of the distance measuring sensors 31 to 38 when the user touches the area 911 in the display area 810 with the finger 910. FIG. 9B is a diagram for explaining output voltages of the distance measuring sensors 31 to 38 when the user touches the area 912 in the display area 810 with the finger 910. FIG. 9C is a diagram for explaining output voltages of the distance measuring sensors 31 to 38 when the user touches the area 913 in the display area 810 with the finger 910.

9A, when the area 911 is touched with a finger 910, the voltage value V2 output from the distance measuring sensor 33 is higher than the voltage value V1 output from other distance measuring sensors. In this case, the CPU 60 specifies the position in the Y direction of the area 911 in the display area 810 based on the voltage value V2 that is the highest voltage value and the association data 71.

Further, the CPU 60 specifies the position in the X direction of the area 911 in the display area 810 based on the position of the distance measuring sensor 33. Specifically, in the memory 70, data in which each distance measuring sensor is associated with the X coordinate in the display area 810 is stored in advance. The CPU 60 specifies the X coordinate associated with the distance measuring sensor having the highest output voltage value based on the data.

9B, when the area 912 is touched with the finger 910, the voltage value V3 output from the distance measuring sensor 35 is higher than the voltage value V1 output from other distance measuring sensors. In this case, the CPU 60 specifies the position in the Y direction of the area 912 in the display area 810 based on the voltage value V3 and the association data 71. Further, the CPU 60 specifies the position in the X direction of the area 912 in the display area 810 based on the position of the distance measuring sensor 35. Since the region 912 is farther from the sensor array 30 than the region 911 illustrated in FIG. 9A, the voltage value V3 is smaller than the voltage value V2.

With reference to FIG. 9C, when the region 913 is touched with the finger 910, the voltage value V4 output from the distance measuring sensors 34 and 35 is higher than the voltage value V1 output from the other distance measuring sensors. . In this case, the CPU 60 specifies the position of the area 913 in the Y direction in the display area 810 based on the voltage value V4 and the association data 71.

Further, the CPU 60 specifies the position in the X direction of the area 913 in the display area 810 based on the positions of the distance measuring sensors 34 and 35. Specifically, the CPU 60 sets the average value of the X coordinate associated with the distance measuring sensor 34 and the X coordinate associated with the distance measuring sensor 35 as the position of the region 913 in the X direction.

Note that, since the region 913 is farther from the sensor array 30 than the region 911 shown in FIG. 9A, the voltage value V4 is smaller than the voltage value V2. In addition, the voltage value V4 is larger than the voltage value V3 because it is closer to the sensor array 30 than the region 912 shown in FIG.

With this configuration, when the user touches the display area 810 with the finger 910, the CPU 60 can detect which position in the display area 810 has been touched. That is, the CPU 60 can detect the position of the finger 910 in the aerial image displayed in the display area 810.

For example, when a selectable object is displayed in the display area 810, it is assumed that the user touches the object with the finger 910. At this time, the CPU 60 calculates the coordinate value of the position of the finger 910 in the display area 810 based on the output of the sensor array 30. Based on the coordinate value, the CPU 60 can determine that a command for selecting the object has been input by the user.

FIG. 10 is a flowchart showing the flow of processing in the display system 1. In the following description, the distance measuring sensors 31 to 38 are referred to as a first distance measuring sensor, a second distance measuring sensor,..., An eighth distance measuring sensor, respectively. Referring to FIG. 10, in step S2, CPU 60 sets the value of variable j to 1. In step S4, the j-th ranging sensor emits infrared rays. In step S <b> 6, the j-th distance measuring sensor receives infrared light (reflected light) reflected by the finger 910.

In step S8, the j-th ranging sensor outputs a voltage value based on the distance to the object (that is, the finger 910). More specifically, the j-th distance measuring sensor outputs a voltage value based on the received infrared light and a voltage value based on the received room light or external light. In step S10, the CPU 60 increases the value of j by one. That is, the CPU 60 increments j. In step S12, the CPU 60 determines whether j is greater than 8.

When CPU 60 determines that the value is large (YES in step S12), it acquires the voltage value output from each distance measuring sensor 31 to 38 from A / D converter 90 in step S14. Specifically, the value (digital value) obtained by A / D converting the voltage value (analog value) output from the distance measuring sensors 31 to 38 by the A / D converter 90 is acquired for each distance measuring sensor. . If the CPU determines that it is not large (NO in step S12), the process proceeds to step S4.

In step S <b> 16, the CPU 60 detects the position of the finger 910 in the two-dimensional image based on each voltage value output by each distance measuring sensor and the association data 71.

As described above, in the display system 1, the sensor array 30 in which the plurality of distance measuring sensors 31 to 38 are arranged in a row is installed in the downward direction of the two-dimensional image (aerial image), thereby the object of the two-dimensional image is displayed. The position can be determined. Therefore, the display system 1 can detect the position of the object in the two-dimensional image with a simple configuration without surrounding the two-dimensional image displayed in the air with a frame.

<First Modification>
FIG. 11 is a diagram illustrating an appearance and a usage state of a display system 1A which is a first modification of the display system 1. With reference to FIG. 11, the display system 1 </ b> A includes a housing 10 </ b> A, an opening 20 </ b> A, a sensor array 30, and a microlens array 40.

The opening 20A has a rectangular shape. Similarly to the opening 20, the opening 20 </ b> A is formed below the display area 810 along the side 810 b of the display area 810. The length of the opening 20 </ b> A in the X direction is the same as that of the opening 20. The length in the Z direction of the opening 20 </ b> A is longer than that of the opening 20. The sensor array 30 is configured to be movable in the Z direction.

FIG. 12 is a cross-sectional view taken along line XII-XII in FIG. Referring to FIG. 12, the display system 1A includes a sensor array 30, a microlens array 40, and a display 50 in a housing 10A.

The display system 1A includes a moving mechanism 110 (see FIG. 12) that translates the display in the direction of the arrow 711 (third direction). In addition, the display system 1A includes a moving mechanism 120 (see FIG. 12) that translates the sensor array in the direction of the arrow 713 (fourth direction). The direction of the arrow 711 and the direction of the arrow 713 are the Z direction. That is, the direction of the arrow 711 and the direction of the arrow 713 are directions perpendicular to the display surface of the display 50. The moving mechanisms 110 and 120 are configured using, for example, an actuator.

When the display 50 is moved by the moving mechanism 110, the position of the display area 810 is also moved in the direction of the arrow 712. Specifically, when the display 50 moves in the negative direction of the Z axis, the display area 810 moves in parallel in the positive direction of the Z axis by a distance corresponding to the amount of movement of the display 50. When the display 50 moves in the positive direction of the Z axis, the display area 810 moves in parallel in the negative direction of the Z axis by a distance corresponding to the amount of movement of the display 50.

The sensor array 30 is moved by the moving mechanism 120 in the same direction as the moving direction of the display area 810 by the same amount as the moving amount of the display area 810. In other words, the sensor array 30 moves in the direction corresponding to the movement of the display 50 by the amount corresponding to the movement amount of the display 50. Thus, in the display system 1A, the sensor array 30 moves in conjunction with the movement of the display 50.

FIG. 13 is a block diagram showing a part of the hardware configuration of the display system 1A. Referring to FIG. 13, display system 1A includes a sensor array 30, a display 50, a CPU 60, a memory 70A, a display driving device 80, an A / D converter 90, a moving mechanism 110, and a moving mechanism 120. Is provided.

The memory 70 </ b> A stores positional relationship data 72 in addition to the programs and data stored in the memory 70. The positional relationship data 72 is data indicating the correspondence between the position of the display 50 and the position of the sensor array 30.

When the CPU 60 receives a command for moving the moving mechanism 110 from an input device (for example, an operation key) (not shown), the CPU 60 sends a command for moving the display 50 to the moving mechanism 110 by a direction and an amount corresponding to the command. The moving mechanism 110 moves the display 50 based on a command from the CPU 60.

When the CPU 60 receives an instruction to move the moving mechanism 110 as described above, the CPU 60 further determines the position of the sensor array 30 based on the position of the display 50 and the positional relationship data 72. When the position of the sensor array 30 is determined, the CPU 60 moves the sensor array 30 to the determined position using the moving mechanism 120.

As described above, the display system 1A can change the position of the display area 810 in the Z-axis direction. That is, the display system 1A can change the position of the aerial image. Further, the display system 1A causes the sensor array 30 to follow the movement of the display area 810. Therefore, the display system 1A can detect the position of the object in the two-dimensional image while allowing the position of the display area 810 to be changed.

<Second Modification>
FIG. 14 is a diagram illustrating an appearance and a usage state of a display system 1B which is a second modification of the display system 1. Referring to FIG. 14, display system 1B includes a housing 10B, an opening 20B, an opening 20C, a sensor array 30, a microlens array 40, and a sensor array 140.

The opening 20B and the opening 20C have the same shape as the opening 20. Similar to the opening 20, the opening 20 </ b> B is formed below the display area 810 along the side 810 b of the display area 810. The opening 20C is formed so as to be parallel to the opening 20B in the negative direction of the Z axis relative to the opening 20B.

The sensor array 30 is disposed along the opening 20B in the housing 10B. As described above, the sensor array 30 is installed such that the sensing surfaces of the distance measuring sensors 31 to 38 face the display area 810.

The sensor array 140 has the same configuration as the sensor array 30. That is, the sensor array 140 has eight distance measuring sensors arranged in a row in the X-axis direction. The sensor array 140 is arranged in parallel with the sensor array 30 at a position where it intersects with a virtual plane (not shown) parallel to the display area 810.

Specifically, the sensor array 140 is disposed at a position obtained by translating the sensor array 30 in the negative direction of the Z axis. That is, the X coordinate values of the eight distance measuring sensors of the sensor array 140 and the eight distance measuring sensors 31 to 38 of the sensor array 30 are all the same. Also, the Y coordinate values of the sensor array 140 and the sensor array 30 are the same.

15 is a cross-sectional view taken along line XV-XV in FIG. Referring to FIG. 15, the display system 1B includes a sensor array 30, a microlens array 40, a display 50, and a sensor array 140 in a housing 10B.

The sensing area 610 by the sensor array 140 is formed in the virtual plane. That is, the virtual plane includes the sensing area 610. The sensor array 140 is installed so that the sensing surface of each distance measuring sensor of the sensor array 140 faces the Y direction. That is, the sensor array 140 emits infrared rays in the same direction as the sensor array 30 (the positive direction of the Y axis).

FIG. 16 is a block diagram showing a part of the hardware configuration of the display system 1B. Referring to FIG. 16, a display system 1B includes a sensor array 30, a display 50, a CPU 60, a memory 70, a display driving device 80, an A / D converter 90, a sensor array 140, and an A / D converter. 150.

Sensor array 140 outputs an analog voltage value as a sensing result to A / D converter 150. The A / D converter 150 converts an analog voltage value into a digital voltage value. The A / D converter 150 sends the converted digital voltage value to the CPU 60.

The CPU 60 detects the position of the finger 910 in the virtual plane based on the voltage value output by each of the distance measuring sensors included in the sensor array 140 and the association data 71. More precisely, the CPU 60 detects the position of the finger 910 in the sensing area 610 based on the voltage value output by each of the distance measuring sensors included in the sensor array 140 and the association data 71.

CPU 60 determines whether or not the position on the plane including the display area 810 corresponding to the position of the finger 910 on the virtual plane is included in the area for displaying the object in the display area 810. More precisely, the CPU 60 determines whether or not the position in the display area 810 corresponding to the position of the finger 910 in the sensing area 610 is included in the area for displaying the object in the display area 810.

The CPU 60 changes the display mode of the object on the display 50 from the first display mode to the second display mode based on the determination that it is included in the area for displaying the object.

FIG. 17 is a diagram for explaining the first display mode and the second display mode. FIG. 17A is a diagram illustrating a first example of the display mode after the change. FIG. 17B is a diagram showing a second example of the display mode after the change.

Referring to FIG. 17A, as a result of the user moving the finger 910 in the positive direction of the Z-axis (see FIG. 15), the position in the plane including the display area 810 corresponding to the position of the finger 910 in the virtual plane is It is assumed that the object B is included in the display area 810 in the display area. More specifically, it is assumed that the value of the XY coordinate of the finger 910 in the sensing area 610 is a value included in the display area of the object B in the display area 810. In this case, the CPU 60 changes the color of the object B on the display 50 from a normal color to a color different from the normal color.

Referring to FIG. 17B, when the value of the XY coordinate of finger 910 in sensing area 610 is a value included in the display area of object B in display area 810, CPU 60 determines that object B in display 50 The size is changed from the normal size to a size larger than the normal size.

FIG. 18 is a flowchart showing the flow of display mode change processing in the display system 1B. More specifically, FIG. 18 shows processing performed when the objects A to F shown in FIG. 17 are displayed.

Referring to FIG. 18, in step S <b> 22, CPU 60 determines whether finger 910 has touched sensing region 610 based on the output of sensor array 140. When CPU 60 determines that finger 910 has touched sensing area 610 (YES in step S22), in step S24, is the position of finger 910 in sensing area 610 corresponding to objects AF in display area 810? Judge whether or not. When CPU 60 determines that finger 910 has not touched sensing area 610 (NO in step S22), CPU 60 advances the process to step S22.

When CPU 60 determines that the position corresponds to objects A to F (YES in step S24), it changes the color of the object corresponding to the position of finger 910 in step S26. If CPU 610 determines that the position does not correspond to objects A to F (NO in step S24), it generates an error sound in step S34.

In step S28, based on the output of the sensor array 30, the CPU 60 determines whether or not the finger 910 has touched any of the objects A to F in the display area 810. If CPU 60 determines that the object has been touched (YES in step S28), it determines in step S30 whether the object at the position touched by finger 910 is the same as the object whose color has been changed. When CPU 60 determines that it is not touched (NO in step S28), the process proceeds to step S28.

When CPU 60 determines that they are the same object (YES in step S30), in step S32, CPU 60 performs an operation corresponding to the object touched by finger 910. When CPU 60 determines that they are not the same object (NO in step S30), it generates an error sound in step S36.

As described above, when the finger 910 reaches the sensing area 610, the display system 1B changes the display mode of the object at the position corresponding to the reached position. Therefore, the user can know in advance which object can be selected by further moving the finger 910 in the negative direction of the Z axis. Therefore, the display system 1B is superior in operability than the display system 1.

The arrangement of the sensor array 30 and the sensor array 140 may be reversed. That is, the sensor array 140 may be arranged so that the sensing area 610 is sandwiched between the display area 810 and the microlens array 40.

<Third Modification>
Next, a third modification of the display system 1 will be described. The display system according to the third modification (hereinafter referred to as “display system 1C”) is different from the display system 1 in the configuration of the sensor array. More specifically, the sensor array of the display system 1C is different from the display system 1 in the configuration of the distance measuring sensor.

Each distance measuring sensor in the display system 1C includes one light emitting device and two light receiving devices. In this respect, each distance measuring sensor is different from the display system 1 including one light emitting device and one light receiving device. Note that the number of distance measuring sensors is the same between the display system 1 </ b> C and the display system 1.

FIG. 19 is a diagram for explaining the configuration of the distance measuring sensor 31A of the display system 1C. Referring to FIG. 19, the light emitting device 311 includes the infrared LED 311a and the lens 311b as described above. The light receiving device 312 includes an infrared light receiving element 312a and a lens 312b. The light receiving device 313 includes an infrared light receiving element 313a and a lens 313b. The characteristics of the infrared light receiving element 313a are different from those of the infrared light receiving element 312a.

FIG. 20 shows the output characteristics of the sensor of the display system 1C. Specifically, FIG. 20 is a diagram showing the characteristics of the infrared light receiving element 313a and the characteristics of the infrared light receiving element 312a. Referring to FIG. 20, infrared light receiving element 313a has a characteristic that when the distance d is greater than di (di ≦ d2), the output voltage decreases as the distance to the sensing object increases. Therefore, the distance measuring sensor 31A can sense an object existing between the distances d1 to d3.

Referring back to FIG. 19, the infrared LED 311a emits infrared rays to the lens 311b. The infrared rays emitted from the lens 311b are reflected by the object 950 that is a sensing target. The reflected infrared light passes through the lens 312b and enters the infrared light receiving element 312a. When there is an object 970 instead of the objects 950 and 960, the infrared light emitted from the infrared LED 311a is reflected by the object 970. The infrared light reflected by the object 970 passes through the lens 313b and enters the infrared light receiving element 313a. In this way, the display system 1C can also sense the object 970 existing at the distances d2 to d3.

<Fourth Modification>
Next, a fourth modification of the display system 1 will be described. The display system according to the fourth modification (hereinafter referred to as “display system 1D”) is different from the display system 1 in the configuration of the sensor array. More specifically, the sensor array of the display system 1D is different from the display system 1 in the number of distance measuring sensors.

FIG. 21 is a diagram for explaining the sensor array 30A of the display system 1D. Referring to FIG. 21, CPU 60 executes a program for displaying an object at a predetermined position on display 50. For example, the display 50 displays the objects G to N at eight predetermined positions. The positions of the objects H and G, the positions of the objects J and I, the positions of the objects L and K, and the positions of the objects N and M are the same.

When it is assumed that the CPU 60 executes such a program, the sensor array 30 includes distance measuring sensors 31, 33, 36, and 38 at four positions corresponding to each object displayed on the display 50. It only has to be.

In such a configuration, the number of distance measuring sensors can be reduced as compared with the display system 1. Therefore, in the display system 1D, the manufacturing cost can be reduced more than the manufacturing cost of the display system 1.

[Embodiment 2]
FIG. 22 is a diagram illustrating an appearance and a usage state of the display system 2. With reference to FIG. 22, the display system 2 includes a housing 11, an opening 21, a sensor array 30, and an optical element 41.

The optical element 41 transmits light emitted from the display (see FIG. 23) in the housing 11 and displays a two-dimensional image (aerial image) in the rectangular display area 820 in the air. As the optical element 41, for example, the imaging element of Patent Document 3 described as the background art can be used.

The display area 820 is an area surrounded by four sides 820a, 820b, 820c, and 820d. Note that the side 820a and the side 820b are parallel, and the side 820c and the side 820d are parallel. The direction of the normal line of the display area 820 is the z direction in the xyz coordinate system. The display area 820 is parallel to the xy plane.

Note that the x direction of the xyz coordinate system and the X direction of the XYZ coordinate system are parallel. The xyz coordinate system is a coordinate system obtained by rotating the XYZ coordinate system by a predetermined angle about the X axis as a rotation axis.

The opening 21 has a rectangular shape. The opening 21 is formed along the side 820b of the display area 820 below the display area 820 (in the negative y-axis direction).

The sensor array 30 has a plurality of ranging sensors 31 to 38 arranged in a row in the x direction. The sensor array 30 is arranged along the opening 21 in the housing 11. Specifically, the sensor array 30 is installed so that the sensing surfaces of the distance measuring sensors 31 to 38 face the display area 820.

In the display system 2, the user touches the aerial image displayed in the display area 820 with, for example, the user's finger 910. Specifically, the user touches the object with the finger 910 in order to select the object included in the aerial image.

23 is a cross-sectional view taken along line XXIII-XXIII in FIG. Referring to FIG. 23, the display system 2 includes a sensor array 30 and a display 50 in the housing 11. The display system 2 includes an optical element 41 in an opening provided on the surface of the housing 11.

The display 50 displays an image in the direction of the optical element 41. The image displayed on the display 50 is displayed in the display area 820 as an aerial image by the optical element 41. More detailed description is as follows.

The display 50 is installed with an inclination of 90 ° -θa with respect to the optical element 41. The incident angle of the light emitted from the display 50 with respect to the optical element 41 is also 90 ° −θa. The optical element 41 emits the light emitted from the display 50 at an emission angle of 90 ° −θb. As a result, the image displayed on the display 50 is displayed in the display area 820 as an aerial image.

Each of the distance measuring sensors 31 to 38 of the sensor array 30 is in the y direction perpendicular to the arrangement direction (x direction) of the distance measuring sensors and the normal direction (z direction) of the display area 820, and is in the display area 820. Light is emitted in the direction of approach. More specifically, the sensor array 30 is disposed at a position that intersects a plane including the display area 820. That is, the sensor array 30 is arranged at a position parallel to the sides 820a and 820b of the display area 820 (see FIG. 22).

It should be noted that between the angle θc formed by the optical path of the light emitted from the sensor array 30 and the optical element 41 (that is, the angle formed by the display area 820 and the optical element 41) and θb, θc = 90 ° −θb. A relationship is established.

The sensor array 30 may be arranged so that the light emitted from the distance measuring sensors 31 to 38 passes through the display area 820 (that is, the light overlaps the display area 820), or the light is displayed in the display area 820. You may arrange | position so that it may follow along. Hereinafter, a case where the light emitted from the distance measuring sensors 31 to 38 is arranged so as to pass through the display area 820 will be described as an example.

FIG. 24 is a block diagram showing a part of the hardware configuration of the display system 2. Referring to FIG. 24, display system 2 includes a sensor array 30, a display 50, a CPU 60, a memory 70, a display driving device 80, and an A / D converter 90. That is, the display system 2 has the same configuration as the display system 1 according to the first embodiment. Therefore, the description about each hardware of the display system 2 is not repeated here.

FIG. 25 is a diagram showing a correspondence relationship between each of the distance measuring sensors 31 to 38 and the display area 820. Referring to FIG. 25, each of the distance measuring sensors 31 to 38 emits light in the y direction. Each of the distance measuring sensors 31 to 38 receives the light reflected by the object among the emitted light, and outputs a voltage value based on the distance to the object to the A / D converter 90.

As described above, in the display system 2, the position of the object in the two-dimensional image can be determined by installing the sensor array 30 in which the plurality of distance measuring sensors 31 to 38 are arranged in a row in the downward direction of the two-dimensional image. . Therefore, the display system 2 can detect the position of the object in the two-dimensional image with a simple configuration without surrounding the two-dimensional image displayed in the air with a frame.

<First Modification>
FIG. 26 is a diagram illustrating an appearance and a usage state of a display system 2A which is a first modification of the display system 2. With reference to FIG. 26, the display system 2 </ b> A includes a housing 11 </ b> A, an opening 21 </ b> A, a sensor array 30, and an optical element 41.

The opening 21A has a rectangular shape. Similarly to the opening 21, the opening 21 </ b> A is formed below the display area 820 along the side 820 b of the display area 820. The length of the opening 21A in the X direction is the same as that of the opening 21. The length of the opening 21 </ b> A in the Y direction is longer than that of the opening 21. The sensor array 30 is configured to be movable in the Y direction.

FIG. 27 is a sectional view taken along line XXVII-XXVII in FIG. Referring to FIG. 27, display system 2A includes a sensor array 30 and a display 50 in housing 11A. The display system 2A includes an optical element 41 in an opening provided on the surface of the housing 11A. The position of the optical element 41 in the display system 2A is the same as the position of the optical element 41 in the display system 2.

The display system 2A includes a moving mechanism 110A (see FIG. 28) that translates the display 50 in the direction of the arrow 721 (third direction). In addition, the display system 2A includes a moving mechanism 120A (see FIG. 28) that translates the sensor array 30 in the direction of the arrow 723 (fourth direction). Note that the direction of the arrow 721 is a direction in which the incident angle of the light emitted from the display 50 to the optical element 41 does not change. The direction of the arrow 723 is the Y-axis direction. Note that the direction of the arrow 723 is also a direction having a component in the normal direction of the display region 820.

When the display 50 is moved by the moving mechanism 110A, the position of the display area 820 is also moved in the direction of the arrow 722. In other words, the display area 820 moves in the direction of the normal line of the display area 820 based on the movement of the display 50. Note that the direction of the normal line is different from the direction of the arrow 721.

Specifically, when the display 50 moves in a direction away from the optical element 41, the display area 820 moves in parallel in a direction away from the optical element 41 by a distance corresponding to the amount of movement of the display 50. When the display 50 moves in the direction approaching the optical element 41, the display area 820 moves in parallel by a distance corresponding to the amount of movement of the display 50 in the direction approaching the optical element 41.

The sensor array 30 is moved in the direction according to the movement of the display area 820 by the moving mechanism 120A. In other words, the sensor array 30 moves in the direction corresponding to the movement of the display 50 by the amount corresponding to the movement amount of the display 50. Thus, in the display system 2A, the sensor array 30 moves in conjunction with the movement of the display 50.

FIG. 28 is a block diagram showing a part of the hardware configuration of the display system 2A. Referring to FIG. 28, display system 2A includes sensor array 30, display 50, CPU 60, memory 70A, display driving device 80, A / D converter 90, moving mechanism 110A, and moving mechanism 120A. Is provided.

The memory 70A stores the positional relationship data 72 as described above. The positional relationship data 72 is data indicating the correspondence between the position of the display 50 and the position of the sensor array 30.

When the CPU 60 receives a command for moving the moving mechanism 110A from an input device (for example, an operation key) (not shown), the CPU 60 sends a command for moving the display 50 by the direction and amount corresponding to the command to the moving mechanism 110A. The moving mechanism 110 </ b> A moves the display 50 based on a command from the CPU 60.

When the CPU 60 receives a command to move the moving mechanism 110A as described above, the CPU 60 further determines the position of the sensor array 30 based on the position of the display 50 and the positional relationship data 72. When the position of the sensor array 30 is determined, the CPU 60 moves the sensor array 30 to the determined position using the moving mechanism 120A.

By the way, in the display system 2A, the distance from the display area 820 changes according to the position of the sensor array 30. For this reason, even when the same position in the two-dimensional image is in contact with the finger 910, the voltage value output by the sensor array 30 differs depending on the position of the sensor array 30. Therefore, in order to accurately detect the y coordinate value of the position of the finger 910 in the two-dimensional image, the CPU 60 performs processing in consideration of the position of the sensor array 30 when specifying the y coordinate value of the position of the finger 910. Necessary.

Therefore, in the display system 2A, the CPU 60 determines the finger 910 in the two-dimensional image based on each voltage value output from each of the distance measuring sensors 31 to 38, the association data 71, and the position of the sensor array 30. The position of is detected. For example, a program for adding a distance considering the moving distance of the sensor array 30 to the distance obtained from the output voltage in the characteristics of the distance measuring sensor (see FIG. 8) to obtain the y coordinate value in the display area 820. And may be stored in the memory 70A.

As described above, the display system 2A can change the position of the display area 820 in the z-axis direction. That is, the display system 2A can change the position of the aerial image. Further, the display system 2A causes the sensor array 30 to follow the movement of the display area 820. Therefore, the display system 2A can detect the position of the object in the two-dimensional image while allowing the position of the display area 820 to be changed.

The CPU 60 detects the position of the finger 910 in the two-dimensional image based on each voltage value output from each of the distance measuring sensors 31 to 38, the association data 71, and the position of the display 50. The display system 2A may be configured. In this case, for example, the distance obtained by considering the moving distance of the display 50 is added to the distance obtained from the output voltage in the characteristics of the distance measuring sensor (see FIG. 8) to obtain the y coordinate value in the display area 820. The program may be stored in the memory 70A.

<Second Modification>
FIG. 29 is a diagram illustrating an appearance and a usage state of a display system 2B which is a second modification of the display system 2. Referring to FIG. 29, display system 2B includes a casing 11B, an opening 21B, a sensor array 30, and an optical element 41.

The opening 21B has a rectangular shape. Similar to the opening 21A, the opening 21B is formed below the display area 820 along the side 820b of the display area 820. The length of the opening 21B in the X direction is the same as that of the opening 21. The length of the opening 21 </ b> B in the Y direction is longer than that of the opening 21. The sensor array 30 is configured to be movable in a predetermined direction (see FIG. 30). For example, the opening 21B has the same shape as the opening 21A of the display system 2A.

FIG. 30 is a cross-sectional view taken along line XXX-XXX in FIG. Referring to FIG. 30, the display system 2B includes a sensor array 30 and a display 50 in a housing 11B. The display system 2B includes an optical element 41 in an opening provided on the surface of the housing 11B. Note that the position of the optical element 41 in the display system 2B is the same as the position of the optical element 41 in the display system 2.

The display system 2B includes a moving mechanism 110B (see FIG. 31) that translates the display 50 in the direction of the arrow 731 (third direction). Further, the display system 2A includes a moving mechanism 120B (see FIG. 31) that translates the sensor array 30 in the direction of the arrow 733 (fourth direction). Note that the direction of the arrow 731 is a direction in which the incident angle of the light with respect to the optical element 41 based on the image displayed on the display 50 is changed. The direction of the arrow 733 is a direction in which the emission angle of each light emitted from the plurality of distance measuring sensors 31 to 38 is changed.

More specifically, the display 50 rotates within a predetermined first range around a predetermined first rotation axis (not shown). In addition, the sensor array 30 rotates within a predetermined second range around a predetermined second rotation axis J1.

When the display 50 is moved by the moving mechanism 110 </ b> B, the position of the display area 830 is also moved in the direction of the arrow 732. Specifically, the display area 820 rotates around the second rotation axis J <b> 1 based on the movement of the display 50.

Specifically, when the display 50 moves in a direction away from the optical element 41, the display area 820 moves in a direction away from the optical element 41 by a distance corresponding to the movement amount (rotation amount) of the display 50. When the display 50 moves in a direction approaching the optical element 41, the display area 820 moves by a distance corresponding to the movement amount (rotation amount) of the display 50 in a direction approaching the optical element 41.

The sensor array 30 is moved in a direction according to the movement of the display area 820 by the moving mechanism 120B. More specifically, the sensor array 30 rotates in the direction corresponding to the movement of the display 50 by an amount corresponding to the rotation amount of the display 50. Thus, in the display system 2A, the sensor array 30 moves in conjunction with the movement of the display 50.

FIG. 31 is a block diagram showing a part of the hardware configuration of the display system 2B. Referring to FIG. 31, display system 2B includes sensor array 30, display 50, CPU 60, memory 70A, display driving device 80, A / D converter 90, moving mechanism 110B, and moving mechanism 120B. Is provided.

When the CPU 60 receives a command to move the moving mechanism 110B from an input device (for example, an operation key) (not shown), the CPU 60 sends a command for moving the display 50 by the direction and amount corresponding to the command to the moving mechanism 110B. The moving mechanism 110 </ b> B moves the display 50 based on a command from the CPU 60.

When the CPU 60 receives a command to move the moving mechanism 110B as described above, the CPU 60 further determines the position of the sensor array 30 based on the position of the display 50 and the positional relationship data 72. When the position of the sensor array 30 is determined, the CPU 60 moves the sensor array 30 to the determined position using the moving mechanism 120B.

As described above, the display system 2B can change the angle of the display area 820 with respect to the optical element 41. That is, the display system 2B can change the angle of the aerial image with respect to the optical element 41. Further, the display system 2B causes the sensor array 30 to follow the movement of the display area 820. For this reason, the display system 2B can detect the position of the object in the two-dimensional image while allowing the angle of the display area 820 to the optical element 41 to be changed.

<Third Modification>
The characteristics of the distance measuring sensors included in the sensor array 30 and the distance measuring sensors included in the sensor array 140 are not necessarily the same. In this case, data (correspondence data 71) indicating the correspondence between the output voltage and distance in each of the distance measuring sensors 31 to 38 included in the sensor array 30 and the output voltage of each distance measuring sensor included in the sensor array 140. And the data indicating the correspondence between the distance and the distance may be stored in the memories 70 and 70A.

<Fourth Modification>
The display system 2 may be modified like a display system 1B (see FIG. 15) which is a second modification of the display system 1. That is, in the display system 2, the sensor array 140 may be arranged in parallel with the sensor array 30.

In the above description, the display areas 810 and 820 are described as rectangular display areas. However, the display areas are not limited to rectangles. The shape of the display area 810 is a shape corresponding to the shape of the display surface of the display 50, the shape of the microlens array 40, and the shape of the optical element 41. For this reason, the shape of the display surface of the display used in the display system, the shape of the microlens array, and the shape of the optical element are different from the shape of the display surface of the display 50, the shape of the microlens array 40, and the shape of the optical element 41. In this case, the display area has a shape corresponding to the different shape. Even if the display area is not rectangular, according to the above-described configuration, the position of the object in the two-dimensional image can be detected with a simple configuration without surrounding the two-dimensional image displayed in the air with a frame.

The embodiment disclosed this time is an example, and is not limited to the above contents. The scope of the present invention is defined by the terms of the claims, and is intended to include any modifications within the scope and meaning equivalent to the terms of the claims.

1, 1A, 1B, 1C, 1D, 2, 2A, 2B Display system 10, 10A, 10B, 11, 11A, 11B housing, 20, 20A, 20B, 20C, 21, 21A, 21B opening, 30, 30A, 140 sensor array, 31-38, 31A ranging sensor, 40 microlens array, 41 optical elements, 50 display, 70, 70A memory, 71 association data, 72 positional relationship data, 80 display drive device, 90 A / D converter, 110, 120, 110A, 110B, 120A, 120B moving mechanism, 150 A / D converter, 311 light emitting device, 312, 313 light receiving device, 312a, 313a infrared light receiving element, 610 sensing area, 810, 820, 830 display Area, 910 fingers, A ~ Objects, 311a infrared LED, 312a, 313a infrared receiving component.

Claims (12)

A display (50);
An optical element (40, 41) for displaying a two-dimensional image in an aerial display area (810, 820) based on the image displayed on the display;
A processor (60);
A first sensor array (30, 30A) in which a plurality of distance measuring sensors are arranged in a row in a first direction;
A memory (70, 70A) storing first data indicating a correspondence relationship between an output voltage and a distance in each distance measuring sensor;
The normal direction of the display area is a direction perpendicular to the first direction;
Each of the plurality of distance measuring sensors emits light in a second direction perpendicular to the first direction and the normal direction and approaching the display area,
Each of the plurality of distance measuring sensors receives light reflected by an object out of the emitted light, and outputs a voltage value based on a distance to the object,
The processor detects a position of the object in the two-dimensional image based on each voltage value output from each of the plurality of distance measuring sensors and the first data, a display system (1, 1A, 1B, 1C, 1D, 2, 2A, 2B). The display system according to claim 1, wherein the first sensor array is disposed at a position intersecting with a plane including the display area. The display system according to claim 2, wherein the light emitted from each of the plurality of distance measuring sensors passes through the display area. The display system includes:
A first moving mechanism (110, 110A, 110B) for moving the display in a third direction;
A second moving mechanism (120, 120A, 120B) for moving the first sensor array in a fourth direction;
The memory further stores second data indicating a correspondence relationship between the position of the display and the position of the first sensor array;
The display area moves based on the movement of the display,
The processor is
Determining a position of the first sensor array based on the position of the display and the second data;
The display system (1A, 2A, 2B) according to any one of claims 1 to 3, wherein the first sensor array is moved to the determined position using the second moving mechanism. The third direction is a direction perpendicular to a display surface of the display;
The display area moves in the third direction based on the movement of the display in the third direction,
The display system (1A) according to claim 4, wherein the fourth direction is the same direction as the third direction. The third direction is a direction perpendicular to a display surface of the display;
The direction of the normal of the display area is different from the third direction,
The display area moves in the direction of the normal based on the movement of the display in the third direction,
The fourth direction is a direction having a component in the direction of the normal line,
The processor is configured to output the two-dimensional image based on each voltage value output from each of the plurality of ranging sensors, the first data, and the position of the first sensor array or the position of the display. The display system (2A) according to claim 4, wherein the position of the object is detected. The third direction is a direction for changing an incident angle of light with respect to the optical element based on an image displayed on the display;
The display system (2B) according to claim 4, wherein the fourth direction is a direction in which an emission angle of each light emitted from the plurality of distance measuring sensors is changed. The display system includes:
A second sensor array (140) in which a plurality of distance measuring sensors are arranged in a row in the first direction;
The second sensor array is arranged in parallel with the first sensor array, emits light in the same direction as the first sensor array,
The processor is
Displaying at least one object in the display area;
Detecting the position of the object based on the voltage value output from each of the plurality of distance measuring sensors included in the second sensor array and the first data;
Determining whether a position on a plane including the display area corresponding to the position of the detected object is included in an area displaying the at least one object in the display area;
The display mode of the object in the display area is changed from the first display mode to the second display mode based on the determination that the object is included in the display area. The display system (1B) according to item. The processor displays at least one object in the display area;
The display system (1D) according to any one of claims 1 to 8, wherein a distance measuring sensor in the first sensor array (30A) is provided at a position corresponding to the at least one object. 2. Each of the plurality of distance measuring sensors in the first sensor array includes one light emitting element and two or more light receiving elements that respectively receive reflected light of light emitted from the light emitting element. The display system (1C) according to any one of 1 to 9. The processor is
Displaying at least one object in the display area;
The process associated with the object is executed based on the fact that the position of the object detected by the first sensor array is included in an area displaying the at least one object. The display system according to any one of 1 to 10. A detection method in a display system (1, 1A, 1B, 1C, 1D, 2, 2A, 2B) for detecting the position of an object in a two-dimensional image displayed in an aerial display area (810, 820),
The display system includes a processor (60), a sensor array (30, 30A) in which a plurality of distance measuring sensors are arranged in a row in a first direction, and a correspondence relationship between an output voltage and a distance in each of the distance measuring sensors. And a memory (70, 70A) that stores data indicating the normal direction of the display area is a direction perpendicular to the first direction,
The detection method is:
Each of the plurality of distance measuring sensors emits light in a second direction perpendicular to the first direction and the normal direction and approaching the display area (S4);
Each of the plurality of distance measuring sensors receives light reflected by an object out of the emitted light, and outputs a voltage value based on the distance to the object (S6, S8);
The processor includes a step (S16) of detecting the position of the object in the two-dimensional image based on each voltage value output by each of the plurality of distance measuring sensors and the data. .

Images