TWI754899B - Floating image display apparatus, interactive method and system for the same - Google Patents

Abstract

An interactive method using a floating image display apparatus and a system thereof are provided. The method is performed in the display apparatus. The display apparatus links to a computer system that executes a driver for establishing a communication interface there-between. In the method, the display apparatus receives image data from the computer system and floating image data can be obtained therefrom. The floating image display apparatus displays a floating image via a display panel after a conversion process. After that, a user can manipulates directly on the floating image by gesture. A computer sequence running in the display apparatus can determines an interactive instruction according to a change of three-dimensional coordinates. When the interactive instruction is transmitted to the computer system, a new image data is formed responsive to the instruction. The display apparatus then displays a new floating image after receiving new data.

Description

Floating image display device, interactive method and system thereof

The specification discloses the application of a floating image display technology, and particularly proposes a floating image display device, an interactive method performed by connecting the device to a computer system, and a combined system.

The technology adopted by the generally known stereoscopic display devices is mostly the method of combining human binocular parallax to form a deep stereoscopic image in the brain. The image displayed by the display is designed as two different pictures. When the user uses Certain devices, such as red and blue glasses, polarized glasses or shutter glasses, or even a head-mounted display, can project the two different images displayed by the display to the eyes respectively, and then combine them in the brain to form a stereo image .

There is also a naked-view stereoscopic display technology. Through the design of optical elements (lenses, gratings), the images in the display can be directly projected on the eyes of people, so that the stereoscopic images can be directly felt without special equipment.

In all kinds of stereoscopic display technologies, they are still display technologies, and there is no interactive technology that allows users to directly manipulate stereoscopic images. The main reason is that common stereoscopic image display technologies are virtual images assembled in the brain, which cannot be Interact directly with images.

In terms of image interaction technology, one of them is gesture operations performed by the user on a capacitive or resistive display. In this type of interaction technology through touch technology, gestures are sensed by the touch display, and then a software program is used to perform gesture operations. The interactive image is displayed according to the gesture, and the interactive behavior is, for example, an action of panning and rotating on a two-dimensional plane with a single-point or multi-point gesture on the flat-panel display.

In addition, there are also interactive operation methods in Augmented Reality (AR) or Virtual Reality (VR) technologies. Augmented reality or virtual reality is generally performed through flat-panel displays or head-mounted devices. The Head-Mounted Display (HMD) first displays the image, and the user can operate the image with gestures, and additionally use the camera to capture the gestures. After image processing, an interactive image can be generated to produce an interactive effect.

The specification discloses an interactive method and system using a floating image display device, and a floating image display device therein, wherein the floating image display device is used to display a floating image, and after being connected to a computer system, it provides users with the ability to actually operate the floating three-dimensional image. New user experience.

The floating image display system mainly includes a floating image display device and a driver running in the computer system, wherein the floating image display device is provided with a gesture sensor, and the floating image display device can be connected to the computer system in a built-in or external manner.

According to an embodiment, the interactive method using a floating image display device runs in the floating image display device, and the floating image display device is connected to a computer system, including a computer device that can be connected to a computer device through a connection, or a modularized device Built into the computer device.

In the method, the floating image display device receives image data from the computer system, and then obtains floating image data according to the image data, and displays a floating image on the display panel. Afterwards, the user is allowed to operate the floating image with a gesture, and the gesture is sensed by a gesture sensor, and the change amount of the three-dimensional coordinate generated by the gesture is obtained, and an interactive command can be judged accordingly.

Then, the interactive command is sent to the computer system, and the computer system responds to the interactive command, and a new image data corresponding to the change of the three-dimensional coordinate can be obtained, and then the new image data is sent to the floating image display device, so that it can be A new floating image data is obtained according to the new image data, and after conversion, the new floating image is displayed on the display panel.

Further, a software program is run in the computer system, and the software program can execute a corresponding program according to the received interactive command, so that the computer system generates new image data according to the change of the three-dimensional coordinate, and this new image data is also floating. The image display device displays a new floating image, and generates a new image due to the gesture operation.

Preferably, the gesture is to reflect the changes of the three-dimensional coordinates of the user's fingers, palm positions and knuckles, and the interactive instruction generated corresponding to the change of the three-dimensional coordinates will indicate one or any combination of moving, rotating and zooming actions, Make the software program execute the corresponding program.

Further, a driver program is run in the computer system, and the driver program establishes an interface for communication between the computer system and the floating image display device, and is used for interpreting and compiling the information exchanged between the computer system and the floating image display device. In the computer system, according to the information generated by the computer system in response to the interactive command, the driver can send a request to an image server to download the image data or new image data provided to the floating image display device. Preferably, the color information and three-dimensional space information describing the floating image are recorded in the data.

According to the embodiment of the floating image display device, the main circuit includes an image processor for processing the image data forming the floating image; an input interface for connecting with the gesture sensor to receive gestures generated by the gesture sensor. information, so that the variation of the three-dimensional coordinate can be calculated by the image processor to form an interactive command; a display driving unit is used to obtain the floating image according to the description of the floating image data, and then display the floating image through the display unit; and a computer connection unit, The floating image display device is connected to the computer device through the computer connection unit.

When the floating image display device receives the image data from the computer system, the floating image data can be obtained according to the image data. After the floating image data is converted by the display driving unit, the floating image can be displayed through the display unit, and the user can directly display the floating image on the floating image. The gesture operation is performed, and the user can operate the floating image and control the software program in the computer system through the exchange of information with the computer system.

For a further understanding of the features and technical content of the present invention, please refer to the following detailed descriptions and drawings of the present invention. However, the drawings provided are only for reference and description, and are not intended to limit the present invention.

The following are specific embodiments to illustrate the embodiments of the present invention, and those skilled in the art can understand the advantages and effects of the present invention from the content disclosed in this specification. The present invention can be implemented or applied through other different specific embodiments, and various details in this specification can also be modified and changed based on different viewpoints and applications without departing from the concept of the present invention. In addition, the drawings of the present invention are merely schematic illustrations, and are not drawn according to the actual size, and are stated in advance. The following embodiments will further describe the related technical contents of the present invention in detail, but the disclosed contents are not intended to limit the protection scope of the present invention.

It should be understood that although terms such as "first", "second" and "third" may be used herein to describe various elements or signals, these elements or signals should not be limited by these terms. These terms are primarily used to distinguish one element from another element, or a signal from another signal. In addition, the term "or", as used herein, should include any one or a combination of more of the associated listed items, as the case may be.

The disclosure discloses a floating image display device, an interaction method performed by using the display device, and a related floating image display system. The technology of using the floating image for interaction is to provide users with the floating image displayed on the floating image display device while viewing the floating image. The floating image can also be manipulated through gestures. The main method is to use the gesture sensor to sense the gestures on the floating image, such as moving, rotating or zooming, and then obtain the floating coordinate change corresponding to the gesture. The determined interaction instruction corresponds to displaying the floating image generated with the gesture, so as to provide the user with the technical purpose of interacting with the floating image through the gesture. It should be mentioned that the floating image may be a floating image of a static object, or a continuous floating image of a dynamic object.

In one embodiment, the display technology in the floating image display device may be a technology for floating and displaying a three-dimensional image. This technology uses a flat display panel to display a plurality of sub-images (unit images), a plurality of sub-images. It forms a complete image (integrated image). When displayed, the image displayed on the flat display panel is displayed through a layer of optical elements (such as a lens array), and each sub-image is imaged through a corresponding lens, and can be displayed on the display device through multiple lenses. A complete image is formed on it, so that a display effect of a floating image can be produced.

It is worth mentioning that, different from the conventional display technologies of augmented reality (AR) or virtual reality (VR), in addition to the different display devices, the technology and effects of projecting stereoscopic images are also different. The display technology used in the AR or VR is a display technology that projects a stereoscopic image on the screen in front of you and reflects the stereoscopic effect on the image through the left and right eyes of a person. The optical element is projected at the position in front of the user's eyes, allowing the user to operate the floating image with gestures. The operation gesture is obtained through gesture sensing, and then a new floating image generated by the gesture is calculated to provide a real experience of the interactive effect of operating on the image. .

FIG. 1 shows a schematic diagram of an embodiment of a floating image display device.

This example shows a floating image display device 10. As in the above-described embodiment, the floating image display device 10 is composed of a layer of optical elements 101 and a display panel 102, and has an image processor, which can process display content and has communication capabilities. The floating image data can be obtained from an external image providing source, and the floating image 103 is displayed after image processing.

Further, the floating image display device 10 is provided with a gesture sensor 105 for sensing the movement of the user's hand 12 on the floating image 103 . The gesture sensor 105 can be achieved by an image detection method. The main method is to use a camera to obtain continuous pictures of several key parts of the hand 12 (such as fingers, palms, and knuckles). Through image processing, it can be obtained at a time. The image changes before and after the interval are mainly the three-dimensional coordinates of each key part in the gesture sensor coordinate system (such as X _sensor , Y _sensor , Z _sensor represented by the rectangular coordinate system, or γ, θ, φ represented by the spherical coordinate system) , after continuous acquisition, the variation of the three-dimensional coordinates can be obtained, and then converted into the variation of the three-dimensional coordinates (X _device , Y _device , Z _device ) of the coordinate system of the floating image display device 10 , such as vector (displacement, direction), velocity (distance and time) and other information, and finally can determine one or any combination of interactive commands for moving, rotating, and zooming.

In another embodiment, the gesture sensor 105 can use the principle of light blocking to sense the gesture. In the related art, a light source is used to first project the sensing light. When the hand 12 enters the light range, the light can sense the light The detector detects the light spot generated on the hand when the light is interrupted, and then obtains the amount of reflected light (reflecting the change part) and the time difference (reflecting the depth) of each key part from it. Similarly, according to the continuous acquisition of these From the light information, it is possible to obtain the three-dimensional coordinate change caused by the gesture, and then determine one or any combination of interactive commands for moving, rotating, and zooming.

In yet another embodiment, the gesture sensor 105 may include a sound wave generator and a receiver, and use the information reflected by the sound wave to determine the gesture. When the hand 12 is in the sensing range of the gesture sensor 105, the sound wave will be blocked, and the reflected wave will be generated in the same way, so that the sound wave receiver can detect several key parts of the hand 12 in space according to the change of the sound wave. The change amount of , and then converted into the change amount of the three-dimensional coordinate, according to which one or any combination of interactive commands of moving, rotating and zooming can be determined.

In this way, the interaction method implemented by the gesture sensor 105 using various gesture sensing technologies can determine the movement, rotation and scaling according to the changes in the three-dimensional coordinates of the fingers, the palm position and the knuckles of the hand 12 reflected by the gesture. One of the actions or any combination of interactive instructions, after that, by comparing the floating image data recorded in the image database, the next floating image data of the floating image 103 under various gesture operations is obtained, and the next floating image is displayed, The floating image data may include a static floating image, or a floating movie formed from continuous images. When changes are sensed, the processing procedures for coordinate transformation and stereoscopic image calculation can be calculated at high speed, and can instantly respond to gestures to produce real-time interactive effects.

FIG. 2 then shows a circuit embodiment diagram of the floating image display device.

The floating image display device 20 shown in the figure is connected to the gesture sensor 22. According to an embodiment, the two can be integrated into one device. The floating image display device 20 and the gesture sensor 22 are two built-in modules, or Yes, the floating image display device 20 and the gesture sensor 22 are two independent devices, which are connected by a connection line and can transmit messages to each other.

The floating image display device 20 is provided with an image processor 203 for performing image processing and the disclosed interaction method, and is electrically connected to other circuit elements, such as an input interface 201 , and the floating image display device 20 senses gestures from gestures through the input interface 201 . The controller 22 receives the information of the sensing gesture, and the input interface 201 is a communication interface. According to the embodiment, the information received through the input interface 201 is the amount of the three-dimensional coordinate change formed by the gesture sensor 22 sensed by the gesture, so that the image processor 203 can determine the interactive command according to the amount of the three-dimensional coordinate change, and then according to the interactive command The command calculates the image coordinate information of the new floating image.

In the image processor 203, after obtaining the three-dimensional coordinate change corresponding to the gesture obtained by the gesture sensor 22, interactive instructions such as rotation, zooming, and zooming can be calculated. The image database is used to obtain the floating image data corresponding to the change, or the image processor 203 can calculate the new floating image data in real time according to the interactive command. The floating image data is used to describe the color information and three-dimensional space information of the floating image displayed by the floating image display device 20 . The image processor 203 is electrically connected to the display driving unit 204 , and can obtain the floating image according to the description of the floating image data, and display the floating image through the display unit 205 .

The floating image display device 20 is connected to the interface unit 223 in the gesture sensor 22 by wired or wireless communication, which is a communication interface, thereby transmitting information. The gesture sensor 22 is provided with a sensing element 221, which realizes the technology of sensing gestures by means of light, sound waves, and images. The gesture position represented by the joint.

The control unit 222 is a control circuit in the gesture sensor 22 , and is connected to the floating image display device 20 through the interface unit 223 .

It should be mentioned here that the three-dimensional coordinate change corresponding to the gesture sensed by the gesture sensor 22 is the coordinate information of the coordinate system of the gesture sensor 22, and then provided to the floating image display device 20, it should be executed The coordinate conversion is to convert the coordinate information of the interactive command in the coordinate system formed by the gesture sensor 22 to the coordinate system formed by the floating image display device 20, so that the image coordinate information of the floating image can be continuously compared and displayed.

Among them, in the floating image displayed by the floating image display device 20, each position of the image is known, so the correlation of the gesture in the same coordinate system after the coordinate transformation can be determined, and this correlation expresses the three-dimensional image formed by the gesture. The relationship between the coordinates and the floating image in the display. For example, the moving gesture causes the floating image to move in a certain direction, so the three-dimensional coordinates of the floating image should be added with the amount of change in this direction, so a new floating image is obtained at the new position. In the image processor 203, Real-time query or real-time calculation of a new floating image data is provided to the display driving unit 204 to form a display image. If it is a rotation gesture, it will cause the floating image to rotate in one direction. Therefore, the three-dimensional coordinates of the original floating image should be added with the rotation change, and new floating image data will be formed to display the new floating image data. Similarly, if the zoom gesture is used to change the size of the floating image at the same reference point position, the relevant three-dimensional coordinates also change accordingly. The image processor 203 queries the image database or performs real-time calculation to generate new floating image data. Provided to the display driving unit 204 to form a display image.

The above gestures such as moving, rotating, and zooming can be combined arbitrarily to form interactive instructions, and then the image processor 203 calculates to generate new floating image data. The floating image data is the data that provides the floating image display device 20 to directly display the floating image.

In yet another embodiment of the application of the floating image display device to perform the interaction method proposed in the disclosure, the floating image display device 20 is provided with a computer connection unit 202, and the computer connection unit 202 is electrically connected to the image processor 203 in the display device, and the computer is connected to The unit 202 allows the floating image display device 20 to be connected to a computer device 25 by means of wireless (such as WiFi, Bluetooth, etc.) or wired (such as USB, RS232 and other industry standard interfaces) through a specific communication interface, and the computer device 25 runs a computer system, and Execute the driver program that communicates with the floating image display device 20, so that the interactive command generated by the gesture operation on the floating image display device 20 can be transmitted to the computer system, and the computer system then responds to the interactive command, and generates a new one through the program calculation in the computer system. The floating image data is sent back to the floating image display device 20 to display the new floating image.

According to an embodiment, the driver program running on the computer system is used to establish an interface for communication between the computer system and the floating image display device 20 , so as to interpret and compile the information exchanged between the computer system and the floating image display device 20 . Further, according to the information generated by the computer system in response to the interactive command, the driver can send a request to an external system (such as an image server) to download the image data or new image data provided to the floating image display device 20. The image data such as The above-mentioned floating image resource file records the color information and three-dimensional space information describing the floating image, so that the image processor 203 in the display device can calculate the floating image having a distance height from the display panel accordingly.

Continue according to the schematic diagram of the embodiment of the floating image display device connected to the external system through the computer system shown in FIG. 3, this example shows the image server 301 connected by the floating image display device 10 through the computer system 35 through the network 30, the floating image data can be obtained from the image server 301. The server 301 provides, and the image server 301 is provided with an image database 303, which provides query to obtain floating image data corresponding to various interactive instructions.

According to the actual operation mode, when the user performs interaction on the floating image, he can first inquire whether the data currently stored in the floating image display device 10 includes the next display mode corresponding to the interactive command, and if it already includes a new display mode mode, the floating image display device 10 can generate and display floating image data after self-calculation; if the next display mode generated by the interaction is not included, the generated interactive command is transmitted to the computer system 35 and can be immediately passed through the network 30 A request for a new floating image resource file is sent to the image server 301 to download new floating image data, and then loaded into the floating image display device 10 via the computer system 35 to display the next interactive floating image.

Referring to FIG. 1 , when the user interacts with the floating image on the floating image display device 10 with the hand 12 , the generated gestures form a three-dimensional coordinate change, and the interactive command is obtained accordingly, and at this time, the user is sent to the computer through a specific communication interface. In the system 35, the software program executed therein can execute a specific program according to the interactive instruction, and can also be transmitted to the image server 301 through the network 30, and the image server 301 provides new floating image data according to the interactive instruction.

Next, as shown in FIG. 4 , the flow chart describes an embodiment of an interactive method implemented by a floating image display device in cooperation with a computer system.

One of the ways of connecting the floating image display device to the computer device is to be built into the computer device through modularization, or to connect the computer device externally through a specific communication method. In the process of this embodiment, the floating image display device displays the The floating image is mainly an image generated according to the interaction of a specific software program running in the computer system. At the beginning of the process of the embodiment, after the floating image display device is connected to the computer system, it can receive data from the computer system (step S401 ), wherein It is recorded that the color information and the three-dimensional space information in the floating image are calculated, so that the image processor in the floating image display device can obtain the image data (step S403 ), and convert the floating image data (step S405 ).

Next, in step S407, the floating image is displayed on the floating image display device, and the user is allowed to operate on the floating image display device. In step S409, the gesture sensor in the floating image display device will sense the gesture, such as In step S411, the coordinate change corresponding to the gesture can be obtained, and in step S413, through the operation of the image processor of the floating image display device, the three-dimensional coordinate change formed by the gesture, including the movement, rotation and/or movement of a specific part in a specific direction, can be obtained. Or the amount of change in the three-dimensional coordinate generated by scaling, to determine the interactive command.

Then, in step S415, the interactive instruction is transmitted to the computer system through the communication interface with the computer system, and in step S417, the software program running in the computer system executes a corresponding program according to the interactive instruction, and this program drives the computer system to display A new picture, such as the picture of the next action of the software program, at this time, as in step S419, the computer system converts the action generated by the software program in response to the interactive command (the action corresponding to the change of the three-dimensional coordinate) to form a floating image display. The image data or new image data of the device; or, after the software program receives the interactive command formed by gestures from the floating image display device, and after executing the corresponding program according to the interactive command, the user operates the software program in the computer system to generate a new command , the operation program can transform and form new image data provided to the floating image display device according to the newly generated instruction of the software program.

In step S421, the computer system will obtain new image data corresponding to the interactive command formed by the gesture, that is, new color and three-dimensional coordinate information, and transmit it to the floating image display device. The image display device obtains new floating image data, and then displays the new floating image through the display function therein.

FIG. 5 uses another flowchart to describe the flow of an embodiment of generating an interaction instruction according to a gesture with a gesture sensor to execute an interaction method.

In this example, initially in step S501, the user views the floating image through the floating image display device, and can perform gestures on the image. The gesture on the floating image, at this time, as in step S503, the three-dimensional coordinate changes of key parts (such as fingers, knuckles, palms, etc.) can be obtained. As in step S505, the three-dimensional coordinate changes are sensed by the gesture sensor for a duration of time. If necessary, the coordinate information of the coordinate system formed by the gesture sensor should be converted into the coordinate system formed by the floating image display device, so that in the same coordinate system, the image processor can know that the gesture is formed. The correlation of interactive commands with floating images.

In step S507, the change amount of the three-dimensional coordinates of one or more key parts is determined from the gesture, and the interactive command can be determined accordingly. After the image coordinate information of the floating image is obtained, the image coordinate information of the new floating image is calculated (step S509 ). According to one of the embodiments, in the floating image display device, when the interactive command is determined according to the gesture, the image processor in the display device can query the image data in the memory unit according to the interactive command. The image data refers to the floating image data corresponding to the variation of the three-dimensional coordinate, or the floating image data corresponding to the variation of the three-dimensional coordinate can be obtained by real-time calculation by the image processor according to the interactive command.

In one embodiment, the driver program running on the computer system can send a request to the image server to query the image database according to the information generated by the computer system in response to the interactive command, and then download the corresponding image data to make the floating image The display device obtains the next floating image data of the floating image under various gesture operations from the computer system (step S511 ), so as to display the new floating image (step S513 ).

FIG. 6 then shows a flow chart of an embodiment of the interaction method performed by the floating image display device. This flowchart describes the interaction flow between the gesture sensor 61 , the floating image display device 62 , the computer system 63 and the application server 64 .

According to this process embodiment, the computer device applying the interactive method is connected to the floating image display device 62 . When the user operates an application program in the computer system 63 , a software program is formed, and the software program is loaded from the application server 64 . The requested content, such as a web page content (step S601 ), then displays a computer image in the computer system 63 (step S603 ). During the operation, command data is generated, which includes relevant information for displaying the floating image, and is processed through a specific communication interface. It is transmitted to the floating image display device 62 (step S605 ), so that the image processor in the floating image display device 62 calculates the floating image data according to the color information and the three-dimensional space information describing the floating image (step S607 ), and can display the floating image data accordingly (step S607 ). The panel displays a floating image.

The user can watch the floating image generated by the floating image display device 62. At this time, the floating image can be manipulated by gestures. The floating image display device 62 senses the gesture through the gesture sensor 61 (step S609), and calculates the movement from the moving image. , the amount of change of the three-dimensional coordinates generated by the rotation and scaling actions, so as to generate corresponding interactive commands (step S611 ).

Then, the floating image display device 62 transmits the interactive command to the computer system 63 (step S613 ), so that the software program running in the computer system 63 can generate a corresponding program according to the interactive command, including displaying the interactive computer image (step S615 ). At this time, the process of responding to the interactive command by the computer system 63 may include obtaining further content from the application server 64, and at the same time obtaining new image data corresponding to the change of the three-dimensional coordinate, and the formed command packet is sent to the floating image display device again. 62 (step S617).

In the floating image display device 62 , the image data is obtained according to the received command packet, and the data for displaying the floating image is calculated and formed, and then the floating image formed by the result of this interaction is displayed on the display panel (step S619 ), and continues Wait and sense the next gesture (step S621).

According to an embodiment, the floating image display device can be matched with a computer device in a variety of ways, including a structure that can be built into a computer system after being modularized, and the gesture sensor is installed in a suitable computer according to the sensing technology used. Sensing the position of the gesture.

One of the embodiments may refer to the schematic diagram of the embodiment of setting the floating image display device in the computer device shown in FIG. 7 .

The figure shows a computer device 70 like a notebook computer, and a modular floating image display device is installed on the part of its keyboard structure, that is, the figure shows the floating image display module 72 built in the computer device 70 , the structure and circuit may refer to the above-mentioned embodiments shown in FIG. 1 and FIG. 2 .

The processing circuit in the floating image display module 72 can convert and display the floating image 721 according to the image data provided by the computer system. A gesture sensor 74 is installed at a specific position of the computer device 70. The gesture sensor 74 It is electrically connected to the floating image display module 72, and can judge the user's gesture by means of image detection, light blocking or sound wave reflection, and the obtained gesture reflects the user's fingers, palm position and knuckles changes in the three-dimensional coordinates of , or the interaction of other objects or body parts displayed on the floating image. To be precise, the technology adopted by the gesture sensor 74 realizes a three-dimensional space touch technology, wherein a space sensor is realized, such as a 2D camera, a depth camera, a raster sensor An optical sensor implemented by a device, etc., or a sensor implemented by a radar (Radar), an ultrasonic (Ultrasonic), or the like to implement a hovering touch (Hovering touch). In this way, the gesture sensing technology described above can continuously track the movements of the user's hand (or a specific object, part), including performing tracking calculations, hand joint position analysis, coordinate conversion, etc. The gesture sensor 74 can be based on A pre-defined comparison table of gestures and interactive commands determines the interactive command according to one or any combination of movement, rotation and zooming of a specific part within the display range of the floating image.

FIG. 8 shows a schematic diagram of an embodiment of a floating image display module 82 built in the computer device 80 , which is an embodiment of a floating image display device organized by in-organization modeling. The floating image display module 82 is used in the computer device 80 A floating image 821 is displayed at a certain position on the computer device 80 , and the corresponding computer image 801 can be displayed in comparison with the software program running in the computer device 80 .

FIG. 9 shows a schematic diagram of another embodiment. This figure shows that the computer device 90 obtains content from an external application server. This example is an e-commerce platform. In the figure, a product image 901 is displayed on the computer device 90, and the product name and the name of the product can be marked. Price and other information, and allow users to make other choices (favorites, add to shopping cart). At the same time, the floating image display device 92 connected to the computer device 90 in an external manner simultaneously displays a floating image 921 of a commodity.

At this time, in addition to seeing the floating image 921 of the product through the floating image display device 92, the user can also view the three-dimensional images of the product from various angles through gesture operations. Because of the three-dimensional image of the floating image 921 of the product, the gesture can refer to Fig. The arrows shown in the figure may include rotations in various directions in space, such as horizontal rotation left and right, vertical rotation up and down, and other rotations in other directions. The method for forming a floating image according to gesture calculation is as described in the above embodiment, including the floating image display device 92. The complete image data of the product is available, or the image data needs to be obtained from the computer device 90 or an external image server according to the user's operation.

For example, the floating image display device 92 can display a single or several items of interest to the user, the user can select through a single or multiple combined gestures, and the item can be moved from various directions, including forward and backward and left and right horizontal movements. , move vertically up and down, and freely rotate the product to view the product from various angles, including zooming in on the product to see more details or zooming out to see more other products, or browsing the previous or next three-dimensional product list by gestures , including selecting a three-dimensional product to complete the purchase.

The schematic diagram shown in FIG. 10 shows that the computer system in the computer device 90 executes one or more application programs, and the display screen displays the application program image 902. In this example, the external floating image display device 92 displays the corresponding one or more application programs simultaneously. The software floating image 922 can also display the icons (icons) of multiple applications through a menu, so that the user can gesture to operate the software floating image 922 to select the application content to be executed or viewed, and the relevant interactive instructions will be It is transmitted to the computer system, and the corresponding application image 902 is displayed.

For example, when the user views a plurality of application patterns through the floating image display device 92, he can operate the software floating image 922 in the air, so that the floating image display device 92 can display a single or multiple floating designs at a time, the user can Use single or multiple combined gestures to zoom the floating image you want to see.

FIG. 11 then shows that an audio and video playback program is executed in the computer device 90, and the corresponding screen is an audio and video playback image 903. The floating image displayed by the attached floating image display device 92 is a floating image of a music player that provides the user to operate the audio and video playback function. The image 923 is schematically represented as several buttons for video and audio playback, such as function buttons such as pause, stop, play, forward fast forward, and backward fast forward. The user can use the floating image 923 displayed on the floating image display device 92 to play, pause, stop or fast-forward the audio and video content.

For example, the user can choose to play the music video by operating the floating image 923 of the music playing tool displayed on the floating image display device 92, and the screen of the music video can also be directly displayed, such as the video playing image 903' in the figure, so that the user can play the music video directly. The floating image display device 92 can be used to rotate the video with gestures to view content from different angles, including zooming in or zooming out of the floating image, and performing actions such as floating play, pause, previous track, and next track.

FIG. 12 shows a computer device 120. A floating image display module 122 is built in a certain position on the computer device 120. Through the operation of the computer system, the floating image display module 122 displays a virtual assistant floating image 123. The synchronization can be A computer image 121 is displayed on the computer device 120 , for example, the result of operating the virtual assistant floating image 123 .

For example, the computer system displays a virtual assistant floating image 123 that realizes the virtual assistant through the floating image display module 122. The user can interact with the floating assistant, and can generate different follow-ups by touching different parts of the virtual assistant floating image 123. Image responses, such as greeting when you meet your head, or falling when you touch your feet, etc.

To sum up, the specification discloses the application of a floating image display technology, and particularly proposes a floating image display device, an interactive method executed by connecting the device to a computer system, and a system formed by combination. The floating image generated by the floating image display device is a real image generated by the projection of each beam, not a stereoscopic effect caused by binocular parallax. Therefore, the interactive effect generated by the application of this display device and the interactive program with the computer system are relatively different from conventional ones. Known technology has novel features.

The contents disclosed above are only preferred feasible embodiments of the present invention, and are not intended to limit the scope of the present invention. Therefore, any equivalent technical changes made by using the contents of the description and drawings of the present invention are included in the application of the present invention. within the scope of the patent.

12: Hands 10: Floating Image Display Device 101: Optical Components 102: Display panel 103: Floating Image 105: Gesture Sensor 20: Floating Image Display Device 201: Input interface 203: Image Processor 204: Display driver unit 205: Display unit 206: Memory Unit 22: Gesture Sensor 221: Sensing element 222: Control Unit 223: Interface Unit 202: Computer connection unit 25: Computer System 30: Internet 301: Image server 303: Image Database 35: Computer System 61: Gesture Sensor 62: Floating Image Display Device 63: Computer System 64: Application Server 70: Computer device 72: Floating image display module 721: Floating Image 74: Gesture Sensor 80: Computer device 801: Computer Imaging 82: Floating image display module 821: Floating Image 90: Computer Devices 901: Product Image 92: Floating Image Display Device 921: Commodity Floating Image 902: Application Image 922: Soft Floating Image 903, 903’: Video playback video 923: Music player tool floating image 120: Computer Devices 121: Computer Imaging 122: Floating Image Display Module 123: Virtual Assistant Floating Image Steps S401-S423: the flow of using the floating image display device to execute the interactive method Steps S501-S513: the flow of using the floating image display device to execute the interactive method Steps S601-S621: the flow of using the floating image display device to execute the interactive method

FIG. 1 shows a schematic diagram of an embodiment of a floating image display device;

FIG. 2 shows a circuit embodiment diagram of the floating image display device;

FIG. 3 shows a schematic diagram of an embodiment in which the floating image is connected to an external system through a computer system;

FIG. 4 shows one of the flow charts of an embodiment of performing an interaction method with a floating image display device;

FIG. 5 shows the second flow chart of the second embodiment of implementing the interaction method by the floating image display device;

FIG. 6 shows the third embodiment of the flow chart of the floating image display system for executing the interaction method through the display device;

FIG. 7 shows a schematic diagram of an embodiment of setting a floating image display device in a computer device;

FIG. 8 shows a schematic diagram of an embodiment of a floating image display module built in a computer device;

FIG. 9 shows one of the schematic diagrams of the embodiment of the connection between the computer device and the floating image display device;

FIG. 10 shows the second schematic diagram of the second embodiment of the connection between the computer device and the floating image display device;

FIG. 11 shows a schematic diagram of the third embodiment of the connection between the computer device and the floating image display device; and

FIG. 12 shows a schematic diagram of an embodiment of generating a virtual assistant with a floating image display device.

20: Stereoscopic floating image display device

201: Input interface

203: Image Processor

204: Display driver unit

205: Display unit

206: Memory Unit

22: Gesture Sensor

221: Sensing element

222: Control Unit

223: Interface Unit

202: Computer connection unit

25: Computer device

Claims (16)

An interactive method using a floating image display device, executed in a floating image display device, the floating image display device is connected to a computer system, comprising: receiving an image data from the computer system, and obtaining a floating image data according to the image data , wherein the floating image data is used to describe the color information and three-dimensional space information of a floating image displayed by the floating image display device; after converting the floating image data, an image processor of the floating image display device is used to display the floating image according to the floating image. The description of the data derives the floating image, wherein the floating image display device includes a flat display panel, and the flat display panel displays an integrated image composed of a plurality of unit images, each unit image passing through a lens of the flat display panel Each corresponding lens in the array is imaged, that is, the floating image is displayed on the floating image display device through the lens array; a gesture is sensed with a gesture sensor, and a three-dimensional coordinate change of the gesture is obtained; according to the judging an interactive command by the change of the three-dimensional coordinate; sending the interactive command to the computer system, and the computer system responding to the interactive command to obtain a new image data corresponding to the change of the three-dimensional coordinate; receiving the new image data from the computer system Image data, obtaining a new floating image data according to the new image data; and after converting the new floating image data, displaying a new floating image through the lens array on the flat display panel. The interactive method using a floating image display device as claimed in claim 1, wherein a software program is run in the computer system, and the software program executes a corresponding program according to the interactive instruction, so that the computer system generates the generated image according to the variation of the three-dimensional coordinate The new image data. The interactive method using a floating image display device according to claim 2, wherein, The computer system converts the image data or the new image data provided to the floating image display device according to the instructions generated by the software program. The interactive method using a floating image display device as claimed in claim 3, wherein after the software program executes the corresponding program, a user operates the software program to generate a new command, so that the computer system updates a new command according to the software program The generated commands are transformed to form the new image data provided to the floating image display device. The interactive method using a floating image display device as claimed in claim 4, wherein the gesture reflects changes in the three-dimensional coordinates of the fingers of a hand, the position of the palm and the knuckles, and the interactive instruction indicates one of movement, rotation and scaling. or any combination, so that the software program executes the corresponding program. The interactive method using a floating image display device according to any one of claims 1 to 5, wherein a driver is run in the computer system, and the driver establishes an interface between the computer system and the floating image display device, using to interpret and compile the information exchanged between the computer system and the floating image display device. The interactive method using a floating image display device according to claim 6, wherein the driver sends a request to an image server according to the information generated by the computer system in response to the interactive command, and downloads the information provided to the floating image display device. the image data or the new image data. A floating image display device, comprising: an image processor for processing image data forming a floating image; an input interface, electrically connected to the image processor and connected to a gesture sensor, for receiving sensing by the gesture sensor The information generated by a gesture is used by the image processor to calculate a three-dimensional coordinate change to form an interactive command; a display driving unit is electrically connected to the image processor to obtain the floating image according to the description of a floating image data, displaying the floating image through a display unit; a computer connection unit, electrically connected to the image processor, and connected to a computer device through a communication interface, the computer device runs a computer system, wherein the floating image data is used to describe the color of the floating image displayed by the floating image display device information and three-dimensional space information, the display unit includes a flat display panel, the flat display panel displays an integrated image composed of a plurality of unit images, and each unit image passes through a corresponding lens array of the flat display panel. Lens imaging, that is, displaying the floating image on the floating image display device through the lens array; wherein the floating image display device receives image data from the computer system, obtains the floating image data according to the image data, and is converted by the display driving unit After the floating image data, the floating image is displayed by the display unit, and then the interaction method of using the floating image display device includes: sensing the gesture with the gesture sensor, and obtaining the three-dimensional coordinate change of the gesture; Determine the interactive command according to the three-dimensional coordinate change; transmit the interactive command to the computer system through the computer connection unit, and the computer system responds to the interactive command to obtain a new image data corresponding to the three-dimensional coordinate change; automatically The computer system receives the new image data, obtains a new floating image data according to the new image data; and displays a new floating image through the display unit after converting the new floating image data. The floating image display device as claimed in claim 8, wherein the floating image display device is built into the structure of the computer system after being modularized, and the gesture sensor is arranged at a position where the gesture can be sensed. The floating image display device according to claim 9, wherein the gesture sensor judges the gesture sensor by an image detection means, a light blocking means or a sound wave reflection means Gestures, the gestures reflect changes in the three-dimensional coordinates of the fingers, palm positions and knuckles of a hand, and the interactive instructions indicate one or any combination of movement, rotation and scaling. The floating image display device according to claim 8, wherein the image processor inquires the image data in a memory unit according to the interactive instruction to obtain the floating image data corresponding to the change of the three-dimensional coordinate; or the image processor According to the interactive command, the floating image data corresponding to the change of the three-dimensional coordinate is calculated in real time. A floating image display system, comprising: a floating image display device provided with a gesture sensor, the floating image display device comprising: an image processor for processing image data forming a floating image; an input interface electrically connected to the floating image An image processor, connected to the gesture sensor, receives information generated by the gesture sensor sensing a gesture, and calculates a three-dimensional coordinate change through the image processor to form an interactive command; a display driving unit, which is electrically connected to the image processor, obtains the floating image according to the description of a floating image data, and then displays the floating image through a display unit, wherein the floating image data is used to describe the color of the floating image displayed by the floating image display device information and three-dimensional space information, the display unit includes a flat display panel, the flat display panel displays an integrated image composed of a plurality of unit images, and each unit image passes through a corresponding lens array of the flat display panel. Lens imaging, that is, displaying the floating image on the floating image display device through the lens array; a computer connection unit, electrically connected to the image processor, and connected to a computer device through a communication interface, the computer device runs a computer system, includes a driver running within the computer system that creates the An interface for communication between the computer system and the floating image display device, used for interpreting and compiling the information exchanged between the computer system and the floating image display device; wherein the floating image display device receives image data from the computer system, and obtains according to the image data After the floating image data is converted by the display driving unit, the floating image is displayed by the display unit, and then the interaction method of using the floating image display device includes: sensing the gesture with the gesture sensor, and obtain the three-dimensional coordinate change of the gesture; determine the interactive command according to the three-dimensional coordinate change; transmit the interactive command to the computer system through the computer connection unit, respond to the interactive command, and obtain a comparison of the three-dimensional coordinate change a new image data; receive the new image data from the computer system, obtain a new floating image data according to the new image data; and display a new floating image data through the display unit after converting the new floating image data New floating image. The floating image display system of claim 12, wherein a software program runs in the computer system, and the software program executes a corresponding program according to the interactive instruction, so that the driver running in the computer system changes according to the three-dimensional coordinates amount to generate the new image data. The floating image display system of claim 13, wherein after the software program receives the interactive command formed by the gesture from the floating image display device, executes a corresponding program according to the interactive command, and then the computer system executes a corresponding program according to the software The commands newly generated by the program are converted to form the new image data provided to the floating image display device. The floating image display system of claim 14, wherein the gesture reflects a The position of the fingers, the palm and the three-dimensional coordinates of the knuckles of the hand are changed, and the interactive instruction indicates one or any combination of movement, rotation and scaling, so that the software program executes the corresponding program. The floating image display system according to any one of claims 12 to 15, wherein the driver sends a request to an image server according to the information generated by the computer system in response to the interactive command, and downloads and provides the floating image display The image data or the new image data of the device.

Images