WO1996029677A1

WO1996029677A1 - An interactive display and input device

Info

Publication number: WO1996029677A1
Application number: PCT/GB1996/000523
Authority: WO
Inventors: Clive Van Heerden
Original assignee: Central Research Laboratories
Priority date: 1995-03-21
Filing date: 1996-03-07
Publication date: 1996-09-26
Also published as: GB9505664D0

Abstract

An interactive display apparatus includes an input device, the device comprising a source of electromagnetic radiation, and an array of detectors (facing the image plane and lying in a different plane) arranged to receive electromagnetic radiation from the source reflected by an object, apparatus producing a real image suspended in space in the image plane, and control means arranged such that a user may interact with the image by placing an object in or adjacent the image plane. This arrangement enables a user to interact with an apparently three-dimensional suspended image in a way similar to using a touch screen but without having a physical screen to touch.

Description

AN INTERACTIVE DISPLAY AND INPUT DEVICE

This invention relates to an input device for a display having at least one image plane, the device comprising a source of electromagnetic radiation and an array of detectors being arranged to receive electromagnetic radiation from the source being reflected by an object being placed in or adjacent the image plane. The invention also relates to an interactive display apparatus.

Input devices for visual displays in the form of 'touch screens' are an increasingly popular way of interacting with computer systems. This is because of the inherent robustness of the devices and the elimination of electromechanical devices such as keyboards, mice and joy-sticks. One known system is disclosed in GB-A-2 268 263. In this device a light beam from a source is scanned across a plane by a shutter, and a photodetector in the same plane detects light reflected by a stylus which light returns back through the shutter. Such a system may be placed on the front of a cathode ray tube to allow interactive control of, for example, a computer system.

However, all touch screen systems have the disadvantage of being associated with a screen which has to be touched. Such screens can be damaged and tend to get dirty which can lead to spurious data input. Also, such displays are in a single plane, and cannot give 3 dimensional effects because of the presence of flatness cues which are recognized by the brain of a user. The known system disclosed in the above mentioned patent also uses expensive components which have to be carefully aligned. Furthermore, if two objects are present at the same time one can produce a shadow so that the other cannot be reliably detected.

It is an object of the present invention to enable the above disadvantages to be mitigated.

According to a first aspect of the invention, there is provided an input device as defined in the first paragraph above characterized in that the array of detectors faces, and does not lie in, the image plane. This enables the device to detect a plurality of objects simultaneously and substantially simplifies construction. Preferably, a plurality of substantially parallel planes are illuminated by at least one source. This gives the advantage of enabling the speed of an object passing through the planes to be measured. Very preferably the electromagnetic radiation being reflected by an object in a plane is characteristic of that plane.

According to a second aspect of the invention, there is provided an interactive visual display apparatus comprising the input device defined above and apparatus for providing a real image suspended in space in the image plane. This enables the apparatus to generate three dimensional effects and enables the touch screen to be dispensed with to provides a 'touch-space' which is a more intuitive interface.

In order that the present invention may be more readily understood, reference will now be made, by way of example, to the accompanying drawings, in which: Figure 1 is a perspective view of an apparatus for providing a real image suspended in space,

Figures 2 and 3 are cross-sectional side views of the apparatus of Figure 1,

Figures 4 and 5 both show an input device and an interactive display apparatus according to the invention Before describing the invention in detail, an apparatus for forming a real image suspended in space for use with the invention will be described. Referring to Figures 1 and 2, a beamsplitter 4 is mounted for example in an aperture in an opaque surface 2.

In this embodiment, retroreflector means comprising an array of retroreflectors, such as a sheet 6 of comer-cubes, is positioned in a path for light which has been reflected by the beamsplitter 4. In this example it is positioned at an acute angle thereto, which is about 45°, so as to form the second side of a triangle therewith. An object 8 such as a picture displayed on a VDU screen is positioned along the third side of the triangle, at approximately 90° to the beamsplitter. For example, the screen 8 may be positioned at an angle in the range from 70° to 110° to the beamsplitter 4. In one specific arrangement, means for screening the retroreflector 6 from the object 8 is provided, such as a louvre film 12 in front of the object 8. As indicated by arrows in figure 2, the beamsplitter 4 reflects light from the object 8 towards the retroreflector 6. In addition some of the light is transmitted by the beamsplitter 4 and is "lost". The beamsplitter 4 also transmits light reflected from the retroreflector 6 to form the suspended image 14 opposing the object 8 with respect to the beamsplitter 4.

In addition, some of the light is reflected by the beamsplitter back towards the original image 8.

In the apparatus shown in Figure 3, a further array of retroreflectors 16 is positioned in a path for light which has been transmitted by the beamsplitter. In this example, the further array opposes the retroreflector 6 with respect to the beamsplitter

4; that is, it forms a triangular arrangement with the beamsplitter 4 and the suspended image 14. The further array of retroreflectors 16 directs light which is transmitted by the beamsplitter 4 directly from the original image 8 back towards the beamsplitter 4 to prevent 'loss' of this light. In an alternative arrangement, the array of retroreflectors 6 in a path for light which is reflected by the beamsplitter 4 could be omitted, leaving only the array in the path of transmitted light.

In the example shown in Figure 3, the object 8 is a projection screen, and an image is formed thereon by a projector 18. The projector 18 is shown positioned in the gap 10 between the retroreflector array 6 and the object 8, but may have other positions. For example it may be positioned in an aperture in the retroreflector array 6. As indicated by plus and minus signs adjacent arrows in the Figure, the beamsplitter may be polarization selective, transmitting light having one type of circular polarization and reflecting light having the other type. For example, it may comprise layers of cholesteric liquid crystal polymers reflecting different spectral regions of circularly polarized light of one sense so as to reflect white light (i.e. the whole visible spectrum) of this sense.

Additionally, the object may be arranged to emit circularly polarized light, in which case one of the arrays 6, 16 may be omitted, since all of the light from the object could be of the type which is either reflected or is transmitted by the selective beamsplitter. This could be achieved by using a transmissive or reflective projection screen which either preserves or reverses polarization, and a projector which produces linearly polarized light, such as an LCD projector, combined with a quarter wave plate. A transmissive such screen is disclosed in GB 1509180.

An apparatus for producing a suspended image 14 in an image plane has been described above. Figures 4 and 5 show such an image together with an input device according to the present invention.

In Figure 4, the image 14 is produced using apparatus as shown in Figure 2. In addition, an array of detectors, in the present example a CCD camera 20, which is sensitive to infra-red light, is constructed and arranged to face the image plane. A source 23 of infra red light 24 is provided (recessed into the surface 2) for illuminating the plane which contains the suspended image 14 with infra red light. The CCD camera is connected to a computer 30, which may optionally also be connected to the apparatus producing the image 14. The camera points in a direction substantially perpendicular to the plane of the image.

In use, if the observer 21 just looks at the image, the CCD camera does not receive any infra red light scattered from the source 23. Only general background radiation from the observer and the surroundings will be received. If, however, a user interacts with the image by placing an object (for example a stylus or a finger 22) into the plane of the image, the sheet of infra red light will be interrupted, and some light 25 will be scattered in the direction of the CCD camera. The CCD camera records the position from which the light is scattered. This information is then inputted to the computer 30. If this computer is connected to the image producing apparatus this information may be used to change the information being displayed in response to the presence of the object. In order to enhance contrast between the infra red light being scattered from the object 22 and other ambient radiation, a filter 31 which selectively transmits wavelength emitted by the source 23 may be placed in front of the CCD camera.

Other techniques to enhance contrast may be employed alternatively or in addition to the above mentioned filter. For example, the light source and camera can be operated intermittently and in synchrony, in this way a higher power infra-red pulse can be employed without the background radiation reducing the signal to noise ratio between pulses. In another alternative a snap-shot of the radiation being scattered from a time modulated or non-modulated source can be captured using 'frame- grabbing' software. This snap-shot can then be compared with another snap-shot (for example taken when the rnodulated source is not radiating). This will make infra red light scattered by the object more distinguishable from ambient light. Software can be employed to look for large changes between successive frames, simplifying the amount of data analysis required. A lens with a large aperture and hence a restricted field of focus may be used for the CCD camera so that only objects in the vicinity of the image plane are focussed sharply.

Figure 5 shows a further embodiment of the invention. A suspended image 14 is produced from an image source 29 such as a cathode ray tube using a beam splitter 26 and a sheet of comer cube retroreflector 27. In this example two sources of infra red light 23 and 28 are provided, such that one source 28 produces a sheet of light parallel to the plane of the image 14 but spaced from the plane a few centimeters towards the position of an observer, and the other source 23 produces a sheet of light in the plane of the image. Using this embodiment, the user can initiate one form of data input by interrupting the light radiated from source 28, and another form of data input by interrupting the light radiated from source 23. In order to enable the CCD camera 20 to differentiate the first form of data from the second form of data, the sources may be constructed to provide electromagnetic radiation of different respective frequencies. Alternatively or in addition other methods, for example time sequential switching of the sources, may be employed to aid in differentiation of the two forms of data.

The apparatus shown in Figure 5 may alternatively or in addition be used to measure the speed of an object passing through the plane of the image by measuring the time between detection of one form of data and the other. If the object does not move directly towards the array of photodetectors (the CCD camera in the present example), the position of the two forms of data will be shifted laterally, so that it will be possible to compute the direction of travel as well as the speed of the object.

The CCD camera can record the size of the object placed in the image plane by measuring the size of the image produced in the camera. In addition the image produced in the camera can be processed using computer software to remove any distortions produced by having the detector array or CCD camera not normal to the image plane (e.g. keystone distortion).

In order to provide an interesting and efficient system of user interaction, it is desirable to include sensory feedback from the system to the user in response to objects being detected. This may be achieved aurally via sounds, visually by changes in the image or ambient lighting, tangibly by means of air currents, or by olfaction of released scents, or through a combination of any of these methods.

Although in the above examples, the image plane is illuminated from below, the illumination may be provided from the top or sides as an alternative or in addition.

The invention provides an interesting way of interacting with suspended images (including for example, holograms, etc.) in two or in three dimensions and as a function of time. It enables the construction of "touchless screens" for diverse applications including dispensing booths, video games, and novel musical instrument interfaces. The apparatus can measure many variables so that it can be employed as a natural, intuitive "soft" interface allowing lots of expression. As well as being used to signal the presence, and detect the position and speed of an object moveable relative to the display and/or suspended image, a detector array such as a CCD camera can identify the shape and orientation of objects such as fingers or hands, thereby increasing the flexibility of methods of interacting with the display and any associated computer.

The term 'suspended image' in the present description denotes a real image formed in real space which does not require the presence of a reflective or transmissive screen for its observation. The priority document of the present application, namely GB 9505664.4 is hereby incorporated herein by reference.

In summary, an interactive display apparatus includes an input device, the device comprising a source of electromagnetic radiation, and an array of detectors (facing the image plane and lying in a different plane) arranged to receive electromagnetic radiation from the source reflected by an object, apparatus producing a real image suspended in space in the image plane, and control means arranged such that a user may interact with the image by placing an object in or adjacent the image plane. This arrangement enables a user to interact with an apparently three dimensional suspended image in a way similar to using a touch screen but without having a physical screen to touch.

Claims

1. An input device for a display having at least one image plane, the device comprising a source of electromagnetic radiation, and an array of detectors being arranged to receive electromagnetic radiation from the source being reflected by an object being placed in or adjacent the image plane, characterized in that the array of detectors faces, and does not lie in, the image plane.

2. An input device as claimed in claim 1 in which the electromagnetic radiation is infra-red radiation and the detectors are responsive to infra-red radiation.

3. An input device as claimed in any previous claim in which the array of detectors is arranged to face in a direction approximately perpendicular to the image plane.

4. An input device as claimed in any preceding claim in which the source of electromagnetic radiation illuminates the image plane or an adjacent plane.

5. An input device as claimed in claim 4 in which a plurality of substantially parallel planes are illuminated by at least one source.

6. An input device as claimed in claim 5 in which the electromagnetic radiation being reflected by an object in a plane is characteristic of that plane.

7. An input device as claimed in claim 6 further comprising means for determining from which plane electromagnetic radiation is being reflected by a given object, and inputting this information to a further device.

8. An input device as claimed in any preceding claim in which the array of infra¬ red detectors comprises a CCD camera.

9. An interactive display apparatus comprising an input device as claimed in any preceding claim and apparatus for providing a real image suspended in space in the at least one image plane.

10. An interactive display apparatus as claimed in claim 10, further comprising control means being arranged such that a user is capable of interacting with the said real image by placing an object in or adjacent said at least one image plane.

SUBS. TTTUTE SHEET (RULE 26)