HK1227205A1 - High luminance projection displays and associated methods - Google Patents

Description

Highlight projection display and related method

The application is a divisional application of PCT international application entering the China national phase with the international application date of 2012, 4 and 11, and the national application number of 201280019001.X, and the invention name of 'highlight projection display and related method'.

Cross reference to related applications

Priority of U.S. provisional patent application No.61/476,949, filed 2011, 4/19, is claimed, the entire contents of which are incorporated herein by reference.

Technical Field

The present invention relates to projection displays. Example embodiments provide a digital cinema display. Other embodiments provide displays such as televisions, computer displays, and special purpose displays such as advertising displays, virtual reality displays, gaming displays, and medical imaging displays.

Background

There is a trend of increasing interest in providing displays capable of reproducing realistically viewed images. One aspect of achieving realistic images is to provide high peak brightness and high dynamic range. A typical natural scene includes very bright areas, such as the sun in the air and highlights of brightly illuminated objects, and dim areas, such as objects in shadows. It is not possible to achieve realistic images of a general scene on displays that cannot have high peak brightness.

Current projection technology does not scale efficiently to high light. For example, in many common projector designs, a light source, such as a xenon lamp, illuminates one or more spatial light modulators. The spatial light modulator directs some light to the screen while attracting or redirecting other light. Achieving high light requires scaling up the power of the light source. The increased power consumption of the light source becomes an obstacle to increasing the brightness of the light source to a level sufficient to provide peak brightness at typical levels of natural scenes. Also, high light sources can cause problems of overheating spatial light modulators and other components in the projector, among other problems.

For example, current digital cinema projectors may have a power draw of 8 kilowatts to illuminate generating 48 nits (48 cd/m)²) A large screen light source with a peak luminance. To achieve a peak brightness of 12,000 nits (a brightness typically encountered in daily life), the power of the light source would need to be scaled up to over 2 megawatts. It is obvious that this is impractical in most cases.

Yet another problem preventing a significant increase in peak brightness for many conventional projection displays is that the contrast does not increase with increasing peak brightness. In many such displays, increasing the intensity of the light source to achieve increased peak brightness also causes black levels. Thus, attempting to increase the peak brightness above the threshold will result in an unacceptably high black level.

Yet another obstacle to providing a display with sufficient highlight to present a realistic image is that the response of the human visual system to light is approximately logarithmic. In contrast, power needs scale approximately linearly with brightness. Doubling the brightness of the image requires doubling the power, assuming the same efficiency of the light source. However, doubling the brightness does not result in an image perceived by the viewer as being twice as bright. Doubling the apparent brightness requires approximately the square of the brightness.

The above examples of related art and the limitations related thereto are intended to be illustrative and not exclusive. Other limitations of the related art will become apparent to those of skill in the art upon a reading of the specification and a study of the drawings.

Disclosure of Invention

The present invention has a number of aspects. Embodiments of the invention provide a projection display, a method for operating a projection display, a dual modulation display, a medium containing computer readable instructions which, when executed by a data processor, cause the data processor to perform a method according to the invention, a method for displaying an image, and a method for processing image data for display, etc.

An exemplary aspect of the present invention provides a display system, including: a main projector arranged to project an image defined by the base image data onto the screen and a highlight image defined by the highlight image data onto the screen in alignment with the base image. The image processor is configured to process the image data to generate highlight image data.

In some embodiments, the highlight projector comprises a scanning beam projector. The scanning beam projector may provide, for example, a plurality of laser beams of primary colors (e.g., red, green, and blue beams). The beams may be scanned together or independently to provide the highlight areas with the desired apparent brightness and color. In other embodiments, the scanned beam projector provides a scannable beam of white light.

In some embodiments, the highlight projector comprises a 2D holographic projector.

Another aspect provides a highlight projector system comprising an image processor configured to process image data to output a highlight image; and a light projector operable to project light according to the highlight image in alignment with the base image.

Another example aspect provides a display comprising a source of spatially modulated light arranged to illuminate a spatial light modulator, wherein the source of spatially modulated light comprises a 2D holographic light source.

Another example aspect provides a method for displaying an image defined by image data. The method includes concentrating light from a light source to output light that has been spatially modulated in a manner based on image data; illuminating the spatial light modulator with spatially modulated light; and controlling the spatial light modulator to display an image according to the image data. For example, concentrating the light may include generating a computer-generated 2D hologram. In some embodiments, the light comprises coherent light, and concentrating the light comprises adjusting a phase of the light in a fourier plane of the optical system.

Another example aspect provides a method for displaying an image from image data. The method includes processing image data to generate a base image and a highlight image comprising highlight pixels; operating the main projector to display the base image; and operating the highlight projector to display the highlight image superimposed with the base image.

In addition to the exemplary aspects and embodiments described above, further aspects and embodiments will become apparent by reference to the drawings and by study of the following detailed description.

Drawings

The drawings show non-limiting exemplary embodiments of the invention.

Fig. 1 is a schematic diagram of a display system according to an example embodiment.

Fig. 1A is a schematic diagram of a display system according to another example embodiment.

Fig. 2A and 2B are exemplary histograms showing the number of pixels in an image according to the luminance of those pixels for a bright image and a dark image, respectively.

Fig. 3 is a schematic diagram of an example apparatus that combines a highlight projector with a main projector.

FIG. 4 is a block diagram illustrating image data processing components of a display system according to an example embodiment.

Fig. 5 is a schematic diagram illustrating a highlight projector including a light redirecting projector according to an example embodiment. A highlight projector of the type shown in fig. 5 may be configured to project light onto, for example, a screen, a component of a main projector, or a spatial light modulator.

Fig. 6A is a schematic diagram showing an example holographic projector configured to project a desired highlight image onto a spatial light modulator controlled to either redirect, divert, or absorb light in an image area outside of the highlight region.

Fig. 6B is a schematic diagram illustrating a holographic projector configured to project a highlight image directly onto a spatial light modulator of a main projector according to an example embodiment.

Fig. 7 is a schematic diagram illustrating a highlight projector comprising a light source arranged to illuminate a 2D spatial light modulator with a spatial filter in the fourier plane for eliminating light leakage according to an example embodiment.

Fig. 8 is a schematic diagram illustrating a highlight projector including a light source illuminating a spatial light modulator according to an example embodiment.

FIG. 9 shows a display according to another embodiment. The display having the general structure schematically shown in fig. 9 may be used as a stand-alone display (e.g., as a television, computer monitor, special purpose display, etc.) or as part of a display system including a highlight projector.

Detailed Description

Throughout the following description, details are set forth in order to provide a more thorough understanding of the present invention to those skilled in the art. However, well known elements may not have been shown or described in detail to avoid unnecessarily obscuring the disclosure. The specification and drawings are, accordingly, to be regarded in an illustrative rather than a restrictive sense.

Some embodiments of the present invention provide a projection display including a main projector and a highlight projector. The main projector may have a relatively low peak brightness and may be used to project a complete image. The brightness of highlights in the image projected by the main projector is lower than desired. The highlight projector may project concentrated light to increase the light at the position of the highlight, thereby remarkably increasing the brightness of the highlight.

Fig. 1 shows a projection system 10 according to a first exemplary embodiment. Projection system 10 includes a main projector 12 having a lens 14 that projects an image 16 onto a screen 18. The screen 18 may be a front projection screen or a rear projection screen. The system 10 also includes a separate highlight projector 20 having a lens 22 that projects the image 16A onto the screen 18. Images 16 and 16A are superimposed so that the viewer sees the image resulting from the combination of images 16 and 16A.

The main projector 12 may include any suitable image projector. For example, the main projector 12 may include a DLP-based projector, a projector that uses one or more Liquid Crystal On Silicon (LCOS) spatial light modulators, a projector that includes a transmissive Liquid Crystal Display (LCD) panel to modulate light, a Cathode Ray Tube (CRT) projector, and so forth.

Highlight projector 20 is of a type that can deliver concentrated light to at least some areas within the area of image 16, preferably without significantly increasing the brightness level of other areas within image 16. For example, the highlight projector 20 may include one or more scanning beams that may be directed to add further illumination to only selected highlight regions of the image 16.

Highlight projector 20 and main projector 12 are co-aligned so that highlight projector 20 can deliver additional light precisely to small highlight areas within image 16 projected by main projector 12. In some embodiments, highlight projector 20 has a spatial resolution equal to or greater than main projector 12. In other embodiments, projector 20 may have a spatial resolution less than that of main projector 12. In other embodiments, projector 20 may have a spatial resolution less than that of main projector 12.

In some embodiments, highlight projector 20 and image processor are provided to be used as an add-on to an existing main projector, such as a commercially available digital cinema projector. The image processor may be configured to receive image data for projection and generate a highlight image for display by the highlight projector. In some embodiments, the image processor may modify the image data to provide a base image for display by an existing main projector. The highlight projector may be calibrated at installation to produce a highlight image that is aligned with the image produced by the existing main projector.

Advantageously, in a typical scene, only a relatively very small proportion of the pixels in the image need to be displayed with a brightness greater than the peak brightness of the standard projector 12 for enhanced realism. It has been found that enhanced realism can be achieved by providing a bright highlight very selectively. Fig. 2A and 2B are histograms showing the number of pixels in an image according to the luminance of those pixels of a bright image and a dark image prepared by a human colorist respectively for viewing on a display having high peak luminance. In each case, the colorist adjusts the image to obtain an appearance that the colorist deems optimal.

Somewhat surprisingly, the average brightness of all pixels in a bright image is still relatively low. Even in bright images with the histogram of fig. 3A, it can be seen that only a relatively very small proportion of the pixels have high luminance (e.g., luminance in excess of about 1,000 or 2,000 nits). A few very bright pixels at high and very high brightness may produce an image with a much more realistic appearance without significantly affecting the light adaptation of the observer's eye. This is different from a real scene in nature where many or all pixels may have very high brightness. For example, a real scene on glaciers on a sunny day may be so bright that viewing for an extended period of time without dark sunglasses is uncomfortable or even harmful. The colorist may prepare such scenes in a way that produces a relatively low average brightness while providing high brightness in several key areas to provide a more realistic viewing experience.

Some embodiments exploit the fact that even very bright scenes can be reproduced with a surprisingly low average brightness while maintaining a realistic viewing print if small highlight areas are presented with a peak brightness much higher than the average brightness used to present the image to the viewer. Some embodiments use a highlight projector that is much less powerful than raising all pixels of image 16 to the brightest highlight level, but is capable of magnifying the illumination of the highlight to the desired level. In such embodiments, the light from the highlight projector is concentrated into the highlights to provide the desired brightness in the highlights.

There are various ways to arrange the main projector and the highlight projector in a combined manner. For example, a system may be provided that provides a projector and a highlight projector for selectively amplifying the brightness of highlight areas, arranged with any combination of the following features:

the main projector and highlight projector may use the same general technology or different technologies.

The main projector and highlight projector may be provided in separate units or in a combined unit (e.g., integrated form factor). In providing the main projector and the highlight projector as a combined unit, the main projector and the highlight projector may share certain optical components and/or certain optical paths. For example, the main projector and the highlight projector may share one or more of a projection lens, relay optics, one or more spatial light modulators, and the like. Various examples of shared components and optical paths are set forth below.

The system may comprise one or more main projectors that collectively project the base image. For example, system 10 may include multiple primary projectors 12 that collectively illuminate screen 18 to provide image 16.

The system may include a highlight projector that may collectively project highlight images to magnify the brightness of highlight regions. For example, the highlight projector 20 may include multiple units that may be controlled to collectively direct light onto the highlight region of the image 16.

Highlight projectors can be monochromatic (e.g. white light can be projected) or polychromatic.

Highlight projectors may optionally include filtering (e.g., a spatial filter in the example fourier plane) to suppress illumination outside of the highlight region.

The highlight projector may optionally comprise one or more spatial light modulators. The spatial light modulator may be controlled to perform one or more of the following: the method includes directing light to illuminate a highlight region, correcting errors in a projected highlight image, suppressing illumination outside the highlight region, adjusting the highlight image to blend smoothly into a base image projected by a main projector, and redirecting light from outside the highlight region into the highlight region. In embodiments where the highlight projector includes one or more spatial light modulators, the spatial light modulator may include a spatial light modulator shared by the main projector and/or may include a spatial light modulator dedicated to the highlight projector.

The main projector and highlight projector may be arranged for front projection or rear projection. Highlight projector 20 and main projector 12 are not forced to illuminate screen 18 from the same side. In embodiments where screen 18 is translucent (e.g., where screen 18 comprises a rear projection type screen), highlight projector 20 and main projector 12 may illuminate screen 18 from opposite sides.

These various methods and their arrangements and combinations are not limiting but are intended to provide examples of some embodiments within the scope of the invention.

Advantageously, the combined image as viewed by the viewer includes some highlights where the peak brightness significantly exceeds the peak brightness of the main projector 12. For example, the main projector may have a peak luminance of 500 nits or less, and the highlight region may have a peak luminance of 2000 nits or more. For example, some main projectors intended for use in dark viewing environments (e.g., movie theaters) may provide peak brightness of about 15 to 50 nits. Some such projectors are designed to image onto a large area of the screen. For example, some main projectors intended for use in bright viewing environments may provide peak brightness of about 100 to 300 nits.

Since the highlight region illuminated by the highlight projector 20 may only include a very small portion (e.g., less than 10%, 5%, 1%, or even less than 0.1%) of the area of the image 16, the highlight projector 20 may be able to achieve the desired highlight in the highlight region without requiring an impractical power input.

Fig. 1A shows a projector system according to an exemplary embodiment in which a highlight projector comprises a point light source 20A producing a narrow light beam 21 and a deflector 23 comprising scanning mirrors 23A and 23B. Mirrors 23A and 23B are rotatably mounted and operated by an actuator (not shown) so that the beam 21 can be directed to form a small spot 25 at any desired position in the image 16. The intensity of the light beam 21 and the position of the display point 25 can be controlled by the controller 24 to achieve increased brightness in selected highlight regions. In some embodiments, the brightness of the highlight region is controlled at least in part by varying the amount of time the control point 25 stays on the highlight region. In some embodiments, the brightness of the highlight region is controlled at least in part by controlling the intensity and/or duty cycle of the light beam 21 when the light beam 21 is illuminating the highlight region.

The light beam 21 may for example comprise a laser beam. In some embodiments the highlight projector comprises three different colored laser beams that can be combined to constitute a white highlight. For example, a highlight projector may include red, green, and blue laser beams. In such embodiments, the light beam may be manipulated by a single deflector assembly (e.g., a single set of mirrors 23A, 23B) to illuminate the high light region. In an alternative embodiment, a separate deflection assembly is provided for each of the plurality of beams 21.

Since highlights generally only occur in a small proportion of the total area of the image 16, the laser can increase the perceived brightness of the highlight areas by staying longer in those areas. The laser need not illuminate any portion of image 16 outside the high light region.

In embodiments where the highlight projector includes a steerable light beam, a controller (e.g., controller 24) that causes the light beam to steer may be configured to control mirrors 23A and 23B (or an alternative light beam steering mechanism such as a mechanism that utilizes a digital light deflector, a grating light valve, etc.) to cause the spot 25 to follow a trajectory that depends on the position of the highlight area to be illuminated. The beam steering mechanism need not scan in a raster or other pattern that covers all of the pixels of image 16. By manipulating the dot 25 in a trajectory that brings the dot 25 to the highlight region while avoiding at least some pixels outside the highlight region, the controller 25 may cause the dot 25 to dwell on the highlight region for a period of time sufficient to achieve a desired brightness for the highlight region.

The main projector 12 and highlight projector 20 may optionally be combined with each other so that the two projectors share some common optical path. For example, the optical systems of the highlight projector 20 and the main projector 12 may be arranged to share a common projection lens 14. One such example is shown in fig. 3. Fig. 3 is schematic in nature. Optical components that may be present in the optical path, such as relay lenses, mirrors, filters, etc., have been omitted for clarity.

In the embodiment shown in fig. 3, main projector 12 includes a light source 30 that can emit light 31 to illuminate a spatial light modulator 32. The light source 30 may comprise a uniform light source or a light source that may be spatially modulated according to the image data (e.g., the base image). The light modulated by spatial light modulator 32 is directed by projection lens 14 onto screen 18 (not shown in fig. 3) to provide image 16 (not shown in fig. 3).

In this embodiment, the highlight projector includes a high intensity narrow beam light source 34 that can be controlled to emit a narrow beam 35 that is manipulated by an X-Y deflector 36 and a light combiner 37 to produce a brightly illuminated spot 38 on the spatial light modulator 32. By controlling the intensity of the light source 34 and/or turning the light source 34 on or off while scanning with the X-Y scanner 36, a plurality of different highlight areas can be illuminated on the spatial light modulator 32 with light from the light source 34. This additional light, as modulated by spatial light modulator 32, is also imaged by lens 14 to increase the brightness of the highlight region within image 16.

In an alternative embodiment, a light combiner 37 is located between spatial light modulator 32 and projection lens 14 such that point 38 is projected directly onto screen 18. In this alternative embodiment, the optical paths of the main projector and the highlight projector may only have the projection lens 14 in common.

FIG. 4 is a block diagram illustrating image data processing components of a display system according to an example embodiment. The image processing system 40 receives image data 42 and processes the image data 42 to identify highlight regions. The processing may comprise, for example, comparing pixel luminance values with a first threshold value and identifying those pixels having luminance values exceeding the first threshold value as belonging to a highlight region. In some embodiments, the highlight region may be limited to a region including a predetermined region of connected pixels having luminance values exceeding a first threshold. As another example, the process may identify the highlight region as being made up of M highest luminance pixels (where M is a number) or those pixels that are at or above the nth percentile for luminance (where N is a percentile such as the 90 th percentile or the 95 th percentile or the 98 th percentile or the 99 th percentile or the 99.9 th percentile). Processing may include applying a plurality of such criteria (e.g., highlight regions may be identified as up to M pixels with brightness exceeding a threshold).

In some embodiments, the processing comprises: a trade-off is made between the region included in the highlight region and the peak luminance. Such processing may include histogram analysis. For example, for images in which processing identifies a relatively large number of pixels as belonging to highlight regions according to a first criterion, the processing may select between retaining the highlight regions at the expense of reduced peak brightness achievable in the highlight regions according to the first criterion or applying a second criterion to reduce the number of pixels included in the highlight regions. Such processing may include histogram analysis.

In some embodiments, the processing is performed with reference to the adaptation point. The adaptation point may for example comprise or be determined from a log mean luminance of the image. In the case of video images, the adaptation points may include a time average over some of the above-mentioned images. In such embodiments, processing to identify highlight regions may include identifying pixels having at least a threshold amount of brightness above the adaptation point.

The image processing system 40 generates a highlight image 43 that is delivered to the highlight projector 20. The highlight image 43 is displayed by the highlight projector 20 to provide increased brightness in the highlight region. Pixels outside the highlight region may have very small or zero values in the highlight image 43. The image processing system 40 also delivers a base image 44 for projection by the main projector 12.

In some embodiments, base image 44 is the same as image data 42. In other embodiments, base image 44 is processed to provide a smooth transition between the highlight region that is primarily illuminated by highlight projector 20 and the base region of image 16 that is primarily or entirely illuminated by main projector 12. For example, this processing may include extracting highlight components from image data 42 to provide base image 44. In some embodiments, the processing includes estimating the brightness that will be delivered to the image pixels by highlight projector 20 when driven to display highlight image 43 and compensating the estimated brightness in generating base image 44. In some embodiments, the estimation may model characteristics of the optical system of the highlight projector 20. In some embodiments, the estimation may estimate the light delivered by the highlight projector 20 to pixels outside of the highlight region. The color and brightness of the main projector 12 and highlight projector 20 may be calibrated to facilitate such smooth transitions.

The highlight image 43 may take a variety of forms. In some embodiments, highlight image 43 may comprise or be considered a binary image ("all pixels of an ON" are set to the same level). Such embodiments may be used in conjunction with a process for selecting a highlight region to be a highlight region composed of pixels having a luminance well beyond the adaptation point. Such embodiments may take advantage of the fact that the human visual system similarly responds to light well beyond the accommodation point. For example, the viewer cannot tell much or any difference between an image in which some highlight pixels have a luminance of 10000 nits and another image in which the same highlight pixels have a luminance of 15000 nits, as long as the highlight pixels have a luminance that greatly exceeds the adapted points in the two images. Some such embodiments may operate by distributing the brightness from the highlight projector equally over the highlight pixels and/or by clipping the brightness of the highlight pixels to a set level.

In other embodiments, the highlight projector may be controlled to provide different brightnesses to different highlight pixels or regions. In other embodiments, the highlight projector may be controlled according to a combination of methods. For example, the highlight processor may be controlled to provide different luminances to highlight pixels for which the image data specifies a luminance in the first range and to provide the same luminance to highlight pixels for which the image data specifies a luminance exceeding the first range. The first range may be fixed or may be changed. For example, the variable first range may be based on a current adaptation point, on a number of pixels identified as being in highlight region, on a statistical value of pixels identified as being in highlight region (e.g., a maximum, average, mean, etc. of highlight pixels), on a combination of these, and so forth.

Image data processing may be distributed in various ways. For example, in some embodiments, image processing system 40 is integrated with a highlight projector such that image data 42 is provided directly to the highlight projector from which highlight image 43 is derived. In some alternative embodiments, processing is performed upstream such that highlight image data 43 is provided along with base image data 44. For example, highlight image data 43 may be encoded along with base image data 44 in a stream, file, or other data structure. In such embodiments, the projector system may be configured to extract highlight image data 43 and control the highlight projector with base image data 43 while causing the main projector to display an image according to base image data 44.

Highlight projectors can take many different forms. Some examples of different technologies that may be used for highlight projectors include: a scanning spot projector (some example embodiments of such projectors are described above); a holographic projector (e.g., a projector that phase-modulates light in the fourier plane of an optical system and thereby concentrates the light to form an image on an image surface).

Alternative types of scanning projectors include 1D light modulators that produce stripes of spatially modulated light on a screen and scanners that scan the stripes across the screen 18. By way of non-limiting example, the 1D modulator may comprise a 1D polarization modulator in combination with a polarizing beam splitter and a scanning mirror.

Another example embodiment is shown in fig. 5. Fig. 5 schematically shows a highlight projector 50 comprising a projector that redirects the light. One general type of such projectors includes projectors that concentrate light in some areas outside other areas by applying diffraction/phase based modulation methods. This method is sometimes referred to as "holographic 2D projection".

In the embodiment shown in fig. 5, the highlight projector comprises a coherent light source 51 (in the illustrated embodiment, the light source 51 comprises a laser 51A and a beam expander 51B), a phase modulation panel 52 located in an optical fourier plane in the optical path of the projector, and a controller 54 that spatially varies the phase shift effect of the phase modulator 52 according to the real component of the inverse fourier transform of the desired highlight image. The controller 54 may be configured to determine a fourier-based hologram (sometimes referred to as a computer-generated hologram) corresponding to the highlight image and set the phase at different locations on the phase modulation panel 52 from the computer-generated hologram. The interaction of the light from the light source 51 with the phase modulation panel 52 controlled according to the fourier based hologram generated by the controller 54 results in the re-creation of a highlight image. The lens 22 projects the generated image onto the screen 18 (not shown in fig. 5).

In some embodiments, the highlight projector comprises one or more holographic projectors with a variable intensity light source. The intensity of the light source may be controlled to provide further control of the display of the highlight image.

In some embodiments, the highlight projector comprises a plurality of holographic projectors, each projecting a different color of light. For example, one holographic projector may include a red light source 51 and be controlled to display the red channel of the highlight image. Such projectors may be used in combination with holographic projectors comprising green and blue light sources and individually controlled to image the green and blue channels of a highlight image.

Current projectors of the type that generate images by varying the phase modulator have the disadvantage that there may be significant light leakage due to the limited resolution of the phase modulator and/or because the desired image inverse fourier transform will generally have both real and imaginary parts and generally the phase modulator only performs one part of the inverse fourier transform. In embodiments where the phase modulated light is imaged to illuminate a spatial light modulator (such as a DMD array, LCOS modulator, LCD panel, etc.), such light leakage may be partially or substantially fully compensated in the highlight projector. In such embodiments, the spatial modulator may be operated to clear the projected highlight image by reducing the amount of light outside the highlight region. The spatial modulator used for this purpose may be the same or different from the spatial modulator used for the main projector.

Light leakage can be reduced by providing the phase modulator panel 52 with high spatial resolution. In some embodiments, the phase modulator panel 52 has a spatial resolution that exceeds the highlight image. In some embodiments, the number of controllable elements of the phase modulator panel 52 is 9 times or more the number of pixels in the highlight image.

The holographic projector may optionally be configured to project the highlight image on a non-planar focal plane. The controller 54 may be configured to generate drive signals for causing the phase modulator to focus on a desired non-planar surface. For example, holography may be configured to produce a focused image on a curved screen or spatial light modulator.

In the embodiment shown in fig. 6A, a holographic projector 72 projects a desired highlight image on a spatial light modulator 74 that is controlled to either redirect or clear or attract light in image areas outside of the highlight region. The light from the spatial light modulator 74 is then imaged onto the screen 18, for example by the projection lens 22. The spatial light modulator may be controlled, for example, by performing a simulation of the operation of the holographic projector 72 to obtain an estimate of the actual distribution of light produced by the holographic projector 72. This estimate is then compared to the highlight image. The comparison may include, for example, determining a ratio or difference of the estimate to the highlight image. Depending on the result of the comparison, the spatial light modulator 74 may be controlled to compensate for the difference between the light pattern actually projected by the holographic projector 72 and the desired highlight image. The calculation of the estimates may be performed, for example, using a programmed data processor, hard-configured logic circuitry, and/or configurable logic circuitry, such as a Field Programmable Gate Array (FPGA). The calculation may include estimating a phase-shifted light field produced by the phase modulator of the holographic projector 72 and calculating a fourier transform of the estimated light field.

In some embodiments, the propagation of light outside the high light region is reduced by blocking the DC component in the fourier plane. In the example embodiment shown in fig. 6B, the holographic projector 72 projects the highlight image directly onto the spatial light modulator 76 of the main projector. The spatial light modulator 76 is also illuminated by the light source 73.

Fig. 7 schematically shows a projector 60 having an alternative configuration. Projector 60 includes a light source 62 (which need not be a coherent light source). The light source 62 illuminates a 2D spatial light modulator 64 such as an analog DMD mirror array. The spatial light modulator 64 has controllable elements that can direct light to different locations on the screen 18. In some embodiments, spatial light modulator 64 is imaged by condenser lens 66 directly on screen 18 to provide a highlight image. In some embodiments, the spatial light modulator 64 illuminates another spatial light modulator 65. For example, spatial light modulator 65 may include a spatial light modulator (not shown in FIG. 7) that is also used by the main projector.

A highlight projector 80 according to a further alternative embodiment is shown in fig. 8. Highlight projector 80 includes a light source 82 that illuminates a spatial light modulator 83. In an example application, the spatial light modulator 83 is controlled such that all pixels outside the highlight region are set so as not to pass light through the screen 18. Because the spatial light modulator 83 is not perfect, some light is passed by pixels outside the highlight region. A viewer may perceive this light leakage as the black level increases (e.g., black takes a gray appearance across the entire image). The highlight projector 80 comprises a spatial filter 84, which in the illustrated embodiment comprises a shield 85 of fourier planes provided in the optical path between the spatial light modulator 83 and the screen 18. The shield 85 blocks DC spatial frequency components (i.e., components that affect the signal of all pixels in the displayed image) to reduce black levels while still delivering high light.

Systems in which the light source for the main projector or holographic projector comprises a coherent light source may include one or more optical components configured to reduce the occurrence of laser speckle in the projected image. Any suitable technique for reducing speckle may be applied. For example, many techniques for reducing laser speckle are known in the art. These include techniques such as providing: a vibrating diffuser in the optical path; randomizing the phase of the coherent light source, and randomizing the polarization of the coherent light source.

A highlight projector as described herein may be applied to 3D projection systems as well as to 2D projection systems. In embodiments where the observer wears polarization or spectrum sensitive glasses such that different components of the projected light are directed to the left and right eyes of the observer, the highlight projector may be controllable to emit light for viewing by the left, right, or both eyes of the observer. In the alternative, separate highlight projectors may be provided to project highlight images for the left and right eyes of the user. In some embodiments, the highlight projector emits light having different spectral components for viewing by the left and right eyes of the viewer. For example, a projection system as described herein may be used with, for example, the system described in WO 2008/140787; WO 2011/002757; and the 3D image projection system described in US 7784938; all incorporated herein by reference.

Fig. 9 shows a display 100 according to another embodiment. The display 100 may be, for example, a television, a computer display, an advertising display, etc. The display 100 may be used with or without a highlight projector. The display 100 includes a spatial light modulator panel 102 illuminated by a backlight assembly 104. For example, the spatial light modulator panel 102 may comprise a transmissive light modulation panel such as an LCD control panel. For example, the backlight assembly 104 comprises a holographic projector as described herein. The holographic projector includes a coherent light source 106, and a phase adjustment panel 108. Light from the light source 106 is phase modulated by the panel 108 and directed onto the spatial light modulator panel 102.

The display controller 109 receives an image to be displayed, determines a desired backlight light distribution, and controls the holographic projector to project the desired backlight light distribution on the spatial light modulator panel 102. The desired backlight light distribution may vary slowly (i.e. mainly consisting of low spatial frequencies). A shield 107 (which may be fixed or controllable) may optionally be provided in the fourier plane to attenuate or eliminate fourier components corresponding to higher spatial frequencies. For example, the controller 109 may determine the desired backlight light distribution by low-pass spatially filtering the image data, applying a blur filter to the image data, and/or calculating a local average or weighted average of local groups of pixels in the image data, among others. The drive values for the pixels of the phase modulation panel 108 may be determined by calculating an inverse fourier transform of the desired backlight light distribution.

In some embodiments, the controller calculates an estimate of the actual light distribution at the spatial light modulator panel 102. This estimate may be used to set the pixels of the spatial light modulator panel 102 to provide an image based on the image data. For example, the value of the pixel of the spatial light modulator panel 102 is set by comparing the estimated intensity of light to be incident on the pixel from the backlight 104 with the intensity of light specified for the pixel by the image data, and setting the pixel of the spatial light modulator panel to reduce the intensity of the incident light to the intensity specified by the image data. For example, the comparison may include dividing the image data by the estimated incident light intensity.

Calculating the estimated incident light intensity may include estimating how the light modulation panel 108 will affect light from the light source 106 when driven by the drive signal established by the controller and using this information to calculate the light field resulting from the signal applied to the phase adjustment panel 108. The light field at the spatial light modulator 102 may then be estimated by computing the fourier transform of the light field.

In some embodiments, display 100 comprises a color display. In some such embodiments, the spatial light modulator panel 102 comprises a monochromatic spatial light modulator. In such embodiments, the backlight 104 may include three or more monochromatic light sources (e.g., red, green, and blue lasers), each of which may be operated to illuminate the phase modulation panel 108. The images can be displayed by time multiplexing the images of different colors. For example, a red image may be displayed based on a red channel of the image data using red light source 106. A green image may then be displayed based on the green channel of the image data using the green light source 106 and a blue image based on the blue channel of the image data using the blue light source 106. In setting the pixels of the phase modulation panel 108 to the phase modulated light from the light sources, the controller may consider the wavelength of the light from each light source 106. In some embodiments, the backlight 104 includes a separate unit (e.g., a holographic projector) for each of a plurality of primary colors.

It is not mandatory that the highlight image data used to drive the highlight projector be derived from the image data in real time during the display of the image. Highlight image data may be predetermined and provided as part of the image data, or provided separately. In embodiments employing a holographic highlight projector, the image values used to control the phase modulation panel may be predetermined and provided as part of the image data.

Certain embodiments of the invention include a computer information processor that executes software instructions that cause the processor to perform the method of the invention. For example, one or more processors in a display system may perform an image processing method as described herein that executes software instructions (which may be or include firmware instructions) in a program memory accessible to the processors. The present invention may also be provided as a program product. The program product may comprise any medium carrying a set of non-transitory computer-readable signals comprising instructions which, when executed by a data processor, cause the data processor to perform the method of the invention. The program product according to the invention may be in any of a number of forms. For example, the program product may include physical media such as magnetic data storage media including floppy disks, hard disk drives, optical data storage media including CDROMs, DVDs, electronic data storage media including ROMs, PROMs, EPROMs, flash RAMs, etc. The computer readable signal on the program product may be selectively compressed or encrypted.

When a component (e.g., a software module, processor, component, device, circuit, etc.) is referred to above, unless otherwise indicated, reference to the component (including a reference to a "method") should be interpreted as including as equivalents of the component any component which performs the function of the described component (i.e., that is functionally equivalent), including components which are not structurally equivalent to the disclosed structure which performs the function in the illustrated exemplary embodiments of the invention.

While a number of exemplary aspects and embodiments have been discussed above, those of skill in the art will appreciate certain modifications, permutations, additions and sub-combinations thereof. It is therefore intended that the following appended claims and claims hereafter introduced are interpreted to include all such modifications, permutations, additions and sub-combinations as are within their true spirit and scope.

Claims (62)

1. A holographic projector, comprising:

a coherent light source;

a phase modulation panel configured to phase-modulate light from the light source and project the phase-modulated light onto the spatial light modulation panel;

wherein the spatial light modulation panel is configured to amplitude modulate the phase modulated light according to image data.

2. The holographic projector of claim 1, further comprising:

a display controller configured to receive image data of an image to be projected from the spatial light modulator, determine a desired light distribution to be projected onto the spatial light modulation panel, and control the holographic projector to project the desired light distribution on the spatial light modulator panel.

3. The holographic projector of claim 2, wherein the desired light distribution varies slowly at the spatial light modulation panel.

4. The holographic projector of claim 2, wherein the desired light distribution is predominantly of lower spatial frequency.

5. The holographic projector of claim 2, wherein the desired light distribution is shielded such that the distribution is of lower spatial frequency.

6. The holographic projector of claim 1 or 2, wherein the phase-modulated light is shielded in the fourier plane to attenuate fourier components corresponding to higher spatial frequencies.

7. The holographic projector of claim 2, wherein the controller determines the desired light distribution by at least one of low-pass spatial filtering the image data, applying a blur filter to the image data, and/or calculating a local average or a weighted average of local pixel groups in the image data.

8. The holographic projector of claim 2, wherein the controller energizes the spatial light modulation panel based on the image data and the phase modulated light at the spatial light modulation panel.

9. The holographic projector of claim 2, wherein the controller energizes the spatial light modulation panel based on the image data, the phase modulated light at the spatial light modulation panel, and a histogram including pixel intensities associated with the image.

10. The holographic projector of claim 2, wherein the controller is configured to determine the drive values for the pixels of the phase modulation panel by calculating an inverse fourier transform of the desired light distribution.

11. The holographic projector of claim 2, wherein the controller is configured to calculate an estimate of an actual light distribution at the spatial light modulation panel and use this estimate to set pixels of the spatial light modulation panel to provide an image in accordance with the image data.

12. The holographic projector of claim 2, wherein the pixel values of the spatial light modulation panel are set by: the estimated intensity of light to be incident on the pixel from the phase modulation panel is compared with the intensity of light specified for the pixel by the image data, and the pixel of the spatial light modulator panel is set to reduce the intensity of incident light to the intensity specified by the image data.

13. The holographic projector of claim 2, wherein a pixel value of the spatial light modulation panel is set by dividing image data of the pixel by an estimated incident light intensity to be incident on the pixel from the phase modulation panel.

14. The holographic projector of claim 2, wherein the pixel values of the spatial light modulation panel are set to reduce the intensity of light from the phase modulation panel at the pixel to match the intensity specified for the pixel by the image data.

15. The holographic projector of claim 13, wherein the intensity of light from the phase modulation panel is calculated based on an inverse fourier transform at the pixel.

16. The holographic projector of claim 13, wherein the intensity of the light from the phase modulation panel is calculated based on a light field simulation of the light incident on the spatial light modulation panel from the phase modulation panel.

17. The holographic projector of claim 16, wherein the light field simulation includes an inverse fourier transform of the light field projected from the phase modulation panel.

18. The holographic projector of any of claims 1 to 17, wherein the coherent light source comprises red, green, and blue lasers.

19. The holographic projector of any of claims 1 to 17, wherein the images are displayed by time multiplexing the images of different colors.

20. A method of encoding image data for holographic projection, comprising the steps of:

determining image data; and

the image data is processed prior to displaying the image data to determine base image data and highlight image data.

21. The method of claim 20, further comprising the steps of: the base image data and highlight image data are encoded into a format for display by the holographic projector.

22. The method of claim 22, further comprising the steps of: the base image data and highlight image data are stored in preparation for display by the holographic projector.

23. The method of claim 20, further comprising the steps of: the method includes receiving encoded image data for display by a holographic projector, decoding the encoded image data into base image data and highlight image data, and generating a holographic projection of the highlight image data to supplement the projection of the base image data.

24. The method of claim 22, wherein the holographic projector comprises a phase modulator configured to wavefront modulate the coherent light so as to project an image represented by the highlight image data in correspondence with the base image data.

25. The method of claim 22, wherein the holographic projector comprises LCOS-based wavefront modulation of highlight image data.

26. The method of claim 21, wherein the step of encoding comprises encoding highlight image data differently than base image data.

27. The method of claim 21, wherein the step of encoding comprises encoding the highlight image data based on an intended holographic modulation scheme of the projector.

28. The method of claim 21, wherein the step of encoding comprises encoding the highlight image data based on an LCOS-based wavefront modulation scheme of the projector.

29. The method of claim 21, wherein the step of encoding comprises encoding highlight image data according to an LCOS-based wavefront modulation scheme and encoding base image data based on an amplitude modulation scheme.

30. A projector, comprising:

a projection device configured to project a base image from the image data;

a highlight projector configured to amplify brightness in a highlight region of a base image.

31. The projector of claim 30 wherein the highlight projector comprises one of a steerable beam projector, a holographic projector, and a spatial light modulated projector.

32. The projector of claim 30 further comprising an image processing device configured to process the image data to determine a base image and a highlight image that, when projected in alignment with the base image, magnifies the brightness in the highlight region.

33. The projector of claim 30 wherein the highlight projector comprises a holographic projector.

34. The projector of claim 30 wherein the highlight projector comprises an LCOS modulator configured as a wavefront modulator configured to cause interaction of light projected from the LCOS modulator so as to cause an amplified illumination.

35. The projector of claim 34 wherein only a small percentage of pixels of the image data comprise amplified illumination.

36. A projector according to claim 30 wherein the projection device comprises an amplitude modulation device configured to amplitude modulate the base image to the output of the projector and the highlight projector comprises a wavefront modulation device configured to wavefront modulate the highlight image data to the output of the projector.

37. A projector according to any one of claims 30 to 36 wherein the amplified illumination is shielded to eliminate high spatial frequencies.

38. A projector according to any one of claims 30 to 36 wherein the amplified illumination varies smoothly and has a low spatial frequency.

39. A projector according to any one of claims 30 to 36 wherein the projection device comprises a DMD based projector and the highlight projector comprises an LCOS based projector.

40. A method of displaying highlighted image data comprising the steps of:

receiving image data;

processing the image data to generate base image data and highlight image data;

projecting the base image data; and

highlight image data is projected in alignment with the base image data.

41. The method of claim 40, wherein the step of projecting highlight image data comprises wavefront modulating light such that an image formed in a plane of the display image comprises highlights defined by the highlight image data that magnify the brightness of corresponding highlight areas in the image formed by the aligned base image and highlight image.

42. The method of claim 40, further comprising the step of further modulating the aligned base image and highlight image.

43. The method of claim 40, wherein highlight image data corresponds to a small percentage of pixels.

44. The method of claim 40, wherein the base image is projected via amplitude modulation and the highlight image is projected based on wavefront modulation.

45. The method of claim 40, wherein highlight images are projected as smoothly varying images.

46. The method of claim 40, wherein the step of projecting a highlight image comprises beam splitting a highlight image path from a base image path.

47. The method of claim 40, wherein the step of projecting comprises beam splitting a highlight image path from a combined highlight image and base image path.

48. The method of claim 40, wherein the step of projecting comprises amplitude projecting a base image and holographically projecting a highlight image.

49. A method of displaying an enhanced image, comprising the steps of:

receiving image data, the image data comprising pre-processed image data, the pre-processed image data comprising base image data and highlight image data;

modulating light according to the base image data and projecting the base image modulated light onto a surface;

coherent light is phase modulated according to highlight image data to generate phase modulated light that interacts to generate a corresponding highlight image on the surface in alignment with the base image.

50. The method of claim 49, wherein the surface comprises a spatial light modulator configured to further modulate the aligned image to generate a final image.

51. The method of claim 49, wherein the step of phase modulating includes masking to cause the highlight image distribution to have a lower spatial frequency.

52. The method of any of claims 49-51, wherein the phase-modulated coherent light is generated in conjunction with a Fourier plane mask configured to attenuate Fourier components corresponding to higher spatial frequencies.

53. A highlight projector comprising:

a wavefront modulator;

an amplitude modulator;

a coherent light source configured to illuminate the wavefront modulator;

a controller configured to actuate the wavefront modulator to cause wavefront modulation of the coherent light so as to cause an interference pattern in the coherent light so as to generate a smoothly varying image comprising only highlights of the desired image at the amplitude modulation device.

54. A highlight projector according to claim 53 wherein the wavefront modulator is an LCOS modulator, the amplitude modulator is a DMD and the smoothly varying image comprising only highlights is a small percentage of the final image to be displayed.

55. A highlight projector according to claim 53 wherein an amplitude modulator amplitude modulates the smoothly varying image.

56. A highlight projector according to claim 53 wherein an amplitude modulator amplitude modulates the smoothly varying image and a base image aligned with the smoothly varying image.

57. A highlight projector according to claim 53 wherein amplitude modulator amplitude modulation comprises only a small percentage of pixels of the smoothly varying image of the final image to be displayed and amplitude modulator amplitude modulates the base image in alignment with the smoothly varying image.

58. A highlight projector according to claim 53 wherein the controller activates the spatial light modulation panel according to image data, wavefront modulated light at the spatial light modulation panel, and a histogram comprising pixel brightness associated with the image.

59. A highlight projector according to claim 53 wherein the controller is configured to determine the drive values for the pixels of the wavefront modulator by calculating an inverse Fourier transform of the desired light distribution.

60. A highlight projector according to claim 53 wherein the controller is configured to calculate an estimate of the actual light distribution at the spatial light modulation panel and use this estimate to set the pixels of the spatial light modulation panel to provide an image in dependence on the image data.

61. A highlight projector according to claim 53 wherein the pixel values of the spatial light modulation panel are set by: the estimated intensity of light to be incident on the pixel from the wavefront modulator is compared with the intensity of light specified for the pixel by the image data, and the pixel of the spatial light modulator panel is set to reduce the intensity of incident light to the intensity specified by the image data.

62. A highlight projector according to claim 53 wherein the intensity of light from the wavefront modulator is calculated based on the inverse Fourier transform at the pixel.