HK1227205B - High luminance projection displays and associated methods - Google Patents

Description

High-gloss projection displays and related methods

This application is a divisional application of the PCT international application entering the Chinese national phase, with the international application date of April 11, 2012, national application number 201280019001.X, and invention name “High-gloss projection display and related methods”.

Cross-reference to related applications

This application claims priority to U.S. Provisional Patent Application No. 61/476,949, filed April 19, 2011, which is hereby incorporated by reference in its entirety.

Technical Field

The present invention relates to projection displays. Example embodiments provide digital cinema displays. Other embodiments provide displays such as televisions, computer monitors, and special-purpose displays such as advertising displays, virtual reality displays, gaming displays, and medical imaging displays.

Background Art

There is a growing interest in providing displays capable of reproducing realistic-looking images. One aspect of achieving realistic images is providing high peak brightness and a high dynamic range. Typical natural scenes include very bright areas, such as the sun in the sky and highlights of brightly lit objects, as well as dim areas, such as objects in shadow. Achieving realistic images of typical scenes is impossible on displays that cannot achieve high peak brightness.

Current projection technology doesn't efficiently scale for high light. For example, in many common projector designs, a light source, such as a xenon lamp, illuminates one or more spatial light modulators. The spatial light modulator directs some light toward the screen while attracting or redirecting other light. Achieving high light requires proportionally increasing the power of the light source. The increased power consumption of the light source becomes an obstacle to increasing the brightness of the light source to a level sufficient to provide peak brightness at levels typical of natural scenes. Furthermore, high light sources can cause overheating of the spatial light modulator and other components in the projector, among other issues.

For example, a current digital cinema projector may have a light source that consumes 8 kilowatts of electrical power to illuminate a large screen producing a peak brightness of 48 nits (48 cd/ ^m2 ). To achieve a peak brightness of 12,000 nits (brightness typically encountered in everyday life), the power of the light source would need to be scaled up to over 2 megawatts. Clearly, this is impractical in most situations.

Another problem that prevents significant increases in peak brightness in many conventional projection displays is that contrast does not increase with increasing peak brightness. In many such displays, increasing the intensity of the light source to achieve increased peak brightness also increases the black level. Therefore, attempting to increase peak brightness beyond a threshold will result in unacceptably high black levels.

Another obstacle to providing displays with high enough brightness to render realistic images is that the human visual system's response to light is roughly logarithmic. In contrast, power needs to scale approximately linearly with brightness. Assuming the same efficiency of the light source, doubling the brightness of the image requires doubling the power. However, doubling the brightness does not result in an image perceived by the viewer as twice as bright. Doubling the apparent brightness requires approximately the square of the brightness.

It is intended that the above examples of the related art and limitations related thereto are illustrative and not exclusive. Other limitations of the related art will become apparent to those skilled in the art upon a reading of the specification and a study of the drawings.

Summary of the Invention

The present invention has a number of aspects. Embodiments of the present invention provide, inter alia, projection displays, methods for operating projection displays, dual modulation displays, media containing computer-readable instructions that, when executed by a data processor, cause the data processor to perform a method according to the present invention, methods for displaying images, and methods for processing image data for display.

One exemplary aspect of the present invention provides a display system comprising: a main projector configured to project an image defined by base image data onto a screen; and a main projector configured to project a highlight image defined by highlight image data onto the screen in alignment with the base image. An image processor is configured to process the image data to generate highlight image data.

In some embodiments, the highlight projector comprises a scanning beam projector. A scanning beam projector can provide, for example, laser beams of multiple primary colors (e.g., red, green, and blue beams). The beams can be scanned together or independently to provide a desired apparent brightness and color for the highlight area. In other embodiments, the scanning beam projector provides a scannable beam of white light.

In some embodiments, the highlight projector comprises a 2D holographic projector.

Another aspect provides a highlight light projector system including an image processor configured to process image data to output a highlight image; and a light projector operable to project light in alignment with a base image according to the highlight image.

Another example aspect provides a display comprising a light source of spatially modulated light arranged to illuminate a spatial light modulator, wherein the source of spatially modulated light comprises a 2D holographic light source.

Another example aspect provides a method for displaying an image defined by image data. The method includes concentrating light from a light source to output light that has been spatially modulated in a manner based on the image data; illuminating a spatial light modulator with the spatially modulated light; and controlling the spatial light modulator to display an image based on the image data. For example, concentrating the light may include generating a computer-generated 2D hologram. In some embodiments, the light includes coherent light, and concentrating the light includes adjusting the phase of the light in a Fourier plane of the optical system.

Another example aspect provides a method for displaying an image based on image data, the method comprising processing the image data to generate a base image and a highlight image including highlight pixels; operating a main projector to display the base image; and operating a highlight projector to display the highlight image superimposed on the base image.

In addition to the exemplary aspects and embodiments described above, further aspects and embodiments will become apparent by reference to the drawings and study of the following detailed description.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings illustrate non-limiting example embodiments of the invention.

FIG. 1 is a schematic diagram of a display system according to an example embodiment.

FIG. 1A is a schematic diagram of a display system according to another example embodiment.

2A and 2B are exemplary histograms showing the number of pixels in an image according to the brightness of those pixels for a bright image and a dark image, respectively.

FIG3 is a schematic diagram of an example arrangement combining a highlight projector with a main projector.

FIG4 is a block diagram illustrating image data processing components of a display system according to an example embodiment.

Figure 5 is a schematic diagram illustrating a highlight projector including a light redirection projector according to an example embodiment. A highlight projector of the type shown in Figure 5 can be configured to project light onto, for example, a screen, a component of a main projector, or a spatial light modulator.

6A is a schematic diagram illustrating an example holographic projector configured to project a desired highlight image onto a spatial light modulator controlled to either redirect, divert, or absorb light in image regions outside of the highlight region.

6B is a schematic diagram illustrating a holographic projector configured to project a highlight image directly onto a spatial light modulator of a main projector according to example embodiments.

7 is a schematic diagram illustrating a high light projector including a light source arranged to illuminate a 2D spatial light modulator using a spatial filter in a Fourier plane for eliminating leaked light according to example embodiments.

FIG. 8 is a schematic diagram illustrating a highlight projector including a light source illuminating a spatial light modulator according to example embodiments.

Figure 9 shows a display according to another embodiment. A display having the general structure schematically shown in Figure 9 can be used as a standalone display (eg as a television, computer monitor, special purpose display, etc.) or as part of a display system including a high light projector.

DETAILED DESCRIPTION

Throughout the following description, details are set forth to provide those skilled in the art with a more thorough understanding of the present invention. However, well-known elements may not be shown or described in detail to avoid unnecessarily obscuring the present disclosure. Therefore, the specification and drawings should be considered in an illustrative rather than a restrictive sense.

Some embodiments of the present invention provide a projection display including a main projector and a highlight projector. The main projector may have a relatively low peak brightness and may be used to project a full image. The brightness of highlights in the image projected by the main projector may be lower than a desired brightness. The highlight projector may project concentrated light to boost the light at the location of the highlight, thereby significantly increasing the brightness of the highlight.

FIG1 shows a projection system 10 according to a first exemplary embodiment. Projection system 10 includes a main projector 12 having a lens 14 that projects an image 16 onto a screen 18. Screen 18 can be a front projection screen or a rear projection screen. System 10 also includes a separate highlight projector 20 having a lens 22 that projects an image 16A onto screen 18. Images 16 and 16A are superimposed so that a viewer sees an image resulting from the combination of images 16 and 16A.

The main projector 12 may include any suitable image projector. For example, the main projector 12 may include a DLP-based projector, a projector using one or more liquid crystal on silicon (LCOS) spatial light modulators, a projector including a transmissive liquid crystal display (LCD) panel to modulate light, a cathode ray tube (CRT) projector, and the like.

Highlight projector 20 is of a type that can deliver concentrated light to at least some areas within the area of image 16, preferably without significantly increasing the brightness levels of other areas within image 16. For example, highlight projector 20 may include one or more scanning light beams that can be directed to add further illumination to only selected highlight areas of image 16.

Highlight projector 20 and main projector 12 are co-aligned so that highlight projector 20 can precisely deliver additional light to small highlight areas within image 16 projected by main projector 12. In some embodiments, highlight projector 20 has a spatial resolution that is equal to or greater than main projector 12. In other embodiments, projector 20 may have a spatial resolution that is less than main projector 12. In other embodiments, projector 20 may have a spatial resolution that is less than main projector 12.

In some embodiments, a highlight projector 20 and an image processor are provided for use as an add-on to an existing master projector, such as a commercially available digital cinema projector. The image processor can be configured to receive image data for projection and generate a highlight image for display by the highlight projector. In some embodiments, the image processor can modify the image data to provide a base image for display by the existing master projector. The highlight projector can be calibrated upon installation to produce a highlight image that aligns with the image produced by the existing master projector.

Advantageously, in typical scenes, only a relatively small proportion of the pixels in the image need to be displayed with a brightness greater than the peak brightness of a standard projector 12 for enhanced realism. It has been found that enhanced realism can be achieved by very selectively providing bright highlights. Figures 2A and 2B are histograms showing the number of pixels in an image according to the brightness of those pixels for a bright image and a dark image, respectively, prepared by a human colorist for viewing on a display with a high peak brightness. In each case, the colorist adjusted the image to achieve what the colorist considered to be the optimal appearance.

Somewhat surprisingly, the average brightness of all pixels in a bright image is still relatively low. Even in a bright image with the histogram of FIG3A , it can be seen that only a relatively small proportion of pixels have high brightness (e.g., brightness exceeding about 1,000 or 2,000 nits). A few very bright pixels at high and very high brightness can produce an image with a much more realistic appearance without significantly affecting the light adaptation of the observer's eye. This is different from real scenes in nature, where many or all pixels can have very high brightness. For example, a real scene on a glacier on a clear day can be so bright that it is uncomfortable or even harmful to view for a sustained period without dark sunglasses. A colorist can prepare such a scene in a way that produces a relatively low average brightness while providing high brightness in a few key areas to provide a more realistic viewing experience.

Some embodiments exploit the fact that if a small highlight area is rendered with a peak brightness much higher than the average brightness used to present the image to the viewer, even very bright scenes can be reproduced with surprisingly low average brightness while maintaining a realistic viewing impression. Some embodiments use a highlight projector that is much less powerful and cannot boost all pixels of image 16 to the brightest highlight level, but is able to amplify the illumination of the highlights to a desired level. In such embodiments, light from the highlight projector is focused on the highlights to provide the desired brightness in the highlights.

There are many ways to arrange the main projector and the highlight projector in combination. For example, a system can be provided that provides a projector and a highlight projector for selectively amplifying the brightness of the highlight area, arranged to have any combination of the following features:

The main projector and the highlight projector may use the same overall technology or different technologies.

The main projector and the highlight projector can be provided as separate units or as a combined unit (e.g., an integrated form factor). In providing the main projector and the highlight projector as a combined unit, the main projector and the highlight projector can share certain optical components and/or certain optical paths. For example, the main projector and the highlight projector can share one or more of a projection lens, relay optics, one or more spatial light modulators, etc. Various examples of shared components and optical paths are described below.

• The system may include one or more than one master projector that collectively project a base image. For example, the system 10 may include a plurality of master projectors 12 that collectively illuminate a screen 18 to provide an image 16.

The system may include one or more highlight projectors that collectively project a highlight image to amplify the brightness of the highlight area. For example, the highlight projector 20 may include multiple units that can be controlled to collectively direct light onto the highlight area of the image 16.

High light projectors can be monochromatic (eg, can project white light) or polychromatic.

• The highlight projector may optionally include filtering (eg spatial filters in the example Fourier plane) to suppress illumination outside the highlight area.

The highlight projector may optionally include one or more spatial light modulators. The spatial light modulator may be controlled to perform one or more of the following: direct light to illuminate the highlight area, correct errors in the projected highlight image, suppress illumination outside the highlight area, adjust the highlight image to smoothly blend into the base image projected by the main projector, and redirect light from outside the highlight area into the highlight area. In embodiments where the highlight projector includes one or more spatial light modulators, the spatial light modulator may include a spatial light modulator shared by the main projector and/or may include a spatial light modulator dedicated to the highlight projector.

The main projector and the highlight projector can be arranged for front or rear projection. It is not mandatory that the highlight projector 20 and the main projector 12 illuminate the screen 18 from the same side. In embodiments where the screen 18 is translucent (e.g., where the screen 18 comprises a rear projection type screen), the highlight projector 20 and the main projector 12 can illuminate the screen 18 from opposite sides.

These various methods and their permutations and combinations are not limiting but are intended to provide examples of some embodiments within the scope of the present invention.

Advantageously, the combined image as viewed by the viewer includes some highlights in which the peak brightness significantly exceeds the peak brightness of main projector 12. For example, the main projector can have a peak brightness of 500 nits or less, and the highlight area can have a peak brightness of 2000 nits or more. For example, it is expected that some main projectors used in a dark viewing environment (such as a movie theater) can provide a peak brightness of about 15 to 50 nits. Some such projectors are designed to be imaged onto a screen of a large area. For example, it is expected that some main projectors used in a bright viewing environment can provide a peak brightness of about 100 to 300 nits.

Because the highlight area illuminated by highlight projector 20 may comprise only a very small portion of the area of image 16 (e.g., less than 10%, 5%, 1%, or even less than 0.1%), highlight projector 20 may be able to achieve the desired high brightness in the highlight area without requiring impractical power input.

FIG1A shows a projector system according to an example embodiment in which a highlight projector includes a point light source 20A that produces a narrow light beam 21 and a deflector 23 that includes scanning mirrors 23A and 23B. Mirrors 23A and 23B are rotatably mounted and operated by actuators (not shown) so that the light beam 21 can be directed to form a small spot 25 at any desired location in the image 16. The intensity of the light beam 21 and the position of the displayed spot 25 can be controlled by a controller 24 to achieve increased brightness in a selected highlight area. In some embodiments, the brightness of the highlight area is at least partially controlled by varying the amount of time that the control spot 25 resides over the highlight area. In some embodiments, the brightness of the highlight area is at least partially controlled by controlling the intensity and/or duty cycle of the light beam 21 when the light beam 21 is illuminating the highlight area.

The light beam 21 may, for example, comprise a laser beam. In some embodiments, the highlight projector comprises three laser beams of different colors that can be combined to form a white highlight. For example, the highlight projector may comprise red, green, and blue laser beams. In such embodiments, the light beams may be manipulated by a single deflector assembly (e.g., a single set of mirrors 23A, 23B) to illuminate the highlight area. In an alternative embodiment, a separate deflection assembly is provided for each of the multiple light beams 21.

Since highlights typically appear in only a small proportion of the total area of image 16, the laser can increase the perceived brightness of the highlight areas by dwelling on those areas for a longer time. The laser need not illuminate any portion of image 16 outside the highlight areas.

In embodiments where the highlight projector includes a steerable beam, a controller (e.g., controller 24) that enables beam steering can be configured to control mirrors 23A and 23B (or alternative beam steering mechanisms, such as mechanisms utilizing digital light deflectors, grating light valves, etc.) so that spot 25 follows a trajectory that depends on the location of the highlight area to be illuminated. The beam steering mechanism need not necessarily scan in a raster or other pattern that covers all pixels of image 16. By steering spot 25 in a trajectory that brings spot 25 into the highlight area while avoiding at least some pixels outside the highlight area, controller 25 can cause spot 25 to remain over the highlight area for a period of time sufficient to achieve the desired brightness of the highlight area.

The main projector 12 and the highlight projector 20 can optionally be combined with each other so that the two projectors share some common optical paths. For example, the optical systems of the highlight projector 20 and the main projector 12 can be arranged to share a common projection lens 14. An example of this is shown in FIG3 . FIG3 is naturally schematic. Optical components that may be present in the optical path, such as relay lenses, mirrors, filters, etc., have been omitted for clarity.

In the embodiment shown in FIG3 , main projector 12 includes a light source 30 that can emit light 31 to illuminate a spatial light modulator 32. Light source 30 can include a uniform light source or a light source that can be spatially modulated according to image data (e.g., a base image). The light modulated by spatial light modulator 32 is directed by projection lens 14 onto screen 18 (not shown in FIG3 ) to provide image 16 (not shown in FIG3 ).

In this embodiment, the highlight projector includes a high-intensity narrow-beam light source 34 that can be controlled to emit a narrow light beam 35 that is manipulated by an X-Y deflector 36 and a light combiner 37 to produce a brightly illuminated spot 38 on a spatial light modulator 32. By controlling the intensity of the light source 34 and/or turning the light source 34 on and off while scanning with the X-Y scanner 36, a plurality of different highlight areas can be illuminated on the spatial light modulator 32 with light from the light source 34. This additional light, as modulated by the spatial light modulator 32, is also imaged by the lens 14 to increase the brightness of the highlight area within the image 16.

In an alternative embodiment, light combiner 37 is located between spatial light modulator 32 and projection lens 14 so that spot 38 is projected directly onto screen 18. In this alternative embodiment, the optical paths of the main projector and the highlight projector may only have projection lens 14 in common.

FIG4 is a block diagram illustrating image data processing components of a display system according to an example embodiment. An image processing system 40 receives image data 42 and processes the image data 42 to identify a highlight region. The processing may include, for example, comparing pixel brightness values to a first threshold and identifying pixels having brightness values exceeding the first threshold as belonging to a highlight region. In some embodiments, the highlight region may be limited to an area comprising a predetermined region of connected pixels having brightness values exceeding the first threshold. As another example, the processing may identify a highlight region as consisting of the M highest brightness pixels (where M is a number) or pixels at or above the Nth percentile for brightness (where N is a percentile such as the 90th percentile, the 95th percentile, the 98th percentile, the 99th percentile, or the 99.9th percentile). The processing may include applying multiple such criteria (e.g., a highlight region may be identified as up to M pixels having brightness exceeding a threshold).

In some embodiments, processing includes weighing the area included in the highlight region against the peak brightness. Such processing may include a histogram analysis. For example, for an image in which processing identifies a relatively large number of pixels as belonging to the highlight region according to a first criterion, processing may choose between retaining the highlight region at the expense of a reduced peak brightness achievable in the highlight region according to the first criterion or applying a second criterion to reduce the number of pixels included in the highlight region. Such processing may include a histogram analysis.

In some embodiments, processing is performed with reference to an adaptation point. The adaptation point may, for example, comprise or be determined from the logarithmic mean luminance of the image. In the case of video images, the adaptation point may comprise a temporal mean over some of the aforementioned images. In such embodiments, processing to identify highlight regions may comprise identifying pixels having a luminance that is at least a threshold amount higher than the adaptation point.

Image processing system 40 generates a highlight image 43 that is delivered to highlight projector 20. Highlight image 43 is displayed by highlight projector 20 to provide increased brightness in the highlight area. Pixels outside the highlight area may have very small or zero values in highlight image 43. Image processing system 40 also delivers a base image 44 for projection by main projector 12.

In some embodiments, base image 44 is the same as image data 42. In other embodiments, base image 44 is processed to provide a smooth transition between a highlight area illuminated primarily by highlight projector 20 and a base area of image 16 illuminated primarily or entirely by main projector 12. For example, this processing may include extracting a highlight component from image data 42 to provide base image 44. In some embodiments, the processing includes estimating the brightness that would be delivered to image pixels by highlight projector 20 when driven to display highlight image 43 and compensating for the estimated brightness in generating base image 44. In some embodiments, the estimation may model characteristics of the optical system of highlight projector 20. In some embodiments, the estimation may estimate the light delivered by highlight projector 20 to pixels outside the highlight area. The color and brightness of main projector 12 and highlight projector 20 may be calibrated to facilitate such a smooth transition.

Highlight image 43 can take a variety of forms. In some embodiments, highlight image 43 can include or be considered a binary image (all pixels that are "ON" are set to the same level). Such embodiments can be used in conjunction with a process for selecting a highlight region to be a highlight region consisting of pixels having a brightness that well exceeds the adaptation point. Such embodiments can take advantage of the fact that the human visual system responds similarly to light that well exceeds the adaptation point. For example, a viewer cannot tell much or any difference between an image in which some highlight pixels have a brightness of 10,000 nits and another image in which the same highlight pixels have a brightness of 15,000 nits, as long as the highlight pixels have a brightness that well exceeds the adaptation point in both images. Some such embodiments can operate by distributing the brightness from the highlight projector equally over the highlight pixels and/or by clipping the brightness of the highlight pixels to a set level.

In other embodiments, the highlight projector can be controlled to provide different brightness to different highlight pixels or areas. In other embodiments, the highlight projector can be controlled according to a combination of methods. For example, the highlight processor can be controlled to provide different brightness to highlight pixels for which the image data specifies a brightness in a first range and to provide the same brightness to highlight pixels for which the image data specifies a brightness exceeding the first range. The first range can be fixed or can be changed. For example, the variable first range can be based on the current adaptation point, based on the number of pixels identified as being in the highlight area, based on statistics of pixels identified as being in the highlight area (e.g., maximum value, average, mean, etc. of highlight pixels), based on combinations of these, and the like.

Image data processing can be distributed in various ways. For example, in some embodiments, the image processing system 40 is integrated with the highlight projector so that image data 42 is provided directly to the highlight projector, which exports a highlight image 43 internally. In some alternative embodiments, processing is performed upstream so that the highlight image data 43 is provided along with the base image data 44. For example, the highlight image data 43 can be encoded in a stream, file, or other data structure along with the base image data 44. In such embodiments, the projector system can be configured to extract the highlight image data 43 and use the base image data 43 to control the highlight projector while causing the main projector to display an image based on the base image data 44.

High-beam projectors can take many different forms. Some examples of different technologies that can be used for high-beam projectors include: scanning point projectors (some example embodiments of such projectors are described above); holographic projectors (e.g., projectors that phase-modulate light in the Fourier plane of an optical system and thereby focus the light to form an image on an image surface).

Alternative types of scanning projectors include a 1D light modulator that produces stripes of spatially modulated light on the screen and a scanner that scans the stripes across the screen 18. By way of non-limiting example, the 1D modulator may include a 1D polarization modulator combined with a polarization beam splitter and a scanning mirror.

Another example embodiment is shown in FIG5 . FIG5 schematically illustrates a high-light projector 50 including a projector that redirects light. One general type of such projector involves applying a diffraction/phase-based modulation method to focus light in some areas and away from other areas. This method is sometimes referred to as "holographic 2D projection."

In the embodiment shown in FIG5 , the highlight projector includes a coherent light source 51 (in the embodiment shown, the light source 51 includes a laser 51A and a beam expander 51B), a phase modulation panel 52 located in the optical Fourier plane of the projector's optical path, and a controller 54 that spatially varies the phase shifting effect of the phase modulator 52 according to the real components of the inverse Fourier transform of the desired highlight image. The controller 54 can be configured to determine a Fourier-based hologram (sometimes called a computer-generated hologram) corresponding to the highlight image and set the phase at different locations on the phase modulation panel 52 according to the computer-generated hologram. The interaction of light from the light source 51 with the phase modulation panel 52, which is controlled according to the Fourier-based hologram generated by the controller 54, results in the reconstruction of the highlight image. The lens 22 projects the resulting image onto the screen 18 (not shown in FIG5 ).

In some embodiments, the highlight projector includes one or more holographic projectors having a light source with variable intensity. The intensity of the light source can be controlled to provide further control over the display of the highlight image.

In some embodiments, the highlight projector includes multiple holographic projectors, each projecting a different color of light. For example, one holographic projector may include a red light source 51 and be controlled to display the red channel of the highlight image. Such a projector may be used in combination with a holographic projector including green and blue light sources and each controlled to image the green and blue channels of the highlight image.

Current projectors that generate images by changing the type of phase modulator have the following disadvantages: there may be significant light leakage due to the limited resolution of the phase modulator and/or because the desired image inverse Fourier transform will generally have both real and imaginary parts and generally the phase modulator only performs one part of the inverse Fourier transform. In embodiments where the phase modulated light is imaged to illuminate a spatial light modulator (such as a DMD array, LCOS modulator, LCD panel, etc.), such light leakage can be partially or substantially fully compensated in the highlight projector. In such embodiments, the spatial modulator can be operated to clean up the projected highlight image by reducing the amount of light outside the highlight area. The spatial modulator used for this purpose can be the same as or different from the spatial modulator used for the main projector.

Light leakage can be reduced by providing a phase modulator panel 52 with high spatial resolution. In some embodiments, the phase modulator panel 52 has a spatial resolution that exceeds that of the highlight image. In some embodiments, the number of controllable elements of the phase modulator panel 52 is nine times or more the number of pixels in the highlight image.

The holographic projector can optionally be configured to project a highlight image onto a non-planar focusing surface. The controller 54 can be configured to generate a drive signal for the phase modulator to cause focusing on the desired non-planar surface. For example, the holographic projection can be configured to produce a focused image on a curved screen or a spatial light modulator.

In the embodiment shown in FIG6A , holographic projector 72 projects a desired highlight image onto a spatial light modulator 74 that is controlled to either redirect, remove, or attract light from image areas outside the highlight region. Light from spatial light modulator 74 is then imaged onto screen 18, for example, by projection lens 22. The spatial light modulator can be controlled, for example, by performing a simulation of the operation of holographic projector 72 to obtain an estimate of the actual distribution of light produced by holographic projector 72. This estimate is then compared to the highlight image. The comparison can include, for example, determining a ratio or difference between the estimate and the highlight image. Based on the results of the comparison, spatial light modulator 74 can be controlled to compensate for the difference between the light pattern actually projected by holographic projector 72 and the desired highlight image. Calculating the estimate can be performed, for example, using a programmed data processor, hard-configured logic circuitry, and/or configurable logic circuitry (e.g., a field-programmable gate array (FPGA)). The calculation can include estimating the phase-shifted light field produced by a phase modulator of holographic projector 72 and calculating a Fourier transform of the estimated light field.

In some embodiments, the propagation of light outside the highlight region is reduced by blocking the DC component in the Fourier plane. In the example embodiment shown in FIG6B , the holographic projector 72 projects the highlight image directly onto the spatial light modulator 76 of the main projector. The spatial light modulator 76 is also illuminated by the light source 73.

FIG7 schematically shows a projector 60 having an alternative structure. The projector 60 includes a light source 62 (which need not be a coherent light source). The light source 62 illuminates a 2D spatial light modulator 64, such as an analog DMD mirror array. The spatial light modulator 64 has controllable elements that can direct light to different locations on the screen 18. In some embodiments, the spatial light modulator 64 is imaged directly on the screen 18 by a focusing lens 66 to provide a highlight image. In some embodiments, the spatial light modulator 64 illuminates another spatial light modulator 65. For example, the spatial light modulator 65 may include a spatial light modulator (not shown in FIG7 ) that is also used by the main projector.

A highlight projector 80 according to a further alternative embodiment is shown in FIG8 . The highlight projector 80 includes a light source 82 that illuminates a spatial light modulator 83. In an example application, the spatial light modulator 83 is controlled so that all pixels outside the highlight area are set so as not to pass light through the screen 18. Because the spatial light modulator 83 is not perfect, some light is passed through the pixels outside the highlight area. An observer can perceive this light leakage as the black level increases (e.g., black takes on a gray appearance across the entire image). The highlight projector 80 includes a spatial filter 84, which in the embodiment shown includes a shield 85 in a Fourier plane provided in the optical path between the spatial light modulator 83 and the screen 18. The shield 85 blocks DC spatial frequency components (i.e., components of the signal that affect all pixels in the displayed image) thereby reducing the black level while still passing the highlights.

Systems in which the light source for a main projector or a holographic projector comprises a coherent light source may include one or more optical components configured to reduce the appearance of laser speckle in the projected image. Any suitable speckle reduction technique may be applied. For example, numerous techniques for reducing laser speckle are known in the art. These include techniques such as providing a vibrating diffuser in the optical path, randomizing the phase of the coherent light source, and randomizing the polarization of the coherent light source.

The highlight projector as described herein can be applied to 3D projection systems as well as 2D projection systems. In embodiments where the observer wears polarized or spectrally sensitive glasses so that different components of the projected light are directed to the observer's left and right eyes, the highlight projector can be controllable to emit light for viewing by the observer's left eye, right eye, or both eyes. In an alternative, a separate highlight projector can be provided to project highlight images for the user's left and right eyes. In some embodiments, the highlight projector emits light having different spectral components for viewing by the viewer's left and right eyes. For example, the projection system as described herein can be used in combination with 3D image projection systems described, for example, in WO2008/140787; WO2011/002757; and US7784938; all of which are incorporated herein by reference.

FIG9 shows a display 100 according to another embodiment. Display 100 can be, for example, a television, a computer monitor, an advertising display, or the like. Display 100 can be used with or without a high-light projector. Display 100 includes a spatial light modulator panel 102 illuminated by a backlight assembly 104. For example, spatial light modulator panel 102 can include a transmissive light modulating panel such as an LCD control panel. Backlight assembly 104 can include, for example, a holographic projector as described herein. The holographic projector includes a coherent light source 106 and a phase adjustment panel 108. Light from light source 106 is phase modulated by panel 108 and directed onto spatial light modulator panel 102.

The display controller 109 receives the image to be displayed, determines the desired backlight distribution, and controls the holographic projector to project the desired backlight distribution onto the spatial light modulator panel 102. The desired backlight distribution can vary slowly (i.e., be primarily composed of low spatial frequencies). A shield 107 (which can be fixed or controllable) can be optionally provided in the Fourier plane to attenuate or eliminate Fourier components corresponding to higher spatial frequencies. For example, the controller 109 can determine the desired backlight distribution by low-pass spatial filtering the image data, applying a blur filter to the image data, and/or calculating a local average or weighted average of a local group of pixels in the image data, etc. The drive value for the pixel of the phase modulation panel 108 can be determined by calculating the inverse Fourier transform of the desired backlight distribution.

In some embodiments, the controller calculates an estimate of the actual light distribution at the spatial light modulator panel 102. This estimate can be used to set the pixels of the spatial light modulator panel 102 to provide an image according to the image data. For example, the values of the pixels of the spatial light modulator panel 102 are set by comparing the estimated intensity of light to be incident on the pixel from the backlight 104 with the intensity of light specified for that pixel by the image data, and setting the pixels of the spatial light modulator panel to reduce the intensity of the incident light to the intensity specified by the image data. For example, the comparison can include dividing the image data by the estimated incident light intensity.

Calculating the estimated incident light intensity may include estimating how the light modulating panel 108 will affect the light from the light source 106 when driven by a drive signal established by the controller and using this information to calculate the light field caused by the signal applied to the phase adjustment panel 108. The light field at the spatial light modulator 102 may then be estimated by calculating the Fourier transform of the light field.

In some embodiments, the display 100 comprises a color display. In some such embodiments, the spatial light modulator panel 102 comprises a monochrome spatial light modulator. In such embodiments, the backlight 104 may comprise three or more monochromatic light sources (e.g., red, green, and blue lasers), each of which may be operated to illuminate the phase modulation panel 108. An image may be displayed by time-multiplexing images of different colors. For example, a red light source 106 may be used to display a red image based on the red channel of the image data. A green light source 106 may then be used to display a green image based on the green channel of the image data and a blue light source 106 may be used to display a blue image based on the blue channel of the image data. In setting the pixels of the phase modulation panel 108 to the phase-modulated light from the light source, the controller may take into account the wavelength of the light from each light source 106. In some embodiments, the backlight 104 comprises a separate unit (e.g., a holographic projector) for each of the multiple primary colors.

It is not mandatory that the highlight image data used to drive the highlight projector be derived from the image data in real time during the display of the image. The highlight image data can be predetermined and provided as part of the image data, or provided separately. In embodiments employing a holographic highlight projector, the image values used to control the phase modulation panel can be predetermined and provided as part of the image data.

Certain embodiments of the present invention include a computer information processor that executes software instructions that cause the processor to perform the method of the present invention. For example, one or more processors in a display system can execute the image processing method described herein executing software instructions (which can be or include firmware instructions) in a program memory accessible to the processor. The present invention can also be provided in the form of a program product. The program product can include any medium that carries a set of non-transitory computer-readable signals including instructions that, when executed by a data processor, cause the data processor to perform the method of the present invention. The program product according to the present invention can be in any of a variety of forms. For example, the program product can include physical media such as magnetic data storage media including floppy disks, hard drives, optical data storage media including CD ROMs, DVDs, electronic data storage media including ROMs, PROMs, EPROMs, flash RAMs, and the like. The computer-readable signals on the program product can be selectively compressed or encrypted.

When components (e.g., software modules, processors, parts, devices, circuits, etc.) are mentioned above, unless otherwise stated, references to components (including references to "methods") should be interpreted to include equivalents of the components (i.e., functionally equivalent) of any components that perform the functions of the described components, including components that are not structurally equivalent to the disclosed structures that perform the functions of the exemplary embodiments shown in the invention.

Although many exemplary aspects and embodiments have been discussed above, those skilled in the art will appreciate certain modifications, permutations, additions, and sub-combinations thereof. It is therefore intended that the following appended claims and claims hereafter introduced be interpreted as including all such modifications, permutations, additions, and sub-combinations as are within their true spirit and scope.

Claims (21)

1. A display system, comprising: Holographic projector; Spatial light modulation panel; and Display controller, The holographic projector mentioned above includes: Coherent light source; A phase modulation panel is configured to phase modulate light from the coherent light source and project the phase-modulated light onto the spatial light modulation panel. The spatial light modulation panel is configured to amplitude modulate the phase-modulated light according to image data, and The display controller is configured to receive image data of an image to be displayed by the display system, determine a desired light distribution to be projected onto the spatial light modulation panel by the holographic projector, and control the holographic projector to project the desired light distribution onto the spatial light modulation panel. 2. The display system of claim 1, wherein the desired light distribution varies slowly at the spatial light modulation panel. 3. The display system according to claim 1, wherein the desired light distribution is primarily of lower spatial frequency. 4. The display system of claim 1, wherein the desired light distribution is shielded such that the distribution is of a lower spatial frequency. 5. The display system of claim 1, wherein the phase-modulated light is shielded in the Fourier plane to attenuate the Fourier components corresponding to higher spatial frequencies. 6. The display system of claim 1, wherein the controller determines the desired light distribution by at least one of low-pass spatial filtering of image data, applying a blur filter to the image data, and calculating a local average or weighted average of local pixel groups in the image data. 7. The display system of claim 1, wherein the controller excites the spatial light modulation panel based on image data and phase-modulated light at the spatial light modulation panel. 8. The display system of claim 1, wherein the controller excites the spatial light modulation panel based on image data, phase-modulated light at the spatial light modulation panel, and a histogram including pixel brightness related to the image. 9. The display system of claim 1, wherein the controller is configured to determine the drive value for the pixels of the phase modulation panel by calculating the inverse Fourier transform of the desired light distribution. 10. The display system of claim 1, wherein the controller is configured to calculate an estimate of the actual light distribution at the spatial light modulation panel and use this estimate to set the pixels of the spatial light modulation panel to provide an image based on image data. 11. The display system of claim 1, wherein the pixel value of the spatial light modulation panel is set by comparing an estimated intensity of light to be incident on the pixel from the phase modulation panel with an intensity of light specified for the pixel by the image data, and setting the pixel of the spatial light modulator panel to reduce the intensity of the incident light to the intensity specified by the image data. 12. The display system of claim 1, wherein the pixel value of the spatial light modulation panel is set by dividing the image data of the pixel by an estimated intensity of incident light to be incident on the pixel from the phase modulation panel. 13. The display system of claim 1, wherein the pixel value of the spatial light modulation panel is set to reduce the intensity of light from the phase modulation panel at that pixel to match the intensity specified for that pixel by the image data. 14. The display system of claim 12, wherein the intensity of light from the phase modulation panel is calculated based on the inverse Fourier transform at the pixel. 15. The display system of claim 12, wherein the intensity of the light from the phase modulation panel is calculated based on a light field simulation of the light incident from the phase modulation panel onto the spatial light modulation panel. 16. The display system of claim 15, wherein the light field simulation comprises the inverse Fourier transform of the light field projected from the phase modulation panel. 17. The display system according to any one of claims 1 to 16, wherein the coherent light source comprises red, green, and blue lasers. 18. The display system according to any one of claims 1 to 16, wherein the image is displayed by time-multiplexing images of different colors. 19. The display system of claim 1, wherein the spatial resolution of the phase modulation panel exceeds the spatial resolution of the image data. 20. The display system of claim 1, wherein the number of controllable elements of the phase modulation panel is nine times or more the number of pixels in the image data. 21. The display system of claim 1, wherein the display controller is further configured to determine a Fourier-based hologram corresponding to the image data and to set the phase at different positions on the phase modulation panel according to the Fourier-based hologram.