WO2025122434A1

WO2025122434A1 - Display levels and calibration of display levels

Info

Publication number: WO2025122434A1
Application number: PCT/US2024/058111
Authority: WO
Inventors: Jerome D. Shields
Original assignee: Dolby Laboratories Licensing Corp
Current assignee: Dolby Laboratories Licensing Corp
Priority date: 2023-12-04
Filing date: 2024-12-02
Publication date: 2025-06-12
Anticipated expiration: 2026-06-04

Abstract

Systems and methods for calibrating display levels created using dual modulation digital micromirror devices. In one example, multi-modulation display system includes a light source, a first modulator including a first plurality of mirrors to modulate light from the light source, and a second modulator including a second plurality of mirrors to modulator light from the first modulator. A first image is captured while the first modulator is off and the second modulator is on, creating a first display level. A second image is captured while the first modulator is on and the second modulator is on, also creating the same first display level. A difference between the first image and the second image is determined, and control signals for controlling the second modulator may be adjusted based on the difference.

Description

DISPLAY LEVELS AND CALIBRATION OF DISPLAY LEVELS

BACKGROUND

1 . Cross-Reference to Related Applications

[0001] This application claims the benefit of priority from U.S. Provisional Application Ser. No. 63/605,841, filed on 4 December 2023, U.S. Provisional Application Ser. No. 63/556.167, filed on 21 February- 2024, and European Patent Application No. EP 24165364.1, filed on 21 March 2024, each of which is incorporated by reference herein in its entirety.

2. Field of the Disclosure

[0002] This application relates generally to bit sequences for dual modulation display systems and. particularly, to calibrating display levels created using dual modulation digital micromirror devices.

3. Description of Related Art

[0003] Digital projection systems ty pically utilize a light source and an optical system to project an image onto a surface or screen. The optical system includes components such as mirrors, lenses, waveguides, optical fibers, beam splitters, diffusers, digital micromirror devices (DMDs), spatial light modulators (SLMs), phase light modulators (PLMs), and the like. A bit sequencing pattern is used to control periods of time that the DMD micromirrors are on, and periods of time that the DMD micromirrors are off. Modulators may switch multiple times within a single frame. Image levels are specified by a video signal that has code values specifying the levels. For example, a video signal may encode image levels using perceptual quantizer (PQ) encoding such that a code value indicates a light level on the image.

BRIEF SUMMARY OF THE DISCLOSURE

[0004] Various aspects of the present disclosure relate to devices, systems, and methods for projection display. One example embodiment provides a method for calibrating a multimodulation display system. The multi-modulation display system includes a light source, a first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source, and a second modulator being illuminated by light from the first modulator, the second modulator including a second plurality- of mirrors to modulator light from the first modulator. The method includes sending first control signals to the first modulator such that the first plurality of mirrors are controlled to an off position during a first period of time, sending second control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the first period of time, and capturing a first image of the light modulated by the second modulator during the first period of time. The method includes sending third control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light during a second period of time, sending fourth control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the second period of time, and capturing a second image of the light modulated by the first modulator and the second modulator during the second period of time. The method includes determining a difference between the first image and the second image.

[0005] Another example embodiment provides a method for calibrating a multimodulation display system. The multi-modulation display system includes a light source, a first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source, and a second modulator being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulator light from the first modulator. The method includes sending first control signals to the first modulator such that the first plurality of mirrors are controlled to off positions during all sub-intervals in a first period of time, sending second control signals to the second modulator such that the second plurality of mirrors are controlled to on positions during a first fraction of the sub-intervals in the first period of time, and capturing a first image of the light modulated by the second modulator during the first period of time. The method includes sending third control signals to the first modulator such that the first plurality of mirrors are controlled to on positions during a second fraction of all sub-intervals in a second period of time, sending fourth control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during a third fraction of all the sub-intervals in the second period of time, and capturing a second image of the light modulated by the first modulator and the second modulator during the second period of time. The method includes determining a difference between the first image and the second image.

[0006] Another example embodiment provides an apparatus for calibrating a multimodulation display system, the multi-modulation display system including a light source, a first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source, and a second modulator being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulator light from the first modulator. The apparatus includes an electronic processor configured to perform operations including sending first control signals to the first modulator such that the first plurality of mirrors are controlled to off positions during all sub-intervals in a first period of time, sending second control signals to the second modulator such that the second plurality of mirrors are controlled to on positions during a first fraction of the sub-intervals in the first period of time, and capturing a first image of the light modulated by the second modulator during the first period of time. The operations include sending third control signals to the first modulator such that the first plurality of mirrors are controlled to on positions during a second fraction of all sub-intervals in a second period of time, sending fourth control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during a third fraction of all the sub-intervals in the second period of time, and capturing a second image of the light modulated by the first modulator and the second modulator during the second period of time. The operations include determining a difference between the first image and the second image.

[0007] Another example embodiment provides an apparatus for calibrating a multimodulation display system, the multi -modulation display system including a light source, a first modulator being illuminated by the light source, the first modulator including a first plurality' of mirrors to modulate light from the light source, and a second modulator being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulator light from the first modulator. The apparatus includes an electronic processor configured to perform operations including sending first control signals to the first modulator such that the first plurality of mirrors are controlled to off positions during all sub-intervals in a first period of time, sending second control signals to the second modulator such that the second plurality of mirrors are controlled to on positions during a first fraction of the sub-intervals in the first period of time, and capturing a first image of the light modulated by the second modulator during the first period of time. The operations include sending third control signals to the first modulator such that the first plurality of mirrors are controlled to on positions during a second fraction of all sub-intervals in a second period of time, sending fourth control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during a third fraction of all the sub-intervals in the second period of time, and capturing a second image of the light modulated by the first modulator and the second modulator during the second period of time. The operations include determining a difference between the first image and the second image.

[0008] Another example embodiment provides a non-transitory computer-readable storage medium recording a program of instructions that is executable by a device to perform the method including sending first control signals to the first modulator such that the first plurality of mirrors are controlled to an off position during a first period of time, sending second control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the first period of time, and capturing a first image of the light modulated by the second modulator during the first period of time. The method includes sending third control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light during a second period of time, sending fourth control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the second period of time, and capturing a second image of the light modulated by the first modulator and the second modulator during the second period of time. The method includes determining a difference between the first image and the second image.

[0009] Another example embodiment provides a non-transitory computer-readable storage medium recording a program of instructions that is executable by a device to perform the method including sending first control signals to the first modulator such that the first plurality of mirrors are controlled to off positions during all sub-intervals in a first period of time, sending second control signals to the second modulator such that the second plurality of mirrors are controlled to on positions during a first fraction of the sub-intervals in the first period of time, and capturing a first image of the light modulated by the second modulator during the first period of time. The method includes sending third control signals to the first modulator such that the first plurality of mirrors are controlled to on positions during a second fraction of all sub-intervals in a second period of time, sending fourth control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during a third fraction of all the sub-intervals in the second period of time, and capturing a second image of the light modulated by the first modulator and the second modulator during the second period of time. The method includes determining a difference between the first image and the second image. [0010] In this manner, various aspects of the present disclosure provide for the display of images having a high dynamic range and high resolution, and effect improvements in at least the technical fields of image projection, holography, signal processing, and the like.

DESCRIPTION OF THE DRAWINGS

[0011] These and other more detailed and specific features of various embodiments are more fully disclosed in the following description, reference being had to the accompanying drawings, in which:

[0012] FIG. 1 illustrates a block diagram of an exemplary image projector display system according to various aspects of the present disclosure.

[0013] FIG. 2A illustrates a plan view of an exemplary spatial light modulator for use with various aspects of the present disclosure.

[0014] FIG. 2B illustrates a cross-sectional view taken along the line I-B of FIG. 2A.

[0015] FIG. 3 illustrates a high-level diagram of a switching scheme that matches a bitsequence repeating pattern.

[0016] FIG. 4A illustrates an example bit sequence for a pre-modulator.

[0017] FIG. 4B illustrates an example bit sequence for a primary modulator.

[0018] FIG. 4C illustrates the transmittance resulting from the bit sequences of FIGS. 4A and 4B.

[0019] FIG. 5A illustrates another example bit sequence for a pre-modulator.

[0020] FIG. 5B illustrates another example bit sequence for a primary modulator.

[0021] FIG. 5C illustrates the transmittance resulting from the bit sequences of FIGS. 5A and 5B.

[0022] FIG. 6A illustrates another example bit sequence for a pre-modulator.

[0023] FIG. 6B illustrates another example bit sequence for a primary modulator.

[0024] FIG. 6C illustrates the transmittance resulting from the bit sequences of FIGS. 6A and 6B. [0025] FIG. 7 illustrates bit sequence levels for various PQ code values.

[0026] FIG. 8 illustrates ramp image levels for various PQ code values.

[0027] FIGS. 9A-9C illustrate example premodulator off-state levels for color channels normalized for a white image.

[0028] FIG. 10 illustrates a block diagram of an example method by the image projector display system of FIG. 1.

DETAILED DESCRIPTION

[0029] This disclosure and aspects thereof can be embodied in various forms, including hardware, devices, or circuits controlled by computer-implemented methods, computer program products, computer systems and networks, user interfaces, and application programming interfaces; as well as hardware-implemented methods, signal processing circuits, memory arrays, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), and the like. The foregoing summary' is intended solely to give a general idea of various aspects of the present disclosure, and does not limit the scope of the disclosure in any way.

[0030] In the following description, numerous details are set forth, such as optical device configurations, timings, operations, and the like, in order to provide an understanding of one or more aspects of the present disclosure. It will be readily apparent to one skilled in the art that these specific details are merely exemplary and not intended to limit the scope of this application.

[0031] Moreover, while the present disclosure focuses mainly on examples in which the various circuits are used in digital projection systems, it will be understood that this is merely one example of an implementation. It will further be understood that the disclosed systems and methods can be used in any device in which there is a need to project light; for example, cinema, consumer, and other commercial projection systems, heads-up displays, virtual reality' displays, and the like.

Projector Systems

[0032] FIG. 1 illustrates one possible embodiment of a suitable image proj ector display system. In the illustrated embodiment, the projector display system is constructed as a dual/multi-modulator projection system 100. The projection system 100 employs a light source 102 that supplies the projector system with a desired illumination such that a final projected image will be sufficiently bright for the intended viewers of the projected image. Light source 102 may comprise any suitable light source, such as, but not limited to, Xenon lamps, laser(s), coherent light sources, and partially-coherent light sources. Additionally, optical systems described herein may implement optical fibers to transfer light from the light source 102 to optics within the optical system.

[0033] Light 104 from the light source 102 may illuminate a first modulator 106 (for example, a pre-modulator) that may, in turn, illuminate a second modulator 110 (for example, a primary modulator) via a set of optional optical components 108. Light from the second modulator 110 may be projected by a projection lens 112 (or other suitable optical components) to form a final projected image upon a screen 114. In some instances, the projection lens 112 includes an optical filter for filtering the light from the second modulator 110, as described below in more detail. The first modulator 106 and the second modulator 110 may be controlled by a controller 116. The controller 116 may receive input image and/or video data and may perform certain image processing algorithms, gamut mapping algorithms or other such suitable processing upon the input image/video data and output control/data signals to the first modulator 106 and the second modulator 110 in order to achieve a desired final projected image on the screen 114. In addition, in some projector systems, it may be possible, depending on the light source, to modulate light source 102 (control line not shown) in order to achieve additional control of the image quality of the final projected image.

[0034] The controller 116 may include electrical components such as an electronic processor and a memory to perform methods and operations described herein. The electronic processor may be implemented as a microprocessor with a separate memory or as a microcontroller with memory on the same chip. The electronic processor may be implemented with multiple processors and may be implemented partially or entirely as, for example, a field-programmable gate array (FPGA) or an applications specific integrated circuit (ASIC). The memory of the controller 116 includes non-transitory, computer-readable memory that stores instructions that are received and executed by the electronic processor of the controller 116 to carry⁷ out the functionality⁷ of the projection system 100 described herein. The memory may include, for example, combinations of different ty pes of memory, such as read-only memory and random-access memory. [0035] Light recycling module 103 is depicted in FIG. 1 as a dotted box that may be placed in the light path from the light source 102 to the first modulator 106. It may be appreciated that light recycling may be inserted into the projector system at various points in the projector system. For example, light recycling may be placed between the first and second modulators. In addition, light recycling may be placed at more than one point in the optical path of the display system.

[0036] Although the embodiment of FIG. 1 depicts a single light channel, it will be appreciated that the first and second modulators may be replicated for each of a series of color channels within the projector such that each color channel includes two optically offset reflective modulators. The series of color channels may comprise a red channel, a green channel, and a blue channel. The light source may comprise, for example, a plurality of colored laser light sources. In one embodiment, the light sources may be modulated either globally (in brightness) and/or spatially (locally) dimmed according to signals (not shown) from a controller (e.g., 116).

[0037] The intermediate signals to the second modulator may be, for example, based on a light field simulation comprising a point spread function of light reflected by the first modulator and the offset. For example, the intermediate signals to the second modulator may be based on a point spread function of light reflected by the first modulator in each channel and the offset in each channel. The offset in the channels may be the same, or the offset of at least two channels is different and the intermediate signals to the second modulator in each channel is based on at least one of the offset and differences in offset between channels.

[0038] While the embodiment of FIG. 1 is presented in the context of a dual, multimodulation projection system, it should be appreciated that the techniques and methods of the present application will find application in other dual, multi-modulation display systems. For example, a dual modulation display system comprising a backlight, a first modulator (e.g., LCD or the like), and a second modulator (e.g., LCD or the like) may employ suitable optical components and image processing methods and techniques to affect the performance and efficiencies discussed herein in the context of the projection systems.

[0039] The first modulator 106 and the second modulator 1 10 may be configured as digital micromirror devices (DMDs) composed of a plurality of mirrors used to adjust the angle of incidence of light. To illustrate the effects of the angle of incidence and the DMD mirrors, FIGS. 2A-2B show an exemplary DMD 200 in accordance with various aspects of the present disclosure. In particular, FIG. 2A illustrates a plan view of the DMD 200, and FIG. 2B illustrates partial cross-sectional view of the DMD 200 taken along line I-B illustrated in FIG. 2A. The DMD 200 includes a plurality of square micromirrors 202 arranged in a two- dimensional rectangular array on a substrate 204. Each micromirror 202 may correspond to one pixel of the eventual projection image, and may be configured to tilt about a rotation axis 208, shown for one particular subset of the micromirrors 202, by electrostatic or other type of actuation. The individual micromirrors 202 have a width 212 and are arranged with gaps of width 210 therebetween. The micromirrors 202 may be formed of or coated with any highly reflective material, such as aluminum or silver, to thereby specularly reflect light. The gaps between the micromirrors 202 may be absorptive, such that input light which enters a gap is absorbed by the substrate 204.

[0040] While FIG. 2A expressly shows only some representative micromirrors 202, in practice the DMD 200 may include many more individual micromirrors in a number equal to a resolution of the projection system 100. In some examples, the resolution may be 2K (2048x 1080), 4K (4096x2160), 1080p (1920x 1080), consumer 4K (3840x2160). and the like. Moreover, in some examples the micromirrors 202 may be rectangular and arranged in the rectangular array; hexagonal and arranged in a hexagonal array, and the like. Moreover, while FIG. 2A illustrates the rotation axis 208 extending in an oblique direction, in some implementations the rotation axis 208 may extend vertically or horizontally.

[0041] As can be seen in FIG. 2B, each micromirror 202 may be connected to the substrate 204 by a yoke 214, which is rotatably connected to the micromirror 202. The substrate 204 includes a plurality of electrodes 216. While only two electrodes 216 per micromirror 202 are visible in the cross-sectional view of FIG. 2B, each micromirror 202 may in practice include additional electrodes. While not particularly illustrated in FIG. 2B, the DMD 200 may further include spacer layers, support layers, hinge components to control the height or orientation of the micromirror 202, and the like. The substrate 204 may include electronic circuitry associated with the DMD 200, such as complementary metal-oxide semiconductor (CMOS) transistors, memory' elements, and the like.

[0042] Depending on the particular operation and control of the electrodes 216, the individual micromirrors 202 may be switched between an “on” position, an “off’ position, and an unactuated or neutral position. If a micromirror 202 is in the on position, it is actuated to an angle of (for example) -12° (that is, rotated counterclockwise by 12° relative to the neutral position) to specularly reflect input light 206 into on-state light 218. If a micromirror 202 is in the off position, it is actuated to an angle of (for example) +12° (that is, rotated clockwise by 12° relative to the neutral position) to specularly reflect the input light 406 into off-state light 220. The off-state light 220 may be directed toward a light dump that absorbs the off-state light 220. In some instances, a micromirror 202 may be unactuated and lie parallel to the substrate 204. The particular angles illustrated in FIGS. 2A-2B and described here are merely exemplary and not limiting. In some implementations, the on- and off- position angles may be between ±11 and ±13 degrees (inclusive), respectively. In other implementations, the on- and off-position angles may be between ±10 and ±18 degrees (inclusive), respectively.

[0043] While examples described herein primarily refer to DMDs. the first modulator 106 and/or the second modulator 1 10 may be different suitable modulation devices, such as MEMS arrays or other devices including a plurality of analog mirrors or digital mirrors.

Bit Sequencing

[0044] Normally, a typical DMD employs a single bit sequence per frame to obtain a certain bit per pixel (e.g., 16 bit/pixel) modulation. '‘Bit sequence” as used herein refers to how the micromirrors 202 are turned on and off. A frame period is partitioned into bit-times, or bit-planes, such that pixels may be on or off for each bit-time. In many embodiments, the bit sequence may be modified such that the higher order bits are spread across the frame period, and therefore, they may be repeated multiple times. For example, in one embodiment, the top bits (e.g., the top 12 bits of 16 bits) may be repeated for each subframe. This would allow a pattern with the top 12 bits to repeat - e.g., 16 times (in a 1/16 subframe subdivision embodiment). The lower significant bits would remain unaffected (e.g., spread across the entire frame period).

[0045] In one embodiment, it may be possible to have the beam steering device (e.g., mirrors or other elements) - e.g., the first modulator 106 - switch quickly and/or at a desired rate (e.g., 10-100 microseconds). This may be desirable so that the first modulator 106 may switch in a “dark” time between sequence repeats. During such a dark time, the display system may not be outputting any light to be rendered and/or projected. This may help to avoid noticeable and/or undesired visual effects. The fraction of time a pixel is on determines the level that is made. Bit-times are short enough that the bit on-times are not seen distinctly, and the level seen is the average on-time for the frame period. The frame rate for bit- sequencing may be different than the video frame rate. For example, the bit-sequencing frame rate for a 24 fps video could be four times that rate, at 96 fps. Additionally, the number of levels that can be made by bit sequencing is limited by the portioning of the frame period into bit-times. The shortest duration bit-time determines the precision of the levels.

[0046] FIG. 3 depicts a high-level switching scheme 300 that matches the bit-sequence repeating pattern. Pattern 302 depicts the time periods (e.g., 302a and 302b) during which the bit sequence is repeating. Between these time periods is a period of dark time 303 between sequence repeats. As may be seen in pattern 304, the first modulator is able to switch multiple times (e.g., 304a, 304b, . .. , 304n) during the entire period and the dark time.

[0047] Dual modulation DMDs, such as the first modulator 106 and the second modulator 110, may use synchronized bit sequencing to make display levels. In some instances, the bit sequencing uses the same bit sequences for both the first modulator 106 and the second modulator 110 such that they are both on or off at the same time. In other instances, different bit sequences are used such that both the first modulator 106 and the second modulator 110 are on, both the first modulator 106 and the second modulator 110 are off, or only one of the first modulator 106 and the second modulator 110 are on at any time.

[0048] When the bit sequencing uses the same bit sequences for both the first modulator 106 and the second modulator 110 such that they are both on or off at the same time, the fraction of a frame time that both the first modulator 106 and the second modulator 110 are on determines the display level that is created. When both the first modulator 106 and the second modulator 110 are off, the contribution to the display level is non-zero, but is instead a very small display level that may be significant only when both the first modulator 106 and the second modulator 110 are off for the whole frame time. For example, let / be the display level and let both on time be the fraction of a frame time that both the first modulator 106 and the second modulator 110 are on. Let Y on be the display level when both the first modulator 106 and the second modulator 110 are on. Let Y off e the display level when a single modulator is off such that, with dual modulation, both the first modulator 106 and the second modulator 110 are off and create a display level of Y off. It should be understood that to simplify these examples, Y off is chosen to be the same for both modulators, and that Eq. 1 and Eq. 2 would be appropriately modified if Y off were different for the two modulators. Accordingly, the created display level for a particular bit sequence is provided by Equation 1: Y = both_on_time * Y_on + (1 — both_on_time) * Y_off² [Eq. 1]

[0049] The second term, (1 -both on time) *Y off. is likely insignificant compared to the first term, except for when establishing a black level when the first term is zero.

[0050] FIGS. 4A-4C illustrate an example bit sequence that partitions a frame into ten equal-sized sub-intervals. In other embodiments, the bit sequences may have significantly more sub-intervals or sub-intervals of different sizes. FIG. 4A provides the bit sequence for the first modulator 106 over the frame. FIG. 4B provides the bit sequence for the second modulator 110 over the frame. FIG. 4C provides the dual-modulation transmittance level over the frame. As referred to herein, transmittance level may be interpreted as normalized display level.

[0051] In the example of FIGS. 4A-4C, the same bit sequence is used for both the first modulator 106 and the second modulator 110. The value of Y on is 1 (full transmittance), and the value of Y off is 1/3000 (minimal transmittance). These values are merely examples. In particular, the Y off value may vary based on the contrast of the modulators (e.g., 1/contrast). For examples described herein, the first modulator 106 and the second modulator 110 have a contrast value of 3000: 1. The bit sequence has the first modulator 106 and the second modulator 110 on at the sub-intervals 3 and 8. Accordingly, the both on time value is 2/10. Solving for Equation 1 it is found that the display level is:

The second term (l-both_on_time) *Y off is a value visually insignificant compared to the first term, and can be ignored. Given that the bit sequence is partitioned into ten equal-sized sub-intervals, the display levels increase in tenths (0/10, 1/10, 2/10... 10/10) and only provide as many levels as the number of partitioned sub-intervals plus one. These display levels maybe referred to as "‘coarse levels”.

[0052] Due to the limitations of switching the micromirrors 202 on and off, both on time may be a value from a limited set of discrete values including zero and some smallest value. The smallest value and the precision of other small values may not be sufficient for making all levels needed for a display. The other needed values may be made using other complimentary schemes such as dithering. [0053] Additionally, bit sequencing may be modified to create additional levels. Rather than limiting the set of bit sequences to those having both the first modulator 106 and the second modulator 110 on or off at the same time, bit sequences may be included that have a single modulator on while the other modulator is off. For example, let (me an time be the fraction of a frame time that only one modulator is on. The display level Y for a particular bit sequence is then provided by Equation 2:

[0054] When both on time is low, the second term one on time *Y off is not insignificant compared to both on time *Y on. and one on time may be manipulated to create many more bit sequences that make many more display levels. This relationship between both on time *Y_on and one on time *T off may be considered a course level with a fine adjustment, where the fine adjustment is used to make more precise display levels than simply using the course level. The fine adjustment may henceforth be referred to as “fine levels”.

[0055] FIGS. 5A-5C illustrate another example bit sequence that partitions a frame into ten equal-sized sub-intervals. In other embodiments, the bit sequences may have significantly more sub-intervals or sub-intervals of different sizes. FIG. 5A provides the bit sequence for the first modulator 106 over the frame. FIG. 5B provides the bit sequence for the second modulator 110 over the frame. FIG. 5C provides the transmittance level over the frame.

[0056] In the example of FIGS. 5A-5C, a different bit sequence is used for the first modulator 106 than the second modulator 110. The value of Y on is 1, and the value of Y off is 1/3000. A first bit sequence has the first modulator 106 on from 2.5 to 3.5 and 7.5 to 8.5 (or on at the sub-intervals 3 and 8). A second bit sequence has the second modulator 110 on at the sub-intervals 3, 5, 6, and 8. Accordingly, the both on time is 2/10 and the one on time is 2/10. Solving for Equation 2, it is found that the display level is:

[0057] In this example, both the second term one_on_time*Y_off and the third term l- bo^ on time-one one time) *Y_off are visually insignificant compared to the first term both on time *Y_on. [0058] These fine adjustments have additional benefits when the desired display levels are less than the display level made by both on time set at its minimum non-zero discrete level. To make these lowest display levels with the fine adjustment, both on time is set to zero and (me an time is manipulated to create smaller display levels. Fine adjustments may also make other needed levels such as those between both on time first and second non-zero levels, or between other greater but low coarse levels. As coarse levels increase, their precision approaches that needed for a display, and the utility of a fine adjustment diminishes.

[0059] FIGS. 6A-6C illustrate another example bit sequence that partitions a frame into ten equal-sized sub-inten als. In other embodiments, the bit sequences may have significantly more sub-intervals or sub-intervals of different sizes. FIG. 6A provides the bit sequence for the first modulator 106 over the frame. FIG. 6B provides the bit sequence for the second modulator 110 over the frame. FIG. 6C provides the transmittance level over the frame.

[0060] In the example of FIGS. 6A-6C, a different bit sequence is used for the first modulator 106 than the second modulator 110. The value of Y on is 1, and the value of Y off is 1/3000. A first bit sequence maintains the first modulator 106 off, and accordingly both on time is set to zero. A second bit sequence has the second modulator 1 10 on at the sub-intervals 5 and 6. Accordingly, one on time is 2/10. Solving for Equation 2, it is found that the display level is:

[0061] In this example, as both on time*Y on is zero, the second term one on time*Y off is visually significant, creating a useful dark level. With the first modulator 106 off for the entire frame, the second modulator 110 may be manipulated to create eleven useful dark discrete display levels.

[0062] The eleven dark levels are created by manipulating the second modulator 110 with the first modulator 106 being maintained off. These fine adjustment levels are visually significant but are much less than the first non-zero coarse level when both the first modulator 106 and the second modulator 110 are on, even if the modulators are on for only one subinterval. The eleven fine adjustment levels alone do not completely fill the gap between coarse level zero and the first non-zero level. To have the fine adjustment fill-in the gaps between coarse levels, the coarse levels may be spaced more closely together. [0063] As one example, consider a bit sequence for a display that is one-bit sequencing method among many. Let the bit sequence be similar to that as previously shown in FIGS. 4A-4C. 5A-5C. and 6A-6C, but the time frame is portioned into 2¹² sub-intervals such that the coarse levels that are made correspond to the values of a 12-bit number. Accordingly, the smallest coarse level has a value of l/(2¹²). Another example may make the same 2¹² levels or other similarly precise levels by using a coarser bit sequencing method complimented by other methods such as dithering. Dithering is intentionally applied noise used to randomize quantization error and may also be used to create more display levels. While embodiments described herein primarily refer to the use of DMDs, the use of dithering is not restricted to displays using DMDs as dithering operates on frames made by any display technology. Dithering utilizes discrete levels that are created by some other means, such as the previously described bit sequencing for DMDs. Dithering alternately shows those discrete levels such that their average over time is an in-between level, providing additional precision to the discrete levels created by bit sequencing.

[0064] FIG. 7 shows perceptual quantizer (PQ) signal inputs for the display levels F (in nits) that should be displayed, and the coarse discrete levels that would be displayed by the 12-bit bit sequencing method if the Y value corresponding to the maximum 12-bit number were 108 nits. In the example of FIG. 7, the lowest 200 PQ values are shown. The number of discrete PQ levels is much greater than the number of coarse 12-bit linear levels (indicated by “X’^?s in FIG. 7). Assuming the same Y off\eNQ\ of the previous examples, the maximum fine adjustment that can be made is sufficient to fill in the gaps betw een all the coarse levels, and the number of fine levels is sufficient to make all PQ levels. By controlling only one modulator during the off time of the other, nearly 2¹² fine levels are added to each coarse level. If the second modulator 110 (e.g.. the primary modulator) is controlled on for the entire time the first modulator 106 (e.g., the pre-modulator) is maintained off, a maximum fine adjustment level is achieved (indicated by “O”s in FIG. 7). If the maximum fine level added to the coarse level is greater than the next greater coarse level, then the gaps between coarse levels can be filled to make all PQ levels.

[0065] For dual-modulation DMD display, such as that provided by projection system 100. light from the first modulator 106 is cast onto the second modulator 110, but not at pixel -to- pixel precision. Rather, light from a pixel of the first modulator 106 (e.g., a premodulator pixel) is cast onto an adjacent group of pixels of the second modulator 110 (e.g., primary pixels). When making a pixel level with both the first modulator 106 and the second modulator 110 on or off at the same, there are existing methods to reduce errors from this condition. However, when making a pixel level with the first modulator 106 off and the second modulator 110 on, an adjacent pixel with the first modulator 106 on may cast light onto the primary pixel that relies on the first modulator 106 being off, corrupting the expected level. This condition may occur whenever pixel levels are being made with the first modulator 106 off, particularly when creating dark levels that must be created only with the first modulator 106 off.

[0066] To avoid this corruption, the frame time may be partitioned into two intervals, one for coarse levels created with both the first modulator 106 and the second modulator 110 on or off at the same time, and another for fine levels made with only either the first modulator 106 or the second modulator 110 on. These two intervals may be further divided into subintervals for two bit sequences, one sequence for the coarse levels and another sequence for the fine levels. During the fine levels frame interval, all premodulator pixels are off, preventing any premodulator pixel from casting light onto a primary' pixel that relies on the premodulator pixel being off. The fine levels frame interval is henceforth referred to as the premodulator off time.

[0067] Dedicating a fraction of the frame interval as a premodulator off time provides for making fine levels while preventing some spatial artifacts. How ever, during the premodulator off time, the peak brightness of the display is reduced by the fraction of the frame time that is the premodulator off time. Accordingly, the duration of the premodulator off time may be limited to achieve the necessary level precision while maintaining a sufficient peak brightness level.

Image Artifacts and Calibration

[0068] As noted, the first modulator 106 and the second modulator 110 include components (e.g., the micromirrors 202) that are manipulated by controller 116 that transmits signal code values to create the correct image levels. How ever, inaccurate display levels may result in visible image artifacts. For example, consider a ramp image displayed by the projection system 100. A ramp image includes levels that gradually increase over the displayed image moving from one end of the image to another end of the image. FIG. 8 illustrates an example range of display levels shown in a ramp image for provided PQ code values. When the display levels are accurately shown, the ramp image appears to change brightness smoothly, as shown by the function with solid circles. However, discontinuities are possible within the projection system 100, resulting in artifacts and discontinuities within the ramp image where the change in display levels are no longer smooth, as shown by the function with hollow circles. Particularly, the levels of the lower part of the range may be made incorrectly, producing a discontinuity where the lower range (created when the first modulator 106 is always off) meets the upper range (created when the first modulator 106 may be on for particular sub-frame intervals).

[0069] For example, a first set of display levels may be generated by maintaining the first modulator 106 off and only operating the second modulator 1 10 to modulate light. The second set of display levels is then generated by operating both the first modulator 106 and the second modulator 110 to modulate light. When the first modulator 106 is off to generate the first set of display levels, the levels are much less predictable than when the first modulator 106 is on and modulating light. The first modulator off level, henceforth referred to as the premodulator floor level or premodulator floor image, may vary over the image frame and over different displays. Accordingly, the display level at which the low range (e.g., the first modulator 106 being off) and the high range (e.g., the first modulator 106 being on) of display levels meet, henceforth referred to as the match display level, may be inconsistent and difficult to match, as shown by jump 802 in FIG. 8. To account for these variations, the first modulator 106 and the second modulator 110 may be calibrated to make the low range of levels meet the higher range of levels at the match display level without a discontinuity. Calibration may identify a parameter that alters operation of the second modulator 110 to ensure the match level is the same both when the first modulator 106 is off and when the first modulator 106 is on.

[0070] To calibrate the projection system 100, the controller 116 using the unadjusted parameter creates nearly the same image both with (i) the first modulator 106 off and (ii) with the first modulator 106 on. The images may be, for example, flat gray images at the match level. A camera is used to capture the two images, and the pictures are compared to relate the two images and adjust the parameter. The first picture is of the level of the top of the lower range of levels created with the first modulator 106 off. This may, as an example, be achieved by turning the first modulator 106 off for an entire frame or for the entire duration of capturing the first picture. The second picture is of the same display level at the bottom of the higher range of levels created with the first modulator 106 on. In some embodiments, the first picture may be captured while the projection system 100 is displaying a first calibration image (e.g., an image of a level near the top of the lower range of levels created with the first modulator 106 off) and the second picture may be captured while the projection system 100 is displaying a second calibration image (e.g., an image of a level near the bottom of the higher range of levels created with the first modulator 106 on). As an example, the first calibration image may be displayed by controlling the first modulator 106 with bit sequences where the first modulator 106 is turned off for each sub-interval of a first series of frames and by controlling the second modulator 110 with bit sequences where the second modulator 110 is turned on for a first fraction of sub-intervals over the first series of frames. The first fraction may, in some examples, be determined such that the display level of the first calibration image is at or just above the bottom of the higher range of levels created with the first modulator 106 on (as the levels with first modulator 106 on versus off should preferably match or overlap rather than have a gap). As one non-limiting example, the first fraction at which the second modulator 110 is turned on may be 100% of the sub-intervals over the first series of frames. Further, the second calibration image may be displayed by controlling the first modulator 106 with bit sequences where the first modulator 106 is turned on for a second fraction of sub-intervals over a second series of frames and by controlling the second modulator 110 with bit sequences where the second modulator 110 is turned on for a third fraction of sub-intervals over the second series of frames. As one non-limiting example, the second fraction at which the first modulator 106 is turned on may be 1% of the sub-intervals over the second series of frames. In such an instance, the second modulator 110 may be modulated with an image that is based on an estimate of the premodulator floor, thereby attempting to create a second calibration image that is nearly the same as the first. The second fraction, the third fraction, and the extent to which the second and third fractions are overlapping versus non-overlapping may, in some examples, be determined such that the display level of the second calibration image is at or just below the top of the lower range of levels created with the first modulator 106 off (as the levels with first modulator 106 on versus off should preferably match or overlap rather than have a gap). In this example, overlapping refers to sub-intervals where both modulators are on (such as sub-interval 3 of FIGS. 5A-5C) while non-overlapping refers to sub-intervals where only one modulator is on (such as in sub-intervals 5 and 6 of FIGS. 5A-5C and as in sub-intervals 5 and 6 of FIGS. 6A- 6C). It may be noted that for this calibration image the partial level made with one modulator on is visually insignificant compared to the partial level made with both modulators on. Additionally, it should be noted that the first fraction of the first calibration image and the second fraction, the third fraction, and the extent to which the second and third fractions are overlapping versus non-overlapping for the second calibration image may vary spatially across the first and second modulators in order to reduce or eliminate spatial variations in the match between the first and second calibration images (see, e.g., FIGS. 9A-9B). The two pictures are compared to create (or in situations the parameter is already created, improve) the parameter that alters operation of the second modulator 110 and/or the first modulator 106. In some examples, the first fraction of the first calibration image and/or the second fraction, the third fraction, and the extent to which the second and third fractions are overlapping versus non-overlapping for the second calibration image may be iteratively adjusted until the first and second calibration images match to a desirable extent. It should be understood that the two images displayed may not be the example image described, and if so, their values are considered when creating or improving the parameter.

[0071] In some instances, the two pictures are converted to linear luminance (Y) values and the ratio of the two pictures is used to scale the initial estimate of the premodulator floor image to produce anew estimate of the premodulator floor image. This estimate of the parameter may be used to make the lower range of levels meet with the upper range without discontinuity. Since the same camera and display are used for both pictures, any distortion caused by the camera itself or by the display other than the inaccurate estimate of the parameter is ignored.

[0072] As one example, FIG. 9A illustrates a red lightfield, FIG. 9B illustrates a green light!! eld, and FIG. 9C illustrates a blue lightfield that form an estimate of the premodulator floor image. The lightfields of FIGS. 9A-9C represent a brightest display level that can be created by maintaining the first modulator 106 off and only controlling the second modulator 110. In the example of FIGS. 9A-9C, the second modulator is controlled using the estimate to display a flat gray image that has a value of 1 x 1 O’⁴ over the entire image frame. The first picture is of this displayed image. When the second picture is captured where the first modulator 106 is controlled on during at least a sub-frame interval to attempt to create the same display image, the pictures may be different. The parameter is created or improved by determining the difference between the image in the first picture and the image in the second picture. In some instances, the difference is a ratio of the first picture and the second picture. In the example described above, the parameter is equal to the hghtfields of FIGS. 9A-9C and is improved by scaling by the ratio of the pictures.

[0073] The parameter is used to adjust control signals sent to the second modulator 110 when the display creates any level in the lower range of levels made with the first modulator 106 off. It may also be used when the first modulator 106 is nearly off when the premodulator floor image is visually significant relative to the level made by the first modulator 106 when it is slightly on.

[0074] In some instances, the calibration is performed multiple times (e.g., iteratively) to tune the parameter. For example, once the parameter is applied to control signals for the second modulator 110, the calibration is performed again. Any further differences between the two pictures are identified and used to update the parameter. This process is repeated until there is no difference between the two pictures.

[0075] The calibration process may be repeated with the roles of the first modulator 106 and the second modulator 110 interchanged. The result may be a different parameter that is the second modulator floor image, and it could be represented by a value as in FIGS. 9A-9C. The second modulator floor image is useful when the second modulator 110 is turned off or nearly off. As an example, assume the second modulator floor image is the same as the premodulator floor image as shown in FIGS. 9A-9C, and consider the display raw black image made with both modulators 106, 110 off. The raw black image would be the product of the premodulator floor image and the second modulator floor image. Since the premodulator floor image and the second modulator floor image are the same in this example, the raw black image would have a value that is the square of the value shown in FIGS. 9A- 9C. If the raw black image were used as the display black image, it would undesirably not be flat in luminance and would have a chromatic non-uniformity. For this example, a display black level could be defined to be at (3x1 O'⁴)². It would be everywhere over the image frame greater than the raw black image and the lowest level the display w ould allows under controlled conditions. Using the first and second parameters, and with the first modulator 106 off. the second modulator 110 may be controlled to turn on slightly to make the display black image.

[0076] The calibration process may also be used to improve a parameter that is not a modulator floor image. One image can be made using an estimate of the parameter and another reference image made without the parameter. The two images may be the same or have some known difference. The two images can be captured and compared to improve the parameter. The parameter is isolated as the cause of the difference between the two images. For example, the parameter could be a PSF model used to make a lightfi eld estimate. The first image would use the PSF model, and the second would not. The difference of pictures of the images would be due to errors in the estimate of the PSF.

[0077] FIG. 10 illustrates a block diagram of an example method 1000 performed by the projection system 100 of FIG. 1. The method 1000 may be performed, for example, by the controller 116. The steps provided within FIG. 10 are merely examples, and may instead be conducted in a different order or simultaneously.

[0078] At step 1002, the controller 116 sends (e.g.. transmits) first control signals to the first modulator 106 during a first period of time. The first control signals may control the first modulator 106 such that the plurality of micromirrors 202 of the first modulator 106 are controlled to an off position during the first period of time.

[0079] At step 1004, the controller 116 sends second control signals to the second modulator 110 during the first period of time. The second control signals may control the second modulator 110 such that the plurality of micromirrors 202 of the second modulator 110 are controlled to modulate light (e.g., to an on position) during the first period of time.

[0080] At step 1006, the controller 116 captures a first image of the light modulated by the second modulator 110 during the first period of time. For example, the controller 116 controls a camera to capture an image projected by the projection lens 112 onto the screen 114 during the first period of time. In some embodiments, the controller 116 receives an image of the light modulated by the second modulator 110 as an input from a separate, external device communicatively connected to the controller 116.

[0081] At step 1008, the controller 116 sends third control signals to the first modulator 106 during a second period of time. The third control signals may control the first modulator 106 such that the plurality of micromirrors 202 of the first modulator 106 are controlled to modulate light during a second period of time.

[0082] At step 1010, the controller 116 sends fourth control signals to the second modulator 110 during the second period of time. The fourth control signals ma control the second modulator 110 such that the plurality of micromirrors 202 of the second modulator 110 are controlled to module light during the second period of time.

[0083] At step 1012, the controller 116 captures a second image of the light modulated by both the first modulator 106 and the second modulator 110 during the second period of time. For example, the controller 116 controls a camera to capture an image projected by the projection lens 112 onto the screen 114 during the second period of time. In some embodiments, the controller 116 receives an image of the light modulated by the second modulator 1 10 as an input from a separate, external device communicatively connected to the controller 116.

[0084] At step 1014, the controller 116 determines a difference between the first image and the second image. For example, the first image and the second image are compared to generate a parameter used to alter operation of the second modulator 1 10, as previously described. In some instances, the difference between the first image and the second image is then used to adjust operation of the projection system 100, such as adjusting the second control signals and/or the fourth control signals for controlling the second modulator 110, applying a correction to a PSF model (as described with respect to FIGS. 9A-9C), adjusting a stored parameter value used by the projection system 100, or the like.

[0085] Systems, methods, and devices in accordance with the present disclosure may take any one or more of the following configurations.

[0086] (1) A method for calibrating a multi-modulation display system, the multimodulation display system including a light source, a first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source, and a second modulator being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulate light from the first modulator, wherein the method comprises: sending first control signals to the first modulator such that the first plurality⁷ of mirrors are controlled to an off position during a first period of time; sending second control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the first period of time; capturing a first image of the light modulated by the second modulator during the first period of time; sending third control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light during a second period of time: sending fourth control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the second period of time; capturing a second image of the light modulated by the first modulator and the second modulator during the second period of time; and determining a difference between the first image and the second image. [0087] (2) The method according to (1), wherein the first period of time includes a first plurality of sub-intervals forming a first series of frames, wherein, during the first period of time, the first plurality of mirrors are controlled to the off position for each sub-interval of the first series of frames, and wherein, during the first period of time, the second plurality of mirrors are controlled to modulate light for a first fraction of sub-intervals of the first series of frames.

[0088] (3) The method according to (2), wherein the second period of time includes a second plurality of sub-intervals forming a second series of frames, wherein, during the second period of time, the first plurality of mirrors are controlled to modulate light for a second fraction of sub-intervals of the second series of frames, and wherein, during the second period of time, the second plurality of mirrors are controlled to modulate light for a third fraction of sub-intervals of the second series of frames.

[0089] (4) The method according to (3), wherein the second fraction of sub-intervals and the third fraction of sub-intervals are overlapping such that both the first plurality of mirrors and the second plurality of mirrors are controlled to modulate light for at least one subinterval of the second series of frames.

[0090] (5) The method according to any one of (1) to (4), further comprising converting the first image and the second image to linear luminance values.

[0091] (6) The method according to any one of (1) to (5), wherein the first image and the second image are captured using a same camera.

[0092] (7) The method according to any one of (1) to (6), wherein the first image and the second image are related by a known difference.

[0093] (8) The method according to any one of ( 1) to (7), wherein the second control signals cause the second modulator to perform a dithering operation.

[0094] (9) The method according to any one of (1) to (8), wherein the first modulator is one selected from the group consisting of a DMD. a SLM, and a PLM.

[0095] (10) The method according to any one of (1) to (9). further comprising adjusting the second control signals based on the difference. [0096] (11) The method according to (10), wherein adjusting the second control signals includes applying a correction to a PSF model.

[0097] (12) The method according to any one of (1) to (11), further comprising adjusting at least one stored parameter value used by the multi-modulation display system during projection of images based on the determined differences between the first image and the second image.

[0098] (13) A method for calibrating a multi-modulation display system, the multimodulation display system including a light source, a first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source, and a second modulator being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulate light from the first modulator, wherein the method comprises: sending first control signals to the first modulator such that the first plurality of mirrors are controlled to off positions during all sub-intervals in a first period of time; sending second control signals to the second modulator such that the second plurality of mirrors are controlled to on positions during a first fraction of the subintervals in the first period of time; capturing a first image of the light modulated by the second modulator during the first period of time; sending third control signals to the first modulator such that the first plurality of mirrors are controlled to on positions during a second fraction of all sub-intervals in a second period of time; sending fourth control signals to the second modulator such that the second plurality of mirrors are controlled to on positions during a third fraction of all the sub-intervals in the second period of time; capturing a second image of the light modulated by the first modulator and the second modulator during the second period of time; and determining differences between the first image and the second image.

[0099] ( 14) The method according to (13), wherein the second fraction of sub-intervals and the third fraction of sub-intervals are overlapping such that both the first plurality of mirrors and the second plurality of mirrors are controlled to modulate light for at least one subinterval in the second period of time.

[00100] ( 15) The method according to any one of (13) to (14), further comprising converting the first image and the second image to linear luminance values. [00101] (16) The method according to any one of (13) to (15), wherein the first image and the second image are captured using a same camera.

[00102] (17) The method according to any one of (13) to (16), wherein the first image and the second image are related by a known difference.

[00103] (18) The method according to any one of (13) to (17), wherein the second control signals cause the second modulator to perform a dithering operation.

[00104] (19) The method according to any one of (13) to (18), wherein the first modulator is one selected from the group consisting of a DMD, a SLM, and a PLM.

[00105] (20) The method according to any one of (13) to (19), further comprising adjusting the second control signals based on the difference.

[00106] (21 ) The method according to any one of (13) to (20), wherein adjusting the second control signals includes applying a correction to a PSF model.

[00107] (22) The method according to any one of (13) to (21), further comprising adjusting at least one stored parameter value used by the multi-modulation display system during projection of images based on the determined differences between the first image and the second image.

[00108] (23) An apparatus for calibrating a multi-modulation display system, the multimodulation display system including a light source, a first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source, and a second modulator being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulator light from the first modulator, the apparatus comprising: an electronic processor configured to perform operations including the method according to any one of (1) to (22).

[00109] (24) A non-transitory computer-readable storage medium recording a program of instructions that is executable by a device to perform the method according to any one of (1) to (22).

[00110] (25) A computer-readable medium comprising a program of instructions that is executable by a device to perform the method according to any one of (1) to (22). [00111] (26) A computer program product comprising instructions that are executable by a device to perform the method according to any one of (1) to (22).

[00112] With regard to the processes, systems, methods, heuristics, etc. described herein, it should be understood that, although the steps of such processes, etc. have been described as occurring according to a certain ordered sequence, such processes could be practiced with the described steps performed in an order other than the order described herein. It further should be understood that certain steps could be performed simultaneously, that other steps could be added, or that certain steps described herein could be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating certain embodiments, and should in no way be construed so as to limit the claims.

[00113] Accordingly, it is to be understood that the above description is intended to be illustrative and not restrictive. Many embodiments and applications other than the examples provided would be apparent upon reading the above description. The scope should be determined, not with reference to the above description, but should instead be determined with reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is anticipated and intended that future developments will occur in the technologies discussed herein, and that the disclosed systems and methods will be incorporated into such future embodiments. In sum, it should be understood that the application is capable of modification and variation.

[00114] All terms used in the claims are intended to be given their broadest reasonable constructions and their ordinary meanings as understood by those knowledgeable in the technologies described herein unless an explicit indication to the contrary' in made herein. In particular, use of the singular articles such as “a,” “the,” “said,” etc. should be read to recite one or more of the indicated elements unless a claim recites an explicit limitation to the contrary.

[00115] The Abstract of the Disclosure is provided to allow the reader to quickly ascertain the nature of the technical disclosure. It is submitted with the understanding that it will not be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing Detailed Description, it can be seen that various features are grouped together in various embodiments for the purpose of streamlining the disclosure. This method of disclosure is not to be interpreted as reflecting an intention that the claimed embodiments incorporate more features than are expressly recited in each claim. Rather, as the following claims reflect, inventive subject matter lies in less than all features of a single disclosed embodiment. Thus, the following claims are hereby incorporated into the Detailed Description, with each claim standing on its own as a separately claimed subject matter.

Claims

1. A method for calibrating a multi -modulation display system, the multi-modulation display system including a light source, a first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source, and a second modulator being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulate light from the first modulator, wherein the method comprises: sending first control signals to the first modulator such that the first plurality of mirrors are controlled to an off position during a first period of time; sending second control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the first period of time; capturing a first image of the light modulated by the second modulator during the first period of time; sending third control signals to the first modulator such that the first plurality of mirrors are controlled to modulate light during a second period of time; sending fourth control signals to the second modulator such that the second plurality of mirrors are controlled to modulate light during the second period of time; capturing a second image of the light modulated by the first modulator and the second modulator during the second period of time; and determining a difference between the first image and the second image.

2. The method of claim 1, wherein the first period of time includes a first plurality' of sub-intervals forming a first series of frames, wherein, during the first period of time, the first plurality of mirrors are controlled to the off position for each sub-interval of the first series of frames, and wherein, during the first period of time, the second plurality of mirrors are controlled to modulate light for a first fraction of sub-intervals of the first series of frames.

3. The method of claim 2. wherein the second period of time includes a second plurality of sub-intervals forming a second series of frames, wherein, during the second period of time, the first plurality of mirrors are controlled to modulate light for a second fraction of subintervals of the second series of frames, and wherein, during the second period of time, the second plurality of mirrors are controlled to modulate light for a third fraction of sub-intervals of the second series of frames.

4. The method of claim 3. wherein the second fraction of sub-intervals and the third fraction of sub-intervals are overlapping such that both the first plurality of mirrors and the second plurality of mirrors are controlled to modulate light for at least one sub-interval of the second series of frames.

5. The method of any one of claims 1 to 4, further comprising converting the first image and the second image to linear luminance values.

6. The method of any one of claims 1 to 5, wherein the first image and the second image are captured using a same camera.

7. The method of any one of claims 1 to 6. wherein the first image and the second image are related by a known difference.

8. The method of any one of claims 1 to 7, wherein the second control signals cause the second modulator to perform a dithering operation.

9. The method of any one of claims 1 to 8. w herein the first modulator is one selected from the group consisting of a DMD, a SLM, and a PLM.

10. The method of any one of claims 1 to 9, further comprising adjusting the second control signals based on the difference.

11. The method of claim 10, wherein adjusting the second control signals includes applying a correction to a PSF model.

12. The method of any one of claims 1 to 11, further comprising adjusting at least one stored parameter value used by the multi-modulation display system during projection of images based on the determined differences between the first image and the second image.

13. A method for calibrating a multi-modulation display system, the multi-modulation display system including a light source, a first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source, and a second modulator being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulate light from the first modulator, wherein the method comprises: sending first control signals to the first modulator such that the first plurality of mirrors are controlled to off positions during all sub-intervals in a first period of time; sending second control signals to the second modulator such that the second plurality of mirrors are controlled to on positions during a first fraction of the sub-intervals in the first period of time; capturing a first image of the light modulated by the second modulator during the first period of time; sending third control signals to the first modulator such that the first plurality of mirrors are controlled to on positions during a second fraction of all sub-intervals in a second period of time; sending fourth control signals to the second modulator such that the second plurality⁷ of mirrors are controlled to on positions during a third fraction of all the sub-intervals in the second period of time; capturing a second image of the light modulated by the first modulator and the second modulator during the second period of time; and determining differences between the first image and the second image.

14. The method of claim 13, wherein the second fraction of sub-intervals and the third fraction of sub-intervals are overlapping such that both the first plurality of mirrors and the second plurality⁷ of mirrors are controlled to modulate light for at least one sub-interval in the second period of time.

15. The method of claim 13 or claim 14, further comprising converting the first image and the second image to linear luminance values.

16. The method of any one of claims 13 to 15. wherein the first image and the second image are captured using a same camera.

17. The method of any one of claims 13 to 16, wherein the first image and the second image are related by a known difference.

18. The method of any one of claims 13 to 17, wherein the second control signals cause the second modulator to perform a dithering operation.

19. The method of any one of claims 13 to 18. wherein the first modulator is one selected from the group consisting of a DMD, a SLM, and a PLM.

20. The method of any one of claims 13 to 19. further comprising adjusting the second control signals based on the difference.

21. The method of claim 20, wherein adjusting the second control signals includes applying a correction to a PSF model.

22. The method of any one of claims 13 to 21, further comprising adjusting at least one stored parameter value used by the multi -modulation display system during projection of images based on the determined differences between the first image and the second image.

23. An apparatus for calibrating a multi-modulation display system, the multi-modulation display system including a light source, a first modulator being illuminated by the light source, the first modulator including a first plurality of mirrors to modulate light from the light source, and a second modulator being illuminated by light from the first modulator, the second modulator including a second plurality of mirrors to modulator light from the first modulator, the apparatus comprising: an electronic processor configured to perform operations including the method of any one of claims 1 to 22.