RU2844349C2 - System and method of projection with dynamic target geometry - Google Patents

Abstract

FIELD: physics.

SUBSTANCE: invention relates to projection systems and to methods of controlling a projection system. Projection system includes a light source configured to emit light in response to image data; a phase light modulator configured to receive light from the light source and apply spatially varying phase modulation to the light to thereby adjust the light and generate projecting light; and a controller configured to dynamically determine, based on one of the user input or the sensor signal, the target geometry of the projection surface onto which the projecting light is projected, determine, based on the target geometry, a phase configuration for a frame of image data and provide a phase control signal to a phase light modulator, wherein the phase control signal is configured to instruct the phase light modulator to generate projecting light in accordance with the phase configuration for the frame.

EFFECT: images projection on surface or surfaces with dynamic geometry.

26 cl, 7 dwg

Description

Cross-references to related applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63/144,027 and European Patent Application No. 21154553.8, both filed February 1, 2021, each of which is incorporated herein by reference in its entirety.

State of the art

1. Field of technology to which the invention relates

[0002] This application generally relates to projection systems and to methods for controlling a projection system.

2. Description of the prior art

[0003] Digital projection systems typically use a light source and an optical system to project an image onto a surface or screen. The optical system includes components such as mirrors, lenses, waveguides, optical fibers, beam splitters, diffusers, spatial light modulators (SLMs), etc. Some projection systems are based on SLMs that implement spatial amplitude modulation. In this system, the light source provides a light field that produces the brightest level that can be reproduced in an image, and the light is attenuated (e.g., rejected) to create the desired scene levels. In such a configuration, light that is not projected to make up any part of the image is attenuated or rejected. The light to be projected passes through a lens on one side of the projection system, which is aligned with a predetermined area of the screen by adjusting the elements of the projection system (e.g. the position of the system itself, the configuration of the various lens optics, etc.) to ensure that the screen is perpendicular to the optical axis of the projector and that the projected image is in focus.

[0004] In this regard, comparative digital projection systems require a static screen geometry. If the screen geometry changes (for example, if the position or orientation of the projection system changes, either intentionally or unintentionally), the image may change.

Brief summary of the invention

[0005] Various aspects of the present disclosure relate to circuits, systems, and methods for a projection display using phase modulation of light to provide projection of images onto a surface or surfaces with dynamic geometry.

[0006] In one exemplary aspect of the present disclosure, a projection system is provided, comprising a light source configured to emit light in response to image data; a phase light modulator configured to receive light from the light source and apply spatially varying phase modulation to the light to thereby adjust the light and generate projection light; and a controller configured to dynamically determine, based on one of a user input or a sensor signal, a target geometry of a projection surface onto which the projection light is projected, determine, based on the target geometry, a phase configuration for a frame of image data, and provide a phase control signal to the phase light modulator, wherein the phase control signal is configured to cause the phase light modulator to generate projection light in accordance with the phase configuration for the frame.

[0007] In another exemplary aspect of the present disclosure, a projection method is provided, comprising emitting light by a light source in response to image data; receiving the light by a phase light modulator; applying a spatially varying phase modulation to the light by the phase light modulator to thereby adjust the light and generate projection light; dynamically determining, based on one of a user input or a sensor signal, a target geometry of a projection surface onto which the projection light is projected; determining, based on the target geometry, a phase configuration for a frame of image data; and providing a phase control signal to the phase light modulator, wherein the phase control signal is configured to cause the phase light modulator to generate projection light in accordance with the phase configuration for the frame.

Brief description of the drawings

[0008] These and other more detailed and specific features of various embodiments are more fully disclosed in the following description, with reference to the accompanying drawings, in which:

[0009] Fig. 1 illustrates a block diagram of an exemplary projection system in accordance with various aspects of the present disclosure;

[0010] Fig. 2 illustrates an example phase modulator in accordance with various aspects of the present disclosure;

[0011] Fig. 3 illustrates another exemplary phase modulator in accordance with various aspects of the present disclosure;

[0012] Fig. 4 illustrates an exemplary arrangement for a projection system according to various aspects of the present disclosure;

[0013] Fig. 5 illustrates another exemplary arrangement for a projection system according to various aspects of the present disclosure;

[0014] Fig. 6 illustrates an exemplary flow chart of processing operations according to various aspects of the present disclosure; and

[0015] Fig. 7 illustrates another exemplary flow chart of processing operations in accordance with various aspects of the present disclosure.

Detailed description of the invention

[0016] This disclosure and its aspects may be embodied in various forms, including hardware or circuits controlled by computer-implemented methods, computer program products, computer systems and networks, user interfaces and application programming interfaces; as well as hardware-implemented methods, signal processing circuits, arrays of memory elements, application-specific integrated circuits (ASICs), field-programmable gate arrays (FPGAs), and the like. The above summary of the invention is intended only to provide a general idea of the various aspects of the present disclosure and does not limit the scope of the disclosure in any way.

[0017] In the following description, numerous details are set forth, such as circuit configurations, timing, operations, and the like, in order to provide an understanding of one or more aspects of the present disclosure. It will be apparent to those skilled in the art that these specific details are merely exemplary and are not intended to limit the scope of this application.

[0018] Furthermore, although the present disclosure focuses primarily on examples in which various circuits are used in digital projection systems, it should be understood that this is merely one example of implementation. It should further be understood that the disclosed systems and methods can be used in any device in which there is a need to project light; for example, in cinematic, consumer and other commercial projection systems, head-up displays, virtual reality-style displays, and the like.

[0019] Projector Systems

[0020] In a projection system based on SLM, the device must be accurately aligned with a predetermined area of the screen. The position of the device and the lens optics must, in particular, be adjusted to ensure that the image is correctly aligned on the screen (for example, the screen must be perpendicular to the optical axis of the device) and is in focus. If the projector is moved, intentionally or unintentionally, the alignment may be reset. In environments such as home or office use, this can occur frequently. In addition, such projection systems typically require a flat screen, and any changes or deviations from the plane of the screen can also lead to distortions of the projected image. Even in a cinema environment, in which the devices are large and typically located in a dedicated room, technical personnel spend a lot of time aligning the devices with the screen and / or ensuring the plane of the screen. In environments in which several projection devices are used on one screen (for example, in 3D cinema projection), this time increases accordingly.

[0021] The use of a phase light modulator (PLM) can reduce these problems. For example, whereas the reference projection system operates by blocking a common light source at each pixel, the PLM projection system operates by adjusting light from areas that are desired to be darker compared to areas that are intended to be brighter. In addition, based on the PLM architecture itself, such projection systems can have significantly lower response times (i.e., the time required to change the projected image) compared to the reference projection systems.

[0022] Fig. 1 illustrates a block diagram of an exemplary projection system 100 according to various aspects of the present disclosure. In particular, Fig. 1 illustrates a projection system 100 that includes a light source 101 configured to emit a first light 102; illumination optics 103 configured to receive the first light 102 and redirect or otherwise modify it to thereby generate a second light 104; a PLM 105 configured to apply spatially varying phase modulation to the second light 104 to thereby adjust the second light 104 and generate a third light 106; first projection optics 107 configured to receive the third light 106 and redirect or otherwise modify it to thereby generate a fourth light 108; a filter 109 configured to filter the fourth light 108 to thereby form a fifth light 110; and a second projection optics 111 configured to receive the fifth light 110 and project it as a sixth light 112 onto a screen 113.

[0023] The projection system 100 further includes a controller 114 configured to control various components of the projection system 100, such as the light source 101 and/or the PLM 105. In some implementations, the controller 114 may additionally or alternatively control other components of the projection system 100, including, but not limited to, the lighting optics 103, the first projection optics 107, and/or the second projection optics 111. The controller 114 may be one or more processors, such as a central processing unit (CPU) of the projection system 100. The lighting optics 103, the first projection optics 107, and the second projection optics 111, respectively, may include one or more optical components, such as mirrors, lenses, waveguides, optical fibers, beam splitters, diffusers, and the like. Furthermore, whereas FIG. 1 illustrates one modulator, the projection system 100 may include other modulators in addition to the PLM 105. For example, the first projection optics 107 may include an amplitude SLM that can be controlled by the controller 114. With the exception of the screen 113, the components illustrated in Fig. 1 may be integrated into a housing to provide a projection device. Such a projection device may include additional components such as a memory device, input/output ports, a communication circuit, a power source, etc.

[0024] The light source 101, for example, can be a laser light source, etc. In general, the light source 101 is any light emitter that emits coherent light. In some aspects of the present disclosure, the light source 101 can comprise several individual light emitters corresponding to different wavelengths or a range of wavelengths. The light source 101 emits light in response to an image signal provided by the controller 114. The image signal includes image data corresponding to a plurality of frames that are to be sequentially displayed. The image signal can come from an external source in a streaming or cloud manner, can come from an internal storage device of the projection system 100, such as a hard drive, can come from a removable medium that is operatively connected to the projection system 100, or from combinations of the above.

[0025] The filter 109 may be provided to mitigate the effects caused by internal components of the projection system 100. In some systems, the PLM 105 (which is described in more detail below) may include a cover glass and cause reflections, switching devices may temporarily cause unwanted adjustment angles, and various components may cause scattering. To counteract this and reduce the floor level of the projection system 100, the filter 109 may be a Fourier filtering ("DC") component configured to block a portion of the fourth light 108. Thus, the filter 109 can increase contrast by reducing the floor level relative to the light near the zero angle, which must correspond to such elements as reflections from the cover glass, transients based on the length of the path, and the like. This region of the DC block may be actively used by the algorithm to prevent certain light from reaching the screen. In some aspects of the present disclosure, filter 109 prevents unwanted light from reaching the screen by directing said light into a light trap located outside the active image area, in response to control from controller 114.

[0026] The second projection optics 111 may include optical components configured to expand the field of view (FOV) of the projection device 101. Such optical components may include, but are not limited to, a fisheye lens or other wide-angle optical component.

[0027] Phase Light Modulator

[0028] As illustrated in Fig. 1, the controller 114 also controls the PLM 105, which receives light from the light source 101. The PLM 105 applies a spatially varying phase modulation to the light and redirects the modulated light to the second projection optics 111. The PLM 105 may be of a reflective type, in which the PLM 105 reflects incident light with a spatially varying phase; alternatively, the PLM 105 may be of a transmissive type, in which the PLM 105 applies a spatially varying phase to the light as it passes through the PLM 105. In some aspects of the present disclosure, the PLM 105 has a liquid crystal on silicon (LCOS) architecture or may be a microelectromechanical system (MEMS), such as a digital micromirror device (DMD).

[0029] Fig. 2 illustrates one example of a PLM 105 implemented as a reflective LCOS PLM 200 and shown in a partial cross-sectional view. As illustrated in Fig. 2, the PLM 200 includes a silicon circuit board 210, a first electrode layer 220, a second electrode layer 230, a liquid crystal layer 240, a cover glass 250, and spacers 260. The silicon circuit board 210 includes electronic circuitry associated with the PLM 200, such as complementary metal oxide semiconductor (CMOS) transistors and the like. The first electrode layer 220 includes an array of reflective elements 221 disposed in a transparent matrix 222. The reflective elements 221 can be formed from any optically highly reflective material, such as aluminum or silver. The transparent matrix 222 can be formed from any optically highly transmitting material, such as a transparent oxide. The second electrode layer 230 can be formed from any optically transparent electrically conductive material, such as a thin film of indium tin oxide (ITO). The second electrode layer 230 can be provided as a common electrode corresponding to a plurality of reflective elements 221 of the first electrode layer 220. In such a configuration, each of the plurality of reflective elements 221 should be connected to the second electrode layer 230 through a corresponding electric field, thereby dividing the PLM 200 into an array of modulation elements (for example, pixels). In this way, individual (or subsets) of the plurality of reflective elements 221 can be addressed through an electronic circuit located in the silicon mounting board 210 in order to thereby modify the state of the corresponding reflective element 221.

[0030] The liquid crystal layer 240 is located between the first electrode layer 220 and the second electrode layer 230 and includes a plurality of liquid crystals 241. The liquid crystals 241 are particles that exist in an intermediate phase element between a solid and a liquid; in other words, the liquid crystals 241 exhibit a degree of directional order, but not a positional order. The direction in which the liquid crystals 241 tend to point is called a "director". The liquid crystal layer 240 modifies incident light entering from the cover glass 250 based on the birefringence Δn of the liquid crystals 241, which can be expressed as the difference between the refractive index in the direction parallel to the director and the refractive index in the direction perpendicular to the director. From this, the maximum difference in the optical paths can be expressed as the birefringence multiplied by the thickness of the liquid crystal layer 240. This thickness is set by means of the spacer 260, which seals the PLM 200 and provides a given distance between the cover glass 250 and the silicon mounting board 210. The liquid crystals 241 generally orient themselves along the electric field lines between the first electrode layer 220 and the second electrode layer 230. As illustrated in Fig. 2, the liquid crystals 241 near the center of the PLM 200 are oriented in this way, while the liquid crystals 241 near the periphery of the PLM 200 are practically unoriented in the absence of electric field lines. By addressing individual of the plurality of reflective elements 221 via a phase control signal, the orientation of the liquid crystals 241 can be determined on a per-pixel basis.

[0031] Fig. 3 illustrates another example of the PLM 105 implemented as a DMD PLM 300 and shown in a partial cross-sectional view. As illustrated in Fig. 3, the PLM 300 includes a mounting board 310 and a plurality of controllable reflective elements as modulation elements, each of which includes a yoke 321, a mirror plate 322 and a pair of electrodes 330. Although only two electrodes 330 are visible in the cross-sectional view of Fig. 3, each reflective element in practice may include additional electrodes. Although not specifically illustrated in Fig. 3, the PLM 300 may further include spacer layers, carrier layers, hinge components for controlling the height or orientation of the mirror plate 322, and the like. The circuit board 310 includes electronic circuitry associated with the PLM 300, such as CMOS transistors, a memory element array, etc.

[0032] The yoke 321 may be formed or include an electrically conductive material so as to allow the application of a bias voltage to the mirror plate 322. The mirror plate 322 may be formed from any highly reflective material, such as aluminum or silver. The electrodes 330 are configured to receive a first voltage and a second voltage, respectively, and may be separately addressable. Depending on the values of the voltage on the electrodes 330 and the voltage (for example, the bias voltage) on the mirror plate 322, there is a potential difference between the mirror plate 322 and the electrodes 330, which creates an electrostatic force that controls the mirror plate 322. The yoke 321 is configured to allow the mirror plate 322 to move vertically in response to the electrostatic force. The equilibrium position of the mirror plate 322, which occurs when the electrostatic force and the spring force of the yoke 322 are equal, determines the length of the optical path of the light reflected from the upper surface of the mirror plate 322. Thus, individual of the plurality of controllable reflective elements are controlled in such a way as to provide a certain number (as illustrated, three) of discrete heights and therefore a certain number of discrete phase configurations or phase states. As illustrated, each of the phase states has a flat profile. In some aspects of the present disclosure, the electrodes 330 can contain different voltages with respect to each other, so as to apply a tilt to the mirror plate 322. Such a tilt can be used with a light trap of the type described above.

[0033] The PLM, for example, 300 can allow high switching speeds such that the PLM 300 switches from one phase state, for example, in a time of the order of tens of μs. In order to provide a complete phase control cycle, the total difference in optical paths between the state in which the mirror plate 322 is at the upper point and the state in which the mirror plate 322 is at the lower point must be approximately equal to the wavelength λ of the incident light. Thus, the range of heights between the upper point and the lower point must be approximately equal to λ/2.

[0034] Regardless of what specific architecture is used for the PLM 105, it is controlled by the controller 114 in such a way as to receive specific phase configurations on a pixel-by-pixel basis. Thus, the PLM 105 uses an array of corresponding modulation elements, for example, an array of 960x540. The number of modulation elements in the array can correspond to the resolution of the PLM 105. Due to the nature of the adjustment that can be implemented, the light can be adjusted to any location on the plane of the reconstructed images and is not tied to a pixel grid identical to the pixel grid of the PLM 105. Since the PLM 105 allows for a short response time, high-resolution moving images can be formed on the plane of the reconstructed images. The operation of the PLM 105 may be affected by the data transmission bandwidth of the projection system 100, the quantization based on the stroke length of the PLM 105, and/or the response time of the PLM 105. The maximum resolution may be determined by the point spread function (PSF) of the light source 101 and the parameters of the various optical components in the projection system 100.

[0035] Dynamic Screen Geometry

[0036] The PLM 105, which may be the PLM 200 of FIG. 2 or the PLM 300 of FIG. 3, is capable of creating a projected image at an arbitrary distance from the projection system 100. In addition, the PLM 105 can form a projected image in any arbitrary shape such that the selected projection surface (e.g., the screen 114, the wall of the room containing the projection system 100, etc.) does not necessarily have to be perpendicular to the optical axis. In implementations in which the projection surface is a flat surface, the projected image may be a two-dimensional image. However, in some implementations, the projection surface may not be a flat surface. In such implementations, the projected image is actually a three-dimensional image, which may correspond to a two-dimensional image transformed onto a non-flat surface. In addition, because the PLM 105 is a digitally controlled device, the effective screen geometry can be changed on the fly. In some implementations, screen geometry changes can be made in response to real-time user input.

[0037] Fig. 4 illustrates one example of an arrangement in which a projection system may be implemented. In particular, Fig. 4 illustrates a projection device 401, which may be the projection system 100 illustrated in Fig. 1, excluding the screen 113. The projection device 401 is located in a room, the first wall 411 and the second wall 412 of which are shown. For ease of explanation, various elements of Fig. 4 that are not directly provided in the projection (e.g., furniture) are not separately described here. When stationary, the projection device 401 can dynamically project an image to one or more of several different locations, of which the first projection surface 421 and the second projection surface 422 on the first wall 411 and the third projection surface 423 on the second wall 412 are illustrated. The first projection surface 421, the second projection surface 422 and the third projection surface 423 can be screens or can simply be portions of the first wall 411 and the second wall 412, respectively. Although only three projection surfaces are shown in Fig. 4, in practice the target projection surface can be any surface in the FOV of the projection device 401. Since the projection device 401 has a short response time (for example, due to the PLM 105), the projection surfaces may not be adjacent to each other and may be located on different walls.

[0038] The location, size and/or other parameters of the image can be adjusted via an interface as illustrated in Fig. 5. In Fig. 5, a user-controlled input device 501 is illustrated that projects a calibration code 511 onto a user-selectable location on the screen. The input device 501 can be a remote control associated with the projection device 401, a smartphone, etc. In some implementations, the input device 501 is configured to emit in the infrared (IR) spectrum, and therefore the calibration code 511 is invisible to the human eye. To detect the calibration code 511, the projection device 401 can comprise a camera, such as an IR sensor. The projection device 401 can find and track the calibration code 511 in real time. In other implementations, the input device 501 may not emit the calibration code 511, but may instead control the projection device 401 using an application installed on the input device 501. For example, the user may predefine a plurality of possible locations on the screen (for example, corresponding to the first projection surface 421, the second projection surface 422 and the third projection surface 423) and then select from among the possible locations on the screen. In still other implementations, the projection device 401 may comprise a distance or depth sensor that detects features of the surface of the projection surface, for example, a time-of-flight (TOF) sensor. The surface features may include features of a flat surface (for example, location and geometry) and/or features of a non-flat surface (for example, geometry and relief).

[0039] The projection device 401 may include both a camera and a distance sensor and may use one or both sensors to perform additional operations such as color correction (e.g., to match a paint color) and the like. Additionally or alternatively, the input device 501 may include a camera and/or a distance sensor and communicate (e.g., transmit data representing a map of a room or areas of a room) with the projection device 401.

[0040] Fig. 6 illustrates an example flow chart of a method for projection with dynamic screen geometry according to various aspects of the present disclosure. The flow chart of processes may be performed in or by a controller of a projection system, such as a controller 114 of a projection system 100 (corresponding to a projection device 401) illustrated in Fig. 1. To perform the flow chart of processes, the projection system 100 may comprise instructions stored on a non-volatile computer-readable medium (e.g., a hard disk, a removable storage medium, a random access memory (RAM), etc.) that, when executed by the controller 114, cause the projection system 100 to perform the steps of Fig. 6.

[0041] In step 601, the controller 114 determines the geometry of the location and/or the topography of the surface of the screen or projection surface (generally referred to as the "target geometry" or "target configuration"). This determination may receive, as input, one or more selected from user input, input from a camera, input from a TOF sensor, or similar data. The determination in step 601 may occur in several stages. For example, the controller 114 may first determine an approximate location and/or geometry for the screen and may thereafter refine the location, geometry, and/or topography. Although FIG. 6 illustrates step 601 occurring at the beginning of a series of processing operations, in some implementations step 601 may be repeated continuously throughout the projection of images. The target geometry and/or topography of the projection surface are thereby dynamically determined as they are determined at or near the actual time of projection of the images.

[0042] At step 602, the controller 114 determines a corresponding phase configuration for a frame of image data. The frame may be in the form of an image signal that is provided from an external, internal or removable source. The image signal includes a series of frames with a frequency depending on the frame rate of a particular application. Each frame includes image data for forming an image on a screen with a particular resolution. The present disclosure is not particularly limited in the frame rates and/or resolutions that can be implemented. For example, the frame rate may be 24 Hz or 48 Hz for cinema applications; 30 Hz, 60 Hz or 120 Hz for home applications, etc. In addition, the resolution may be 2K (2048×1080), 4K (4096×2160), 1080p (1920×1080), consumer 4K (3840×2160), etc. The determination in step 602 may receive, as inputs, one or more pixel data selected from the output of step 601, included in the frame of image data, or similar data. In addition to the phase configuration, step 602 may include determining the corresponding brightness levels of the light source 101.

[0043] The determination at step 602 may include computing a beamforming-based control solution for the PLM 105 configuration that, when reconstructed, approximates the target light field. In some examples, the computations may establish a transform (e.g., a bidirectional transform) between the phasor field in the modulating plane (called the "modulating field") and the phasor field in the reconstructing plane (called the "reconstructing field"). The transformation can be represented as any numerical wave propagation, including, but not limited to, the Fresnel or Rayleigh-Sommerfeld methods.

[0044] In step 603, the controller 114 controls the components of the projection system 100 so as to project light in accordance with the phase configuration determined in step 603. Step 603 may include controlling the light source 101 so as to emit light and/or controlling the PLM 105 so as to modulate the light, in accordance with the phase configuration. This control may be performed using an emitting control signal provided to the light source 101 and/or a phase control signal provided to the PLM 105. As a result of step 603, a frame of image data is projected onto a projection surface having an appropriate location and geometry. In implementations in which step 601 occurs continuously during the entire display of images, step 603 may also include the emission of IR illumination multiplexed with visible light, so that the IR illumination can be detected by the projection system 100 and used to refine the determination of the screen geometry. The light source may be configured to emit infrared light multiplexed with visible light.

[0045] In step 604, the controller 114 determines whether or not the screen geometry should be changed. If a change should not occur, the processing flow returns to step 602 to determine an appropriate phase configuration for the next frame in the image data. If a change should occur, the processing flow returns to step 601 to determine an appropriate screen geometry before the next frame is processed and displayed. The controller 114 may determine that the change occurs in response to user input, a signal from a camera and/or a TOF sensor associated with the projection system 100, etc. In some implementations, step 604 may occur once per frame, in which case steps 602-604 are performed multiple times for each frame. In other implementations, step 604 may only occur once every few frames (e.g., once per second), so that steps 602 and 603 are performed multiple times for each frame, and step 604 occurs at the appropriate interval.

[0046] The projection system 100 may be configured to display multiple images on multiple projection surfaces in such a way that they appear to be simultaneous to the viewer. Fig. 7 illustrates an exemplary processing flow of such a step, which may be in addition to or in place of steps 601, 602 and/or 603 illustrated in Fig. 6. To perform the processing flow, the projection system 100 may comprise instructions stored on a non-transitory computer-readable medium (e.g., a hard disk, a removable storage medium, RAM, etc.), which, when the instructions are executed by the controller 114, cause the projection system 100 to perform the steps of Fig. 7.

[0047] In step 701, the controller 114 determines the number of different screens (or other projection surfaces or target areas of projection surfaces), represented by a positive integer N>1. Step 701 may also include determining the corresponding geometry of each screen. In step 702, the index variable n, which is from 1 to N, is initialized or reinitialized to 1.

[0048] In step 703, the controller receives a frame of image data, and in step 704, the controller 114 determines a corresponding phase configuration for the frame of image data corresponding to the n-th projection surface. As described above, the frame may be in the form of an image signal that is provided from an external, internal or removable source. The image signal includes a sequence of frames with a frequency depending on the frame rate of a particular application. Each frame includes image data for forming an image on a screen with a particular resolution. The present disclosure is not particularly limited in the frame rates and/or resolutions that can be implemented. The frame rate may be a base frame rate divided by N. For example, the base frame rate may be 24 Hz or 48 Hz for cinema applications; 30 Hz, 60 Hz or 120 Hz for home applications, etc. In addition, the base resolution may be 2K (2048×1080), 4K (4096×2160), 1080p (1920×1080), consumer 4K (3840×2160), etc. Thus, each frame period (i.e., the inverse of the frame rate corresponding to the frame duration) is divided into N sub-periods. The determination in step 704 may receive, as inputs, one or more pixel data selected from the output of step 701, included in the frame of image data, or similar data. In addition to the phase configuration, step 704 may include determining the corresponding brightness levels of the light source 101.

[0049] In step 705, the controller 114 controls the components of the projection system 100 so as to project light in accordance with the phase configuration determined in step 704. Step 705 may include controlling the light source 101 so as to emit light and/or controlling the PLM 105 so as to modulate the light, in accordance with the phase configuration. This control may be performed using an emitting control signal provided to the light source 101 and/or a phase control signal provided to the PLM 105. As a result of step 705, a frame of image data is projected onto the n-th projection surface having a corresponding location and geometry. Step 703 may also include emitting IR illumination multiplexed with visible light, so that the IR illumination can be detected by the projection system 100 and used to refine the determination of the geometry of the screen.

[0050] In step 706, the controller 114 determines whether steps 703-704 have been performed for all N projection surfaces by comparing the index n with N. If n is less than N, n is gradually increased in step 707, and the processing flow returns to step 703. Otherwise, the processing flow returns to step 702, in which n is reinitialized to 1 for the next frame in the image data.

[0051] Fig. 7 illustrates a method for time-division projection for multiple displays in which each frame is displayed at full brightness or resolution sequentially, but at a frame rate that is a fraction of the base frame rate. In other implementations, the projection system 100 may display frames in a spatially multiplexed manner. In such an implementation, each frame may be displayed, simultaneously for all N projection surfaces, at full frame rate, but at a brightness or resolution that is a fraction of the base brightness or frame rate.

[0052] Advantages

[0053] Due to the beamforming nature and fast response times of the projection systems and methods described herein, it is possible to achieve a projection system that allows for dynamic movement and/or change in image shape and/or screen geometry. Compared to other projection systems, the system described herein is suitable for a wide range of projection applications in a wide range of environments. For example, the projection systems and methods according to the present disclosure may be suitable for home environments, office environments, concert venues and halls, automotive environments, industrial environments, commercial and/or retail environments, or group (e.g., "crowdsourcing") environments.

[0054] Furthermore, although the above description primarily describes dynamically moving and/or changing the shape of the image in order to accommodate the geometry of the screen, the present disclosure is not limited thereto. In some implementations, the projection systems and methods described above may dynamically move and/or change the shape of the image for laser safety reasons. For example, the projection systems may determine whether or not a person moves into the beam path and move the beam in a direction away from the person (e.g., away from the person's eyes).

[0055] Furthermore, the above systems, methods and advantages may be achieved by multiple projection devices operating together, wherein the multiple projection devices are configured according to the above aspects of the present disclosure. In such examples, the multiple projection devices may be operated to cover a larger area of space (e.g., by tiling respective projected images with or without overlapping on a large projection surface) than is achievable by a single projection device, to increase the brightness of a projected image (e.g., by causing two or more of the multiple projection devices to project the same image onto an identical area of a projection surface), and/or to perform stereoscopic projection (e.g., by causing two of the multiple projectors to project views for different eyes of the same image onto an identical area of a projection surface). In one particular example, individual projection devices may be implemented as, in conjunction with, or using automobile headlights, such that one or more automobiles (e.g., in a drive-in theater) may work together to project images onto a large screen from various positions.

[0056] The systems, methods, and devices according to the present disclosure may adopt any one or more of the following configurations.

[0057] (1) A projection system comprising a light source configured to emit light in response to image data; a phase light modulator configured to receive light from the light source and to apply spatially varying phase modulation to the light to thereby control the light and generate projection light; and a controller configured to dynamically determine, based on one of a user input or a sensor signal, a target geometry of a projection surface onto which the projection light is projected, determine, based on the target geometry, a phase configuration for a frame of image data, and provide a phase control signal to the phase light modulator, wherein the phase control signal is configured to cause the phase light modulator to generate projection light in accordance with the phase configuration for the frame.

[0058] (2) The projection system according to (1), in which the phase light modulator includes a plurality of modulation elements arranged in an array, and a circuit configured to modify respective states of the plurality of modulation elements in response to a phase control signal.

[0059] (3) A projection system according to (1) or (2), wherein the phase light modulator is a digital micromirror device.

[0060] (4) The projection system according to (1) or (2), in which the phase light modulator is a liquid crystal semiconductor device.

[0061] (5) The projection system according to any one of (1) to (4), further comprising an infrared sensor configured to detect a calibration code projected onto the projection surface and to generate a sensor signal based on the calibration code.

[0062] (6) The projection system according to any one of (1) to (5), further comprising a depth sensor configured to detect a surface feature of the projection surface and to generate a sensor signal based on the surface feature.

[0063] (7) A projection system according to any one of (1) to (6), wherein the controller is configured to: determine a number of target areas of the projection surface; divide a frame period of image data into a number of sub-periods, wherein the number of sub-periods corresponds to the number of target areas; and, with time division, perform the operations of dynamically determining the target geometry, determining the phase configuration and providing a phase control signal for each sub-period.

[0064] (8) The projection system of (7), wherein the target areas are not adjacent to each other.

[0065] (9) The projection system according to any one of (1) to (8), further comprising a filter configured to block a portion of the projection light.

[0066] (10) A projection system according to any one of (1) to (9), wherein the light source is a coherent light source.

[0067] (11) A projection method comprising emitting light by a light source in response to image data; receiving the light by a phase light modulator; applying a spatially varying phase modulation to the light by the phase light modulator to thereby control the light and generate projection light; dynamically determining, based on one of a user input or a sensor signal, a target geometry of a projection surface onto which the projection light is projected; determining, based on the target geometry, a phase configuration for a frame of image data; and providing a phase control signal to the phase light modulator, wherein the phase control signal is configured to cause the phase light modulator to generate projection light in accordance with the phase configuration for the frame.

[0068] (12) The method according to (11), wherein the phase light modulator includes a plurality of modulation elements arranged in an array, wherein the method comprises: modifying corresponding states of the plurality of modulation elements in response to the phase control signal.

[0069] (13) The method according to (11) or (12), wherein the phase light modulator is a digital micromirror device.

[0070] (14) The method according to (11) or (12), wherein the phase light modulator is a liquid crystal semiconductor device.

[0071] (15) The method according to any one of (11) to (14), further comprising: detecting a calibration code projected onto the projection surface by means of an infrared sensor; and generating a sensor signal based on the calibration code.

[0072] (16) The method according to any one of (11) to (15), further comprising: detecting a surface feature of the projection surface using a depth sensor; and generating a sensor signal based on the surface feature.

[0073] (17) The method according to any one of (11) to (16), further comprising: determining a number of target regions of the projection surface; dividing a frame period of image data into a number of sub-periods, wherein the number of sub-periods corresponds to the number of target regions; and, with time division, performing the operations of dynamically determining the target geometry, determining the phase configuration and providing a phase control signal for each sub-period.

[0074] (18) The method according to (17), wherein the target areas are not adjacent to each other.

[0075] (19) The method according to any one of (11) to (18), further comprising blocking a portion of the projection light by means of a filter.

[0076] (20) A non-transitory computer-readable medium storing instructions that, when executed by a processor of a projection device, cause the projection device to perform operations comprising the method of any of (11)-(19).

[0077] With respect to the processes, systems, methods, heuristics, etc. described herein, it should be understood that although the steps of such processes, etc. are described as being performed according to a certain ordered sequence, such processes may be practiced with the described steps performed in an order different from the order described herein. It should further be understood that certain steps may be performed simultaneously, that other steps may be added, or that certain steps described herein may be omitted. In other words, the descriptions of processes herein are provided for the purpose of illustrating particular embodiments and are not to be construed as limiting the claims in any way.

[0078] Accordingly, it is to be understood that the foregoing description is intended to be illustrative and not restrictive. Numerous embodiments and uses other than the examples provided will become apparent from reading the foregoing description. The scope should not be determined by reference to the foregoing description, but should instead be determined by reference to the appended claims, along with the full scope of equivalents to which such claims are entitled. It is intended and understood that future developments will be made in the technologies described herein, and that the disclosed systems and methods will be included in such future embodiments. In general, it is to be understood that the application is susceptible to modification and variation.

[0079] All terms used in the claims are to be given their broadest reasonable constructions and their ordinary meanings as understood by those skilled in the art described herein, unless otherwise expressly indicated herein. In particular, the use of a singular form such as "mentioned," etc., are to be construed as a recitation of one or more recited elements, unless the claim lists another express limitation.

[0080] The Abstract of the Disclosure is provided to enable the reader to readily ascertain the nature of the technical disclosure. It is presented with the understanding that it is not to be used to interpret or limit the scope or meaning of the claims. In addition, in the foregoing detailed description, it can be seen that various features are grouped in various embodiments for the purpose of simplifying the disclosure. This manner of disclosure should not be interpreted as reflecting an intention that the claimed embodiments include more features than are expressly recited in each claim. To the contrary, as the appended claims reflect, the subject matter of the invention does not reside in all features of a single disclosed embodiment. Accordingly, the appended claims are hereby incorporated into the Detailed Description, with each claim standing independently as separately claimed subject matter.

[0081] Various aspects of the present invention can be taken into account from the following enumerated exemplary embodiments (EEE):

EEE1. A projection system comprising:

- a light source configured to emit light in response to image data;

- a phase light modulator configured to receive light from a light source and apply spatially varying phase modulation to the light to thereby regulate the light and generate projection light; and

- a controller configured to:

- dynamically determine, based on one of the user input or sensor signal, the target geometry of the projection surface onto which the projection light is projected,

- determine, based on the target geometry, a phase configuration for a frame of image data, and

- provide a phase control signal to the phase light modulator, wherein the phase control signal is configured to instruct the phase light modulator to generate projection light in accordance with the phase configuration for the frame.

EEE2. The projection system according to claim 1, wherein the phase light modulator includes a plurality of modulation elements arranged in an array, and a circuit configured to modify the corresponding states of the plurality of elements in response to a phase control signal.

EEE3. The projection system of claim 2, wherein the phase light modulator is a digital micromirror device.

EEE4. The projection system of claim 2, wherein the phase light modulator is a liquid crystal semiconductor device.

EEE5. A projection system according to any one of paragraphs 1-4, further comprising an infrared sensor configured to detect a calibration code projected onto the projection surface and to generate a sensor signal based on the calibration code.

EEE6. A projection system according to any one of claims 1-5, further comprising a depth sensor configured to detect a surface feature of the projection surface and to generate a sensor signal based on the surface feature.

EEE7. A projection system according to any one of paragraphs 1-6, in which the controller is configured to:

- determine the number of target areas of the projection surface;

- divide the frame period of the image data into a number of sub-periods, where the number of sub-periods corresponds to the number of target areas; and

- with time division, perform operations of dynamic determination of target geometry, determination of phase configuration and provision of phase control signal for each sub-period.

EEE8. The projection system of claim 7, wherein the target areas are not adjacent to each other.

EEE9. A projection system according to any one of claims 1-8, further comprising a filter configured to block a portion of the projection light.

EEE10. A projection system according to any one of claims 1 to 9, wherein the light source is a coherent light source.

EEE11. A method for controlling a projection system, comprising:

- emitting light by means of a light source in response to image data;

- reception of light by means of a phase light modulator;

- application of spatially varying phase modulation to light by means of a phase light modulator in order to thereby regulate the light and shape the projected light;

- dynamically determining, based on one of the user input or sensor signal, the target geometry of the projection surface onto which the projection light is projected;

- determining, based on the target geometry, a phase configuration for a frame of image data; and

- providing a phase control signal to a phase light modulator, wherein the phase control signal is configured to instruct the phase light modulator to generate projection light in accordance with a phase configuration for the frame.

EEE12. The method according to claim 11, wherein the phase light modulator includes a plurality of modulation elements arranged in an array, wherein the method comprises:

- modification of the corresponding states of a plurality of modulation elements in response to a phase control signal.

EEE13. The method of claim 12, wherein the phase light modulator is a digital micromirror device.

EEE14. The method of claim 12, wherein the phase light modulator is a liquid crystal semiconductor device.

EEE15. The method according to any one of paragraphs 11-14, further comprising:

- detection of a calibration code projected onto the projection surface by means of an infrared sensor; and

- formation of a sensor signal based on a calibration code.

EEE16. The method according to any one of paragraphs 11-15, further comprising:

- detecting a surface feature of the projection surface using a depth sensor; and

- formation of a sensor signal based on a surface feature.

EEE17. The method according to any one of paragraphs 11-16, further comprising:

- determination of the number of target areas of the projection surface;

- dividing the frame period of the image data into a number of sub-periods, where the number of sub-periods corresponds to the number of target areas; and

- with time division, performing operations of dynamic determination of target geometry, determination of phase configuration and provision of phase control signal for each sub-period.

EEE18. The method of claim 17, wherein the target areas are not adjacent to each other.

EEE19. The method according to any one of claims 11-18, further comprising blocking a portion of the projecting light by means of a filter.

EEE20. A non-transitory computer-readable medium storing instructions that, when executed by a processor of a projection system, cause a projection device to perform operations comprising a method according to any one of paragraphs 11-19.

Claims (49)

1. A projection system comprising: a light source configured to emit light in response to image data; a phase light modulator configured to receive light from a light source and apply spatially varying phase modulation to the light to thereby regulate the light and generate projection light; and a controller configured to: dynamically determine, based on at least one of a user input or a sensor signal, a target geometry and a topography of a projection surface onto which the projection light is projected, determine, based on the target geometry and terrain, a phase configuration for a frame of image data, and provide a phase control signal to the phase light modulator, wherein the phase control signal is configured to cause the phase light modulator to generate projection light in accordance with a phase configuration for the frame. 2. The projection system according to claim 1, wherein the phase light modulator includes a plurality of modulation elements arranged in an array, and a circuit configured to modify the corresponding states of the plurality of elements in response to a phase control signal. 3. The projection system according to claim 2, wherein the phase light modulator is a digital micromirror device or a liquid crystal semiconductor device. 4. A projection system according to any one of paragraphs 1-3, additionally containing an infrared sensor configured to detect a calibration code projected onto the projection surface and to generate a sensor signal based on the calibration code. 5. A projection system according to any one of paragraphs 1-4, further comprising a user-controlled input device configured to project a calibration code onto a user-selectable projection surface. 6. The projection system according to claim 5, wherein the user-controlled input device is configured to emit light in the infrared spectrum. 7. A projection system according to any one of claims 1-4, wherein the light source is configured to emit infrared light multiplexed with visible light. 8. A projection system according to any one of claims 1-7, further comprising a depth sensor configured to detect a surface feature of the projection surface and generate a sensor signal based on the surface feature. 9. The projection system according to claim 8, wherein the surface feature includes features of a non-flat surface. 10. A projection system according to any one of paragraphs 1-9, in which the controller is configured to: determine the number of target areas of the projection surface; divide the frame period of the image data into a number of sub-periods, where the number of sub-periods corresponds to the number of target areas; and with time division, perform operations of dynamically determining the target geometry and relief, determining the phase configuration and providing a phase control signal for each sub-period. 11. The projection system of claim 10, wherein the target areas are not adjacent to each other. 12. A projection system according to any one of paragraphs. 1-11, further comprising a filter configured to block a section of the projection light. 13. A projection system according to any one of claims 1 to 12, wherein the light source is a coherent light source. 14. A method for controlling a projection system, comprising the steps of: emit light via a light source in response to image data; receive light through a phase light modulator; apply spatially varying phase modulation to light by means of a phase light modulator in order to thereby regulate the light and shape the projection light; dynamically determining, based on one of the user input or sensor signal, the target geometry and topography of the projection surface onto which the projection light is projected; determining, based on the target geometry and terrain, a phase configuration for a frame of image data; and providing a phase control signal to a phase light modulator, wherein the phase control signal is configured to cause the phase light modulator to generate projection light in accordance with a phase configuration for the frame. 15. The method according to claim 14, wherein the phase light modulator includes a plurality of modulation elements arranged in an array, wherein the method comprises a step in which: modify the corresponding states of the plurality of modulation elements in response to the phase control signal. 16. The method according to claim 15, wherein the phase light modulator is a digital micromirror device or a liquid crystal semiconductor device. 17. The method according to any one of paragraphs 14-16, further comprising the steps of: detecting a calibration code projected onto the projection surface by means of an infrared sensor; and generate a sensor signal based on a calibration code. 18. The method according to any one of claims 14-17, further comprising the step of projecting, via a user-controlled input device, a calibration code onto a user-selectable projection surface. 19. The method of claim 18, further comprising the step of emitting, via a user-controlled input device, light in the infrared spectrum. 20. The method according to any one of claims 14-17, further comprising the step of emitting, by means of a light source, infrared light and visible light and multiplexing the infrared light with the visible light. 21. The method according to any one of paragraphs 14-20, further comprising the steps of: detecting a surface feature of the projection surface using a depth sensor; and generate a sensor signal based on a surface feature. 22. The method according to claim 21, wherein the surface feature includes features of a non-flat surface. 23. The method according to any one of paragraphs 14-22, further comprising the steps of: determine the number of target areas of the projection surface; dividing the frame period of the image data into a number of sub-periods, wherein the number of sub-periods corresponds to the number of target areas; and with time division, operations of dynamically determining the target geometry and relief, determining the phase configuration and providing a phase control signal for each sub-period are performed. 24. The method of claim 23, wherein the target areas are not adjacent to each other. 25. The method according to any one of claims 14-24, further comprising the step of blocking a portion of the projected light by means of a filter. 26. A non-transitory computer-readable medium storing instructions that, when executed by a processor of a computer system, cause a projection device to perform operations comprising the method of any one of paragraphs 14-25.