WO2020184116A1 - Image output system and image output control method - Google Patents

Abstract

The present invention reduces differences in color space reproduced by a plurality of projectors. An image output system according to an embodiment is provided with: a plurality of projectors (2) which respectively project a plurality of pieces of image data represented in a first color space; adjustment units (11, 12, 13) which adjust, on a second color space, color information about the plurality of pieces of image data respectively to be input to the plurality of projectors; and an output unit (15) which outputs the plurality of pieces of adjusted image data from the adjustment unit to the plurality of respective projectors.

Description

Video output system and video output control method

This disclosure relates to a video output system and a video output control method.

Conventionally, a multi-projection system that projects one large image or video using a plurality of projectors is known. Further, there is known a technique for making an image output from a plurality of projectors close to white on a projection surface (see, for example, Non-Patent Documents 1 and 2). In such a technique, white on a projection surface is reproduced by outputting complementary colors from a plurality of projectors and superimposing them on the same projection surface at the same position.

Yugo Minomo, Yasuaki Kakehi, Makoto lida, and Takeshi Naemura. 2006. Transforming your shadow into colorful visual media: multiprojection of complementary colors. Comput. Entertain. 4, 3, Article 10 (July 2006) .org / 10.1145 / 1146816.1146832 Hiroshi Kato, Takeshi Naemura, and Hiroshi Harashima. 2003. Textured Shadow. In proceedings of the 2nd IEEE / ACM International Symposium on Mixed and Augmented Reality (ISMAR '03). IEEE Computed Reality (ISMAR '03). IEEE Computed S

However, since there are individual differences in projectors, the color space reproduced by each of multiple projectors will differ from one projector to another. Therefore, when the colors output from a plurality of projectors are adjacent to each other or superimposed, there is a problem that the viewer feels uncomfortable due to the difference in the reproduced color space.

Therefore, this disclosure proposes a video output system and a video output control method that make it possible to reduce the difference in color space reproduced by a plurality of projectors.

In order to solve the above problems, the video output system of one form according to the present disclosure includes a plurality of projectors each projecting video data represented in the first color space, and the plurality of projectors for inputting data to the plurality of projectors. It includes an adjustment unit that adjusts the color information of the video data on the second color space, and an output unit that outputs the video data adjusted by the adjustment unit to each of the plurality of projectors.

It is a functional block diagram which shows the structure of the video output system which concerns on one Embodiment. It is a figure which shows the outline of the video output control processing which concerns on one Embodiment. It is a functional block diagram which shows the structure of the video output control apparatus which concerns on one Embodiment. It is a figure which shows the measured value of XYZ on the projection plane of a single RGB channel. It is a figure which shows the measured value of XYZ on the projection plane of a single RGB channel. It is a figure which shows the measured value of XYZ on the projection plane of a single RGB channel. It is a figure explaining the uniform texture which concerns on one Embodiment. It is a figure explaining the appearance of unevenness in an XYZ color space. It is a figure explaining the appearance of unevenness in an XYZ color space. It is a figure which shows an example of the uniformity adjustment which concerns on one Embodiment. It is a figure which shows an example of the uniformity adjustment which concerns on one Embodiment. It is a figure (1) which shows an example of the use of the video output control which concerns on one Embodiment. It is a figure (2) which shows an example of the use of the video output control which concerns on one Embodiment. It is a figure (1) which shows an example of the image which controlled the image output which concerns on one Embodiment. It is a figure (2) which shows an example of the image which controlled the image output which concerns on one Embodiment.

Hereinafter, an embodiment of the video output system and the video output control method disclosed in the present application will be described in detail with reference to the drawings. The present invention is not limited to the embodiments illustrated below.

In addition, the present disclosure will be described according to the order of items shown below.
[Configuration example of video output system according to the embodiment]
[Configuration example of video output control device]
[Use of video output control]
[Example of video with video output control]
[Effect of Embodiment]
[Other]

[Configuration example of video output system according to the embodiment]
FIG. 1 is a functional block diagram showing a configuration of a video output system according to an embodiment. The video output system 9 shown in FIG. 1 includes a video output control device 1, a plurality of projectors 2, a spectroscope 3, a camera 4, and an output board 5.

The projector 2 is a device that displays an image or an image by projecting it on an output board 5 or the like. The output board 5 is, for example, a screen, but may be any one that can be displayed by being projected from the projector 2. Note that the plurality of projectors 2 are arranged apart from each other so that when a real object such as a user exists between the plurality of projectors 2 and the projection surface, an image hidden by the real object and an image not hidden by the real object are presented. The object. With such an arrangement, a video representation as presented in FIGS. 9A and 9B, which will be described later, is presented.

The spectroscope 3 is, for example, a spectroradiometer, and is for measuring the color space of the color projected on the output board 5 by each projector 2. The spectroscope 3 reversibly converts, for example, the colors of the RGB color space (hereinafter referred to as RGB color space) output by the projector 2 into the color space of the color physically reproduced by the projector 2. Is used to enable.

The RGB color space here refers to a color space that expresses colors by mixing the three primary colors of red (Red), green (Green), and blue (Blue).

Further, in the present embodiment, the XYZ color space (hereinafter referred to as XYZ color space) is used as the color space of the physically reproduced color. The RGB color space is a color space for each projector 2 including individual differences, whereas the XYZ color space is a color space common to all projectors 2 not including individual differences.

For this XYZ color space, for example, a CIE1931 color space or the like can be used. However, it is not limited to the XYZ color space, and various color spaces can be used as long as it is a color space capable of expressing an actually reproduced physical color expression such as an L ^* a ^* b ^* color space. It is possible to adopt.

As the camera 4, it is possible to use various cameras capable of acquiring color images, such as a 4K resolution camera.

The video output control device 1 adjusts the color space reproduced by each of the plurality of projectors 2 by using a common color space so that individual differences between the projectors 2 are reduced or eliminated. Then, the video output control device 1 outputs the video data for each projector 2 generated by this adjustment to each projector 2.

Here, an outline of the video output control process executed by the video output control device 1 according to the present embodiment will be described with reference to FIG.

As shown in FIG. 2, the video output control device 1 generates video data to be projected for each projector 2 (a1). The video data for each projector 2 may be generated, for example, by dividing the original video data into RGB or regions, or dividing the three-dimensional video into a plurality of layers, or the like. It may be generated by various methods.

Next, the video output control device 1 converts the color space of the video data generated for each projector 2 from the RGB color space to the XYZ color space (a2). At that time, the video output control device 1 specifies this identification based on the correspondence between the RGB value in the video data and the XYZ value measured by the spectroscope 3 at a certain point on the projection surface (hereinafter referred to as a specific point). The RGB values of the video data are converted into XYZ values so that the individual differences between the projectors 2 at the points are eliminated.

Next, the video output control device 1 corrects the video data for each of the plurality of projectors 2 so that the non-uniformity (unevenness) of the video is adjusted for all the pixels of the video data (a3). This uniformity adjustment is performed on the XYZ color space.

Next, the video output control device 1 converts the video data uniformly adjusted as described above from the XYZ color space to the RGB color space (a4).

Next, the video output control device 1 geometry-corrects the video data so that the distortion of the video projected from each of the projectors 2 is eliminated (a5).

Then, the video output control device 1 outputs video data to each projector 2 so that the video is projected synchronously from the plurality of projectors 2 (a6).

[Configuration example of video output control device]
FIG. 3 is a functional block diagram showing a schematic configuration example of the video output control device according to the present embodiment. The video output control device 1 shown in FIG. 3 has a control unit 10 and a storage unit 20.

The control unit 10 corresponds to electronic circuits such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). Then, the control unit 10 has an internal memory for storing programs and control data that define various processing procedures, and executes various processes by these. The control unit 10 includes a color adjustment unit 11, a uniformity adjustment unit 12, a geometry adjustment unit 13, a video synchronization unit 14, and a video output unit 15.

The storage unit 20 is, for example, a semiconductor memory element such as a RAM (Random Access Memory) or a flash memory (Flash Memory), or a storage device such as a hard disk or an optical disk. The storage unit 20 stores the RGB-XYZ conversion type table 21 and the uniformization type table 22.

The RGB-XYZ conversion formula table 21 is used, for example, in the above-mentioned color space conversion (a2) and color space inverse conversion (a4), and is used to convert the RGB color space into the XYZ color space. The conversion formula is stored for each projector 2. The RGB-XYZ conversion table 21 is generated by, for example, the color adjustment unit 11.

The homogenization table 22 is used, for example, in the uniformity adjustment (a3) described above, and stores the formula used for uniformizing the projection surface so that there is no unevenness for each projector 2. The homogenization table 22 is generated by, for example, the uniformity adjustment unit 12.

The color adjustment unit 11 adjusts the color of a specific pixel (pixel corresponding to the specific point) in the video data for each projector 2.

For example, the color adjusting unit 11 determines that the XYZ values of specific points in the images output from each of the projectors 2 measured by the spectroscope 3 are substantially the same XYZ values in all the projectors 2. An RGB-XYZ conversion formula for each is generated. The specific point may be, for example, a projection plane or a central point of the projected image. Hereinafter, the specific point will be described as the central point of the projection plane. As an example, in the present embodiment, the XYZ values of the center points of the projection planes of the images output from the projector 2 are separately measured using the spectroscope 3. The measurement of the XYZ value is performed for the R, G, and B channels alone.

Then, the color adjustment unit 11 calculates the RGB-XYZ conversion formula used for color matching in the common XYZ color space based on the XYZ values measured by the R, G, and B channels alone. Further, the color adjustment unit 11 calculates an XYZ-RGB conversion formula for converting XYZ to RGB. Then, the color adjustment unit 11 stores the calculated RGB-XYZ conversion formula and XYZ-RGB conversion formula in the RGB-XYZ conversion formula table 21. The RGB-XYZ conversion formula and the XYZ-RGB conversion formula are held in association with each projector 2.

Here, the measurement results of the XYZ values performed by the RGB channel alone are shown in FIGS. 4A to 4C. 4A to 4C are diagrams showing XYZ measurement values on the projection plane of the RGB channel alone. FIG. 4A shows the relationship between the pixel values from 0 to 256 output from the projector 2 of the R channel alone and the XYZ values. FIG. 4B shows the relationship between the pixel values from 0 to 256 output from the projector 2 of the G channel alone and the XYZ values. FIG. 4C shows the relationship between the pixel values from 0 to 256 output from the projector 2 of the B channel alone and the XYZ values.

In the case of the R channel alone, the relationship between the pixel value [0-256] output from the projector 2 and the XYZ value is as follows. The relationship between the pixel value [0-256] and the X value within the XYZ value is expressed as X _R = a _RX R ^g . The relationship between the pixel value [0-256] and the Y value within the XYZ value is expressed as Y _R = a _RY R ^g . The relationship between the pixel value [0-256] and the Z value within the XYZ value is expressed as Z _R = a _RZ R ^g . In addition, g corresponds to the gamma value of the display.

Further, in the case of the G channel alone, the relationship between the pixel value [0-256] output from the projector 2 and the XYZ value is as follows. The relationship between the pixel value [0-256] and the X value within the XYZ value is expressed as X _G = a _GX R ^g . The relationship between the pixel value [0-256] and the Y value within the XYZ value is expressed as Y _G = a _GY R ^g . The relationship between the pixel value [0-256] and the Z value within the XYZ value is expressed as Z _G = a _GZ R ^g .

Further, in the case of the B channel alone, the relationship between the pixel value [0-256] output from the projector 2 and the XYZ value is as follows. The relationship between the pixel value [0-256] and the X value within the XYZ value is expressed as X _B = a _BX R ^g . The relationship between the pixel value [0-256] and the Y value within the XYZ value is expressed as Y _B = a _BY R ^g . The relationship between the pixel value [0-256] and the Z value within the XYZ value is expressed as Z _B = a _BZ R ^g .

Here, each parameter a _RX , a _RY , a _RZ , a _GX , a _GY , a _GZ , a _BX , a _BY , and a _BZ is estimated from the measured RGB-XYZ data.

Using the measurement result of the XYZ value performed by the RGB channel alone, the color adjustment unit 11 calculates the RGB-XYZ conversion formula used for color matching in a common color space. As an example, when RGB is output from the projector 2, the color adjusting unit 11 calculates the X of the physical color XYZ appearing on the projection surface from the measured X value based on the following equation (1). Further, the color adjusting unit 11 calculates the Y of the physical color XYZ appearing on the projection surface from the measured Y value based on the following equation (2). Further, the color adjustment unit 11 calculates the Z of the physical color XYZ appearing on the projection surface from the measured Z value based on the following equation (3). The present inventor has confirmed by prior verification that each value of XYZ is additive in each channel of RGB.
X _{(R, G, B)} = X _R + X _G + X _B ... (1)
Y _{(R, G, B)} = Y _R + Y _G + Y _B ... (2)
Z _{(R, G, B)} = Z _R + Z _G + Z _B ... (3)

Then, the color adjusting unit 11 calculates a conversion formula for converting RGB to XYZ and a conversion formula for converting XYZ to RGB using the formulas (1) to (3) as follows. That is, the color adjusting unit 11 derives the equation (4) from the equations (1) to (3).

Then, the color adjusting unit 11 substitutes the above-mentioned, for example, X _R = a _RX R ^g , into the equation (4) to derive the equation (5).

Then, the color adjusting unit 11 transforms the equation (5) to derive the equation (6).

Therefore, the color adjustment unit 11 can calculate the RGB-XYZ conversion formula for converting from RGB to XYZ as in the formula (6). The RGB-XYZ conversion formula is an example of the first conversion formula.

Here, the matrix of the parameter a is set to the equation (7).

Then, the color adjusting unit 11 obtains the inverse matrix of A of the equation (7), transforms the equation (6), and derives the equation (8). In addition, the color adjusting unit 11 derives the equation (9) from the equation (8).

Therefore, the color adjustment unit 11 can calculate the XYZ-RGB conversion formula for converting from XYZ to RGB as in the formula (9).

Returning to FIG. 3, the uniformity adjusting unit 12 adjusts the unevenness of the image projected on the projection surface so as to be uniform. That is, since the light from the projector 2 has attenuation due to the distance and non-uniformity due to the lens and the element of the projector 2, the uniformity adjusting unit 12 is adjusted so as to be uniform. The uniformity adjusting unit 12 is executed for each projector 2.

For example, the uniformity adjusting unit 12 generates a uniform texture for measuring the non-uniformity of the projected image for each projector 2.

As an example, the uniformity adjustment unit 12 projects gray from the projector 2 in which RGB is (k, k, k). Note that k indicates a value larger than 0 and smaller than 256, and may be preset according to the dynamic range of the camera 4.

Then, the uniformity adjusting unit 12 projects the image projected from the projector 2 based on the image of the projection surface taken by the camera 4 so that the image projected from the projector 2 is even and uniform with reference to a specific point. Obtain a uniformed gray texture for adjusting the video data to be used. Here, it is assumed that the uniformed gray texture is G _RGB (u, v) (u, v indicate the X value and the Y value on the projection plane, and 0 <u <1,0 <v <1). .. In other words, the uniformity adjusting unit 12 obtains a gray texture to be input to the projector 2 in order to make the image of the projector 2 uniform.

Then, the uniformity adjusting unit 12 converts the uniformed gray texture G _RGB (u, v) from the RGB color space to the XYZ color space by using the RGB-XYZ conversion formula, and the uniformed gray texture G in the XYZ color space. Calculate _XYZ (u, v). Then, the uniformity adjusting unit 12 uses the following equation (u, v) to obtain a versatile uniform texture H _XYZ (u, v) other than gray from the uniform gray texture G _XYZ (u, v) in the XYZ color space. Calculate based on 10). The homogenized texture H _XYZ (u, v) is an example of the second conversion formula.
H _XYZ (u, v) = (G _XYZ (u, v) -G _XYZ (0.5, 0.5)) / G _XYZ (0.5, 0.5) ... (10)

Then, the uniformity adjusting unit 12 stores the calculated equation of the homogenized texture H _XYZ (u, v) in the homogenized expression table 22. The equation for the uniformed texture H _XYZ (u, v) is held in association with each projector 2.

Then, the uniformity adjusting unit 12 uses the uniform texture H _XYZ (u, v) to make the image in the XYZ color space output from the projector 2 uniform based on the following equation (11). Adjust to. The I _XYZ (u, v) of the equation (11) is an image in the XYZ color space output from the projector 2. _I'XYZ (u, v) is an image of uniform tuned XYZ color space.
_{I'XYZ (u, v) =} I XYZ (u, v) + H XYZ (u, v) × I XYZ (u, v) ··· (11)

The homogeneity adjusting unit 12 converts the XYZ color space to the RGB color space uniformly adjusted video _I'XYZ (u, v) and using the XYZ-RGB conversion formula, an image in the RGB color space calculate.

Here, the homogenized texture according to the present embodiment will be described with reference to FIG. FIG. 5 is a diagram illustrating a homogenized texture according to the present embodiment.

As shown in the graph of G0 in FIG. 5, when gray in which RGB is (k, k, k) is projected from the projector 2 onto the projection surface, it is assumed that the left side of the projection surface is bright and the right side is dark. .. The graph of G1 is a result of adjusting the output image from the projector 2 so that the image captured by the projection surface taken by the camera 4 is even and uniform with respect to the center point of the projection surface. That is, the graph of G1 represents a uniform gray texture at the camera 4 viewpoint.

The graph of G2 is a graph showing a gray texture to be input to the projector 2 in order to make the image of the projector 2 uniform. That is, the graph of G2 represents the uniformed gray texture G _RGB (u, v) of the projector 2 image.

The graph of G3 is the result of converting the uniformed gray texture G _RGB (u, v) from the RGB color space to the XYZ color space by using the RGB-XYZ conversion formula. That is, the graph of G3 represents the uniformed gray texture G _XYZ (u, v) in the XYZ color space. The position where u indicates 0.5 is the center point of the projection plane.

The graph of G4 is a graph of the uniformed gray texture G _XYZ (u, v) in the XYZ color space with reference to the center point (u = 0.5) of the projection plane.

Then, the graph of G5 represents the uniformed texture H _XYZ (u, v) in the XYZ color space with respect to the center point (u = 0.5) of the projection plane.

Here, how unevenness appears in the XYZ color space will be described with reference to FIGS. 6A and 6B. 6A and 6B are diagrams for explaining how unevenness appears in the XYZ color space. As shown in FIGS. 6A and 6B, the unevenness (color variation) is linearly proportional to the XYZ value of the color at the center point (u = 0.5). In addition, the unevenness increases as the brightness of the light becomes brighter. Here, FIG. 6A is a projection of gray when k is 0.2 to the center point. FIG. 6B is a projection of gray when k is 0.6 to the center point. Then, it can be seen that the unevenness becomes larger when k is 0.6 than when k is 0.2. That is, it can be seen that the brighter the light, the greater the unevenness.

Returning to FIG. 5, the uniformity adjustment of unevenness (color variation) is linearly proportional to the XYZ value of the color at the center point (u = 0.5) depending on the appearance of unevenness in the XYZ color space. That is, the brighter the light, the larger the ratio of the gray texture to the center point. That is, the brighter the brightness of the light, the larger the uniform texture H _XYZ (u, v) from the central point in the XYZ color space.

Here, an example of uniformity adjustment according to the present embodiment will be described with reference to FIGS. 7A and 7B. 7A and 7B are diagrams showing an example of uniformity adjustment according to the present embodiment. FIG. 7A is information before the uniformity adjustment process is executed. As shown in "Color Sampling Position", the gray output from the projector 2 is sampled on the projection surface. The more to the left of the center point, the more blackish the color is than the color of the center point, and the more to the right of the center point, the more whitish the color is than the color of the center point. That is, the projected image of the projected surface is uneven. As shown in "Uniformity Calibration", RGB uniformity is not adjusted.

FIG. 7B is information after the uniformity adjustment process is executed. As shown in "Color Sampling Position", the gray output from the projector 2 is sampled on the projection surface. The color of each position sampled on the projection plane is almost the same as the color of the center point. As shown in "Uniformity Calibration", the uniformity of RGB is adjusted.

As a result, the video output control device 1 performs the adjustment process by the color adjustment unit 11 and the uniformity adjustment unit 12 for each projector 2, so that even among the plurality of projectors 2, all of them are in the same XYZ color space. It is possible to perform video composition.

Returning to FIG. 3, the geometry adjustment unit 13 corrects the geometry of the video data for each projector 2 so that the distortion of the video projected from each of the projectors 2 is eliminated. For example, the geometry adjustment unit 13 corrects the geometry of the video data by using the previously held correspondence between the pixel coordinates of the projector 2 and the coordinates of the capture camera. The correspondence between the pixel coordinates of the projector 2 and the coordinates of the capture camera may be stored in the storage unit 20 as a 32-bit texture in advance using the camera 4.

The video synchronization unit 14 synchronizes the respective images output from the plurality of projectors 2. The synchronization method may be performed by any conventional technique.

The video output unit 15 outputs the synchronized video to the projector 2.

[Use of video output control]
8A and 8B are diagrams showing an example of an application of video output control according to the present embodiment.

As shown in FIG. 8A, the image output from one of the projectors P1 is RGB ₁ . RGB ₁ is an image to be shown on the projection surface. The image output from the other projector P2 is RGB ₂ . RGB ₂ is an image to be hidden by the projection surface.

In such a case, the video output control device 1 converts RGB of RGB ₁ into XYZ. For this conversion, the RGB-XYZ conversion formula associated with the projector P1 may be used. Further, the video output control device 1 converts RGB of RGB ₂ into XYZ. For this conversion, the RGB-XYZ conversion formula associated with the projector P2 may be used.

The video output control unit 1 subtracts the XYZ ₂ converted from XYZ ₁ multiplied by K the RGB ₁ of P1 side XYZ ₁ converted into XYZ color space RGB ₂ of P2 side in XYZ color space. K may be a value greater than 1. The reason for multiplying XYZ ₁ by K is to prevent the XYZ value from becoming an unapplicable value by subtracting the XYZ value.

Then, the video output control device 1 adjusts the uniformity of the XYZ ₁ as a result of the subtraction on the P1 side. This uniformity adjustment may be performed by using the homogenization formula and the formula (11) associated with the projector P1. Further, the video output control device 1 adjusts the uniformity of the XYZ ₂ on the P2 side. This uniformity adjustment may be performed by using the homogenization formula and the formula (11) associated with the projector P2.

Then, the video output control device 1 converts the uniformity-adjusted XYZ ₁ on the P1 side from XYZ to RGB. For this conversion, the XYZ-RGB conversion formula associated with the projector P1 may be used. Further, the video output control device 1 converts the uniformity-adjusted XYZ ₂ on the P2 side from XYZ to RGB. For this conversion, the XYZ-RGB conversion formula associated with the projector P2 may be used.

Then, the video output control device 1 adjusts the geometry of the projector P1 and adjusts the geometry of the projector P2.

Then, the video output control device 1 synchronizes the XYZ-RGB converted RGB ₁ output from the projector P1 with the XYZ-RGB converted RGB ₂ output from the projector P2. Then, the video output control device 1 outputs RGB ₁ that has undergone XYZ-RGB conversion to the projector P1. The video output control device 1 outputs RGB ₂ that has undergone XYZ-RGB conversion to the projector P2.

As a result, the video output control device 1 allows the projector P1 and the projector P2 to display an image obtained by subtracting the image to be hidden from the image to be shown on the projection surface.

As shown in FIG. 8B, the original image output from the projector P1 and the projector P2 is RGB ₁ .

In such a case, the video output control device 1 converts RGB of RGB ₁ on the P1 side into XYZ. For this conversion, the RGB-XYZ conversion formula associated with the projector P1 may be used. Further, the video output control device 1 converts RGB of RGB ₁ on the P2 side into XYZ. For this conversion, the RGB-XYZ conversion formula associated with the projector P2 may be used.

Then, the video output control device 1 generates XYZ ₁ obtained by multiplying XYZ _{1 obtained} by converting RGB ₁ on the P1 side into an XYZ color space by a binary mask. Further, the video output control device 1 generates XYZ ₂ obtained by multiplying XYZ _{1 obtained} by converting RGB ₁ on the P2 side into an XYZ color space by a binary mask.

Then, the video output control device 1 adjusts the uniformity of the XYZ ₁ on the P1 side. This uniformity adjustment may be performed by using the homogenization formula and the formula (11) associated with the projector P1. Further, the video output control device 1 adjusts the uniformity of the XYZ ₂ on the P2 side. This uniformity adjustment may be performed by using the homogenization formula and the formula (11) associated with the projector P2.

Then, the video output control device 1 converts the uniformity-adjusted XYZ ₁ on the P1 side from XYZ to RGB. For this conversion, the XYZ-RGB conversion formula associated with the projector P1 may be used. Further, the video output control device 1 converts the uniformity-adjusted XYZ ₂ on the P2 side from XYZ to RGB. For this conversion, the XYZ-RGB conversion formula associated with the projector P2 may be used.

Then, the video output control device 1 adjusts the geometry of the projector P1 and adjusts the geometry of the projector P2.

Then, the video output control device 1 synchronizes the XYZ-RGB converted RGB ₁ output from the projector P1 with the XYZ-RGB converted RGB ₂ output from the projector P2. Then, the video output control device 1 outputs RGB ₁ that has undergone XYZ-RGB conversion to the projector P1. The video output control device 1 outputs RGB ₂ that has undergone XYZ-RGB conversion to the projector P2.

As a result, the video output control device 1 can cause the projector P1 and the projector P2 to display the image masked on the projection surface, respectively.

[Example of video with video output control]
9A and 9B are diagrams showing an example of a video whose video output is controlled according to the present embodiment.

The video shown in FIG. 9A is an example of a video whose video output is controlled according to the application shown in FIG. 8A. Here, the white background is the “image to be shown” output from the projector P1 with respect to the projection surface. Since a person is standing in front of the projector P1, a shadow is reflected on the projection surface. The image in which the pattern is drawn on the projection surface is the "image to be hidden" output from the projector P2. By subtracting the pattern of the "image to be hidden" of the projector P2 from the "image to be shown" of the projector P1 and synthesizing it with the "image to be hidden" of the projector P2, the periphery of the shadow is displayed on a white background. And a pattern is displayed in the shadow.

If the color of the image projected from each projector 2 is not subjected to the color adjustment process and the homogenization adjustment process, the white around the shadow will be uneven due to individual differences of the projector 2. The video output control device 1 according to the present embodiment projects the colors output from the respective projectors 2 by performing the color adjustment processing and the homogenization adjustment processing on the colors of the video projected from the respective projectors 2. It can be controlled to be white on the surface.

The video shown in FIG. 9B is an example of a video whose video output is controlled according to the application shown in FIG. 8B. Here, the projector P1 projects so as to create a shadow from the left side of the projection surface. The projector P2 projects so as to create a shadow from the right side of the projection surface. And the masked part is displayed in white.

If the color of the image projected from each projector 2 is not subjected to the color adjustment processing and the uniformity adjustment processing, the white of the masked portion will be uneven due to individual differences of the projector 2. The video output control device 1 according to the present embodiment projects the colors output from the respective projectors 2 by performing the color adjustment processing and the homogenization adjustment processing on the colors of the video projected from the respective projectors 2. It can be controlled to be white on the surface.

[Effect of this embodiment]
According to the above embodiment, the video output control device 1 adjusts the color of each video output from the multi-projector using a common color space. The video output control device 1 outputs a video based on the adjustment. According to such a configuration, the video output control device 1 can adjust the color of the video output from the multi-projector so that there is no individual difference between the multi-projectors. That is, the video output control device 1 adjusts the color of each video output from the multi-projector by using a common color space, so that even if the color is the same due to individual differences of the projector 2 or the usage time of the lamp light source, for example. It is possible to adjust different colors so that they are the same color.

Further, according to the above embodiment, the video output control device 1 measures the color information of a specific point on the projection surface with respect to the video output from the multi-projector. The video output control device 1 calculates a first conversion formula used for color matching in a common color space based on the measured color information. According to such a configuration, the video output control device 1 can correct each color of the multi-projector to a common color by using a common color space.

Further, according to the above embodiment, the video output control device 1 further outputs a predetermined gray color from the multi-projector to the projection surface. The video output control device 1 obtains gray texture data to be input to the multi-projector so as to be uniform with reference to a specific point on the projection surface, and uses the gray texture data in a common color space based on the first conversion formula. Convert to texture data. The video output control device 1 calculates a second conversion formula based on the converted texture data, and adjusts the color of the video output from the multi-projector so as to be uniform using the second conversion formula. According to such a configuration, the video output control device 1 can uniformly correct each non-uniformity (unevenness) of the multi-projector by using a common color space.

Further, according to the above embodiment, the video output control device 1 further converts the color-adjusted video from the common color space to the RGB color space. According to such a configuration, the video output control device 1 can apply the color-adjusted video to the multi-projector, and can adjust the color of the video output from the multi-projector so that there is no individual difference between the multi-projectors. ..

[Other]
In the present embodiment, the color adjusting unit 11 measures the XYZ value of a specific point on the projection surface of the image output from the projector 2 by using the spectroscope 3. However, the color adjustment unit 11 is not limited to this, and the spectroscope 3 is replaced with a device capable of measuring the value of a common color space such as the XYZ color space. A value such as an XYZ value at a specific point on the projection surface may be measured.

Further, in the present embodiment, the color adjustment unit 11 calculates an RGB-XYZ conversion formula used for color matching in a common color space based on the XYZ values measured by the R, G, and B channels alone. .. For example, it was explained that the X value measured by the R channel alone is expressed as X _R = a _RX R ^g . It was explained that the Y value measured by the R channel alone is expressed as Y _R = a _RY R ^g . It was explained that the Z value measured by the R channel alone is expressed as Z _R = a _RZ R ^g . However, X _R , Y _R , and Z _R are not limited to this, and an offset on the projection plane may be added. That is, the X value measured by the R channel alone may be expressed as X _R = a _RX R ^g + b _RX . The Y value measured by the R channel alone may be expressed as Y _R = a _RY R ^g + b _RY . The Z value measured by the R channel alone may be expressed as Z _R = a _RZ R ^g + b _RZ . Similarly, an offset may be added to the XYZ value measured by the G channel alone and the XYZ value measured by the B channel alone. These offsets correspond to the ambient light from other than the projector 2 and the XYZ value when the black of the projector 2 is output. Then, if the color adjustment unit 11 calculates the RGB-XYZ conversion formula used for color matching in a common color space based on the XYZ value to which the offset measured by the R, G, and B channels alone is added. Good.

Further, the video output control device 1 is a known information processing device such as a personal computer or workstation, and the color adjustment unit 11, the uniformity adjustment unit 12, the geometry adjustment unit 13, the video synchronization unit 14, and the video output unit 15 described above. It can be realized by installing each function such as.

Further, each component of the video output control device 1 shown does not necessarily have to be physically configured as shown in the figure. That is, the specific embodiment of the distribution / integration of the video output control device 1 is not limited to the one shown in the drawing, and all or part of the distribution / integration is functionally or physically in an arbitrary unit according to various loads and usage conditions. Can be distributed and integrated into the configuration. For example, the video synchronization unit 14 and the video output unit 15 may be integrated as one unit. Further, the color adjustment unit 11 may be separated into a measurement unit that measures the XYZ value of the projection surface and a calculation unit that calculates the RGB-XYZ conversion formula. Further, the storage unit 20 may be connected via a network as an external device of the video output control device 1.

Although the embodiments of the present disclosure have been described above, the technical scope of the present disclosure is not limited to the above-described embodiments as they are, and various changes can be made without departing from the gist of the present disclosure. Moreover, you may combine suitably the component over different embodiment and a modification.

<Modification example 1>
The embodiments of the present disclosure can provide a three-dimensional image representation on a two-dimensional projection surface by a plurality of projectors. Hereinafter, a modification 1 relating to the embodiment of the present disclosure will be described.

Modification 1 is a modification of the video expression described with respect to FIGS. 9A and 9B. Therefore, the description of the same configuration as in FIGS. 9A and 9B is omitted. In the first modification, the video output control device 1 outputs video data related to the three-dimensional virtual space to a plurality of projectors. More specifically, the video output control device 1 outputs short-distance video data and long-distance video data to, for example, a short-distance projector and a long-distance projector among a plurality of projectors, respectively. The short-distance video data in the first modification may be regarded as corresponding to the “image to be hidden” described with respect to FIGS. 9A and 9B.

The short-distance video data represents an image closer to the user's viewpoint of observing the projection surface than the long-distance video data in a three-dimensional virtual space. The short-distance video data and the long-distance video data represent continuous video in the depth direction in the virtual space. The data formats of the short-distance video data and the long-distance video data expressed in the three-dimensional virtual space may be the same as the data formats used for general games and VR videos.

Hereinafter, the user experience provided by the modification 1 will be described by taking as an example a virtual object extending in the depth direction of the virtual space from the long-distance video data to the short-distance video data. As described above, in the video output control device 1 of the present disclosure, at a specific point on the projection surface, the color of the specific point in the image projected on the same projection surface is the same color among the plurality of projectors. Video data can be output. Therefore, in the video output control device 1 of the present disclosure, the output of the short-distance portion and the output of the long-distance portion of the virtual object extending in the depth direction are shared by the short-distance projector and the long-distance projector, respectively, and the user. Seen from the perspective, it can be represented as a single seamless virtual object on the projection plane.

When a person stands in front of the projector 2 that projects the long-distance video data, only the short-distance video data is reflected in the shadow of the person who has moved to the projection surface. Therefore, the user feels as if the virtual object extending in the depth direction of the virtual space penetrates his / her body. When the video output control device 1 moves the virtual object in the depth direction, a higher sense of reality of the virtual object that penetrates the shadow is presented.

<Modification 2>
The embodiment of the present disclosure can provide product information and the like that can be visually recognized only when the user approaches the projection surface by a plurality of projectors. Hereinafter, a second modification of the embodiment of the present disclosure will be described.

Modification example 2 is a modification of the video expression described with respect to FIGS. 9A and 9B, as in the case of modification 1. Therefore, the description of the same configuration as that of FIGS. 9A and 9B or the same configuration as that of the first modification is omitted. In the second modification, the video output control device 1 projects, for example, a general street advertisement onto a projection surface as the “image to be shown” described with respect to FIGS. 9A and 9B on a plurality of projectors. Alternatively, various video contents are projected as "images to be shown". The projection surface may be a wall surface of a building or a storefront window provided with a reflective material that reflects the light of the projector. Further, the video output control device 1 presents different product information or the like for each area of the projection surface as an “image to be hidden”. For example, when the projection surface is a wall surface, the "image to be hidden" related to the area corresponding to the "image to be shown" is projected. More specifically, when the "image to be shown" is a specific product image or a person image wearing a specific product, detailed information related to the product is projected as the "image to be hidden". Alternatively, when the projection surface is a store window, detailed information about the product (real object) arranged in the window is presented as an "image to be hidden" on the projection surface corresponding to the arranged product.

According to the second modification, when a user who is interested in the "image to be shown" approaches the image, detailed information related to the image to be shown is displayed to the user as an "image to be hidden" on the projection surface. Presented within the user's shadow area above. Here, other users who are not close to the projection plane are hidden behind the user who is close to the projection plane, and the "image to be hidden" is still invisible. That is, on a single projection plane, a plurality of users are simultaneously provided with different video experiences according to the distance of the projection plane.

The presentation of the "image to be hidden" is performed by blocking the light of the projector P1 caused by the natural movement of the user. Therefore, it is not necessary to detect the position of the user using a sensor when presenting the “image to be hidden”.

According to the configuration of the modification 2, it is possible to provide a new video experience according to the natural interaction of each user. The "image to be hidden" is not limited to the product information, and may be various information related to the "image to be shown". According to such a configuration, information of a plurality of layers can be projected on a single projection surface, and detailed information according to the user's interest in the "image to be shown", that is, the "image to be hidden". Can be dynamically presented without providing a special sensor.

Further, the effects in each embodiment described in the present specification are merely examples and are not limited, and other effects may be obtained.

The present technology can also have the following configurations.
(1)
Multiple projectors, each projecting video data represented in the first color space,
An adjustment unit that adjusts the color information of the video data input to each of the plurality of projectors in the second color space, and
An output unit that outputs the video data adjusted by the adjustment unit to each of the plurality of projectors, and an output unit.
Video output system with.
(2)
The adjusting unit is the image expressed in the first color space so that the color of a specific point in the image projected on the same projection surface from each of the projectors is the same color among the plurality of projectors. The video output system according to (1) above, wherein the color information of the data is adjusted on the second color space.
(3)
The adjusting part
Correspondence between the color information in the video data of the image projected on the specific point by each of the projectors and the color information in the second color space obtained by measuring the image projected on the specific point. Based on the relationship, the first conversion formula for converting the video data represented in the first color space into the video data represented in the second color space was obtained.
By converting the video data expressed in the first color space into the video data expressed in the second color space using the first conversion formula, the data is input to each of the plurality of projectors. The video output system according to (2) above, which adjusts the color information of video data on a second color space.
(4)
The video output system according to (2) or (3), further comprising a measuring instrument for measuring color information of the video projected on the specific point.
(5)
The video output system according to (4) above, wherein the measuring instrument is a spectroscopic radiation system.
(6)
The adjusting unit inputs the video data to each of the plurality of projectors so that the video projected on the projection surface becomes uniform with respect to the specific point when a monochrome video is projected from each of the projectors. The video output system according to any one of (2) to (5), wherein the color information of the above is adjusted on the second color space.
(7)
The adjusting part
When the monochromatic image is projected from each of the projectors, the image projected on the projection surface is uniform with respect to the specific point based on the ratio of the color information measured at each of the plurality of points on the projection surface. Find the second conversion formula for
By converting the video data using the second conversion formula on the second color space, the color information of the video data input to each of the plurality of projectors can be transmitted on the second color space. The video output system according to (6) above.
(8)
The adjusting unit further converts the video data whose color information is adjusted on the second color space into the video data expressed in the first color space.
The item according to any one of (1) to (7), wherein the output unit inputs the video data converted from the second color space to the first color space into each of the plurality of projectors. Video output system.
(9)
The first color space is an RGB color space.
The video output system according to any one of (1) to (8) above, wherein the second color space is a color space of the XYZ color system.
(10)
It is a video output control method that projects video from a plurality of projectors, each of which projects video data expressed in the first color space.
The color information of the video data input to each of the plurality of projectors is adjusted on the second color space.
A video output control method including outputting the adjusted video data to each of the plurality of projectors.
(11)
It is a video output control method that projects video from a plurality of projectors, each of which projects video data expressed in the first color space.
Each of the projectors measures the color information on the second color space of the image projected on the same projection surface at a specific point, and then measures the color information.
Based on the correspondence between the color information in the video data of the video projected by each of the projectors at the specific point and the color information on the second color space measured at the specific point, the first A first conversion formula for converting the video data represented in the first color space into the video data represented in the second color space was obtained.
Using the first conversion formula, the video data represented in the first color space is converted into video data represented in the second color space.
When the monochromatic image is projected from each of the projectors, the color information at each of the plurality of points on the projection surface is measured.
Based on the ratio of the color information measured at each of the plurality of points, a second conversion formula for making the monochromatic image projected on the projection surface uniform with respect to the specific point was obtained.
A video output control method including converting video data input to each of the projectors on the second color space using the second conversion formula.

1 Video output control device 2 Projector 3 Spectrometer 4 Camera 5 Output board 9 Video output system 10 Control unit 11 Color adjustment unit 12 Uniformity adjustment unit 13 Geometry adjustment unit 14 Video synchronization unit 15 Video output unit 20 Storage unit 21 RGB-XYZ Convertible table 22 Uniformized table

Claims (11)

Multiple projectors, each projecting video data represented in the first color space,
An adjustment unit that adjusts the color information of the video data input to each of the plurality of projectors in the second color space, and
An output unit that outputs the video data adjusted by the adjustment unit to each of the plurality of projectors, and an output unit.
Video output system with. The adjusting unit is the image expressed in the first color space so that the color of a specific point in the image projected on the same projection surface from each of the projectors is the same color among the plurality of projectors. The video output system according to claim 1, wherein the color information of the data is adjusted on the second color space. The adjusting part
Correspondence between the color information in the video data of the image projected on the specific point by each of the projectors and the color information in the second color space obtained by measuring the image projected on the specific point. Based on the relationship, the first conversion formula for converting the video data represented in the first color space into the video data represented in the second color space was obtained.
By converting the video data represented in the first color space into video data represented in the second color space using the first conversion formula, the video data is input to each of the plurality of projectors. The video output system according to claim 2, wherein the color information of the video data is adjusted on the second color space. The video output system according to claim 2, further comprising a measuring instrument that measures the color information of the video projected on the specific point. The video output system according to claim 4, wherein the measuring instrument is a spectroscopic radiation system. The adjusting unit inputs the video data to each of the plurality of projectors so that the video projected on the projection surface becomes uniform with respect to the specific point when a monochrome video is projected from each of the projectors. The video output system according to claim 2, wherein the color information of the above is adjusted on the second color space. The adjusting part
When the monochromatic image is projected from each of the projectors, the image projected on the projection surface is uniform with respect to the specific point based on the ratio of the color information measured at each of the plurality of points on the projection surface. Find the second conversion formula for
By converting the video data using the second conversion formula on the second color space, the color information of the video data input to each of the plurality of projectors can be transmitted on the second color space. The video output system according to claim 6, which is adjusted. The adjusting unit further converts the video data whose color information is adjusted on the second color space into the video data expressed in the first color space.
The video output system according to claim 1, wherein the output unit inputs the video data converted from the second color space to the first color space into each of the plurality of projectors. The first color space is an RGB color space.
The video output system according to claim 1, wherein the second color space is a color space of an XYZ color system. It is a video output control method that projects video from a plurality of projectors, each of which projects video data expressed in the first color space.
The color information of the video data input to each of the plurality of projectors is adjusted on the second color space.
A video output control method including outputting the adjusted video data to each of the plurality of projectors. It is a video output control method that projects video from a plurality of projectors, each of which projects video data expressed in the first color space.
Each of the projectors measures the color information on the second color space of the image projected on the same projection surface at a specific point, and then measures the color information.
Based on the correspondence between the color information in the video data of the video projected by each of the projectors at the specific point and the color information on the second color space measured at the specific point, the first A first conversion formula for converting the video data represented in the first color space into the video data represented in the second color space was obtained.
Using the first conversion formula, the video data represented in the first color space is converted into video data represented in the second color space.
When a monochrome image is projected from each of the projectors, color information is measured at each of a plurality of points on the projection surface, and the color information is measured.
Based on the ratio of the color information measured at each of the plurality of points, a second conversion formula for making the monochromatic image projected on the projection surface uniform with respect to the specific point was obtained.
A video output control method including converting video data input to each of the projectors on the second color space using the second conversion formula.

Images