JP2019028165A - Image display device, control device, image display method, and control method - Google Patents

Abstract

An object of the present invention is to easily perform image quality setting based on a projection plane by operating a user interface without editing image data. An image display device according to the present invention includes image projection means for projecting an image onto a projection plane, one or more superimposed areas where a projected image overlaps with an object that is different from the projection plane, and other superimposition areas. detection means for detecting a non-overlapping area which is an area; image quality setting means for separately setting the image quality of the overlapping area and the non-overlapping area by a user; and correction means for correcting the image data of the superimposed region or the non-superimposed region based on the above. [Selection drawing] Fig. 2

Description

  The present invention relates to an image display device, a control device, an image display method, and a control method.

  In recent years, there has been an effect of projecting an image onto a wall surface on which an object (target object) such as a printed matter, a painting, or an illustration is installed. In such a case, it is desired to display an image with a user-desired image quality, such as increasing the contrast of an object or suppressing color unevenness caused by illumination light or the like.

  Patent Document 1 discloses a technique for projecting an image so as to overlap an object (printed material on which an image is printed) placed on a projection surface. Thereby, the contrast of the image on the printed matter can be increased, and the dynamic range can be expanded. Patent Document 2 discloses a technique for correcting image data based on a captured image of a projection surface. Thereby, it is possible to suppress color unevenness caused by unevenness of the color of the projection surface and the luminance of the illumination light.

JP 2008-083180 A JP 2004-158941 A

  In general, when an image is projected on a projection surface on which an object is installed, the reflection characteristics are different between the object and other regions. In such a case, in order to display an image with a desired image quality, it is necessary to project the image in consideration of the difference in the reflection characteristics of the projection surface.

  However, in Patent Document 1, when the reflection characteristics of the projection surface are not uniform, it is necessary to prepare image data in advance in consideration of the reflection characteristics. However, advanced knowledge and a dedicated editing device are required. Further, if the desired image quality is not obtained even if image data prepared in advance is used, it is necessary to edit the image data again. Further, in Patent Document 2, since the image data is automatically corrected so as to cancel the difference in reflection characteristics, it is not assumed that the contrast of the projection surface is increased. For this reason, the user cannot set the image quality in consideration of the difference in the reflection characteristics of the projection surface.

  SUMMARY An advantage of some aspects of the invention is that it provides a technique capable of easily setting an image quality based on a projection plane by operating a user interface without editing image data.

The first aspect of the present invention is:
Image projection means for projecting an image onto a projection plane;
Detecting means for detecting one or more overlapping regions in which an object that is an object different from the projection plane and a projection image overlap and a non-overlapping region that is another region;
Image quality setting means for setting the image quality of the superimposed area and the non-superimposed area separately by the user,
Correction means for correcting image data of the superimposing region or the non-superimposing region based on a user instruction performed using the image quality setting unit;
It is an image display apparatus characterized by having.

The second aspect of the present invention is:
A control device for controlling a projection device having image projection means for projecting an image onto a projection plane,
Detecting means for detecting one or more overlapping regions in which an object that is an object different from the projection plane and a projection image overlap and a non-overlapping region that is another region;
Image quality setting means for setting the image quality of the superimposed area and the non-superimposed area separately by the user,
Correction means for correcting image data of the superimposing region or the non-superimposing region based on a user instruction performed using the image quality setting unit;
It is a control device characterized by having.

The third aspect of the present invention is:
An image display method in an image display device for projecting an image onto a projection plane,
A detection step of detecting one or more overlapping regions in which an object that is an object different from the projection plane and a projection image overlap and a non-overlapping region that is another region;
An image quality setting step for receiving an input of settings separately from the user for the image quality of the superimposed region and the non-superimposed region,
A correction step of correcting the image data of the superimposing region or the non-superimposing region based on a user instruction performed in the image quality setting step;
An image projecting step of projecting an image to be corrected in the correcting step;
It is the image display method characterized by having.

The fourth aspect of the present invention is:
A control method in a control device for controlling a projection device having image projection means for projecting an image onto a projection plane,
A detection step of detecting one or more overlapping regions in which an object that is an object different from the projection plane and a projection image overlap and a non-overlapping region that is another region;
An image quality setting step for receiving an input of settings separately from the user for the image quality of the superimposed region and the non-superimposed region,
A correction step of correcting the image data of the superimposing region or the non-superimposing region based on a user instruction performed in the image quality setting step;
It is the control method characterized by having.

  A fifth aspect of the present invention is a program for causing a computer to execute each step of the image display method of the image display device or the control method of the control device.

  According to the present invention, it is possible to easily perform image quality setting based on a projection plane by operating a user interface without editing image data.

The figure which shows an example of the image display system which concerns on Embodiment 1 of this invention. 1 is a functional block diagram showing an example of an image projection apparatus according to Embodiment 1 of the present invention. The flowchart which shows an example of the detection process which concerns on Embodiment 1 of this invention. The figure which shows an example of the brightness | luminance histogram of the detection process result which concerns on Embodiment 1 of this invention. The flowchart which shows an example of the detection process which concerns on Embodiment 1 of this invention. The figure which shows an example of the image quality setting menu which concerns on Embodiment 1 of this invention. The figure which shows an example of the image display system which concerns on Embodiment 2 of this invention. Functional block diagram showing an example of an image projection apparatus according to Embodiment 2 of the present invention The flowchart which shows an example of the detection process which concerns on Embodiment 2 of this invention. Functional block diagram showing an example of an image projection apparatus according to Embodiment 3 of the present invention The flowchart which shows an example of the detection process which concerns on Embodiment 3 of this invention.

<Embodiment 1>
Hereinafter, Embodiment 1 of the present invention will be described.

(overall structure)
FIG. 1 is a configuration diagram illustrating an example of an image display system according to the present embodiment.

  The image display system according to the present embodiment includes an image projection device 100 (image display device) and a printed material 160 (object) that is an object different from the wall surface 150 installed on the wall surface 150. The image projection apparatus 100 is installed so as to include the printed material 160 within the range of the projection area 170. In the following description, among the projected images projected onto the projection area 170, the area projected onto the printed material 160 is the superimposed area 171 (horizontal line area in FIG. 1), and the area not projected onto the printed material 160 is the non-superimposed area. 172 (vertical line region in FIG. 1). In the present embodiment, as shown in FIG. 1, an example in which a printed material 160 is installed inside the projection area 170 and an image corresponding to the printed material 160 is projected from the image projection apparatus 100 to the projection area 170 will be described.

  FIG. 2 is a functional block diagram illustrating an example of the image projection apparatus 100 according to the present embodiment. The image projection apparatus 100 according to the present embodiment is an information processing apparatus (computer) including an arithmetic device, a storage device, an input / output device, and the like. When the arithmetic unit executes the program stored in the storage device, the acquisition unit 101, the region detection unit 102, the operation input unit 103, the image quality setting unit 104, the correction unit 105, and the projection image generation unit 106 of the image projection device 100 are executed. The functions of the image projection unit 107 and the like are provided. Some or all of these functions may be implemented by a dedicated logic circuit such as an ASIC or FPGA. The image projection unit 107 includes a liquid crystal panel that modulates transmittance based on a projection image, a light source that projects light onto the liquid crystal panel, and the like.

  The acquisition unit 101 is a functional unit that acquires image data of a projection image corresponding to the printed material 160. The acquisition unit 101 acquires image data from a personal computer, file server, cloud storage, or the like via a cable or wirelessly. Note that a camera (not shown) may be mounted on the acquisition unit 101, and imaging data obtained by imaging the printed matter 160 by the camera may be acquired as image data. The projected image is described as a still image, but may be a moving image or the like.

  The region detection unit 102 is a functional unit that detects the position (superimposed region 171) of the printed material 160 installed on the projection plane. The area detection unit 102 includes, for example, an imaging unit such as a camera and a calculation unit that analyzes image data, and detects the position of the printed material 160 installed on the projection surface. Further, the area detection unit 102 outputs the detected position information of the printed material 160 to the correction unit 105 and the image quality setting unit 104 as coordinate information of the superimposed area 171 converted into a coordinate value relative to the projection area 170. The details of the operation of the area detection unit 102 will be described in the description of processing using the flowcharts of FIGS. 3 and 5 described later.

  The operation input unit 103 is a button or switch, a remote controller, a keyboard, or the like mounted on the image projection apparatus 100. When operated by the user, the operation input unit 103 makes an image quality setting request to the image quality setting unit 104.

The image quality setting unit 104 generates an image quality setting menu in response to an image quality setting request from the operation input unit 103, and outputs it to a projection image generation unit 106 described later. In addition, the image quality setting unit 104 updates the image quality setting menu according to the user's image quality setting operation, and outputs image quality setting parameters according to the operation to the correction unit 105 described later. Details of the operation of the image quality setting unit 104 will be described with reference to FIG.

  The correction unit 105 is a functional unit that corrects the image data acquired by the acquisition unit 101 based on a user image quality setting operation. Here, the correction unit 105 can hold a plurality of image quality setting parameters set by the image quality setting unit 104. The correction unit 105 selects one of the image quality setting parameters held for each of the superimposed region 171 and the non-superimposed region 172 detected by the region detection unit 102 and performs correction processing. The correction processing is general correction processing such as offset / gain correction, gamma correction, color correction, white balance correction, noise reduction, and sharpness correction.

  The projection image generation unit 106 is a functional unit that generates image data of an image projected by the image projection apparatus 100. In the present embodiment, the projection image generation unit 106 uses the corrected image data output from the correction unit 105 and the image quality setting menu image data output from the image quality setting unit 104 to set the image quality for the corrected image. Image data is generated so that the menu is superimposed.

  The image projection unit 107 is a functional unit that projects an image onto a projection plane. The intensity of the light source constituting the image projection unit 107 is controlled based on the projection image, user designation, and the like. The image projection unit 107 may be configured using a MEMS shutter or the like instead of the liquid crystal panel.

(Detection process)
Details of the detection processing of the region detection unit 102 according to the present embodiment will be described with reference to FIG.

  In step S102, the image projection apparatus 100 acquires a projection plane image (captured image) in response to a projection start instruction from the user. For example, the image projection apparatus 100 captures the projection plane using a camera mounted on the area detection unit 102 or the like. The projection start instruction may be a power-on operation or the like.

  In step S <b> 104, the image projection apparatus 100 starts projecting an image based on the image data acquired by the acquisition unit 101 from the image projection unit 107. In the present embodiment, the image projection apparatus 100 acquires the projection plane image before projecting the image in order to prevent the projection image from being reflected in the acquired projection plane image. It is not limited to. For example, the image projection apparatus 100 may acquire a projection plane image during image projection or after completion of image projection.

  In step S <b> 106, the region detection unit 102 extracts the image feature amount of the projection plane image acquired by the acquisition unit 101 in order to detect the position of the printed material 160. Specifically, the image data of the projection plane image is divided into blocks by the calculation unit constituting the region detection unit 102, and a luminance histogram is acquired for each block (region). As the block, for example, a square block of 32 pixels × 32 pixels can be used, but the size and shape are not particularly limited. The acquired histogram may be a color histogram such as RGB instead of the luminance histogram.

FIG. 4 shows an example of a luminance histogram acquired by the region detection unit 102. The horizontal axis in FIG. 4 is the luminance value, and the vertical axis is the frequency. When a color histogram is acquired, the horizontal axis represents the color gradation value. Here, the histogram acquired by the region detection unit 102 has a narrow distribution range as shown in FIG. 4A if the color and brightness of the projection plane are uniform, and various colors and brightnesses are displayed on the projection plane. Is present, the distribution range becomes wide as shown in FIG. In other words, the distribution of the histogram is close to the distribution shown in FIG. 4A on the projection surface where the color and brightness are substantially uniform, such as the wall or screen on which the printed material is placed, and a predetermined image is printed like the printed material. In such a projection plane, the distribution is close to that shown in FIG.

  In step S <b> 108, the area detection unit 102 detects the position of the printed material 160 based on the acquired histogram. Specifically, referring to the histogram of each block (area), if the distribution width of the histogram is larger than a predetermined width, it is detected as a printed area, and if it is smaller than the predetermined width, the printed material 160 is installed. The background area (wall surface, screen, etc.) is determined. Here, the width of the distribution of the histogram in the present embodiment is the number of luminance values equal to or higher than a predetermined frequency in the acquired histogram.

  In step S <b> 110, the area detection unit 102 converts the detected position information of the printed material 160 into a coordinate value relative to the projection area 170 and outputs the converted coordinate value. Specifically, the position information of the printed material 160 is normalized with the resolution of the projection area 170. For example, when the resolution of the image projected by the image projection unit 107 is 1920 × 1080, the upper left of the projection area 170 is the coordinate value (0, 0), the lower left is (0, 1079), and the upper right is (1919, 0). ), And normalize to the coordinate range where the lower right is (1919, 1079). Then, the normalized coordinate value is output as the coordinate information of the superimposed region 171. This normalization process is performed in order to handle the position of the printed matter in the projection plane image as the position of the superimposed region 171 in the projection image (the coordinate value of the pixel constituting the projection image).

  As described above, the area detection unit 102 detects the position of the printed material 160 by the process shown in FIG. 3 and outputs the coordinate information of the superimposed area 171. In addition, although the example which uses a brightness | luminance and a color as an image feature-value was demonstrated, you may use an edge component. In this case, since there is almost no edge component in the wall surface or screen area, and there is an edge component in the printed material area, the position of the printed material 160 can be detected. Moreover, although the example which detects the position of the printed matter 160 per block was demonstrated, the detection method is not restricted to this. For example, in consideration of the fact that general printed materials are often rectangular, the area detected in the processing of S108 may be corrected to be a rectangular area.

  In addition, in FIG. 3, the flow of detecting the printed material region only once at the start of projection has been described, but the timing of executing the processing shown in FIG. 3 and the number of times of execution are not limited to this. For example, the image projection may be stopped every time the user makes an image quality setting request, and the processing from S102 to S110 may be executed. Note that the image projection may be stopped by projecting an image (all black image) in which the gradation value of all pixels is zero, or may be performed by setting the output of the light source to zero. Further, an acceleration sensor (not shown) or the like may be mounted on the image projection apparatus 100, and the image projection may be stopped each time it is detected that the image projection apparatus 100 has moved, and the processing from S102 to S110 may be executed.

(Image quality setting process)
Details of the image quality setting process of the image quality setting unit 104 according to the present embodiment will be described. As the first process, the image quality setting unit 104 according to the present embodiment generates an image quality setting menu in response to an image quality setting request from the operation input unit 103 and outputs the menu to the projection image generation unit 106. Further, as a second process, the image quality setting unit 104 updates the image quality setting menu according to the user's image quality setting operation, and outputs the image quality setting parameter according to the operation to the correction unit 105.

  Here, in the first process, the image quality setting menu 104 illustrated in FIG. 6 is displayed when the coordinate information of the superimposed area 171 is output from the area detection unit 102 when the image quality setting menu is generated. (User interface) is generated. That is, the image quality setting unit 104 generates an image quality setting menu in which the user separately sets the image quality of the superimposed region 171 and the non-superimposed region 172.

  FIG. 6A shows an example in which the image quality setting menu includes an area selection area 1031 for designating an image quality setting target area and an image quality designation area 1032 for designating an image quality setting parameter. The image quality setting unit 104 displays the overlapping area 171 and the non-overlapping area 172 as options in the area selection area 1031. FIG. 6A shows a state in which the overlapping area 171 is designated as the image quality setting target area. The user designates an area for image quality setting in the area selection area 1031, and then designates an image quality setting parameter for the area in the image quality designation area 1032.

  FIG. 6B shows an example in which the image quality setting menu includes an image quality setting menu 1033 for the overlapping area and an image quality setting menu 1034 for the non-overlapping area. FIG. 6B shows a state in which the image quality setting menu 1033 for the overlapping area is active. The user activates a menu for an area for setting image quality and designates each image quality setting parameter.

  Note that the image quality setting unit 104 may display other than the image quality setting menu as shown in FIG. For example, the image quality setting unit 104 may use parameters other than those shown in FIG. The number and arrangement of parameters that can be set are not particularly limited. Further, the image quality setting unit 104 may generate a frame line or the like of the superimposed region 171 or the non-superimposed region 172 on the projected image in order to highlight the region for which the user sets the image quality. Further, the image projection apparatus 100 is further provided with a user position detection unit (not shown) configured with a camera or the like, and the image quality setting unit 104 is closer to the detected user position in the superimposition region 171 and the non-superimposition region 172. The area may be set as a default image quality setting target area.

  In the second process, the image quality setting unit 104 generates an image quality setting parameter for correcting an image quality setting target area designated by the user in accordance with an instruction from the user. For example, it is assumed that the user performs an operation input that increases the brightness of the superimposed region 171 by 5 points. In this case, the image quality setting unit 104 changes the image quality setting parameter used for image correction of the superimposed region 171 to a value that increases the brightness by 5 points among the plurality of image quality setting parameters held by the correction unit 105. To do. In this way, the user can set the image quality of an area designated by the user in the overlapping area 171 or the non-overlapping area 172.

  Note that the image quality setting parameters (setting values) for the superimposed region 171 and the non-superimposed region 172 at the end of the projection may be stored, and the projection may be automatically performed with the same image quality setting parameter at the next projection. Also, the arrangement area of the superimposing area 171 and the non-superimposing area 172 is stored at the same time, and is automatically projected with the same image quality setting parameter only when the recorded arrangement and the current arrangement are the same at the start of the next projection. You may make it perform.

(Operational effect of this embodiment)
Through the operations as described above, the user can easily correct the difference in the appearance of the projected image caused by the difference in the image feature amount on the projection plane by operating the user interface without editing the image data. is there.

  In the present embodiment, the case where there is one printed material 160 has been described, but a plurality of printed materials 160 may exist. In this case, each area may be an area where independent image quality setting is possible, or the printed area and other areas may be areas where independent image quality setting is possible without distinguishing the printed material.

<Modification 1 of Embodiment 1>
As a first modification of the first embodiment, an example in which a marker is used instead of an image feature amount and an example in which a projection plane image and a projection image are used will be described.

  In the above-described example, the example in which the position of the printed matter is detected based on the image feature amount has been described, but the detection method is not limited thereto. For example, while printing by adding markers indicating the vertices of the printed material at the four corners of the rectangular printed material, the region detecting unit 102 may detect the position of the printed material by detecting the marker. In addition, although a two-dimensional code etc. can be used as a marker, it is not limited to this.

  In the above-described example, the example in which the position of the printed matter is detected only from the projection plane image has been described. However, the region detection unit 102 is not limited to one image. For example, assuming that the same image as the printed material or a similar image is superimposed on the printed material, the region detection unit 102 compares the projection surface image with the projected image to determine the position of the printed material (superimposed region). ) May be detected.

  In this case, first, the area detection unit 102 acquires a projection plane image by the same processing as in the above-described example. Then, the area detection unit 102 extracts an edge component from the acquired image data of the projection plane image and generates an edge image. The area detection unit 102 also receives projection image data from the acquisition unit 101 and generates an edge image of the projection image data. Then, the area detection unit 102 performs resolution conversion so that the resolutions of the two coincide with each other, and then divides the edge image into blocks, and compares the image data of each block (area). Then, the region detection unit 102 detects a block having a difference less than a predetermined value as a superposed region (a region that matches or is similar). The block size and shape may be the same as or different from those of the first embodiment. Further, the overlapping region may be detected from other than the edge component, and for example, it may be detected by a luminance value or a color difference.

<Modification 2 of Embodiment 1>
As a second modification of the first embodiment, an example of the detection process shown in FIG. 5 will be described. FIG. 5 is a flowchart in the case where the position of the printed material 160 is detected using an image of the projection surface acquired in advance and in a state where the printed material is not installed. The processing shown in FIG. 5 allows the area detection unit 102 to detect the installed printed matter regardless of the wall surface or screen characteristics on which the printed matter is placed.

  In step S152, the image projection apparatus 100 starts projecting a projection image in response to a projection start instruction from the user, and waits for an operation instruction from the user.

  In step S154, the area detection unit 102 determines whether or not there is an instruction to acquire the first projection plane image from the user before setting the printed material. In the present embodiment, an acquisition instruction from the user is performed via a button or switch mounted on the image projection apparatus 100, a remote controller, or the like. In this embodiment, when there is an acquisition instruction from the user, the process proceeds to step S156. If there is no acquisition instruction from the user, the process proceeds to step S162.

  In step S156, when there is an acquisition instruction from the user in step S154 (S154-YES), the image projection apparatus 100 stops image projection in order to acquire the first projection plane image. This is to prevent the projected image from being reflected in the acquired first projection plane image.

  In step S158, the image projection apparatus 100 acquires a first projection plane image using a camera mounted on the area detection unit 102 or the like. Further, when the first projection plane image has been acquired and there is an acquisition instruction from the user, the image projection apparatus 100 acquires and updates the first projection plane image.

  In step S160, image projection is resumed, and the process proceeds to step S154.

  In step S162, when there is no acquisition instruction of the first projection plane image before installation of the printed material from the user in step S154 (S154-NO), the region detection unit 102 determines the second projection plane image after installation of the printed material from the user. Determine whether there is an acquisition instruction. In the present embodiment, an acquisition instruction from the user is performed via a button or switch mounted on the image projection apparatus 100, a remote controller, or the like. If there is an acquisition instruction from the user, the process proceeds to step S164. If there is no acquisition instruction from the user, the process proceeds to step S176.

  The buttons used for the second projection plane image acquisition instruction may be the same as or different from the buttons used for the first projection plane image acquisition instruction. In the case of the same button or the like, the region detection unit 102 may determine whether the instruction is for obtaining the first or second projection plane image based on the number of times the button is pressed or the like. In step S162, the region detection unit 102 may determine whether there is a projection image quality setting instruction or the like instead of whether there is a second projection plane image acquisition instruction from the user. That is, the region detection unit 102 may acquire or update the second projection plane image when the user sets the image quality of the projection image.

  In step S164, when there is an acquisition instruction from the user in step S162 (S162-YES), the image projection apparatus 100 stops image projection in order to acquire the second projection plane image, similarly to step S156. .

  In step S <b> 166, the image projection device 100 acquires a second projection plane image using the camera that constitutes the region detection unit 102. When the user gives an acquisition instruction in a state where the second projection plane image has already been acquired, the region detection unit 102 acquires and updates the second projection plane image.

  In step S168, the image projection apparatus 100 resumes image projection, and the process proceeds to step S170.

  In step S <b> 170, the region detection unit 102 determines whether or not both the first projection plane image before setting the printed material and the second projection surface image after setting the printed material are acquired. When both are acquired, it progresses to step S172. If both are not acquired, the process proceeds to step S154.

  In step S172, when both the first projection plane image and the second projection plane image are acquired in step S170 (S170-YES), the area detection unit 102 has a difference based on both image feature amounts. The area to be detected is detected. The area detection unit 102 detects the difference area detected by this process as a superimposed area 171 where a printed matter or the like (object) placed on the projection surface is present.

In the present embodiment, the region detection unit 102 specifically detects the difference region as follows. First, the region detection unit 102 extracts an edge as a feature amount from two images and generates an edge image. Next, the area detection unit 102 divides the edge image into blocks and compares the image data of each block (area). Then, the region detection unit 102 detects a block in which a difference equal to or greater than a predetermined value exists as a difference region. The block size and shape may be the same as or different from those in the first embodiment. In addition, the region detection unit 102 may detect the difference region based on other than the edge component, and may detect it based on, for example, a luminance value or a color difference. In this case, the area detection unit 102 may detect the difference area after canceling the influence of the environmental light in consideration of the possibility that the intensity of the environmental light is different before and after installing the printed material. The intensity of the ambient light may be acquired by, for example, an ambient light sensor (not shown) mounted on the image projector 100 and added to the projection plane image as metadata when the projection plane image is acquired.

  In step S <b> 174, the region detection unit 102 converts the detected difference region into positional information relative to the projection region 170, and outputs it to the subsequent stage as coordinate information of the superimposed region 171. Since this process is the same as step S110 in FIG. 3 except that the difference area is used instead of the position information of the printed material 160, the description thereof is omitted.

  If there is no instruction to acquire the second projection plane image in step S176 (S162—NO), the image projection apparatus 100 determines whether there is a projection end instruction. If there is no end instruction, the process proceeds to step S154. If there is an end instruction, the image projection apparatus 100 ends the image projection.

  As described above, the region detection unit 102 can detect the position of the printed matter and output the coordinate information of the superimposed region 171 also by the processing shown in FIG. In addition, although the example which detects the position of printed matter per block was demonstrated, the detection method is not restricted to this. For example, considering that a general printed matter is often rectangular, the area detected in the process of S172 may be corrected to be a rectangular area.

<Embodiment 2>
Hereinafter, Embodiment 2 of the present invention will be described.
In the first embodiment, the example in which the superimposed region is detected based on the feature amount of the projection plane image has been described. In the present embodiment, the superimposed region is detected based on the difference in the distance from the image projection apparatus to the projection. Will be described. Here, the projection indicates a projection surface or a printed matter (object). In this embodiment, there is a difference in the distance from the image projection device to the projection between when projecting an image on a wall or screen and when projecting an image on a printed material, painting, or illustration installed on the same wall or screen. Pay attention to.

  FIG. 7 is a diagram illustrating an example of the image display system according to the present embodiment.

  In FIG. 7, the table 280 is installed on the wall surface 150, and the printed material 160 is installed on the table 280. In the present embodiment, the distance from the image projection device 200 to the projection indicates the distance from the image projection device 200 to a plurality of points in the wall surface 150 or the printed material 160 (arrow 290). Here, at the boundary portion 173 between the superimposed region 171 and the non-superimposed region 172, the distance from the image projection device 200 to the projection is discontinuous.

  FIG. 8 is a functional block diagram illustrating an example of the image projection apparatus 200 according to the present embodiment. The image projection apparatus 200 according to the present embodiment is an information processing apparatus (computer) including an arithmetic device, a storage device, an input / output device, and the like. When the arithmetic device executes the program stored in the storage device, the acquisition unit 101, the region detection unit 202, the operation input unit 103, the image quality setting unit 104, the correction unit 105, and the projection image generation unit 106 of the image projection device 200. The functions of the image projection unit 107 and the like are provided. Further, as a function of the image projection apparatus 200, a distance measuring unit 208 and the like are provided. Some or all of these functions may be implemented by a dedicated logic circuit such as an ASIC or FPGA. In addition, the image projection apparatus 200 according to the present embodiment detects the superimposed region 171 and the like using the distance measurement unit 208 and the region detection unit 202. In FIG. 8, the same functional units as those in the first embodiment (FIG. 2) are denoted by the same reference numerals, and description thereof is omitted.

  The distance measuring unit 208 is a sensor that can measure the distance from the image projection device 200 to the projection, and is configured to be able to measure the distance from the image projection device 200 to a plurality of different points in the projection region 170.

  The area detection unit 202 analyzes distances to a plurality of points in the projection area 170 and detects an area in which independent image quality setting is possible. In the present embodiment, the region detection unit 202 detects discontinuous points from the distance information. Then, the area detection unit 202 sets the area surrounded by the boundary part 173 obtained from the detected points as an area where independent image quality setting is possible. In addition, the region detection unit 202 converts the detected region into positional information relative to the projection region 170 and outputs it to the image quality setting unit 104 and the correction unit 105. If the boundary portion 173 is not a closed region, the region detection unit 202 may set the region separated by the boundary portion 173 as a region where independent image quality setting is possible.

(Processing content)
Next, with reference to the flowchart of FIG. 9, an operation in which the area detection unit 202 detects an area where independent image quality setting is possible (hereinafter referred to as an independent image quality setting area) will be described.

  In step S202, the distance measuring unit 208 acquires the distance from the image projection device 200 to the projection. In the present embodiment, the distance measurement unit 208 measures the distance from the image projection device 200 to a predetermined point in the projection area 170. Specifically, the distance measuring unit 208 uses a laser or the like to move the predetermined point so as to sequentially scan the entire projection area 170 in the horizontal and vertical directions, and to reach a plurality of points in the projection. Measure distance. The distance measurement method is not particularly limited. For example, the distance measuring unit 208 may measure the distance using a stereo camera, or may measure the distance using ultrasonic waves or the like.

  In step S204, the region detection unit 202 analyzes the distance information acquired by the distance measurement unit 208, and detects points where the distance information is discontinuous. By such processing, the area detection unit 202 detects the boundary part 173.

  In step S206, the region detection unit 202 detects the region surrounded by the detected boundary portion 173 and other regions as independent image quality setting regions.

  In step S <b> 208, the area detection unit 202 converts the independent image quality setting area into the coordinate information of the overlapping area 171 and outputs it.

  In the present embodiment, the area detection unit 202 detects the independent image quality setting area by detecting points where the distance information is discontinuous, but the detection method is not limited to this. For example, as in the second modification of the first embodiment, the area detection unit 202 may compare distance distributions before and after installing a printed material, and may set an area where the distance distribution changes as an independent image quality setting area. The area detection unit 202 may detect an area where the distance acquired by the distance measurement unit 208 is shorter than a predetermined first threshold and other areas as the independent image quality setting area. Further, although the case where there are two independent image quality setting areas has been described, the present invention is not limited thereto, and the area detection unit 202 may provide three or more independent image quality setting areas.

(Operational effect of this embodiment)
By using the image projection apparatus 200 as described above, a printed matter, a painting, an illustration, or the like installed on the wall surface 150 can be easily detected from only information on the distance to the projection without using information on the projection surface image. can do. Thereby, the difference in the appearance of the projected image caused by the difference in the distance between the projections can be easily corrected by the operation of the image projection apparatus 200.

<Embodiment 3>
Hereinafter, Embodiment 3 of the present invention will be described.
In the present embodiment, an example in which the independent image quality setting area is detected based on the reflection characteristics of the projection will be described.

  FIG. 10 is a functional block diagram illustrating an example of the image projector 300 according to the present embodiment. The image projection apparatus 300 according to the present embodiment is an information processing apparatus (computer) including an arithmetic device, a storage device, an input / output device, and the like. When the arithmetic device executes the program stored in the storage device, the acquisition unit 101, the region detection unit 302, the operation input unit 103, the image quality setting unit 104, the correction unit 105, and the projection image generation unit 106 of the image projection device 300 are executed. The functions of the image projection unit 107 and the like are provided. As functions of the image projection apparatus 300, a reflection characteristic measurement unit 308, a measurement image generation unit 309, a selection unit 310, and the like are provided. Some or all of these functions may be implemented by a dedicated logic circuit such as an ASIC or FPGA. In FIG. 10, the same reference numerals are given to the same functional units as those in the first embodiment (FIG. 2), and the description thereof is omitted.

  The reflection characteristic measurement unit 308 is constituted by an imaging device such as a camera, for example, and measures the reflection characteristic of the projection. Here, the reflection characteristics of the projection are the reflectance, brightness, color, and the like of the projection. In the present embodiment, the reflection characteristic measurement unit 308 outputs a value obtained by normalizing the reflection characteristic with a predetermined reference value. For example, if the reflectance is extracted as the reflection characteristic, the reference value uses, for example, an average luminance value of a projection plane image acquired in advance in a state where an all white image is projected from the image projection apparatus 300 onto a standard white plate. be able to.

  The area detection unit 302 detects an area in which independent image quality setting is possible based on the reflection characteristic information in the projection area 170. In addition, the region detection unit 302 converts the detected region into positional information relative to the projection region 170 and outputs the information to the image quality setting unit 104 and the correction unit 105.

  The measurement image generation unit 309 generates and outputs image data suitable for measuring the reflection characteristics of the projection. In the present embodiment, the measurement image generation unit 309 generates all white image data for measurement of reflection characteristics, but the color image is not limited to this. For example, a color image such as red, green, blue, and black may be displayed in all pixels.

  The selection unit 310 selects the image data output from the measurement image generation unit 309 when measuring the reflection characteristics of the projection, and selects the image data output from the projection image generation unit 106 otherwise, and outputs the image data to the image projection unit 107. To do.

(Processing content)
Next, an operation in which the area detection unit 302 detects the independent image quality setting area will be described using the flowchart of FIG.

  In step S302, when a projection start instruction is input from the user, the measurement image generation unit 309 generates measurement image data suitable for measuring the reflection characteristics of the projection. Then, the selection unit 310 selects the image data generated by the measurement image generation unit 309 and outputs it to the image projection unit 107. Here, when the reflectance is measured as the reflection characteristic, the measurement image generation unit 309 generates image (all white image) data in which the gradation values of all the pixels are maximum as the measurement image data. When measuring the brightness and color of the projection as the reflection characteristics, the measurement image generation unit 309 does not need to generate the measurement image data and the image projection unit 107 may project the image.

  In step S304, the reflection characteristic measurement unit 308 measures the reflection characteristic of the projection surface from image data (imaging data) acquired using a camera or the like.

  For example, if the reflectance is extracted as the reflection characteristic, the reflection characteristic measurement unit 308 divides the image data acquired in a state where the all white image is projected into blocks, and calculates an average luminance value for each block (region). To do. The block size and shape may be the same as or different from those of the first embodiment. Then, the reflection characteristic measurement unit 308 extracts a value obtained by normalizing the acquired average luminance value using a predetermined reference value as the reflectance. The area for calculating the average luminance value is not limited to the block unit. For example, you may correct | amend so that it may become a rectangular area | region.

  In step S306, after obtaining the reflection characteristics of the projection, the image projection unit 107 starts projecting the projection image.

  In step S308, the region detection unit 302 divides the projection region 170 into regions according to the reflection characteristics of the projection, and detects them as regions where independent image quality settings are possible. For example, in the case of dividing the area according to the reflectance, an area having a reflectance smaller than a predetermined threshold and other areas are detected as areas capable of independent image quality setting.

  In step S <b> 310, the region detection unit 302 converts the detected independent image quality setting region into coordinate information of the superimposition region 171 and outputs the converted information.

(Operational effect of this embodiment)
The image projection apparatus 300 as described above makes it easy to independently set the image quality of a projected image independently for each region having a different reflection characteristic of the projected material, regardless of whether or not a printed material is installed on the projection surface. Can do.

  Even when there is a region having greatly different reflection characteristics in the printed material, independent image quality setting is possible for each region. In general, when an image is projected across a region having greatly different reflection characteristics, it is difficult to predict the image quality of the projected image in advance even if the image is inside the same printed matter. Therefore, according to the present embodiment, even when the desired image quality cannot be realized, it is possible to reduce a user load such as re-editing of image data.

  Further, in the present embodiment, since an area where independent image quality setting is possible is detected according to the reflection characteristic, it is possible to cope with a case where the image is projected on a complicated projection such as a building as well as on the printed matter. For example, in the case of projecting onto a projection object including a concrete wall and a glass window, it is possible to set independent image quality for each region by detecting the reflection characteristics of the concrete wall and the glass window.

<Other embodiments>
In the above-described first to third embodiments, the example in which the image projecting device is used has been described. However, a control device, an imaging device, and a projecting device (a distance measuring device in the case of the second embodiment) are applied instead of the image projecting device. You can also.

The control device is an information processing device (computer) including an arithmetic device, a storage device, an input / output device, and the like. When the arithmetic unit executes the program stored in the storage device, the acquisition unit 101, the area detection units 102, 202, and 302, the operation input unit 103, the image quality setting unit 104, the correction unit 105, and the projection image generation of the control device Functions of the unit 106 and the like are provided. Some or all of these functions may be implemented by a dedicated logic circuit such as an ASIC or FPGA. Furthermore, when applied to the third embodiment, the control device has functions such as a measurement image generation unit 309 and a selection unit 310. In addition, about the same function part as the above-mentioned embodiment, the same code | symbol is attached | subjected and description is abbreviate | omitted. The projection device includes a liquid crystal panel that modulates transmittance based on a projected image, a light source that projects light onto the liquid crystal panel, and the like. The control device, the projection device, and the imaging device communicate via a cable or radio.

  The imaging device is a device having an imaging unit, and outputs image data obtained by imaging to the control device. In the case of the above-described second embodiment, a distance measuring device having the distance measuring unit 208 may be used instead of the imaging device. The projection device is a device having the image projection unit 107, and is a device that projects an image onto a projection surface. The projection apparatus inputs image data output from the projection image generation unit 106 and the like, and projects an image.

  Here, the area detection units 102, 202, and 302 detect the overlapping area and the non-superimposing area based on the image data output from the imaging apparatus (in the case of the second embodiment, the output result of the distance measuring apparatus). Note that the image quality setting unit 104 may display the above-described image quality setting menu on a display unit (display or the like) of the control device and accept an input of image quality setting from the user.

  The present invention supplies a program that realizes one or more functions of the above-described embodiments to a system or apparatus via a network or a storage medium, and one or more processors in a computer of the system or apparatus read and execute the program This process can be realized. It can also be realized by a circuit (for example, ASIC) that realizes one or more functions.

  As mentioned above, although preferable embodiment of this invention was described, this invention is not limited to these embodiment, A various deformation | transformation and change are possible within the range of the summary.

102: Area detection unit 104: Image quality setting unit 105: Correction unit 107: Image projection unit

Claims (19)

Image projection means for projecting an image onto a projection plane;
Detecting means for detecting one or more overlapping regions in which an object that is an object different from the projection plane and a projection image overlap and a non-overlapping region that is another region;
Image quality setting means for setting the image quality of the superimposed area and the non-superimposed area separately by the user,
Correction means for correcting image data of the superimposing region or the non-superimposing region based on a user instruction performed using the image quality setting unit;
An image display device comprising: It further has an imaging means,
The detecting means detects the superposed area and the non-superimposed area based on a captured image obtained by the imaging means;
The image display apparatus according to claim 1. The detection means detects, as the superimposed region, a region in which the width of the luminance histogram or color histogram distribution obtained from the captured image is a predetermined value or more.
The image display device according to claim 2. The detection means is extracted based on image data of an image obtained by capturing the projection surface and the target object in a state where the target object is present and an image obtained by capturing only the projection surface in a state where the target object is absent. A region where the difference between the image feature values is equal to or greater than a predetermined value is detected as the superimposed region;
The image display device according to claim 2. The detection unit sets a region that matches or is similar to the overlapping region based on image data of an image obtained by capturing the projection plane and the object and the projection image in a state where the image projection unit is not performing projection. To detect,
The image display device according to claim 2. The detecting means detects the overlapping region by detecting a position of a marker added to the object;
The image display device according to claim 2. A distance measuring means for measuring a distance between the image projecting means and the projection surface or the object;
The detecting means detects the overlapping region based on the difference in the distance;
The image display apparatus according to claim 1. The image quality setting means can accept setting input from the user for each area when there are a plurality of the overlapping areas,
The correction unit corrects the image data for each region based on a user instruction.
The image display device according to claim 1, wherein: Image projection means for projecting an image;
Detecting means for detecting one or more regions based on a difference in reflection characteristics of a projection surface of an object onto which an image is projected by the image projecting means;
Image quality setting means for the user to set the image quality for each area;
A correction unit that corrects image data of the region based on a user instruction performed using the image quality setting unit;
An image display device comprising: The detection means performs the detection in a state where the image projection means is not performing projection.
The image display device according to claim 1, wherein the image display device is an image display device. The image quality setting means receives a setting input from a user using a user interface superimposed on a projection image.
The image display device according to claim 1, wherein the image display device is an image display device. The user interface can select an area for image quality setting and independent image quality setting for each area.
The image display device according to claim 11. A storage unit for storing the arrangement of each area detected by the detection unit and the set value set using the image quality setting unit at the end of projection;
When the arrangement of each area detected by the detection unit is the same as the arrangement of each area stored in the storage unit, the correction unit corrects the image data based on the stored setting value.
The image display device according to claim 1, wherein the image display device is an image display device. A control device for controlling a projection device having image projection means for projecting an image onto a projection plane,
Detecting means for detecting one or more overlapping regions in which an object that is an object different from the projection plane and a projection image overlap and a non-overlapping region that is another region;
Image quality setting means for setting the image quality of the superimposed area and the non-superimposed area separately by the user,
Correction means for correcting image data of the superimposing region or the non-superimposing region based on a user instruction performed using the image quality setting unit;
A control device comprising: The image quality setting means accepts setting input from a user using a user interface displayed on a display unit, which allows selection of an area for image quality setting and independent image quality setting for each area.
The control apparatus according to claim 14. The detection means detects the superimposed region and the non-superimposed region based on a captured image obtained by an imaging device capturing a projection plane.
The control device according to claim 14 or 15, wherein An image display method in an image display device for projecting an image onto a projection plane,
A detection step of detecting one or more overlapping regions in which an object that is an object different from the projection plane and a projection image overlap and a non-overlapping region that is another region;
An image quality setting step for receiving an input of settings separately from the user for the image quality of the superimposed region and the non-superimposed region,
A correction step of correcting the image data of the superimposing region or the non-superimposing region based on a user instruction performed in the image quality setting step;
An image projecting step of projecting an image to be corrected in the correcting step;
An image display method characterized by comprising: A control method in a control device for controlling a projection device having image projection means for projecting an image onto a projection plane,
A detection step of detecting one or more overlapping regions in which an object that is an object different from the projection plane and a projection image overlap and a non-overlapping region that is another region;
An image quality setting step for receiving an input of settings separately from the user for the image quality of the superimposed region and the non-superimposed region,
A correction step of correcting the image data of the superimposing region or the non-superimposing region based on a user instruction performed in the image quality setting step;
A control method characterized by comprising:   The program for making a computer perform each step of the method of Claim 17 or 18.

Images