JP3727991B2 - Image extraction device - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to an image extraction apparatus, and is particularly suitable for use in an image processing system having an image cutout function and a region extraction function.
[0002]
[Prior art]
Conventionally, as a general technique for image segmentation (extraction), a key signal is generated by a chroma key method using a specific color background or image processing such as histogram processing, difference processing, differentiation processing, contour enhancement processing, and contour tracking. A video mat (Television Society Technical Report, vol. 12, pp. 29-34, 1988) is known.
[0003]
As an apparatus for extracting an image of a specific area from an image, for example, in the technique disclosed in Japanese Patent Publication No. 6-9062, a differential value obtained by a spatial filter is binarized to detect a boundary line, and the detected boundary The connected areas partitioned by lines are labeled, and areas having the same label are extracted.
[0004]
Note that the method of performing image extraction based on the difference from the background-only image is a classic one, and recently, in Japanese Patent Laid-Open No. 4-216181, the difference data between the background-only image and the processing target image is masked. A method is disclosed in which an image (specific processing region) is set, and target objects in a plurality of specific regions in the image are extracted or detected.
[0005]
In addition, in the method related to Japanese Examined Patent Publication No. 7-16250, the existence of an object to be extracted from the color conversion data of the current image including the background using the color model of the object to be extracted, and the difference data of brightness between the image of the background and the current image. The probability distribution is obtained.
[0006]
Furthermore, as an example of a method for optimizing camera operation and operation mode, in the method described in Japanese Patent Laid-Open No. 6-253197, the aperture is set so that the average luminance is appropriate when capturing an image of only the background. To do. Then, the current image is captured using the same set value, and the object image is extracted based on the difference image data.
[0007]
On the other hand, in the imaging means, as the degree of freedom of video information processing and processing increases with the digitization of signal processing, internal processing is compared with brightness level, tone conversion, white balance processing, quantization size conversion, etc. From simple processing to one having an edge extraction function or one having an image extraction function using the sequential growth method of color components (Technical Report of Television Society, vol.18, pp.13-18, 1994), etc. It shows great progress.
[0008]
[Problems to be solved by the invention]
However, among the above-described conventional examples, the methods using difference data with only the background image, except for the method described in Japanese Patent Laid-Open No. 6-253197, are all shooting conditions (such as camera parameters and illumination). (External conditions) are not taken into account, so if the background-only image and the subject-included image to be extracted are not obtained under the same shooting conditions and the same fixed position, the determination error of the extraction target region from the difference data is very high. There was a problem of getting bigger.
[0009]
Further, the method described in Japanese Patent Publication No. 7-16250 is unsuitable for image extraction of an arbitrary unknown object in that a color model to be extracted is required.
[0010]
Also, in the method related to Japanese Patent Laid-Open No. 6-253197, it is premised that the imaging means is at the same fixed position, and that the imaging conditions are the same as when imaging only the background. Only a technique of using the aperture setting value at the time of capturing an image when capturing an image including a subject has been disclosed. In addition, in this system in which priority is given to the imaging condition of only the background image, there is a problem in that the image quality of the extraction target of the subject-inclusive image is generally not guaranteed.
[0011]
Furthermore, the chroma key method has problems such as being unusable outdoors due to the large restrictions on the background and coloring.
In addition, the video mat method has a problem that it is necessary for humans to accurately perform contour designation work in units of pixels, which requires labor and skill.
[0012]
In addition, the method of detecting the boundary line by the differential operation and detecting the area partitioned by the boundary line is difficult to apply to an object having a complex texture pattern (pattern), or a stable and general-purpose boundary. There were problems such as lack of a line detection processing method.
[0013]
In view of the above-mentioned problems, the present invention increases the tolerance for differences in imaging conditions in each image and easily identifies an extraction target region when extracting a specific subject from a comparison between a plurality of images. An object of the present invention is to provide an image extraction apparatus capable of obtaining a good image of an object to be extracted.
[0020]
[Means for Solving the Problems]
The image extraction apparatus of the present invention records an imaging condition control unit for controlling an imaging condition when imaging is performed by the imaging unit, an imaging condition when the imaging unit performs imaging, and the recorded imaging condition. Based on the imaging conditions supplied from the imaging condition recording / reproducing means, the imaging condition recording / reproducing means for reproducing and outputting the image, the image recording means for recording a plurality of images taken by the imaging means, Image data conversion means for converting image data of at least one of the images, image data comparison means for comparing image data of a plurality of images converted by the image data conversion means, and the image data Image extracting means for extracting an image of the specific image area based on the comparison result output from the comparing means, and the image recording means, A scene-only image and a subject-included image in which a cut-out target image is present in the background are temporarily stored and held, and the image data comparison unit stores a plurality of difference image data reproduced and output by the image recording unit. The imaging condition recording / reproducing unit records the imaging conditions when the image to be cut out in a predetermined background is imaged, and the image data converting unit reproduces and outputs the image reproduced by the imaging condition recording / reproducing unit. The image data of the background only image is converted based on the condition.
[0023]
Another feature of the present invention is that the image recording means encodes and records image data of a specific image area obtained by the image cutout means.
[0024]
Another feature of the present invention is that the image data conversion means spatially performs a shift operation so that the similarity between the background-only image and the subject-included image is maximized. .
[0026]
Another feature of the present invention is that the image data conversion means converts an image size, a luminance level, a color component, and a resolution.
[0027]
[Action]
Since the present invention comprises the above technical means, according to the first invention, the imaging conditions are recorded and reproduced, and the image data is converted based on the reproduced imaging conditions. When extracting a specific subject by comparing between a plurality of images such as images or different frame images in a moving image, the tolerance for different imaging conditions of each image can be increased, As a result, it is possible to extract a specific subject even if there is a slight change in the position of the imaging means due to camera shake, a difference in exposure conditions, a gain change in the sensor, or the like.
[0028]
The image including only the background and the subject-included image in which the image to be cut out exists in the background is temporarily stored and held by the image recording unit, and is reproduced and output by the image recording unit. Since the data is extracted by the image data comparison means, and the extracted difference image data is used as comparison data between the background-only image and the subject-included image, the process of cutting out the specific subject from the background can be performed efficiently. It becomes possible.
[0029]
Furthermore, since the image data of the background only image is converted based on the imaging conditions when the image to be cut out in the predetermined background recorded in the imaging condition recording / reproducing means is captured, the high image quality focused on the object to be cut out It is possible to output the image.
[0030]
According to the second invention, the image data of the specific image area obtained by the image cut-out means is encoded and recorded in the image recording means, so that the required image can be efficiently encoded. Is possible.
[0031]
According to the third invention, since the spatial shift operation is performed so that the similarity between the background-only image and the subject-included image is maximized, image extraction that allows fluctuations in the position and orientation of the imaging means is performed. Is possible.
[0033]
According to the fourth invention, since the image size, the luminance level, the color component, and the resolution are converted by the image data conversion unit, normalization of the image data between a plurality of images captured under different imaging conditions is performed satisfactorily. It is possible to extract a specific subject with high accuracy.
[0034]
DETAILED DESCRIPTION OF THE INVENTION
Embodiments of the image extraction apparatus of the present invention will be described below with reference to the drawings.
FIG. 1 is a functional configuration diagram showing the main configuration of the image extraction apparatus of the present invention. In FIG. 1, A is an image pickup means comprising an image pickup optical system and an image pickup element, B is an image pickup condition control means for controlling various image pickup parameters such as zoom, focus, stop, etc. C is a record of control data of the image pickup condition control means B. An imaging condition recording / reproducing means for reproducing, D is an image recording means for recording an image taken by the imaging means A, and E is an image data conversion for converting an image based on various control parameters supplied from the imaging condition recording / reproducing means C. Means F is an image data comparison means for comparing the background image and the image including the subject, and G is an image cutout means for setting an image cutout region based on the output of the image comparison means F. H is a display means including a monitor display for displaying an image, an electronic viewfinder, and the like.
[0035]
The imaging means A is provided for capturing a plurality of images, and the imaging condition control means B is for controlling imaging conditions when the imaging means A performs imaging. is there. In the present embodiment, the imaging conditions include the exposure amount, the focused state, and the presence or absence of strobe light emission.
[0036]
The imaging condition recording / reproducing means C is for recording the imaging conditions when imaging is performed by the imaging means A, and for reproducing and outputting the recorded imaging conditions. The condition recording / reproducing means C records the imaging condition when the clipping target image in the predetermined background is captured.
[0037]
The image recording means D is for recording a plurality of images picked up by the image pickup means A, and temporarily stores and holds a background-only image and a subject-included image in which a clipping target image exists in the background. Further, the image data of the specific image area obtained by the image cutout means G is encoded and recorded.
[0038]
The image data converting means E is provided for converting image data of at least one of the plurality of images based on the imaging conditions supplied from the imaging condition recording / reproducing means C. is there. Further, the image data of the background only image is converted based on the imaging conditions reproduced and output by the imaging condition recording / reproducing means C. Further, the image data conversion means E performs a spatial shift operation on the memory so that the similarity between the background-only image and the subject-included image is maximized, and the image size, luminance level, color component, and Convert resolution.
[0039]
The image data comparison means F is for comparing the image data of a plurality of images converted by the image data conversion means E, and the difference image data of the plurality of images reproduced and output by the image recording means D. To extract.
[0040]
The image cutout means G is provided for extracting an image of a specific image area based on a comparison result of a plurality of image data output from the image data comparison means F.
[0041]
According to the image extraction apparatus of the present embodiment configured as described above, a specific subject is extracted from a comparison between a plurality of images such as a registered image and a current image, or different frame images in a moving image. In doing so, it is possible to increase the tolerance for different imaging conditions in each image.
[0042]
Thereby, when extracting a subject image from an image of only the background, for example, even if there is a slight change in the position of the imaging means A due to camera shake or the like, a difference in exposure conditions, or a change in sensor gain, the specific subject. Can be extracted satisfactorily. In addition, it is possible to increase the tolerance of fluctuations in illumination conditions and the like without using a model relating to an extraction target such as a color model.
[0043]
In addition, since it is possible to increase the tolerance for fluctuations in imaging conditions and camera parameters, it is possible to efficiently perform a process of cutting out a specific subject from the background. Furthermore, since the image data of the background only image is converted using the subject-included image captured under the image capturing condition when the image of the background only is captured, the image capturing condition and the camera parameter are determined from the image including the background only and the image including the subject. The tolerance for fluctuations can be increased. In addition, it is possible to perform high-quality subject extraction that is not affected by the imaging conditions at the time of capturing an image of only the background, and it is possible to output a high-quality image that is focused on the extraction target.
[0044]
According to another feature of the present embodiment, the image data of the specific image area obtained by the image cutout means G is encoded and recorded in the image recording means D, so that the necessary image can be efficiently stored. It is possible to perform encoding and to perform image extraction satisfactorily.
[0045]
Further, according to other features of the present embodiment, the image data conversion means E spatially performs the shift operation so that the similarity between the background-only image and the subject-included image is maximized. Image extraction that allows fluctuations in the position and posture of the image pickup means A is possible, and image extraction that is less adversely affected by camera shake during image pickup can be performed.
[0046]
Further, according to another feature of the present embodiment, the image capturing condition includes exposure amount, in-focus state, and presence / absence of strobe light emission, so that an image of a specific subject is extracted from a plurality of images captured under different image capturing conditions. Thus, it is possible to perform image extraction with increased tolerance for variations in magnification conditions, focus, contrast, illumination conditions, and the like.
[0047]
According to another feature of the present embodiment, since the image data conversion means E converts the image size, the luminance level, the color component, and the resolution, the image between a plurality of images captured under different imaging conditions. Data normalization is performed, and a specific subject extraction process based on comparison between images can be performed with high accuracy, so that subject image extraction with high accuracy can be performed.
[0048]
Next, the configuration and operation of the image extraction apparatus of the present invention will be described more specifically with reference to FIGS.
First, the first embodiment will be described. The basic processing of the first embodiment is to capture an image of a background only and a subject image in the background, and subject the comparison data (difference data) in consideration of the imaging conditions of the two to perform statistical processing. The image of the subject is extracted by detecting the area.
[0049]
In the present embodiment, it is assumed that the operator holds an image pickup means such as a video camera in his / her hand, and an image in which the subject to be extracted is in the background is first picked up, and the image pickup conditions at that time Are recorded together with the image data in the storage means. Next, the same imaging conditions (camera internal parameters including image signal characteristic parameters) are read from the storage means so as to capture an image of only the background.
[0050]
On the other hand, if the environmental conditions are not considered to be the same between the two times of imaging, for example, if the imaging time or external illumination conditions are completely different, the imaging parameters that should be the same conditions are limited to magnification, focus, etc. Thus, it is possible to prevent a situation in which the same condition cannot be obtained.
[0051]
FIG. 2 shows a main configuration of the imaging system according to the present embodiment. In FIG. 2, reference numeral 1 denotes an image pickup means, and 2 denotes an image forming optical system including an image pickup lens, which is a compound eye image pickup system for picking up a stereoscopic image in this embodiment. Reference numeral 3 denotes a lens motor drive controller for driving each lens of the imaging optical system 2, and reference numeral 4 denotes an image sensor. In general, a CCD or the like is used.
[0052]
Reference numeral 5 denotes an imaging parameter measurement control means, which is a focal length detection means for a zoom lens for controlling the magnification, a focus state detection means for detecting the focus state of the lens by a known means, for example, a shutter speed for controlling the accumulation time of the CCD. Detection control means, stop measurement control means for controlling the aperture diameter of the stop, detection means for characteristic quantities (for example, correction coefficient for gamma) of image signal characteristic parameters (gamma, knee, white balance correction, CCD accumulation time, etc.), etc. Is included. Reference numeral 6 denotes an image recording unit comprising a memory or the like, and 7 denotes an electronic viewfinder (EVF or the like) as a display unit.
[0053]
Reference numeral 8 denotes imaging mode recording means for recording information at the time of imaging including imaging parameters, image characteristic parameters, presence / absence of strobe light emission, intentional motion such as scanning, presence / absence of camera shake, and the like. Note that camera motion such as camera shake, scanning, and panning may be determined from the output data or the like by incorporating an acceleration sensor in the imaging means. These incidental data are stored in the image database 18 together with the image data.
[0054]
Reference numeral 9 denotes image data conversion means for converting image data at the time of synthesis based on imaging conditions and the like, and details will be described later. An image signal processing circuit 10 includes gamma, knee, white balance correction, AF (Automatic Focusing), AE (Automatic Exposure), AGC (Automatic Gain Control) processing circuits, and the like. Reference numeral 11 denotes image data comparison means for detecting and outputting the difference between the background-only image and the subject-included image.
[0055]
Reference numeral 12 denotes an image cutout unit, which identifies an extraction target region based on a result of statistical processing of the output from the image data comparison unit 11, and a key signal (or a key for cutting out the identified extraction target region from a subject-included image) (Mask data) is output. An image transfer means 13 is provided for transferring image data or the like to an external database or terminal.
[0056]
Reference numeral 14 denotes a strobe light emitting means, and reference numeral 15 denotes an external synchronizing means. Although a signal line is omitted, a synchronizing clock is supplied to each circuit. Reference numeral 16 denotes a terminal for performing imaging mode control from outside, selection of a cutout image, search and selection of a registered image, and the like. Reference numeral 17 denotes a display, which also functions as a processed image output and a finder display.
[0057]
Reference numeral 18 denotes an image database. As image data captured in the past and associated data, the type as to whether it is a registered image, imaging parameters, imaging conditions (outdoor or indoor distinction, presence / absence of strobe light emission, etc.) Or other information (date, time, location, camera operator, title, etc.).
[0058]
Reference numeral 19 denotes image type setting means for setting a switch for registering as a reference image for extracting a target based on comparison with another image, and a type of subject extraction image to be compared with the reference image. For example, the image type is automatically recorded as incidental information.
[0059]
Reference numeral 20 denotes camera parameter setting means. Normally, an image of only a background and an image including a subject are picked up in the same image pickup mode. However, the camera parameter setting means 20 allows the operator to arbitrarily set the internal characteristics of the image pickup means. This is used when setting to. In addition, although the circuit inside an imaging means was set as the display for every function, each function is operation-controlled by the microprocessor not shown.
[0060]
Next, the basic processing of the image extracting apparatus of the present embodiment configured as described above is shown in the flowchart of FIG. 3, and an example of image extraction will be described with reference to FIG. Although these operations are also displayed for each function in the figure, they are actually processed by the microprocessor.
When the process is started, first, in the first step S1, it is determined whether the image output from the imaging means is a background only image or a subject-included image. The above determination is made for the type of image set by the operator via the image type setting means 19. That is, the operator designates a type of whether the image to be captured is the main subject, only the background, or a combination of both.
[0061]
If it is determined in step S1 that an image including the subject is to be captured, the process proceeds to step S2 to set an imaging mode. Thereafter, in step S3, various parameters for imaging are set and imaging is performed under imaging conditions (described below) that are optimal for the subject. Then, after completing the imaging in step S3, the process proceeds to step S4, and as described above, the magnification (focal length), the degree of focus, the aperture, the shutter speed, the presence / absence of camera shake, the presence / absence of panning / tilting, or the gain The incidental information such as various imaging modes and imaging conditions is measured.
[0062]
Next, proceeding to step S5, the incidental information measured at step S4 is recorded in a predetermined format together with the image data. The supplementary information may be recorded separately in a header file or the like together with the address of the corresponding image data.
[0063]
On the other hand, if the result of determination in step S1 is that only an image of the background is captured, the process proceeds from step S1 to step S6. In step S6, incidental information of the image including the subject is read, and in step S7, the imaging mode is set in the same manner as described above. In this case, the same condition parameter in the process of step S2 is selected, and basically, only the background is captured under the same imaging condition in the next step S8.
[0064]
However, in order to cope with fluctuations in environmental conditions and the like, in step S7 described above, as an imaging condition selection mode recorded by the imaging mode recording means 8 at the time of imaging, the outdoor or indoor distinction, the presence or absence of strobe light emission, etc. An optimal imaging mode is set using the information. The imaging parameter measurement control means 5 determines the presence / absence of a change in environmental conditions by using these imaging conditions and incidental information, and controls the imaging parameters so that the same conditions are satisfied.
[0065]
It should be noted that the illumination means mounted on the imaging means, such as a strobe, when the imaging time of the background only image and the imaging time of the subject-incorporated image are close to each other, whether indoors or outdoors. It is considered that the lighting conditions, background patterns, etc. except for have hardly changed.
[0066]
Therefore, by making the imaging mode excluding the image signal characteristic parameter the same between the two (for example, unified with the imaging mode for the subject-included image), the fluctuation of the image data in the same area of the background pattern between the two images is suppressed. This can increase the reliability of the subsequent statistical processing for image segmentation.
[0067]
However, since the characteristics of the sensor signal processing circuit (gamma, white balance, etc.) change due to the spectral reflectance characteristics of the subject image, noise, etc., the image data in the background area is the same even in the same imaging mode in general situations. May not perfectly match between the two images.
[0068]
In order to deal with such a case, image data conversion processing is performed in step S9. In this case, scale conversion (when the magnification is changed), luminance conversion (when the exposure amount / gamma is changed), color component conversion (white balance characteristics fluctuation), image alignment (when there is a change in the image position), etc. The normalization is performed so that the image data of the portion corresponding to the image area of only the background of the subject-included image in the background only image is substantially the same.
[0069]
Next, in step S10, an image comparison process with the background only image is performed to calculate an image data difference, and an image cutout process is performed in step S11 based on the image difference data. Details of these processes will be described later.
[0070]
As shown in FIG. 4, the image data conversion means 9 includes a data input means 90 for inputting image data and its accompanying data, a luminance value conversion means 91, a color component conversion means 92, a space shift calculation means 93, an image size scaling. It comprises means 94, parameter fluctuation evaluation means 95 and the like. In general, an image including a subject is used as a reference image. For the reference image, the image data is fixed, and the image data of the background only image is converted.
[0071]
In addition, when the imaging mode is the same between both images, that is, when there is no change in environmental conditions, it can be said that conversion of luminance values and color components is generally unnecessary. However, depending on the characteristics of the subject image and the ratio of the area to the background, the image signal characteristic parameters, that is, the gain, white balance characteristic, and gamma characteristic may change.
[0072]
Therefore, when the change width of at least one of these characteristic values exceeds a certain threshold value, the parameter characteristic value of each point in the background-only image and the subject-included image recorded in the predetermined memory is recorded. Based on the difference, the image of only the background is converted so that the luminance and color components thereof correspond to the characteristic values of the image including the subject. Hereinafter, processing contents performed in each converting means will be described.
[0073]
Note that the conversion processing in the image data conversion means 9 is considered that there is no significant change in the imaging environment when the variation of each parameter is relatively small. As shown in FIG. After conversion and alignment of each image to enable basic image composition, the image details are adjusted in order such as luminance conversion and color component conversion. In FIG. 4A, the vertical arrows indicate the order in which processing is performed.
[0074]
However, this is not the case when the imaging mode equalization control is canceled by the imaging parameter setting means 20, and as shown in FIG. 4B, the conversion order setting means 96 for setting the conversion order excluding alignment is used. Alternatively, the conversion may be performed by setting the order in the order of items with the largest fluctuation amount. As a result, the effect of the error is reduced and the accuracy is improved as the item with a small fluctuation amount is converted.
[0075]
The scaling conversion means 94 for performing the scaling conversion converts the background only image so as to have the same viewing angle as the subject-incorporated image, based on the magnification fluctuation amount between the two images. However, it is assumed that the variation in the distance between the imaging means and the subject is sufficiently small. If there is no change in the magnification data, the process may be omitted and the conversion may be started from the alignment means 93.
[0076]
Luminance value conversion means 91 matches or does not match lighting conditions such as gain, gamma characteristic fluctuation amount, exposure amount, outdoor / indoor distinction, presence / absence of strobe light emission, between a background-only image and a subject-included image. Based on the above, luminance level conversion is performed for each pixel.
The color component conversion unit 92 converts the color component of each image data based on the white balance characteristic fluctuation amount and the like.
[0077]
The alignment means 93 performs a spatial shift operation on the memory in order to maximize the similarity between the background-only image and the subject-included image. In other words, in order to allow fluctuations in the position of the imaging means between two images and posture changes (when there is camera shake, a pick-up of the imaging means, or when the position of the foot is slightly changed), The corresponding points are extracted and alignment based on the corresponding points, that is, matching calculation is performed.
[0078]
Normally, it is sufficient to extract corresponding points (at least three points) between the local areas at the four corners of each image for fluctuations in the degree of camera shake during a short time. As an algorithm for extracting the corresponding points between both images as described above, typically, the image is divided into blocks of a predetermined size centered on each point in the image, and the cross-correlation value between the blocks is calculated. It is possible to consider a block matching method in which the maximum points are corresponding points.
[0079]
By the way, in order to perform alignment, it is important that the background portion overlaps between both images at a predetermined ratio or more. As a general guideline, it is a necessary condition that the overlapping area excluding the image region to be extracted is, for example, 50% or more between both images.
[0080]
However, it goes without saying that the minimum value of the ratio of the overlapping areas is variable according to the pattern of the background only image. In particular, when the background and subject are relatively easily separated, for example, when the background is close to plain or has a periodic pattern, etc., and the extracted subject has a distinctly different pattern, etc. The area may be significantly smaller.
[0081]
The image data comparison unit 11 generates difference image data between the background-only image after the image conversion and the subject-included image.
In addition, the image cutout unit 12 performs smoothing processing (such as a median filter) and statistical processing (based on color component deviation and luminance level deviation) on the generated difference image data, so that an area in which background image data varies greatly A subject area is extracted from.
[0082]
Specifically, the converted background-only image I _b , Subject-included image I _t From the R, G, B components and the luminance signal, hue H _b , H _t , Saturation S _b , S _t , Brightness V _b , V _t Are extracted, and the value of the following evaluation function F is binarized with a predetermined threshold value to identify the subject area and the background area.
F (H _b − H _t , S _b − S _t , V _b − V _t )
= Α _h (H _b − H _t ) ² + Α _s (S _b − S _t ) ² + Α _u (V _b − V _t ) ² ... (Formula 1)
[0083]
Where α _h , Α _s , Α _u Is image I _b, I _t Is a function of the S / N value of each component or the variance value of each component when each image is divided into blocks of a predetermined size, for example α _h = P _h (I _b ) ・ P _h (I _t ) Etc. are used.
[0084]
Where P _h (I) represents a monotonically increasing function of the S / N value or a monotonically decreasing function of the variance value (reciprocal number, etc.) relating to the hue component in a predetermined region of the image data I. Similarly, P _s (I), P _u Parameters relating to saturation and lightness are defined as (I). The threshold value T for discriminating each point in the region is set according to Otsu's method (Journal of the Institute of Electronics, Information and Communication Engineers, vol.J63, PP.349-356, 1980), etc. _b − H _t ) ² , (S _b − S _t ) ² , (V _b − V _t ) ² Apply for each, T _h, T _s, T _u T = T _h + T _{s +} T _u It is good.
[0085]
Note that the above evaluation function and each parameter are not limited to the above definitions. Further, the threshold value itself may be a predetermined value and may be a constant value over the entire image area.
[0086]
Next, the connected areas determined as the subject areas as a result of the threshold processing are labeled and displayed on the display 17 as mask image data with different colors, luminance values, hatch patterns, or the like.
[0087]
At this time, the isolated region in the connected region, that is, the background block region may be regarded as the same subject region and converted into the same label as the surrounding subject region. The user selects and designates one of the subject areas to be extracted using a mouse (not shown) or the like (clicks with the mouse or the like), and as a result, only the subject image from which the background is removed is displayed on the display 17. .
[0088]
If there is no problem with the extracted image data, the user gives a confirmation instruction by, for example, clicking a confirmation icon. Thereby, the image data of only the subject is encoded, and then generation of an image data file is executed.
[0089]
(1) in FIG. 6 shows an example of a subject-included image including both a subject and a background, and (2) in FIG. 6 shows an example of an image with only a background. In the subject-incorporated image of (1) in FIG. 6, the image is input in an imaging mode in which the subject (person) is prioritized.
[0090]
In addition, the background-only image in (2) of FIG. 6 is imaged at the same magnification so that the image signal characteristic amount matches the distant view. Among these images, the characteristics (average luminance level and color component) of the image data may be slightly different even in the same background portion, and FIG.
[0091]
(3) of FIG. 6 shows the result of normalizing the image of only the background based on the imaging condition of the subject-included image. The clipping process is performed based on the difference data between the subject-included image (1) and the background-only image (3) (the result of the comparison process in step S10 described above), and the result is shown in (4) of FIG.
[0092]
The registered image may be retrieved from the image database 18 and used. In this case, the registered image is an image including a subject in the present embodiment, and similar processing is performed by using supplementary information at the time of imaging recorded in the header portion or header file of the image.
[0093]
Next, a second embodiment of the image extraction apparatus of the present invention will be described.
In the second embodiment, processing that does not impair the image quality and clipping accuracy of the subject image is performed regardless of the imaging order of the background-only image and the subject-included image.
[0094]
For this reason, in this embodiment, the respective magnification conditions are kept constant, and other image capturing conditions are captured and recorded under conditions that can be set independently. However, when capturing an image including a subject, an imaging mode in which priority is given to the image quality of the subject image is taken.
[0095]
FIG. 5 shows a processing flow of the second embodiment. The first image and the second image in FIG. 5 correspond to the background-only image, the subject-included image, or the reverse image thereof in the above description.
[0096]
In FIG. 5, in the image data conversion process in step S49, the image data conversion unit 9 converts the other image data and matches between the two images so as to match any one of the imaging conditions as in the first embodiment. Perform position alignment.
[0097]
Prior to this processing, both image data may be converted into a low resolution image by thinning processing or local averaging processing. This is because even when the magnification condition is the same, the focus (resolution) may be greatly different between the background region of the image including the subject and the corresponding region of the background-only image. As described above, when both the images are processed with a reduced resolution, the efficiency and accuracy in roughly estimating the subject area can be improved.
[0098]
【The invention's effect】
According to the first invention, the imaging condition is recorded, and the image data is converted based on the recorded imaging condition. For example, the registered image and the current image, or different frame images in the moving image, etc. When a specific subject is extracted from a comparison between a plurality of images as described above, it is possible to increase the tolerance for different imaging conditions in each image. As a result, when extracting a subject image from an image of only the background, even if there is a slight variation in the position of the imaging means due to camera shake, differences in exposure conditions, or slight variations in sensor gain, etc. Can be extracted. In addition, it is possible to increase the tolerance of fluctuations in illumination conditions and the like without using a model relating to an extraction target such as a color model.
[0099]
Since the difference image data is used as the comparison data between the background-only image and the subject-incorporated image, the tolerance for fluctuations in imaging conditions and camera parameters can be increased, and the process of cutting out a specific subject from the background is performed. It can be done efficiently.
[0100]
Furthermore, since the image data of the background only image is converted using the subject-included image captured under the image capturing conditions when the image of the background only is captured, the image capturing condition and the Increases the tolerance for camera parameter fluctuations, enables high-quality subject extraction that is not affected by the imaging conditions when capturing images of only the background, and outputs high-quality images that are in focus for the target to be cut out Can do.
[0101]
According to the second invention, since the image data of the specific image area obtained by the image cutout unit is encoded and recorded in the image recording unit, the necessary image can be efficiently encoded, Image extraction can be performed satisfactorily.
[0102]
According to the third invention, since the shift operation is spatially performed so that the similarity between the background-only image and the subject-included image is maximized, the variation of the position of the imaging unit and the variation of the posture are allowed. Image extraction is possible, and image extraction with less adverse effects due to camera shake during imaging can be performed.
[0104]
According to the fourth invention, since the image data conversion means converts the image size, the luminance level, the color component, and the resolution, the image data between the plurality of images captured under different imaging conditions is normalized. Thus, the extraction process of the specific subject based on the comparison between the images can be performed with high accuracy, and the subject image extraction with high accuracy can be performed.
[Brief description of the drawings]
FIG. 1 is a functional configuration diagram showing a main configuration of an image extraction apparatus according to the present invention.
FIG. 2 is a system configuration diagram showing an embodiment of the present invention.
FIG. 3 is a flowchart showing a procedure of basic processing.
FIG. 4 is a block diagram showing a configuration of image data conversion means.
FIG. 5 is a flowchart showing processing of the second embodiment.
FIG. 6 is a diagram illustrating an example of a processing process.
[Explanation of symbols]
1 Imaging means
2 Imaging optics
3 Lens motor drive controller
4 Image sensor
5. Imaging parameter measurement control means
6 Image recording means
7 Viewfinder
8 Imaging mode recording means
9 Image data conversion means
10 Imaging mode setting means
11 Image data comparison means
12 Image transfer means
13 Image clipping means
14 Strobe flash means
15 External synchronization means
16 terminals
17 Display
18 Image database

Claims (4)

Imaging condition control means for controlling imaging conditions when imaging is performed by the imaging means;
An imaging condition recording / reproducing unit for recording an imaging condition when performing imaging by the imaging unit and reproducing and outputting the recorded imaging condition;
Image recording means for recording a plurality of images taken by the imaging means;
Image data conversion means for converting image data of at least one of the plurality of images based on the imaging conditions supplied from the imaging condition recording / reproducing means;
Image data comparison means for comparing image data of a plurality of images converted by the image data conversion means;
Image extracting means for extracting an image of a specific image region based on the comparison result output from the image data comparing means,
The image recording means temporarily stores and holds a background-only image and a subject-included image in which a clipping target image exists in the background,
The image data comparison unit extracts difference image data of a plurality of images reproduced and output by the image recording unit,
The imaging condition recording / reproducing means records an imaging condition when an image to be cut out in a predetermined background is captured,
The image extraction apparatus characterized in that the image data conversion means converts image data of the background only image based on the imaging conditions reproduced and output by the imaging condition recording / reproducing means.   2. The image extracting apparatus according to claim 1, wherein the image recording unit encodes and records image data of a specific image area obtained by the image cutout unit.   3. The image extraction apparatus according to claim 1, wherein the image data conversion means spatially performs a shift operation so that the similarity between the background-only image and the subject-included image is maximized.   The image extraction apparatus according to claim 1, wherein the image data conversion unit converts an image size, a luminance level, a color component, and a resolution.

Images