JP4597391B2 - Facial region detection apparatus and method, and computer-readable recording medium - Google Patents

Description

[0001]
BACKGROUND OF THE INVENTION
The present invention relates to a face area detecting apparatus and method for detecting a human face area from a color image, and a computer-readable recording medium.
[0002]
[Prior art]
In the development of an autonomous mobile robot, it is one of the important issues to automatically extract a facial region from an image in order to identify humans and recognize facial expressions. For example, the template matching method is one of conventionally known face region extraction methods. In this method, images of only various types of face regions prepared in advance are used as templates, matching with face images in the image to be processed is performed using gray values, and the template with the highest matching degree is selected. Is. As other methods, an Eigenface method using feature vectors called eigenfaces, a skin color region clustering method using color information, and the like are known.
[0003]
[Problems to be solved by the invention]
However, the conventional methods such as the template matching method described above have the following problems. If the size of a person's face varies depending on the distance from the camera taken in the same image, or if the size of the same person's face in each frame changes due to the movement of the person in the moving image, Even the faces of the same person cannot be extracted with the same template. Therefore, in order to be able to detect the face area of a person assuming all cases, it is necessary to prepare a huge number of templates corresponding to faces of various sizes and types in advance.
[0004]
Further, the template matching method has a disadvantage that the calculation cost increases by increasing the number of templates. As a face area recognition method for autonomous mobile robots, it is necessary to search and track multiple faces simultaneously within a practical processing time, so apply a method that requires search time, such as the template matching method. That is not preferable.
Further, when the clustering method using color information is used for detection of a face area, there arises a problem that a face area that is a skin color area cannot be extracted effectively if there is a change in illumination or a skin color portion exists in the background.
[0005]
The present invention has been made in consideration of such circumstances, and it is not necessary to store an enormous number of templates in advance as in the template matching method, and the face can be detected at high speed. An object of the present invention is to provide an area detection apparatus and method, and a computer-readable recording medium.
[0006]
[Means for Solving the Problems]
In order to solve the above-described problem, the invention according to claim 1 includes an extraction unit that extracts a skin color region from a color image acquired by an imaging device. ,face Outline model generation means for generating an outline model that gives an outline of the area, the outline model, and the skin color area According to the height of the evaluation value using as parameters the number of pixels that coincide with the contour pixel and the number of pixels in the skin color region included in the region surrounded by the contour model A face area specifying means for specifying the face area. The contour model generation means generates the contour model having a size corresponding to the depth information of the skin color region using a distance image having depth information of the color image. It is characterized by doing.
[0007]
According to a second aspect of the present invention, in the first aspect of the present invention, the subject within the distance image and within a predetermined distance The area containing The above Evaluation value Area to calculate As It further has search area setting means for setting.
[0008]
The invention according to claim 3 is a face area detection device that detects a face area from one frame of a color image of a continuous scene and tracks the face area in the next frame and thereafter, wherein the skin color area is detected from the color image. Extraction means for extracting ,in front Contour model generation means for generating a contour model that gives the contour of the facial expression region, the contour model and the skin color region According to the height of the evaluation value using as parameters the number of pixels that coincide with the contour pixel and the number of pixels in the skin color region included in the region surrounded by the contour model A face area specifying means for specifying the face area, and a subject within the distance image and within a predetermined distance; The area containing The above Evaluation value Area to calculate As Search area setting means for setting, and the one set in the one frame Evaluation value Reflecting the position of the area where the Evaluation value Tracking means to set the position of the area Then, using the distance image having the depth information of the color image of the one frame, the contour model having a size corresponding to the depth information of the skin color region is generated. It is characterized by doing.
[0009]
According to a fourth aspect of the present invention, in the invention according to any one of the first to third aspects, the extracting means uses a hue component of the color image.
[0010]
The invention according to claim 5 is the invention according to any one of claims 1 to 4, wherein the distance image is composed of two images acquired from two imaging devices associated with each other. It is created using depth information obtained by stereoscopically viewing a stereo image to be viewed.
[0012]
Claim 6 In the first aspect of the present invention, a skin color region is extracted from a color image acquired by an imaging apparatus, and a subject within a predetermined distance that is a region in the distance image. The area containing , Evaluation value Area to calculate As A second step to set up and ,face A third step of generating a contour model that gives a contour of the region, the contour model and the skin color region According to the height of the evaluation value using as parameters the number of pixels that coincide with the contour pixel and the number of pixels in the skin color region included in the region surrounded by the contour model And a fourth process for identifying the face area. In the second step, the contour model having a size corresponding to the depth information of the skin color region is generated using a distance image having the depth information of the color image. It is characterized by doing.
[0013]
Claim 7 The invention described in 1 is a face area detection method for detecting a face area from one frame of a color image of a continuous scene and tracking the face area after the next frame, and extracting a skin color area from the color image A first process and a subject within a predetermined distance in an area in the distance image The area containing , Evaluation value Area to calculate As A second step to set up and ,in front A third step of generating a contour model for giving a contour of the face writing region, the contour model and the skin color region According to the height of the evaluation value using as parameters the number of pixels that coincide with the contour pixel and the number of pixels in the skin color region included in the region surrounded by the contour model A fourth process of identifying the face area, and the one set in the one frame Evaluation value Reflecting the position of the area where the Evaluation value And a fifth process for setting the position of the region where In the third process, the distance model having the depth information of the color image is used to generate the contour model having a size corresponding to the depth information of the skin color region. It is characterized by doing.
[0015]
Claim 8 The invention described in 1 is a first procedure for extracting a skin color region from a color image acquired by an imaging device, and a subject within a predetermined distance that is a region in the distance image. The area containing , Evaluation value Area to calculate As The second step to set and ,face A third procedure for generating a contour model that gives a contour of the region; and the contour model and the skin color region Depending on the height of the evaluation value using as parameters the number of pixels that coincide with the contour pixel of the image and the number of pixels in the skin color region included in the region surrounded by the contour model Causing the computer to execute a fourth procedure for identifying the face area. In the third procedure, the computer is caused to execute a procedure of generating the contour model having a size corresponding to the depth information of the skin color region using a distance image having depth information of the color image. The program is recorded.
[0016]
Claim 9 The invention described in 1 is a face area detection method for detecting a face area from one frame of a color image of a continuous scene and tracking the face area in subsequent frames, and extracting a skin color area from the color image A first procedure and a subject within a predetermined distance that is an area in the distance image; The area containing , Evaluation value Area to calculate As The second step to set and ,in front A third procedure for generating a contour model for giving a contour of the face writing region, the contour model and the skin color region Depending on the height of the evaluation value using as parameters the number of pixels that coincide with the contour pixel of the image and the number of pixels in the skin color region included in the region surrounded by the contour model A fourth procedure for identifying the face area; and the one set for the one frame. Evaluation value Reflecting the position of the area where the Evaluation value And causing the computer to execute a fifth procedure for setting the position of the area in which to calculate The third procedure causes the computer to execute a procedure for generating the contour model having a size corresponding to the depth information of the skin color region using a distance image having depth information of the color image. The program is recorded.
[0018]
Claim 10 The invention described in 1 is a first procedure for extracting a skin color region from a color image acquired by an imaging device, and a subject within a predetermined distance that is a region in the distance image. The area containing , Evaluation value Area to calculate As The second step to set and ,face A third procedure for generating a contour model that gives a contour of the region; and the contour model and the skin color region According to the height of the evaluation value using as parameters the number of pixels that coincide with the contour pixel and the number of pixels in the skin color region included in the region surrounded by the contour model Causing the computer to execute a fourth procedure for identifying the face area. The third procedure causes the computer to execute a procedure for generating the contour model having a size corresponding to the depth information of the skin color region using a distance image having depth information of the color image. It is characterized by that.
[0019]
Claim 11 The invention described in 1 is a face area detection method for detecting a face area from one frame of a color image of a continuous scene and tracking the face area in subsequent frames, and extracting a skin color area from the color image A first procedure and a subject within a predetermined distance that is an area in the distance image; The area containing , Evaluation value Area to calculate As The second step to set and ,in front A third procedure for generating a contour model for giving a contour of the face writing region, the contour model and the skin color region According to the height of the evaluation value using as parameters the number of pixels that coincide with the contour pixel and the number of pixels in the skin color region included in the region surrounded by the contour model A fourth procedure for identifying the face area; and the one set for the one frame. Evaluation value Reflecting the position of the area where the Evaluation value And causing the computer to execute a fifth procedure for setting the position of the area in which to calculate The third procedure causes the computer to execute a procedure for generating the contour model having a size corresponding to the depth information of the skin color region using a distance image having depth information of the color image. It is characterized by that.
[0021]
DETAILED DESCRIPTION OF THE INVENTION
Hereinafter, an embodiment of the present invention will be described with reference to the drawings. FIG. 1 is a block diagram showing a configuration of a face area detecting apparatus according to an embodiment of the present invention.
In FIG. 1, reference numeral 11 denotes a stereo image input means, which is a means for acquiring a color stereo image with two imaging devices such as two CCD cameras associated with each other and inputting the color stereo image to the face area detection device. A distance image generation unit 12 obtains depth information as image information of a subject by stereoscopically viewing two stereo images of the same scene input by the stereo image input unit 11. And it is a means for producing | generating the distance image which matched those depth information with one color image of a stereo image.
[0022]
13 is a skin color area extracting means, which converts a color image acquired by RGB coordinates into HLS coordinates and extracts only the area corresponding to the skin color by using a hue component or the like, thereby extracting the skin color area and other areas. Is a means for creating a skin color region image consisting of
Reference numeral 14 denotes edge extraction means for extracting an outline portion of the skin color area extracted by the skin color area extraction means 13 as an edge and creating an edge image.
[0023]
15 is an outline model determining unit, which is a region in the distance image generated by the distance image generating unit 12, and has a size determined by using depth information of the region corresponding to the target skin color region It is a means for generating a contour model of a region. In the present embodiment, a contour model expressed by an octagonal frame is used.
Reference numeral 16 denotes search area setting means, which extracts only a subject area having distance information designated in advance using the distance image generated by the distance image generation means 12, and determines a rectangular area including the area as a face area. It is a means used as a search area for detecting.
[0024]
Reference numeral 17 denotes matching means, which is included in the number of matching pixels between the contour model determined by the contour model determination means 15 and the contour pixels of the skin color area extracted by the edge extraction means 14, and within the area surrounded by the contour model. It is a means for calculating a correlation value using the number of skin color pixels in the skin color area extracted by the skin color area extraction means 13 as a parameter. This calculation is limited to the search area determined by the search area setting means 16.
Reference numeral 18 denotes a face area determination means, which is a case where the correlation value set based on the number of two types of pixels calculated by the matching means 17 is larger than a preset threshold value, and within the search area. This is means for determining the skin color area in the case of the maximum as the face area.
[0025]
Next, operations of the embodiment will be described with reference to the drawings.
FIG. 2 is a flowchart for explaining the face area detection procedure in the embodiment described above.
First, a color stereo image is acquired by the stereo image input means 11 which is an imaging device such as two CCD cameras associated with each other (step S11). FIG. 3 is a diagram showing an outline of one color image among stereo images of a scene including a human face. In FIG. 3, 3 is a color image of a scene in which three persons 31, 32, and 33 are photographed indoors.
[0026]
Then, a distance image having depth information is generated by stereoscopically viewing the stereo image by the distance image generating means 12 (step S12). When creating a distance image, two monochrome images may be used. FIG. 4 is a diagram showing an outline of a distance image having depth information corresponding to the color image 3 of FIG. In the distance image 4, the distance is expressed so that the portion near the camera is bright and the distance from the camera becomes darker. That is, the persons 31, 32, and 33 that are close to the camera and located at substantially the same distance are expressed with substantially the same brightness value. In this embodiment, a portion near the camera is displayed in white, and a portion far from the camera is displayed with diagonal lines.
[0027]
Next, a skin color area image including the face area is created by extracting the skin color area from one color image 3 of the stereo image by the skin color area extracting means 13 (step S13).
Here, the procedure for creating the skin color region image in step S13 will be described in detail with reference to FIG. FIG. 5 is a flowchart for explaining the procedure for creating the skin color area image in step S13 in FIG.
First, the input color image is subjected to color coordinate conversion from RGB coordinates to HLS color coordinates (step S131). As a result, the value of each pixel represented by the red, green, and blue components can be extracted as the hue component.
Then, a pixel having a size in a range regarded as a skin color pixel is set in advance from the hue component and the like, extracted as a skin color pixel, and a set of the skin color pixels is set as a skin color region (step S132).
[0028]
Next, a binarization process is performed to divide the skin color area into other areas (step S133).
Further, region expansion processing and contraction processing are performed on the portion corresponding to the skin color region in the binarized image (step S134).
Then, a noise removal process is performed in which an area that does not have a certain number of pixels or more out of each area included in the image is not a face area but is regarded as noise (step S135). As a result, a binary skin color region image can be created. FIG. 6 is a diagram showing an outline of the skin color area image created in the above-described procedure. In FIG. 6, reference numerals 61 to 65 denote skin color regions.
[0029]
And the edge image which extracted only the outline of each skin color area | region of a skin color area image as an edge is produced (step S136). FIG. 7 is a diagram showing an outline of the edge image. In FIG. 7, reference numerals 71 to 75 denote edges corresponding to the contour portions of the skin color regions 61 to 65 in FIG.
[0030]
Furthermore, a rectangular area including an area in the distance image having distance information within a preset range is obtained by the search area setting means 16, and the area is set as a search area for performing matching using the contour model ( Step S14). In the present embodiment, only a region having depth information within a distance of 5 meters from the camera is obtained from the distance image 4, and a rectangular region including the region is set as one search region.
FIG. 8 is a diagram for explaining a search area created from the distance image 4. In FIG. 8, the portion indicated by the dotted line 8 is the boundary of the search area, and the inside is the search area. At this time, if areas within 5 meters exist apart in the same image, two or more search areas may be created. The distance from the camera can be appropriately set according to conditions such as the environment to which the present apparatus is applied.
[0031]
If two or more search areas are set, one search area is designated (step S15). If only one search area is set, the following processing is performed for the search area.
The search area 8 specified in step S15 is applied to the skin color area image 6 created in step S13, and one skin color area for performing face area detection processing is specified from each skin color area existing in the search area. (Step S16).
For example, consider the case where the skin color region 61 of FIG. 6 is designated in the present embodiment. Therefore, based on the depth information of the area in the distance image corresponding to the designated skin color area 61, it is determined whether or not the skin color area is in an effective range such as within 5 meters from the camera (step S17).
If the determination result is YES, the process proceeds to step S18. If the determination result is NO, the face area to be detected is determined and the process returns to step S16, and the process of step S17 is performed for another skin color area. In this embodiment, it is determined as YES and the process proceeds to step S18.
[0032]
In step S18, based on the depth information in the distance image corresponding to the target skin color area 61, the contour model determining means 15 generates an octagonal contour model having a size corresponding to the depth information. FIG. 9 is a diagram for explaining the search area 8 and the contour model applied on the skin color area image 6 generated in FIG.
In FIG. 9, 91 is a contour model determined by the contour model determining means 15 based on the skin color region 61. Then, matching with the skin color region is performed while scanning the contour model 91 created in step S18 in the search region 8 pixel by pixel.
[0033]
FIG. 10 is a diagram for explaining the search area 8 and the contour model 91 applied on the edge image 7 generated in FIG. Then, the contour model 91 is scanned pixel by pixel in the search region 8 and matching with the edge that is the contour of each skin color region is performed (step S19).
In this matching, the number of coincident pixels between the created contour model and the edge that is the contour pixel of the skin color region and the number of skin color pixels surrounded by the contour model are calculated. Then, the correlation value using the number of coincident pixels and the number of skin color pixels as parameters is used as an evaluation value for determining whether or not the face region (step S20).
[0034]
In the present embodiment, the evaluation value is obtained as follows. First, as shown in FIG. 9, the number of skin color pixels surrounded by the contour model 91 applied on the skin color region image 6 is calculated, and this is set as M1. Next, the total number of pixels in the region surrounded by the contour model 91 is calculated, and this is set as M2. A value obtained by dividing M1 by M2 is M3.
Further, as shown in FIG. 10, an edge pixel coinciding with the contour model 91 applied on the edge image 7 is calculated, and this is set as N1. Next, the number of pixels constituting the contour model is calculated, and this is set as N2. A value obtained by dividing N1 by N2 is N3.
Then, each value of M3 and N3 is multiplied by a weighting factor designated in advance, and a value obtained by adding both is defined as an evaluation value Z.
[0035]
When this calculation is performed in the search area 8 and the evaluation value Z is the maximum in the search area 8 and equal to or greater than the threshold value, it is determined that the target skin color area is a face area. In other cases, it is determined that the face area is not included (step S21).
Note that threshold values may also be set for M3 and N3, which are the basis for obtaining the evaluation value Z, and the evaluation value Z may be obtained when both are equal to or greater than the threshold value. Thus, in the present embodiment, the skin color area 61 is determined to be a face area based on the contour model 91.
[0036]
When the above process ends, the process proceeds to step S22. In step S22, when another skin color area exists in the same search area (YES), the process returns to step S16 and the processing for face area detection is executed in the procedure described above for the skin color area (step S22). If no other skin color area exists (NO), all face areas in the search area are detected, and the process proceeds to step S23.
[0037]
In step S23, in the present embodiment, since there are skin color regions 62 to 65 in addition to the skin color region 61 in the search region 8, it is determined YES, and in the same manner as described above, the skin color regions 62 to 65 are detected in the face region. It is determined whether it exists. As a result, the three skin color areas 61, 62, and 63 are determined as face areas, and the skin color areas 64 and 65 are not determined as face areas because the evaluation value Z is small.
In this way, as shown in FIG. 11, all face regions can be detected in one image.
[0038]
Next, in step S23, it is determined whether another search area exists in the same image.
If another search area exists (YES), the process returns to step S15 and the same process is performed. If no other search area exists (NO), the process ends because all face areas in the image have been detected. In the present embodiment, the process ends because there is no other search area.
[0039]
Next, a processing procedure for detecting a face area from one frame of a color image of a continuous scene and tracking the face area in the subsequent frames will be described. FIG. 12 is a flowchart for explaining a procedure for tracking a face area in a continuous frame.
In the process related to the tracking of the face area, first, the face area in one frame is detected based on the above-described procedure (step S31). Then, a search area is set in the next successive frame (step S32). At this time, the position of the search area to be set is set to the same position as the previous position or the vicinity thereof based on the position set in the previous frame.
[0040]
When the search area is set in the image of the next frame, using the contour model determined in the previous frame, the number of skin color pixels is calculated in the same manner as the processing procedure shown in step S19 of FIG. Is calculated (step S33).
Next, it is determined whether or not the evaluation value is the maximum in the search area and is equal to or greater than a certain threshold value (step S34).
And if a judgment result is YES, it will consider that the position is a face area and will progress to step S35. On the other hand, if the determination result is NO, the face region corresponding to the contour model is determined not to exist in the search region, and the process ends.
[0041]
In step S35, it is further determined whether or not tracking is performed in the next frame. If the determination result is YES, the process returns to step S32 to execute the tracking process in the next frame. On the other hand, if the determination result is NO, the face region is not tracked and the process ends.
[0042]
In the above-described embodiment, a method for directly inputting a color stereo image captured using an imaging device such as a CCD camera has been described. However, the same procedure is performed by inputting an image file that has been captured and stored in advance. It is also possible to detect the face area.
Further, instead of calculating the number of skin color pixels in the contour model, color histogram calculation may be used.
[0043]
Furthermore, in the above-described embodiment, an octagonal frame model is used as the contour model, but a model having an ellipse or other polygonal shape may be used.
Furthermore, by changing the aspect ratio of the contour model, finer matching processing can be performed on face shapes having individual differences.
Furthermore, by using a contour model having a shape close to the actual face contour, it is possible to specify the exact face position.
[0044]
In the present embodiment, since not only a two-dimensional color image but also a distance image is used, matching with the skin color region image 6 and the edge image 7 is not performed using the contour model, but the distance image is used. It is also possible to perform matching using a three-dimensional face model.
[0045]
Further, by simultaneously performing the initial search and tracking of the face area, it is possible to detect a newly appearing face area while tracking the face area. Alternatively, it is possible to easily cope with environmental changes by changing the threshold value.
Furthermore, by recording a history such as the size of the search area 8 and the contour model used in the past, the face area that has gone out of the screen in the middle of the continuous image or the face area that was mistakenly detected is recorded. It is also possible to delete unnecessary items efficiently.
[0046]
1 is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read into the computer system and executed, thereby executing the processing of the facial area. Detection may be performed. Here, the “computer system” includes an OS and hardware such as peripheral devices. Further, the “computer system” includes a homepage providing environment (or display environment) if a WWW system is used.
[0047]
The “computer-readable recording medium” refers to a portable medium such as a floppy disk, a magneto-optical disk, a ROM, or a CD-ROM, and a storage device such as a hard disk built in the computer system. Further, the “computer-readable recording medium” refers to a volatile memory (RAM) in a computer system that becomes a server or a client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line. In addition, those holding programs for a certain period of time are also included.
[0048]
The program may be transmitted from a computer system storing the program in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the “transmission medium” for transmitting the program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. The program may be for realizing a part of the functions described above. Furthermore, what can implement | achieve the function mentioned above in combination with the program already recorded on the computer system, and what is called a difference file (difference program) may be sufficient.
[0049]
【The invention's effect】
As described above, according to the first aspect of the present invention, the extraction means for extracting the skin color region from the color image acquired by the imaging device; ,face Outline model generation means for generating an outline model that gives an outline of an area, and the outline model and skin color area Depending on the height of the evaluation value using as parameters the number of pixels that match the contour pixels of the image and the number of pixels in the skin color region included in the region surrounded by the contour model A face area specifying means for specifying the face area. The contour model generation means generates a contour model having a size corresponding to the depth information of the skin color region by using the distance image having the depth information of the color image. Therefore, it is not necessary to store a huge number of templates in advance as in the template matching method, and the face area can be detected at high speed.
[0050]
According to the second aspect of the present invention, a subject within a distance image and within a predetermined distance The area containing , Evaluation value Area to calculate As Since the search area setting means for setting is further included, the processing speed for detecting the face area can be further improved.
[0051]
According to the third aspect of the present invention, there is provided a face area detecting device that detects a face area from one frame of a color image of a continuous scene and tracks the face area in the next and subsequent frames. Extraction means for extracting ,face Outline model generation means for generating an outline model that gives an outline of an area, and the outline model and skin color area Depending on the height of the evaluation value using as parameters the number of pixels that match the contour pixels of the image and the number of pixels in the skin color region included in the region surrounded by the contour model Face area specifying means for specifying a face area, and an object in a distance image that is within a predetermined distance The area containing , Evaluation value Area to calculate As Search area setting means to be set and set to one frame Evaluation value Reflecting the position of the area where Evaluation value Tracking means to set the position of the area Then, using a distance image having depth information of the color image of one frame, a contour model having a size corresponding to the depth information of the skin color region is generated. Therefore, it is possible to efficiently track a plurality of face regions from successive frames, and it is suitable for use in an eye part of an autonomous mobile robot. Further, it is not necessary to use a distance image having depth information for tracking the face area in the subsequent frames.
[0052]
According to a fourth aspect of the present invention, the extracting means uses a hue component of a color image. By using the hue component, it is possible to detect the skin color more robustly than the skin color is identified by the normal RGB values.
[0053]
According to the fifth aspect of the present invention, the distance image includes depth information obtained by stereoscopically viewing a stereo image composed of two images acquired from two imaging devices associated with each other. Since it is created by using, a color image and distance information can be acquired at the same time, and it is not necessary to use a special device other than a camera for creating a distance image.
[0054]
This application According to the invention, the face area specifying means includes the contour model and the skin color area. Depending on the height of the evaluation value using as parameters the number of pixels that match the contour pixels of the image and the number of pixels in the skin color region included in the region surrounded by the contour model Since the face area is specified, the matching between the contour model and the skin color area can be evaluated by two parameters, and the human face area can be detected with high reliability.
[Brief description of the drawings]
FIG. 1 is a block diagram showing a configuration of a face area detecting apparatus according to an embodiment of the present invention.
FIG. 2 is a flowchart for explaining a face area detection procedure in the embodiment described above.
FIG. 3 is a diagram showing an outline of one color image 3 among stereo images of a scene including a human face.
FIG. 4 is a diagram showing an outline of a distance image 4 having depth information corresponding to a color image 3;
FIG. 5 is a flowchart for explaining a procedure for creating a skin color area image 6 in step S13 of FIG. 2;
6 is a diagram showing an outline of a skin color region image 6 created in the embodiment described above. FIG.
7 is a diagram showing an outline of an edge image created from a skin color area image 6. FIG.
FIG. 8 is a diagram for explaining a search area 8 created by using depth information of a distance image 4;
FIG. 9 is a diagram for explaining a search area 8 and a contour model applied on a skin color area image 6;
FIG. 10 is a diagram for explaining a search area 8 and a contour model applied on an edge image.
FIG. 11 is a diagram for explaining a result of detecting all face regions in one image.
FIG. 12 is a flowchart for explaining a procedure for tracking a face area in successive frames.
[Explanation of symbols]
3 Color image
4 range image
6 Skin color area image
7 Edge image
8 Search area
11 Stereo image input means
12 Distance image generation means
13 Skin color area extraction means
14 Edge extraction means
15 Contour model determining means
16 Search area setting means
17 Matching means
18 Face area determination means
91, 92, 93 Contour model

Claims (11)

Extraction means for extracting a skin color region from a color image acquired by the imaging device;
Contour model generating means for generating a contour model that gives a contour of a face area;
The height of the evaluation value using the number of pixels in which the contour model and the contour pixels in the skin color region match and the number of pixels in the skin color region included in the region surrounded by the contour model as parameters. Correspondingly possess a face region specifying means for specifying the facial area,
The face area detection device, wherein the outline model generation means generates the outline model having a size corresponding to the depth information of the skin color area using a distance image having depth information of the color image . 2. The search area setting means for setting an area in the distance image that includes a subject within a predetermined distance as an area for calculating the evaluation value. Face area detection device. A face area detection device that detects a face area from one frame of a color image of a continuous scene and tracks the face area in subsequent frames,
Extracting means for extracting a skin color region from the color image;
A contour model generating means for generating a contour model giving the contour of the front Kikao region,
The height of the evaluation value using the number of pixels in which the contour model and the contour pixels in the skin color region match and the number of pixels in the skin color region included in the region surrounded by the contour model as parameters. A face area specifying means for specifying the face area in response ,
Search area setting means for setting an area in the distance image that includes a subject within a predetermined distance as an area for calculating the evaluation value ;
To reflect the position of the region to be calculated the set in one frame the evaluation value, possess a tracking means for setting the position of the evaluation value to be calculated area in the next frame,
An apparatus for detecting a face area , comprising generating a contour model having a size corresponding to the depth information of the skin color area using a distance image having depth information of the color image of the one frame .   The face area detection apparatus according to claim 1, wherein the extraction unit uses a hue component of the color image.   The distance image is created using depth information obtained by stereoscopically viewing a stereo image composed of two images acquired from two imaging devices associated with each other. The face area detection device according to any one of claims 1 to 4. A first step of extracting a skin color region from a color image acquired by the imaging device;
The region including the subject in the distance the distance a region in an image is specified in advance, a second step of setting a region for calculating an evaluation value,
A third step of generating a contour model that gives a contour of the face area;
The height of the evaluation value using the number of pixels in which the contour model and the contour pixels in the skin color region match and the number of pixels in the skin color region included in the region surrounded by the contour model as parameters. Correspondingly it possesses a fourth step of identifying the face region,
In the third step, the distance model having depth information of the color image is used to generate the contour model having a size corresponding to the depth information of the skin color region . A face area detection method for detecting a face area from one frame of a color image of a continuous scene and tracking the face area in the subsequent frames,
A first step of extracting a skin color region from the color image;
The region including the subject in the distance the distance a region in an image is specified in advance, a second step of setting a region for calculating an evaluation value,
A third process of generating a contour model giving the contour of the front Kikao region,
The height of the evaluation value using the number of pixels in which the contour model and the contour pixels in the skin color region match and the number of pixels in the skin color region included in the region surrounded by the contour model as parameters. And a fourth step of identifying the face region in response ,
To reflect the position of the region to be calculated the evaluation value the set in one frame, possess a fifth step of setting the position of should be calculated area the evaluation value in the next frame,
In the third step, the distance model having depth information of the color image is used to generate the contour model having a size corresponding to the depth information of the skin color region . A first procedure for extracting a skin color region from a color image acquired by an imaging device;
A second procedure for setting an area in the distance image that includes an object within a predetermined distance as an area for which an evaluation value is to be calculated;
A third procedure for generating a contour model that gives a contour of the face area;
The height of the evaluation value using the number of pixels in which the contour model and the contour pixels in the skin color region match and the number of pixels in the skin color region included in the region surrounded by the contour model as parameters. In response , the computer executes a fourth procedure for identifying the face area ,
Wherein in the third step, using a range image having depth information of the color image, and the procedure for generating the contour model of a size corresponding to the depth information of the skin color area storing a program Ru is executed by the computer Computer-readable recording medium. A face area detection method for detecting a face area from one frame of a color image of a continuous scene and tracking the face area in the subsequent frames,
A first procedure for extracting a skin color region from the color image;
A second procedure for setting an area in the distance image that includes an object within a predetermined distance as an area for which an evaluation value is to be calculated;
A third step of generating a contour model giving the contour of the front Kikao region,
The height of the evaluation value using the number of pixels in which the contour model and the contour pixels in the skin color region match and the number of pixels in the skin color region included in the region surrounded by the contour model as parameters. And a fourth procedure for identifying the face area in response ,
To reflect the position of the region to be calculated the evaluation value the set in one frame, to execute a fifth step of setting the position of the region to be calculated the evaluation value in the next frame to the computer ,
Wherein in the third step, using a range image having depth information of the color image, and the procedure for generating the contour model of a size corresponding to the depth information of the skin color area storing a program Ru is executed by the computer Computer-readable recording medium. A first procedure for extracting a skin color region from a color image acquired by an imaging device;
A second procedure for setting an area in the distance image that includes an object within a predetermined distance as an area for which an evaluation value is to be calculated;
A third procedure for generating a contour model that gives a contour of the face area;
The height of the evaluation value using the number of pixels in which the contour model and the contour pixels in the skin color region match and the number of pixels in the skin color region included in the region surrounded by the contour model as parameters. In response , the computer executes a fourth procedure for identifying the face area ,
Wherein in the third step, the use of a range image having depth information of a color image, because of the program procedures of generating a magnitude the contour model corresponding to the depth information of the skin color area is performed on the computer. A face area detection method for detecting a face area from one frame of a color image of a continuous scene and tracking the face area in the subsequent frames,
A first procedure for extracting a skin color region from the color image;
A second procedure for setting an area in the distance image that includes an object within a predetermined distance as an area for which an evaluation value is to be calculated;
A third step of generating a contour model giving the contour of the front Kikao region,
The height of the evaluation value using the number of pixels in which the contour model and the contour pixels in the skin color region match and the number of pixels in the skin color region included in the region surrounded by the contour model as parameters. And a fourth procedure for identifying the face area in response ,
To reflect the position of the region to be calculated the evaluation value the set in one frame, to execute a fifth step of setting the position of the region to be calculated the evaluation value in the next frame to the computer ,
Wherein in the third step, the use of a range image having depth information of a color image, because of the program procedures of generating a magnitude the contour model corresponding to the depth information of the skin color area is performed on the computer.

Images