TWI393070B - Human face model construction method - Google Patents

Description

Method of building a face model

The invention relates to a method for establishing a face model, in particular to a method for establishing a face model by using an infrared light source for face reconstruction.

The face recognition system has a very wide application in the security monitoring system. However, the current face recognition technology has great limitations on the applied environment, making it difficult to widely promote this technology in practical applications. Among them, the biggest limitation is that it cannot adapt to changes in the environment. For example, different changes in ambient light, face pose, and expressions result in a low recognition rate of conventional face recognition methods, and have little practical value.

A common method of the conventional face recognition method is to use a single camera to capture the user's face, and transmit the image to the monitoring center through the image capture card, and analyze the feature values in the face image and the face image of the database. Perform face recognition. However, due to the complexity of the environment in practical applications, especially in non-ideal ambient lighting conditions, most face recognition systems are bound to encounter problems of recognition performance degradation. As the light changes, the information on the same face may change greatly, often resulting in differences in facial features. This is also the reason why the existing face recognition system cannot provide the user with the most important use.

Most of the research work is to improve existing visible light face recognition systems to mitigate the effects of ambient light. Although some progress has been made, the results have been poor.

In order to solve the different changes in face posture and expression, it is proposed to use three The Viteron scanner captures the three-dimensional coordinates of the molded object and constructs its three-dimensional model, or uses a structured light camera to generate a fixed pattern of stripes on the object, and then calculates the three-dimensional position of the object and constructs the three-dimensional model according to the shape of the stripe. Or use a face image sequence to construct a three-dimensional face model. Among them, there is a technique for constructing a three-dimensional face by constructing a three-dimensional face model of a molded face using a single or multiple face images, which is prepared by a general three-dimensional face model prepared in advance, according to two-dimensional The image adjusts the shape of the general three-dimensional face model by interactive, semi-automatic or automatically capturing the position of the face feature point, and reconstructs the important feature points of the three-dimensional model according to the feature points of the organs in the face image, and then the general three-dimensional The relationship between the triangular vertices of the face model, plus the texture and mapping techniques of the two-dimensional face image, to create a personal three-dimensional face model. However, the virtual three-dimensional face model of a single face image lacks face depth information.

In the 3D image face recognition method, one of them captures 3D face data through a 3D laser scanner. Although the device system is less affected by ambient light, the device is expensive and the storage and calculation complexity is high. The light will cause more damage to the human eye and will not meet the needs of the actual system. At present, the technology that can overcome the influence of illumination changes is the infrared light source face recognition technology.

Face recognition is mainly based on the characteristics of the human face, but because of the complex changes in the face, it is very difficult to describe and capture features, and these factors make face recognition a challenging topic. The general face recognition system mostly analyzes two-dimensional photos or dynamic video sequences based on image processing technology; however, the surface of the face itself is three-dimensional, and the photo is the result of planar projection of the three-dimensional surface. Zhonghui Losing some important information, so if you use 2D photos to identify, there will be serious recognition obstacles. The biggest difference between using 3D recognition and traditional methods is that the information of the face can be better described, such as the depth information of the feature points of the face and the structure between the points. With more comprehensive information, the false positive rate in the recognition process can be effectively reduced. At the same time, because the three-dimensional face model has the characteristics of illumination independence and attitude independence, it can correctly reflect the basic characteristics of the face, and the main face is The three-dimensional structure is more affected by the expression, thus forming a relatively stable representation of the face features. Therefore, the recognition method of 3D face model can solve the research bottleneck existing in this field.

Based on the above two problems of face recognition, the present invention proposes a method for reconstructing an infrared light source face model by using stereo vision, which is a face shape model obtained by using two predefined images to pass through pre-defined feature points and sample images. A method of recognizing a face that is actually photographed. On the basis of some local point model matching, the statistical model is used to constrain the shape of the face to be recognized, which is transformed into an optimization problem, and it is expected to finally converge to the actual face shape. The three-dimensional face model can be created by storing a small number of face feature points without storing the three-dimensional face image fitted by the multi-pose two-dimensional face image. Therefore, the 3D face model can reduce the influence of facial expressions, and the infrared light source can cooperate with the images captured by the camera to solve the problem of poor image quality caused by the ambient light source, thereby obtaining a better face. Detect the effect.

In view of the above-mentioned problems of the prior art, the object of the present invention is to provide A method for establishing a face model to solve the problem that the face recognition is too high due to changes in ambient illumination, different face poses, and different expressions of expressions.

According to an object of the present invention, a method for establishing a face model is provided, which comprises: after illuminating a face with a plurality of invisible light sources, capturing a plurality of first images reflected by a human face, and performing a first image on the first image After the regional binary image processing forms a plurality of second images, a plurality of second images are subjected to an active face model calculation to obtain a plurality of feature points, and finally a convex hull calculation and a triangle within a plurality of feature points are performed. Insert a calculation and build a face model.

The invisible light source is an infrared light source.

Among them, the number of invisible light sources is two.

The regional binary graphics processing further comprises first dividing the first image into a plurality of blocks, each block is further divided into a sub-block of 3×3, and comparing the plurality of surrounding parts in a clockwise direction for each sub-block. Sub-block grayscale value and a central sub-block grayscale value. If the grayscale value of the surrounding sub-block is greater than the grayscale value of the central sub-block, the surrounding sub-block coding is represented by 1, if the surrounding sub-element If the gray level value of the block is smaller than the gray value of the central sub-block, the surrounding sub-block coding is represented by 0, thereby obtaining a regional binary pattern coding, and finally coding the regional binary pattern, and performing two The regionally converted graphic value of one of the main blocks is obtained by the carry conversion.

Among them, the range of regional binary graphic values is between 0 and 255.

The active face model calculus includes horizontally comparing the second images and performing image matching, and then obtaining a plurality of feature points.

Among them, the number of the plurality of feature points is 50 to 90.

Among them, the number of the plurality of feature points is preferably 68.

As described above, the method of establishing a face model according to the present invention may have one or more of the following advantages:

(1) The method for establishing a face model can solve the problem of poor image capturing caused by the ambient light source by the infrared light source and the image captured by the camera, thereby obtaining a better face detection effect.

(2) The method for establishing a face model can establish a three-dimensional face model by storing a small number of face feature points without storing a three-dimensional face image fitted by the multi-pose two-dimensional face image.

Please refer to FIG. 1 , which is a flowchart of a method for establishing a face model according to the present invention. As shown in the figure, the method for establishing a face model of the present invention comprises the following steps: (S10) illuminating a face with a plurality of invisible light sources;

(S20) capturing a plurality of first images reflected by the human face; (S30) performing a regional binary pattern processing on the first image to form a plurality of second images; (S40) pairing a plurality of second images The image is subjected to an active face model calculus to obtain a plurality of feature points; (S50) a convex face calculus and a triangular interpolation calculus are performed on a plurality of feature points to construct a face model.

The invisible light source is an infrared light source, and the number thereof is two.

Step (S30) Regional Binary Graphics Processing further includes the following steps: (S300) dividing the first image into a plurality of blocks, each block being further divided into a sub-block of 3×3; (S301) for each sub-block Blockwise direction For a plurality of surrounding sub-block grayscale values and a central sub-block grayscale value, if the surrounding sub-block grayscale value is greater than the central sub-block grayscale value, the surrounding sub-block coding is represented by 1. If the gray level value of the surrounding sub-block is smaller than the gray value of the central sub-block, the surrounding sub-block coding is represented by 0, thereby obtaining a regional binary pattern coding; (S302) the regionalization is binarized. The graphic coding performs binary conversion to obtain a regionally binarized graphic value of one of the main blocks. Among them, the range of regional binary graphic values is between 0 and 255.

In the step (S40), the active face model calculation further includes horizontally comparing the second images and performing image matching, and then obtaining a plurality of feature points. The number of the plurality of feature points is 50 to 90, preferably 68.

Please refer to FIG. 2, which is a schematic diagram of an embodiment of a method for establishing a face model according to the present invention. In the figure, (a) to (d) are two sets of two left and right face images captured by the above step (S20), which are first processed in a regional binary pattern for noise cancellation. And get the (e) to (h) map. As shown in Figure 3, the basic Local Binary Patterns (LBP) operands are a fixed-size 3×3 rectangle with a total of nine grayscale values. Then, the grayscale value in the range of 3×3 pixels is compared with the central grayscale value, and the block larger than the central grayscale value is represented by 1, and vice versa, represented by 0, and the clockwise square image is sequentially compared and encoded. , a regional binary pattern coding can be obtained, and then converted into a binary value to obtain a value of 0 to 255, that is, a regional binary pattern value of the 3×3 region. For different types of regional textures, after the regional binary graphics operator calculation, we can get the corresponding regional binary graphics coding, for example: 00000000=0 means there is a bright spot. 11111111=255 means there is a dark spot, and 11100000=224 means there is an edge in the 3×3 area. It can be seen that the regional binary pattern coding has strong extraction performance for local texture. Therefore, the present invention uses a regional binary graphics operator to extract texture features of a human face. In order to get a normalized face image, we cut the face image into many areas. According to the regional dual-characteristic feature extraction of different regions, each region extracts independent feature values, and these feature values are sequentially arranged into one feature vector, and the regional binary image is extracted for each region, and the feature values are extracted. The resulting feature vector represents the original face image.

In addition, if the (e) to (h) images are subjected to pseudo color processing, (i) to (1) images can be obtained, and it can be seen that the face image has been subjected to the regional binary pattern processing. It is not affected by the unevenness of the infrared light source, which is beneficial to the subsequent face feature positioning processing.

Please refer to FIG. 4, which is a schematic diagram of finding the feature points of the face image for the active face model calculation of the present invention. As shown in the figure, the face image subjected to the regional binary pattern processing is subjected to active face model calculation to obtain a plurality of feature points. Among them, (a) is the original image image, (b) is the image processed by the face shape model, (c) is the face detection result, and the feature point and face shape model are described. (d) to (h) are the process of calculating the face shape model, (d) the picture is a predefined feature point and sample image, and (e) the picture is the image picture of the feature point after calculation, (f) The picture shows the number of the special micro point, and the (g) picture shows the face shape model. Finally, (h) is the experimental result obtained by combining the feature points and the face shape model. In this embodiment, the number of feature points is 50 to 90, and preferably 68.

Please refer to FIG. 5 , which is a schematic diagram of establishing a face model for the convex hull calculation and the triangular interpolation calculation of the present invention. As shown in the figure, after searching for feature points in two face images by using the active face model algorithm, the present invention uses the corresponding algorithm of feature point consistency to find the disparity map, and finally uses the convex hull (Convex- Hull) algorithm and delaunay algorithm to establish 3D face model. Figure 5 is a comparison of the processing results of two left and right images after feature point search and the corresponding three-dimensional face model. (a) The picture is the left face image, (b) the picture is the right face image, (c) the figure is the left face image of 68 feature points calculated by the active face model algorithm, and (d) the figure is via The active face model algorithm calculates the face image of the right side of 68 feature points, and then the horizontal alignment (c) and (d) the image to complete the image matching, and finally, the (e) figure (f) through the triangulation algorithm Three-dimensional face reconstruction results can be obtained by inserting other face feature points.

Among them, the convex hull algorithm, which can enclose the smallest convex outer shell of some points, that is, the smallest convex polygon that can pack all the points into it. Convex is defined as the line connecting any two points in the graph does not pass outside the graph. The convex hull algorithm is a basic mathematical and geometric structure. The convex envelope algorithm is used to find the periphery of the feature points of the face, and the three-dimensional model of the face is established by triangulation and interpolation. The facial feature points obtained through the active face model use triangulation to connect each point with the adjacent two points into a triangle, which can separate all points in the geometric space according to the distance and distance relationship of the objects, so that each object can be divided into certain The area, the distance between any point in the area and the object is closer than other objects. Triangulation is usually used to interpolate the data of facial feature points, and this method is effective for triangulation of scattered points.

The purpose of the present invention is to predict an image of a face of a face as a tool for assisting face recognition or expression recognition. The current automatic face recognition system is mostly based on positive face images, which is less effective or unusable for non-positive faces. Such a design limits the user's need to face the camera, greatly limiting the user's convenience and system suitability. Through the prior statistical learning, the present invention predicts the appearance of the reconstructed front through the 3D face shape model information, so as to facilitate the processing of the subsequent analysis and identification. You can choose to mitigate the effects of facial expression changes and contribute to the accuracy of face recognition.

The above is intended to be illustrative only and not limiting. Any equivalent modifications or alterations to the spirit and scope of the invention are intended to be included in the scope of the appended claims.

S10~S50‧‧‧Steps

1 is a flow chart of a method for establishing a face model according to the present invention; FIG. 2 is a schematic diagram of an embodiment of a method for establishing a face model according to the present invention; and FIG. 3 is a regional binary of the present invention. Schematic diagram of the graphics processing; FIG. 4 is a schematic diagram of finding the feature points of the face image by the active face model calculation of the present invention; and FIG. 5 is a method for constructing the face model by using the convex hull calculation and the triangular interpolation calculation of the present invention. Schematic diagram.

S10~S50‧‧‧Steps

Claims (7)

A method for establishing a face model, comprising the steps of: illuminating a face with a plurality of invisible light sources; capturing a plurality of first images reflected by the face; performing a regional binary on the first image Forming a plurality of second images by performing graphics processing; performing an active face model calculation on the plurality of second images to obtain a plurality of feature points; and performing convex hull calculation and triangular interpolation calculation on the plurality of feature points to establish one person a face model; wherein the regional binary graphics processing system divides the first image into a plurality of blocks, and each of the blocks is further divided into a sub-block of 3×3, and the clockwise direction is proportional to each sub-block. For a plurality of surrounding sub-block grayscale values and a central sub-block grayscale value, if the surrounding sub-block grayscale values are greater than the central sub-block grayscale values, the surrounding sub-blocks are encoded 1 means that if the gray level values of the surrounding sub-blocks are smaller than the gray value of the central sub-block, the surrounding sub-block codes are represented by 0, thereby obtaining a regional binary pattern coding, and the area is obtained. Sexually binary graphics coding for binary transposition Each of the blocks obtained by one of the main pattern of regional values binarized. The method for establishing a face model according to claim 1, wherein the invisible light source is an infrared light source. The method for establishing a face model according to claim 2, wherein the number of the invisible light sources is two. The method for establishing a face model according to claim 1, wherein the regional binary pattern value ranges from 0 to 255. The method for establishing a face model according to claim 1, wherein the active face model calculus comprises horizontally aligning each of the second images and performing image matching, and obtaining the plurality of feature points. The method for establishing a face model according to claim 5, wherein the number of the plurality of feature points is 50 to 90. The method for establishing a face model according to claim 6, wherein the number of the plurality of feature points is preferably 68.