CN105719326A - Realistic face generating method based on single photo - Google Patents

Abstract

The invention discloses a method for generating a realistic human face based on a single photo. First, construct a standardized human face model library, interactively select the facial feature points of the input photo, and match the best model based on the facial features; secondly, through The method of triangle deformation and bilinear interpolation realizes the texture mapping from photos to 3D models, and introduces the Alpha map to realize the blending and transition from the face covering texture to the neutral texture of the model; finally, adopts the grid adjustment method from the whole to the level of detail Adjust the model to achieve the purpose of generating realistic 3D faces. The method used in the present invention is user-friendly, requires fewer feature points, and only needs one frontal photo and fewer feature points, so that the user can participate in and complete the realistic face creation method independently.

Description

A realistic face generation method based on a single photo

technical field

The invention relates to a method for quickly generating a three-dimensional human face, in particular to a method for generating a realistic human face based on a single photo.

Background technique

Human faces are an important part of emotional expression and identity recognition. In life, we use faces to identify identities and express emotions through faces. Faces play a pivotal role in our daily lives. With the development of computer graphics, face reconstruction for specific graphics has been widely used in many fields. Constructing a three-dimensional virtual character with a realistic human face has a positive effect on enhancing the user's interest and participation initiative. In the fields of game animation, film and television advertisements, 3D campuses, digital tourist attractions, virtual locker rooms and other fields, as one of the means to improve the user's sensory experience, research on face model generation technology with better realism, low generation cost, and fast and simple generation process is of great significance.

The existing rapid face generation methods mainly use 3D scanners and other equipment to obtain 3D face data. The generated model has high fidelity, but relies on a large amount of point cloud data, which requires high technical requirements and is expensive; A frontal and side image is used to reconstruct the face model through visual related theories and algorithms. This method requires manual calibration of many feature points, and the input conditions are relatively complicated. Therefore, the traditional face model generation method has problems such as expensive equipment, complex implementation process, and insufficient real-time performance, and cannot fully meet the needs of real-time interaction in virtual scenes.

Contents of the invention

The purpose of the present invention is to overcome the defects of the above-mentioned technologies, and provide a method for generating a realistic human face based on a single photo. The method used in this technology is user-friendly and requires fewer feature points. With few feature points, the realistic face creation method can be completed independently by the user. This method introduces the user into the face creation process, selects a small number of feature points interactively, matches the appropriate face model, realizes the mapping from covering texture to neutral texture, and the skin color fusion of the face boundary, and finally through the whole to The detailed model is adjusted in layers to generate a personalized face model.

Its specific technical plan is:

A method for generating a realistic human face based on a single photo, comprising the following steps:

Step1. Feature point calibration: Obtain a frontal two-dimensional face photo, select 13 feature points for 7 parts such as eyes, nose, and mouth to mark, and determine the approximate position of the entire facial features;

Step2. Model matching: compare the feature point distance between the photo and the model, and match the model with the highest similarity in the model library;

Step3. Texture mapping: Simulate the real color of the face according to the texture map of the face, and realize the mapping from the covering texture to the neutral texture. The neutral texture is the default texture of the model, and the covering texture is a face photo. Through 13 features in the two-layer texture Mapping of points and their surrounding areas, adding texture features of face photos to the matching model;

Step4. Skin color fusion: use an Alpha map to form a smooth transition from covering texture to neutral texture at the boundary of face fusion, and at the same time, use the Alpha map to fine-tune the eyebrows, bridge of the nose and other details;

Step5. Model adjustment: Adjust the size and position of facial features such as the mouth, eyes, and face shape of the matching model through the method of vector difference to generate a personalized model that conforms to its facial features.

Preferably, in Step5, construct a model regulator that can be used to adjust the subtle changes of human facial features to generate a personalized realistic human face;

If the vertex V _i = (x, y, z) ^T in the face grid is regarded as a vector group H ₀ = (V ₁ , V ₂ , V ₃ ,..., V _n ), n represents the Quantity, after the eyes are transformed (such as changing the size), the positions of some points are changed to form a vector group H ₁ =(V ₁ ', V ₂ ', V ₃ ',...V _n '), for this The transformation adds a weight W, then all the transformation states H _x from H ₀ to H ₁ can be calculated by linear interpolation;

H _x =H ₀ +W*(H ₁ -H ₀ )(1)

It is difficult for one transformation to meet the needs of the face model. It is necessary to select multiple transformations according to the characteristics of the face, and assign a weight value W to each transformation. Then the mixing formula of m transformations is:

${\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; \&Sigma;</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; *</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

The number of m is subject to the number required in the actual application. In the experiment, 23 grid regulators with different transformations are defined, including: eye height, eye distance, eye size, eyebrow height, eyebrow depth, nose size, Nose Depth, Nose Height, Nose Bridge Depth, Nose Bridge Width, Nostril Width, Mouth Height, Mouth Thickness, Mouth Width, Lip Coverage, Cheekbone Depth, Cheekbone Width, Cheekbone Height, Cheek Width, Chin Height, Chin Depth, Chin Width , jaw width. Conditioners implement mesh deformation by controlling weights.

Preferably, the method of model matching in Step2 is as follows: first, carry out normalization processing to photograph, calculate the distance and the proportional relation between main characteristic points; Finally, match the Euclidean distance between the model and the corresponding feature points of the photo in the model library of the corresponding face shape, and use the formula (3) to find the closest three-dimensional model;

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}^{<span\; class="notranslate">\; ,</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; \&lambda;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Among them, D _i is the distance between certain feature points in the photo, D _i ' is the distance between corresponding feature points in the 3D model, and λ _i is the weight value of the distance of each feature point, which is determined according to its contribution degree and empirical formula. n is the number of distances being compared.

Preferably, in Step3, it is assumed that the positions of the feature points marked by the neutral texture and the covering texture respectively form two two-dimensional point sets V ₁ and V ₂ , and due to the difference in manual labeling, the corresponding points in V ₁ and V ₂ must not Very precisely at the same position, a mapping method is needed to make it merge naturally, take P ₁ ∈ V ₁ , P ₂ ∈ V ₂ . Extending P ₁ and P ₂ to three-dimensional vectors, P ₁ = (X ₁ , Y ₁ , 1), P ₂ = (X ₂ , Y ₂ , 1), and assuming a matrix M, we can get:

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

After obtaining the transformation matrix of each point, the transformation of the feature point is applied to the pixels around the feature point. This process uses the common triangle deformation method to apply the covering texture to the model texture, and connect the feature points to form a mesh covering the face. , the grid at this time contains both triangles and quadrilaterals. By splitting, the polygons are simplified into triangles, and the pixels in the middle of the triangle are affected by the three vertices of the triangle at the same time;

Assume that the three vertices of the triangle are P ₁ , P ₂ , P ₃ , and use them as three columns of the matrix to form a 3*3 matrix (P ₁ , P ₂ , P ₃ ), assuming that the transformed triangle is (P ₁ ', P ₂ ', P ₃ '), then:

M*(P ₁ , P ₂ , P ₃ )=(P ₁ ′, P ₂ ′, P ₃ ′)(5)

M=(P ₁ ', P ₂ ', P ₃ ')*(P ₁ , P ₂ , P ₃ ) ^-1 (6)

For a point i in the middle of the triangle, use M*P _i to get P _i , so as to determine the mapped coordinates of all points in the triangle. For unmapped points, the coordinate values are all integers, but the mapped coordinates may not be is an integer, resulting in a one-to-one correspondence with the pixels on the map.

Preferably, during the actual fusion process in Step4, the model neutral texture, Alpha map, and overlay texture will be processed into the same size, and the pixels in the Alpha map will be used as interpolation weights for blending to participate in the blending, and finally form a naturally blended texture. Three The blending formula for the graph is:

C(x, y)=C _base (x, y)*(255-C _Alpha (x, y))+C _overlay (x, y)*C _Alpha (x, y) where 0≤C _Alpha (x, y)≤255(7)

C _base (x, y), C _Alpha (x, y), and C _overlay (x, y) represent the color values at (x, y) coordinates of the neutral texture, Alpha map, and overlay texture, respectively.

Compared with prior art, the beneficial effect of the present invention is:

The method used in the present invention is user-friendly, requires fewer feature points, and only needs one frontal photo and fewer feature points, so that the user can participate in and complete the realistic face creation method independently. This method introduces the user into the face creation process, selects a small number of feature points interactively, matches the appropriate face model, realizes the mapping from covering texture to neutral texture, and the skin color fusion of the face boundary, and finally through the whole to The detailed model is adjusted in layers to generate a personalized face model.

Description of drawings

Figure 1 is a realistic face construction process;

Figure 2 is the location of feature points;

Fig. 3 is feature point connection;

Figure 4 is the Alpha graph.

detailed description

In order to make the technical means, creative features, goals and effects achieved by the present invention easy to understand, the present invention will be further described below in conjunction with the accompanying drawings and specific examples.

First, build a standardized face model library, interactively select the facial feature points of the input photo, and match the best model based on the facial features; second, realize the texture mapping from the photo to the 3D model by means of triangular deformation and bilinear interpolation , and introduce the Alpha map to realize the fusion transition from the face covering texture to the neutral texture of the model; finally, the grid adjustment method is used to adjust the model from the whole to the details in layers to achieve the purpose of generating a realistic 3D face.

1. Steps of interactive generation method

The realistic 3D face interactive generation method first needs to build a face model library according to the characteristics of Asian faces; then match the 2D frontal photos with the face model, use texture mapping and skin color fusion algorithms, and fine-tune the model adjuster , generating a realistic 3D model.

1.1 Model library construction

Now there are some standardized 3D face model libraries, such as UND, BU-3DFE, BJUT, etc. These model libraries have been widely used in fields such as face generation and face recognition. Different three-dimensional model databases have different standards for race, age, gender, illumination, etc., and the method of the present invention is mainly aimed at Asian races, and establishes a three-dimensional face database in a targeted manner.

There are many standards for the commonly used classification of Asian frontal faces: such as morphological method, font method, Asian method, etc. In order to better match the face shape of the model with the photo, the model library divides the face shape into 5 types: oval, inverted triangle, rectangle, square, and circle according to the obviousness and common use of features. The generation of the face neutral model refers to a neutral face wireframe model CANDIDE-3 ^[8] developed by the image research organization ICG of Linköping University in Sweden. Its model is simple and free for public use. However, this model is only suitable for the face. This method refers to the CANDIDE-3 neutral face model, and specially made a head neutral model suitable for Chinese facial features. After the model library is built, the model needs to be preprocessed, including patching bugs, binding neutral skin tone textures, normalizing size, and calibrating feature points.

1.2 Generation steps

The three-dimensional face mainly reflects its realism from the facial features and texture of the model, and the user participates in the designation of the texture and the adjustment of the model. The specific method flow is shown in Figure 1, which is mainly subdivided into the following five steps:

Step1. Feature point calibration: Obtain a frontal two-dimensional face photo, select 13 feature points for 7 parts such as eyes, nose, and mouth, and determine the approximate position of the entire facial features.

Step2. Model matching: compare the feature point distance between the photo and the model, and match the model with the highest similarity in the model library.

Step3. Texture mapping: Simulate the real color of the face according to the texture map of the face, and realize the mapping from the covering texture to the neutral texture. The neutral texture is the default texture of the model, and the overlay texture is a face photo. By mapping the 13 feature points and their surrounding areas in the two-layer texture, the texture features of the target face are added to the matching model.

Step4. Skin color fusion: Use an Alpha map to form a smooth transition from overlay texture to neutral texture at the boundary of face fusion. At the same time, use the Alpha map to make fine adjustments to the eyebrows, nose bridge and other details.

Step5. Model adjustment: Adjust the size and position of facial features such as the mouth, eyes, and face shape of the matching model through the method of vector difference to generate a personalized model that conforms to its facial features.

1.3 Model Regulator Design

The role of the model adjuster is to locally adjust the layout of the facial features to achieve the goal of being closer to the real face. Commonly used 3D modeling software, such as Maya, 3DSMax, etc., all provide the function of blendshapes (or morph), which is used to transform the model between two or more states to form different facial features or facial expressions. The present invention borrows this technical idea, and generates a personalized and realistic human face by constructing a model adjuster that can be used to adjust subtle changes in facial features.

If the vertex V _i = (x, y, z) ^T in the face grid is regarded as a vector group H ₀ = (V ₁ , V ₂ , V ₃ ,..., V _n ), n represents the Quantity, after the eyes are transformed (such as changing the size), the positions of some points are changed to form a vector group H ₁ =(V ₁ ', V ₂ ', V ₃ ',...V _n '), for this The transformation adds a weight W. Then, all transformation states _Hx from H ₀ to H ₁ can be calculated by linear interpolation.

H _x =H ₀ +W*(H ₁ -H ₀ )(1)

It is difficult for one transformation to meet the needs of the face model. It is necessary to select multiple transformations according to the characteristics of the face, and assign a weight value W to each transformation. Then the mixing formula of m transformations is:

${\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; \&Sigma;</span>}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; m</span>}}{\mathrm{<span\; class="notranslate">\; W</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; *</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; h</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 2</span>}<span\; class="notranslate">\; )</span>$

The number of m is subject to the number required in practical applications. In the experiment, 23 mesh modifiers with different transformations were defined, including: eye height, eye distance, eye size, eyebrow height, eyebrow depth, nose size, nose depth, nose height, nose bridge depth, nose bridge width, nostril width , mouth height, mouth thickness, mouth width, lip coverage, cheekbone depth, cheekbone width, cheekbone height, cheek width, chin height, chin depth, chin width, chin width. Conditioners implement mesh deformation by controlling weights.

2. Key technologies for realistic face generation

The key technologies of realistic face generation mainly include the matching technology of 2D face photos and 3D models, texture mapping technology and skin color fusion technology.

2.1 Feature point calibration and matching

After obtaining the 2D face photo, it is necessary to calibrate the key parts in the photo to match the best face model. The MPEG4 standard specifies 84 feature points on a neutral face. Referring to the MPEG4 standard, for the needs of face model matching and texture mapping, in order to reduce the complexity of manual intervention, it is proposed to select 13 key feature points on the input photo and face model as matching and mapping marks, as shown in Figure 2 Show. According to the research results of 3D face recognition, the relationship between these key feature points can roughly determine the position and shape of the facial features, so as to match the most suitable model. In the experimental part, the validity of 13 feature points is verified by comparing with matching methods with different numbers of feature points.

The method of model matching is as follows: First, normalize the photos to calculate the distance and proportional relationship between the main feature points; secondly, determine the approximate classification of the face shape according to the distance and proportional relationship of the feature points; finally, in the corresponding face shape Match the Euclidean distance between the model and the corresponding feature point of the photo in the model library, and use the formula (3) to find the closest 3D model.

$\mathrm{<span\; class="notranslate">\; E.</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; \&Sigma;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; 1</span>}}^{\mathrm{<span\; class="notranslate">\; no</span>}}{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span>{{\mathrm{<span\; class="notranslate">\; D.</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}}^{<span\; class="notranslate">\; ,</span>}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; \&lambda;</span>}}_{\mathrm{<span\; class="notranslate">\; i</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 3</span>}<span\; class="notranslate">\; )</span>$

Among them, D _i is the distance between certain feature points in the photo, D _i ' is the distance between corresponding feature points in the 3D model, and λ _i is the weight value of the distance of each feature point, which is determined according to its contribution degree and empirical formula. n is the number of distances being compared.

2.2 Texture Mapping

At present, researchers at home and abroad have proposed a variety of texture mapping methods. Kraevoy and others triangulated the parameterized model and the corresponding texture according to the corresponding relationship of feature points, and established the corresponding mapping relationship. This algorithm needs to be established on the basis of a large number of iterations, and the complexity is high. The present invention uses Kraevoy's method for reference, and introduces methods such as triangular deformation and bilinear difference to realize the feature point mapping between covering texture and neutral texture.

It is assumed that the positions of feature points marked by the neutral texture and the covering texture respectively form two two-dimensional point sets V ₁ and V ₂ . Due to the differences in manual marking, the corresponding points in V ₁ and V ₂ will not necessarily be at the same exact position, and a mapping method is needed to make them merge naturally. Take P ₁ ∈ V ₁ , P ₂ ∈ V ₂ . Extending P ₁ and P ₂ to three-dimensional vectors, P ₁ = (X ₁ , Y ₁ , 1), P ₂ = (X ₂ , Y ₂ , 1), and assuming a matrix M, we can get:

$\mathrm{<span\; class="notranslate">\; m</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; *</span>{\mathrm{<span\; class="notranslate">\; P</span>}}_{\mathrm{<span\; class="notranslate">\; 2</span>}}^{<span\; class="notranslate">\; -</span>\mathrm{<span\; class="notranslate">\; 1</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; 4</span>}<span\; class="notranslate">\; )</span>$

After obtaining the transformation matrix of each point, apply the transformation of the feature point to the pixels around the feature point. This process applies the overlay texture to the model texture using the common triangle deformation method. As shown in Figure 3, the feature points are connected to form a grid covering the face. At this time, the grid contains both triangles and quadrilaterals. By splitting, the polygons are simplified into triangles. Pixels in the middle of a triangle are simultaneously affected by all three vertices of the triangle.

Suppose the three vertices of the triangle are P ₁ , P ₂ , P ₃ . Take them as three columns of the matrix to form a 3*3 matrix (P ₁ , P ₂ , P ₃ ). Assuming the transformed triangle is (P ₁ ', P ₂ ', P ₃ '), then:

M*(P ₁ , P ₂ , P ₃ )=(P ₁ ′, P ₂ ′, P ₃ ′)(5)

M=(P ₁ ', P ₂ ', P ₃ ')*(P ₁ , P ₂ , P ₃ ) ^-1 (6)

For a point i in the middle of the triangle, P _{i can be obtained by using M*P i to} determine the mapped coordinates of all points in the triangle. For unmapped points, the coordinate values are all integers, but the coordinates after mapping may not be integers, resulting in a one-to-one correspondence with the pixels on the map. At this time, using the bilinear interpolation sampling method to sample the colors on the image can achieve better results.

2.3 Skin color fusion

After texture mapping, due to the color difference between the neutral texture and the overlay texture, a certain fusion algorithm is required to eliminate the visual difference, so that the effect of skin color fusion is more natural. In the existing image processing software and projection splicing fusion software, the AlphaTransition method is commonly used to realize the smooth transition of overlapping images, which not only has a fast fusion speed, but also has a better effect. Borrowing this idea, the present invention makes a standard Alpha map according to the face model features and texture map distribution, and eliminates the boundary mutation by setting the transition area of the edge. There are two functions of using the Alpha map: first, to produce a smooth transition of the texture boundary; second, to adjust the gradient of other parts of the face, such as eyebrows and the bridge of the nose, to reduce the influence of the overlay texture on the neutral texture. Figure 4 shows 10 hand-painted Alpha images. The color of the white part in the picture is taken from the color of the covering texture, and the color of the black part is taken from the color of the neutral texture. The part between white and black will be obtained by linear interpolation based on the pixels on the Alpha map. Through the coverage test, the Alpha map numbered 10 can achieve the best fusion effect.

In the actual fusion process, the model's neutral texture, Alpha map, and overlay texture will be processed into the same size. The pixels in the Alpha map are used as interpolation weights for mixing, and finally form a naturally blended texture. The blending formula for the three images is:

C(x, y)=C _base (x, y)*(255-C _Alpha (x, y))+C _overlay (x, y)*C _Alpha (x, y) where 0≤C _Alpha (x, y)≤255(7)

C _base (x, y), C _Alpha (x, y), and C _overlay (x, y) represent the color values at (x, y) coordinates of the neutral texture, Alpha map, and overlay texture, respectively.

The above is only the best implementation mode of the present invention, any simple changes or equivalent replacements of the technical solutions that can be clearly obtained by any person skilled in the art within the technical scope disclosed in the present invention all fall into the scope of the present invention within the scope of protection.

Claims (5)

1. the Realistic Human face generating method based on single photo, it is characterised in that comprise the following steps:

Step1. characteristic point is demarcated: obtains front two-dimension human face photo, 7 parts is chosen 13 characteristic points and carries out labelling, it is determined that the approximate location of whole face face；

Step2. Model Matching: compare the characteristic point distance between photo and model, the model that matching similarity is maximum in model library；

Step3. texture maps: the texture mapping simulation of facial true colors according to face, realize the mapping covering texture to neutral texture, neutral texture is the texture of model acquiescence, covering texture is human face photo, by the mapping of 13 characteristic points and peripheral region thereof in two layer texture, add the textural characteristics of target face to Matching Model；

Step4. the colour of skin merges: using an Alpha figure, the boudary portion in face fusion is formed from covering texture seamlessly transitting to neutral texture, meanwhile, utilizes Alpha figure that the details position of eyebrow, the bridge of the nose is carried out trickle adjustment；

Step5. model regulates: by the method for vector differentials, the characteristic of the face size of Matching Model, position is adjusted, and generates the personalized model meeting its facial characteristics.

2. the Realistic Human face generating method based on single photo according to claim 1, it is characterised in that construct a Model Regulator for adjusting human face five-sense-organ slight change in Step5, generate personalized face true to nature；

If the summit V in face grid_i=(x, y, z)^TRegard Vector Groups H as₀=(V₁, V₂, V₃..., V_n), n represents the quantity on summit, and after eyes are converted, the position of some points changes, and forms Vector Groups H₁=(V₁', V₂', V₃' ... V_n'), this conversion is increased a weights W, then, from H₀To H₁Middle all of transition state H_xCan be transferred through linear interpolation to calculate；

H_x=H₀+W*(H₁-H₀)(1)

A kind of conversion is difficult to meet the demand of faceform, it is necessary to the feature according to face, selects multiple conversion, gives a weighted value W for each conversion, then the mixed formulation of m conversion is:

$H_x=H_0+{\&Sigma;}_{i=1}^m{W}_i*(H_i-H_1)---\left(2\right)$

Wherein the quantity of m is as the criterion with the quantity needed in practical application, in an experiment, define the grid actuator of 23 kinds of different conversion, including: the covering of eye-level, eye distance, eyes size, eyebrow height, the eyebrow degree of depth, nose size, the nose degree of depth, nose height, the bridge of the nose degree of depth, nose bridge widths, nostril width, face height, face thickness, face width, lip, the cheekbone degree of depth, cheekbone width, cheekbone height, buccal width, chin height, the chin degree of depth, chin width, lower jaw width, actuator realizes distortion of the mesh by control weight.

3. the Realistic Human face generating method based on single photo according to claim 1, it is characterised in that in Step2, the method for Model Matching is as follows: first, and photo is normalized, and calculates the distance between principal character point and proportionate relationship；Secondly, according to characteristic point distance and proportionate relationship, it is determined that the rough classification of shape of face；Finally, Euclidean distance between Matching Model and photo individual features point in the model library of corresponding shape of face, adopt formula (3) to search out closest threedimensional model；

$E={\&Sigma;}_{i=1}^n{(D_i-{D_i}^{,})}^2*{\mathrm{\&lambda;}}_i---\left(3\right)$

Wherein, D_iFor distance between certain characteristic point in photo, D_i' for distance between individual features point in threedimensional model, λ_iFor the weighted value of each characteristic point distance, determining according to its percentage contribution size and empirical equation, n is the distance quantity compared.

4. the Realistic Human face generating method based on single photo according to claim 1, it is characterised in that assume in Step3 that the characteristic point position of neutral texture and covering texture labelling respectively forms V respectively₁, V₂Two two-dimentional point sets, due to the difference of manual mark, V₁, V₂In corresponding point necessarily will not be exactly accurate at same position, it is necessary to a kind of mapping method makes its natural fusion, takes P₁∈V₁, P₂∈V₂, by P₁, P₂Expand to three-dimensional vector, P₁=(X₁, Y₁, 1), P₂=(X₂, Y₂, 1), and assume there is matrix M, then:

M=P₁*P₂ ^-1(4)

After obtaining the transformation matrix of each point, the conversion of characteristic point is applied in the pixel around characteristic point, this flow process uses common triangle mode of texturing that covering texture is applied to model texture, characteristic point is connected, form the grid covering face, grid now has concurrently triangle and tetragon, by splitting, polygon is all reduced to triangle, is in the pixel in the middle of triangle and is subject to the impact on Atria summit simultaneously；

Three summits assuming triangle are P₁, P₂, P₃, using they three row as matrix, form 3*3 matrix (P₁, P₂, P₃), it is assumed that the triangle after conversion is (P₁', P₂', P₃'), then:

M*(P₁, P₂, P₃)=(P₁', P₂', P₃’)(5)

M=(P₁', P₂', P₃’)*(P₁, P₂, P₃)^-1(6)

Certain i point in the middle of diabolo, uses M*P_iJust obtain P_i, determine the coordinate after mapping a little in triangle with this, for unmapped point, coordinate figure is all integer, but the coordinate after mapping is likely to not be integer, and causing cannot with the pixel one_to_one corresponding on figure.

5. the Realistic Human face generating method based on single photo according to claim 1, it is characterized in that, in actual fused process in Step4, model neutrality texture, Alpha figure, covering texture will be processed into the same size, pixel in Alpha figure participates in mixing as the interpolation weights of mixing, ultimately forming the texture of natural fusion, the mixed formulation of three figure is:

C (x, y)=C_base(x, y) * (255-C_Alpha(x, y))+C_overlay(x, y) * C_Alpha(x, y) wherein 0≤C_Alpha(x, y)≤255 (7)

C_base(x, y), C_Alpha(x, y), C_overlay(x y) represents neutral texture, Alpha figure respectively, covers texture in (x, y) color value at coordinate place.