CN114820894A - A method and system for generating a virtual character - Google Patents

Abstract

The present invention proposes a method and system for generating a virtual character. By acquiring a face image of a target user, a stereoscopic head model of a virtual character is generated according to a stereoscopic head model template, and initialization rendering parameters are obtained to generate a stereoscopic head model for the stereoscopic head model. Perform rendering, project the rendered three-dimensional head model to a specified plane to obtain a plane projection image, match the plane projection image with the face image of the target user, and perform a mapping on the stereo head according to the matching result. Adjust the model and rendering parameters, re-render the adjusted three-dimensional head model using the adjusted rendering parameters, and cyclically execute the above-mentioned steps of projection, matching, model and rendering parameter adjustment and re-rendering until the plane projection image The matching map with the face image of the target user complies with the preset conditions, so that the virtual character of the real image of the user can be quickly generated.

Description

A method and system for generating a virtual character

technical field

The invention relates to the technical field of three-dimensional modeling, in particular to a method and system for generating a virtual character.

Background technique

In recent years, the concept of metaverse has been increasingly sought after by Internet companies and online players. More and more online social activities have begun to use three-dimensional avatars, that is, virtual characters to simulate real interaction scenarios. Go to the office or role-playing games, etc. The avatars in the metaverse are generally constructed using the system preset three-dimensional models. Users can adjust the avatars through the personalized tools provided by the system to customize the avatars with personalized styles, such as personalized hairstyles, hair colors, face shapes, skin tone and texture. In order to make virtual characters closer to reality, some technologies have recently emerged to collect real image data of users to create virtual characters. However, due to the complex construction methods, cumbersome construction procedures, time-consuming and large gap between the model effect and the real ones, these technologies It is difficult to be promoted and applied.

SUMMARY OF THE INVENTION

Based on the above problem, the present invention proposes a method and system for generating a virtual character, which realizes the rapid generation of a virtual character of a user's realistic image.

In view of this, a first aspect of the present invention provides a method for generating a virtual character, including:

Get the face image of the target user;

According to the stereoscopic head model template to generate the stereoscopic head model of the virtual character;

obtaining initialization rendering parameters to render the stereoscopic head model;

Projecting the rendered three-dimensional head model to a specified plane to obtain a plane projection image;

matching the planar projection image with the face image of the target user;

Adjust the three-dimensional head model and rendering parameters according to the matching result;

re-rendering the adjusted three-dimensional head model using the adjusted rendering parameters;

The above steps of projection, matching, model and rendering parameter adjustment, and re-rendering are performed cyclically until the matching map between the planar projection image and the face image of the target user meets preset conditions.

Further, in the above-mentioned virtual character generation method, the step of matching the plane projection image with the face image of the target user specifically includes:

extracting at least one first image feature of the planar projection image and the face image;

Converting the first image feature into two sets of coordinate vectors and two sets of color vectors corresponding to the plane projection image and the face image respectively;

Calculate the difference between the two sets of coordinate vectors and the difference between the two sets of color vectors.

Further, in the above-mentioned virtual character generation method, the first image features include facial features images and facial contour images, and the steps of adjusting the three-dimensional head model and rendering parameters according to the matching results specifically include:

Predicting a plurality of vertex coordinates corresponding to the first image feature in the rendering parameters of the three-dimensional head model according to the difference between the two sets of coordinate vectors;

Modifying a plurality of vertex coordinates corresponding to the first image feature to predicted values.

Further, in the above-mentioned virtual character generation method, before the step of acquiring initialization rendering parameters to render the three-dimensional head model, the method further includes:

Get the initialization texture parameters to generate the initialization plane texture space;

After cyclically performing the above steps of projection, matching, model and rendering parameter adjustment and re-rendering until the matching map between the planar projection image and the face image of the target user meets preset conditions, the method further includes:

Establishing the coordinate correspondence between the face image and the initialization plane texture space according to the three-dimensional head model;

Divide the face image into partial images of multiple regions according to a preset color difference threshold;

The local images of the multiple regions are mapped to the plane texture space according to the coordinate mapping relationship.

Further, in the above-mentioned virtual character generation method, the above-mentioned steps of projection, matching, model and rendering parameter adjustment and re-rendering are performed cyclically until the matching map between the planar projection image and the face image of the target user is in accordance with the pre-set image. After the step of setting the conditions, it also includes:

extracting at least one second image feature from the face image;

calculating an ambient light parameter according to the second image feature;

The rendering parameters of the stereo head model and the texture parameters of the plane texture space are adjusted according to the ambient light parameters.

A second aspect of the present invention provides a virtual character generation system, comprising:

The image acquisition module is used to acquire the face image of the target user;

The model generation module is used to generate the three-dimensional head model of the virtual character according to the three-dimensional head model template;

A model rendering module, configured to obtain initialization rendering parameters to render the stereo head model and use the adjusted rendering parameters to render the adjusted stereo head model after the rendering parameters and the stereo head model are adjusted. to re-render;

a plane projection module for projecting the rendered three-dimensional head model to a specified plane to obtain a plane projection image;

an image matching module for matching the planar projection image with the face image of the target user;

a parameter adjustment module for adjusting the stereo head model and rendering parameters according to the matching result;

The loop execution module is used for loop execution of the steps of projection, matching and rendering parameter adjustment until the matching map between the plane projection image and the face image of the target user meets a preset condition.

Further, in the above-mentioned virtual character generation system, the image matching module includes:

A feature extraction sub-module for extracting at least one first image feature of the planar projection image and the face image. The first image features include, but are not limited to, one or more of facial features and facial contours.

A vector transformation submodule, configured to transform the first image feature into two sets of coordinate vectors and two sets of color vectors corresponding to the plane projection image and the face image, respectively. Exemplarily, taking the eye feature as one of the first image features, in the plane projection image and the face image, the eye features extracted from the two images are converted into corresponding coordinate vectors. and color vector. Assuming that the eye feature consists of n pixels, the coordinate values on the plane projection image are (x1, y1), (x2, y2)...(xn, yn), and the color values are (r1, g1) ,b1), (r2,g2,b2)...(rn,gn,bn), the coordinate values on the face image are (x`1,y`1), (x`2,y` 2)......(x`n,y`n), the color values are (r`1,g`1,b`1), (r`2,g`2,b`2)......(r` n, g`n, b`n), the two sets of coordinate vectors and the two sets of color vectors are constructed as:

Eye feature coordinate vector of plane projection image: [x1,y1,x2,y2,...,xn,yn];

Face image eye feature coordinate vector: [x`1, y`1, x`2, y`2, ..., x`n, y`n];

Eye feature color vector of plane projection image: [r1,g1,b1,r2,g2,b2,...,rn,gn,bn];

Face image eye feature color vector: [r`1,g`1,b`1,r`2,g`2,b`2,...,r`n,g`n,b`n].

A difference calculation submodule, configured to calculate the difference between the two sets of coordinate vectors and the difference between the two sets of color vectors. In some embodiments of the present invention, the eye feature coordinate difference vector [dx1, dy1, dx2 is obtained by subtracting the eye feature coordinate vector and the eye feature color vector of the plane projection image and the face image respectively. ,dy2,...,dxn,dyn] and the eye feature color difference vector [dr1,dg1,db1,dr2,dg2,db2,...,drn,dgn,dbn]. Predict at least one set of adjustment values for the stereoscopic head model and the rendering parameters according to the coordinate difference vector and the color difference vector, and adjust the stereoscopic head model and the rendering parameters accordingly .

Further, in the above virtual character generation system, the first image features include facial features images and facial contour images, and the parameter adjustment module includes:

A coordinate prediction submodule, configured to predict a plurality of vertex coordinates corresponding to the first image feature in the rendering parameters of the three-dimensional head model according to the difference between the two sets of coordinate vectors. Since the change from the three-dimensional head model to the plane projection image is a change from three-dimensional to two-dimensional, corresponding to the coordinate vector difference in the two-dimensional image, after mapping to the three-dimensional space, the three-dimensional head model is open-circuited. For the corresponding region, such as the eye region, there are multiple sets of different prediction values corresponding to the corresponding vertex coordinates. However, using any of the possible vertex coordinate prediction values as the adjusted coordinate value will also affect the color vector of the associated area in the projected image due to the change of the spatial coordinate. The shaded area of the area will also vary, etc.

The coordinate modification submodule is used for modifying the coordinates of a plurality of vertices corresponding to the first image feature into predicted values. In the steps of cyclically executing the projection, matching, model and rendering parameter adjustment and re-rendering, by testing the above-mentioned multiple sets of different predicted values of the vertex coordinates one by one, in the testing process, according to the difference of the color vector coordinates to determine whether the vertex coordinate value meets the preset condition.

Further, in the above virtual character generation system, the parameter acquisition module is further configured to acquire initialization texture parameters to generate an initialization plane texture space, and the virtual character generation system further includes:

A coordinate association module, configured to establish a coordinate correspondence between the face image and the initialized plane texture space according to the three-dimensional head model. In the aforementioned steps of matching the planar projection image with the face image of the target user and the steps of acquiring initialized texture parameters to generate an initialized planar texture space, the face image and the stereo head have been established respectively. The mapping relationship between the head models and the mapping relationship between the plane texture space and the three-dimensional head model, according to the relationship between the three, the coordinate value of each pixel on the face image and the plane can be established. The mapping relationship between the coordinate values of each pixel in texture space.

An image segmentation module, configured to segment the face image into partial images of multiple regions according to a preset color difference threshold. Different areas of the human face, including hair, forehead, eyebrows, eyes, cheeks, bridge of the nose, lips, and chin, have great differences in color. The cheeks may also have dark and high-light parts under the action of different light. , with a certain color tolerance range as the threshold, the regions of different colors are divided into different partial images.

A pixel mapping module, configured to map the partial images of the multiple regions to the plane texture space according to the coordinate mapping relationship. Each of the above-mentioned partial images is mapped into the plane texture space according to the aforementioned coordinate mapping relationship, and then a smoothing function is used to perform smoothing processing on the connecting part to prevent the transition at the connecting position from being displayed very bluntly.

Further, in the above-mentioned virtual character generation system, it also includes:

A feature extraction module for extracting at least one second image feature from the face image. The second image features include dark parts and highlight parts on the face image.

A parameter calculation module, configured to calculate ambient light parameters according to the second image feature. The ambient light parameters include, but are not limited to, the direction of illumination, the color and intensity of ambient light, the intensity and color of diffuse reflection and specular reflection, and the like.

The parameter adjustment module is further configured to adjust the rendering parameters of the three-dimensional head model and the texture parameters of the plane texture space according to the ambient light parameters.

The present invention provides a method and system for generating a virtual character. By acquiring a face image of a target user, a three-dimensional head model of a virtual character is generated according to a three-dimensional head model template, and initialization rendering parameters are obtained to generate the three-dimensional head model for the three-dimensional head model. Rendering, projecting the rendered three-dimensional head model to a designated plane to obtain a plane projection image, matching the plane projection image with the face image of the target user, and applying the stereo head according to the matching result. Adjust the model and rendering parameters, re-render the adjusted three-dimensional head model using the adjusted rendering parameters, and perform the above-mentioned projection, matching, model and rendering parameter adjustment and re-rendering steps in a loop until the plane projection image The matching map with the face image of the target user complies with the preset condition, so that the virtual character of the real image of the user can be quickly generated.

Description of drawings

1 is a schematic flowchart of a method for generating a virtual character provided by an embodiment of the present invention;

2 is a schematic flowchart of a method for matching a projected image and a face image provided by an embodiment of the present invention;

3 is a schematic flowchart of a method for modifying vertex coordinates of a three-dimensional model provided by an embodiment of the present invention;

4 is a schematic flowchart of a texture space generation method provided by an embodiment of the present invention;

5 is a schematic flowchart of a method for adjusting ambient light parameters provided by an embodiment of the present invention;

FIG. 6 is a schematic block diagram of a virtual character generation system provided by an embodiment of the present invention.

Detailed ways

In order to understand the above objects, features and advantages of the present invention more clearly, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments. It should be noted that the embodiments of the present application and the features in the embodiments may be combined with each other in the case of no conflict.

Many specific details are set forth in the following description to facilitate a full understanding of the present invention. However, the present invention can also be implemented in other ways different from those described herein. Therefore, the protection scope of the present invention is not limited by the specific implementation disclosed below. example limitations.

In the description of the present invention, the term "plurality" refers to two or more than two, unless otherwise expressly defined, the orientation or positional relationship indicated by the terms "upper", "lower" etc. is based on the drawings. The orientation or positional relationship is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the indicated device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation of the present invention. The terms "connected", "installed", "fixed", etc. should be understood in a broad sense. For example, "connected" can be a fixed connection, a detachable connection, or an integral connection; it can be directly connected, or through the middle media are indirectly connected. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific situations. In addition, the terms "first", "second", etc. are used for descriptive purposes only, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature defined as "first", "second", etc., may expressly or implicitly include one or more of that feature. In the description of the present invention, unless otherwise specified, "plurality" means two or more.

In the description of this specification, the description of the terms "one embodiment", "some embodiments", "specific embodiment", etc. means that a particular feature, structure, material or characteristic described in connection with the embodiment or example is included in the present invention at least one embodiment or example of . In this specification, schematic representations of the above terms do not necessarily refer to the same embodiment or instance. Furthermore, the particular features, structures, materials or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

The following describes a method and system for generating a virtual character according to some embodiments of the present invention with reference to FIG. 1 to FIG. 6 .

As shown in FIG. 1 , a first aspect of the present invention provides a method for generating a virtual character, including:

S110: Acquire a face image of the target user. In some embodiments of the present invention, the face image of the target user may be a photo taken in advance, and is read from a local or remote storage space by the virtual character generation system. In some other embodiments of the present invention, the face image of the target user may also be captured in real time by a camera device when the generation program of the virtual character generation system is started. After acquiring the face image of the target user in the above manner, the virtual character generation system can generate a virtual character similar to the real image for the user according to the face image.

S120: Generate a three-dimensional head model of the virtual character according to the three-dimensional head model template. Before generating the virtual character according to the face image of the target user, a general three-dimensional head model is first constructed using a general three-dimensional head model template. Further, in some embodiments of the present invention, after obtaining the face image of the target user, image feature extraction and analysis are performed on the face image to obtain the face feature of the target user, so as to Obtain the corresponding stereo head model template according to the facial features. For example, the characteristics of the target user include gender characteristics of the target user, and after acquiring the face image of the target user, perform face recognition on the face image to confirm whether the target user is a male user or a female The user obtains the corresponding stereoscopic head model template of the male avatar or the stereoscopic head model template of the female avatar in the current step. The characteristics of the target user also include ethnic characteristics, age characteristics and the like of the target user. In some embodiments of the present invention, the virtual character generation system presets stereoscopic head model templates corresponding to different races, genders and ages, according to the extraction and analysis of the facial features of the target user in the face image , so as to identify the race, gender and age of the target user, so as to obtain the corresponding stereo head model template according to the above information to generate a preliminary stereo head model of the virtual character, so as to reduce subsequent changes to the stereo head model and rendering parameters. Time spent making adjustments.

S130: Acquire initialization rendering parameters to render the three-dimensional head model. The initialized rendering parameters include, but are not limited to, the number of light sources, the type of light sources, the position of the light sources, the color of the light sources, the light intensity of the light sources, the position of the camera, the angle of the camera, and the reflection/diffuse reflection properties of environmental objects, etc. Specifically, before the step of acquiring initialization rendering parameters to render the three-dimensional head model, the virtual character generation system further acquires initialization texture parameters to generate an initialization plane texture space, and when using the initialization plane texture space Mapping onto the stereo head model so that the stereo head model has preliminary material texture properties.

S140: Project the rendered three-dimensional head model to a specified plane to obtain a plane projection image. Specifically, confirm the relative angle between the camera device and the target user's head when the camera device acquires the face image according to the relative positions of the facial features and the facial contour in the face image, and use the relative angle as the The relative angle of the three-dimensional head model and the specified plane is obtained to obtain a plane projection image of the three-dimensional head model.

S150: Match the planar projection image with the face image of the target user. Determine the difference between the plane projection image and the face image by matching the plane projection image with the face image of the target user, such as differences in hair, face shape and facial features, and position, and each image Differences in color, brightness, contrast, etc. of the area.

S160: Adjust the three-dimensional head model and rendering parameters according to the matching result. Specifically, according to the difference between the plane projection image and the face image, calculate the adjustment method of the three-dimensional head model and rendering parameters so that the plane projection image tends to be the same as the face image, and press This adjustment method adjusts the three-dimensional head model and rendering parameters.

S170: Re-render the adjusted three-dimensional head model using the adjusted rendering parameters. The changes produced by the adjusted three-dimensional head model include, for example, changes in hairstyles, changes in face shapes, and changes in facial features. The changes brought about by the adjusted rendering parameters include the orientation of the head model, the global brightness, the color/size of the shadow of the face, the brightness/color/size of the reflective area of the face, and the like.

S180: Execute the above steps of projection, matching, model and rendering parameter adjustment, and re-rendering in a loop until the matching map between the planar projection image and the face image of the target user meets a preset condition. The coordinates and color of the same image feature on the planar projection image are affected by a plurality of different parameters in the stereo head model and the rendering parameters, and different image features also affect each other, so it cannot be determined according to the The difference between the plane projection image and the face image directly restores the unique three-dimensional head model and the rendering parameters, and the three-dimensional head model and the rendering parameters need to be adjusted many times so that the The projected image tends to be the same as the face image. Due to the complex factors of the shooting environment, the computational resources and time resources consumed by completely restoring the stereoscopic head model and rendering parameters so that the planar projection image is exactly the same as the face image are very huge and unnecessary. When the similarity between the plane projection image and the face image of the target user reaches the expectation, the loop execution steps can be stopped. The preset condition specifically means that the difference between the coordinate vector and the color vector of one or more image features on the planar projection image and the face image of the target user is smaller than a preset threshold.

As shown in Figure 2, in the above-mentioned virtual character generation method, the step of matching the plane projection image with the face image of the target user specifically includes:

S151: Extract at least one first image feature of the planar projection image and the face image. The first image features include, but are not limited to, one or more of facial features and facial contours.

S152: Transform the first image feature into two sets of coordinate vectors and two sets of color vectors corresponding to the planar projection image and the face image, respectively. Exemplarily, taking the eye feature as one of the first image features, in the plane projection image and the face image, the eye features extracted from the two images are converted into corresponding coordinate vectors. and color vector. Assuming that the eye feature consists of n pixels, the coordinate values on the plane projection image are (x1, y1), (x2, y2)...(xn, yn), and the color values are (r1, g1) ,b1), (r2,g2,b2)...(rn,gn,bn), the coordinate values on the face image are (x`1,y`1), (x`2,y` 2)......(x`n,y`n), the color values are (r`1,g`1,b`1), (r`2,g`2,b`2)......(r` n, g`n, b`n), the two sets of coordinate vectors and the two sets of color vectors are constructed as:

Eye feature coordinate vector of plane projection image: [x1,y1,x2,y2,...,xn,yn];

Face image eye feature coordinate vector: [x`1, y`1, x`2, y`2, ..., x`n, y`n];

Eye feature color vector of plane projection image: [r1,g1,b1,r2,g2,b2,...,rn,gn,bn];

Face image eye feature color vector: [r`1,g`1,b`1,r`2,g`2,b`2,...,r`n,g`n,b`n].

S153: Calculate the difference between the two sets of coordinate vectors and the difference between the two sets of color vectors. In some embodiments of the present invention, the eye feature coordinate difference vector [dx1, dy1, dx2 is obtained by subtracting the eye feature coordinate vector and the eye feature color vector of the plane projection image and the face image respectively. ,dy2,...,dxn,dyn] and the eye feature color difference vector [dr1,dg1,db1,dr2,dg2,db2,...,drn,dgn,dbn]. Predict at least one set of adjustment values for the stereoscopic head model and the rendering parameters according to the coordinate difference vector and the color difference vector, and adjust the stereoscopic head model and the rendering parameters accordingly .

As shown in FIG. 3 , in the above-mentioned virtual character generation method, the first image features include facial features images and facial contour images, and the steps of adjusting the three-dimensional head model and rendering parameters according to the matching results specifically include:

S161: Predict a plurality of vertex coordinates corresponding to the first image feature in the rendering parameters of the three-dimensional head model according to the difference between the two sets of coordinate vectors. Since the change from the three-dimensional head model to the plane projection image is a change from three-dimensional to two-dimensional, corresponding to the coordinate vector difference in the two-dimensional image, after mapping to the three-dimensional space, the three-dimensional head model is open-circuited. For the corresponding region, such as the eye region, there are multiple sets of different prediction values corresponding to the corresponding vertex coordinates. However, using any of the possible vertex coordinate prediction values as the adjusted coordinate value will also affect the color vector of the associated area in the projected image due to the change of the spatial coordinate. The shaded area of the area will also vary, etc.

S162: Modify multiple vertex coordinates corresponding to the first image feature to predicted values. In the steps of cyclically executing the projection, matching, model and rendering parameter adjustment and re-rendering, by testing the above-mentioned multiple sets of different predicted values of the vertex coordinates one by one, in the testing process, according to the difference of the color vector coordinates to determine whether the vertex coordinate value meets the preset condition.

As shown in FIG. 4 , in the above-mentioned virtual character generation method, before the step of acquiring initialization rendering parameters to render the three-dimensional head model, the method further includes:

S210: Acquire initialized texture parameters to generate an initialized plane texture space. As mentioned above, the race, gender, age, etc. of the target user are determined according to the analysis of the face image, so that the initialized texture parameters are read from the preset template information to generate the initialized plane texture space. The plane texture space is an unfolded figure in which the surface of the three-dimensional head model is cut along a preset dividing line and then tiled onto a plane, and the coordinates of the plane texture space have a mapping with the coordinates of the surface of the three-dimensional head model. relationship, before the step of rendering the three-dimensional head model is performed each time, the material texture of the plane texture space is mapped to the surface of the three-dimensional head model, so that the surface of the three-dimensional head model has color and gloss. and other properties.

After cyclically performing the above steps of projection, matching, model and rendering parameter adjustment and re-rendering until the matching map between the planar projection image and the face image of the target user meets preset conditions, the method further includes:

S220: Establish a coordinate correspondence between the face image and the initialized plane texture space according to the three-dimensional head model. In the aforementioned steps of matching the planar projection image with the face image of the target user and the steps of acquiring initialized texture parameters to generate an initialized planar texture space, the face image and the stereo head have been established respectively. The mapping relationship between the head models and the mapping relationship between the plane texture space and the three-dimensional head model, according to the relationship between the three, the coordinate value of each pixel on the face image and the plane can be established. The mapping relationship between the coordinate values of each pixel in texture space.

S230: Divide the face image into partial images of multiple regions according to a preset color difference threshold. Different areas of the human face, including hair, forehead, eyebrows, eyes, cheeks, bridge of the nose, lips, and chin, have great differences in color. The cheeks may also have dark and high-light parts under the action of different light. , with a certain color tolerance range as the threshold, the regions of different colors are divided into different partial images.

S240: Map the partial images of the multiple regions to the plane texture space according to the coordinate mapping relationship. Each of the above-mentioned partial images is mapped into the plane texture space according to the aforementioned coordinate mapping relationship, and then a smoothing function is used to perform smoothing processing on the connecting part to prevent the transition at the connecting position from being displayed very bluntly.

As shown in FIG. 5 , in the above-mentioned virtual character generation method, the above-mentioned steps of projection, matching, model and rendering parameter adjustment and re-rendering are performed in a loop until the plane projection image matches the face image of the target user After the steps that the graph meets the preset conditions, it also includes:

S310: Extract at least one second image feature from the face image. The second image features include dark parts and highlight parts on the face image.

S320: Calculate ambient light parameters according to the second image feature. The ambient light parameters include, but are not limited to, the direction of illumination, the color and intensity of ambient light, the intensity and color of diffuse reflection and specular reflection, and the like.

S330: Adjust the rendering parameters of the stereo head model and the texture parameters of the plane texture space according to the ambient light parameters.

As shown in FIG. 6 , a second aspect of the present invention provides a virtual character generation system, including:

The image acquisition module is used to acquire the face image of the target user. In some embodiments of the present invention, the face image of the target user may be a photo taken in advance, and is read from a local or remote storage space by the virtual character generation system. In some other embodiments of the present invention, the face image of the target user may also be captured in real time by a camera device when the generation program of the virtual character generation system is started. After acquiring the face image of the target user in the above manner, the virtual character generation system can generate a virtual character similar to the real image for the user according to the face image.

The model generation module is used for generating the stereo head model of the virtual character according to the stereo head model template. Before generating the virtual character according to the face image of the target user, a general three-dimensional head model is first constructed using a general three-dimensional head model template. Further, in some embodiments of the present invention, after obtaining the face image of the target user, image feature extraction and analysis are performed on the face image to obtain the face feature of the target user, so as to Obtain the corresponding stereo head model template according to the facial features. For example, the characteristics of the target user include gender characteristics of the target user, and after acquiring the face image of the target user, perform face recognition on the face image to confirm whether the target user is a male user or a female The user obtains the corresponding stereoscopic head model template of the male avatar or the stereoscopic head model template of the female avatar in the current step. The characteristics of the target user also include ethnic characteristics, age characteristics and the like of the target user. In some embodiments of the present invention, the virtual character generation system presets stereoscopic head model templates corresponding to different races, genders and ages, according to the extraction and analysis of the facial features of the target user in the face image , so as to identify the race, gender and age of the target user, so as to obtain the corresponding stereo head model template according to the above information to generate a preliminary stereo head model of the virtual character, so as to reduce subsequent changes to the stereo head model and rendering parameters. Time spent making adjustments.

A model rendering module, configured to obtain initialization rendering parameters to render the stereo head model and use the adjusted rendering parameters to render the adjusted stereo head model after the rendering parameters and the stereo head model are adjusted. to re-render. The initialized rendering parameters include, but are not limited to, the number of light sources, the type of light sources, the position of the light sources, the color of the light sources, the light intensity of the light sources, the position of the camera, the angle of the camera, and the reflection/diffuse reflection properties of environmental objects, etc. Specifically, before the step of acquiring initialization rendering parameters to render the three-dimensional head model, the virtual character generation system further acquires initialization texture parameters to generate an initialization plane texture space, and when using the initialization plane texture space Mapping onto the stereo head model so that the stereo head model has preliminary material texture properties. The changes produced by the adjusted three-dimensional head model include, for example, changes in hairstyles, changes in face shapes, and changes in facial features. The changes brought about by the adjusted rendering parameters include the orientation of the head model, the global brightness, the color/size of the shadow of the face, the brightness/color/size of the reflective area of the face, and the like.

The plane projection module is used for projecting the rendered three-dimensional head model to a specified plane to obtain a plane projection image. Specifically, confirm the relative angle between the camera device and the target user's head when the camera device acquires the face image according to the relative positions of the facial features and the facial contour in the face image, and use the relative angle as the The relative angle of the three-dimensional head model and the specified plane is obtained to obtain a plane projection image of the three-dimensional head model.

An image matching module, configured to match the planar projection image with the face image of the target user. Determine the difference between the plane projection image and the face image by matching the plane projection image with the face image of the target user, such as differences in hair, face shape and facial features, and position, and each image Differences in color, brightness, contrast, etc. of the area.

A parameter adjustment module, configured to adjust the stereoscopic head model and rendering parameters according to the matching result. Specifically, according to the difference between the plane projection image and the face image, calculate the adjustment method of the three-dimensional head model and rendering parameters so that the plane projection image tends to be the same as the face image, and press This adjustment method adjusts the three-dimensional head model and rendering parameters.

The loop execution module is used for loop execution of the steps of projection, matching and rendering parameter adjustment until the matching map between the plane projection image and the face image of the target user meets a preset condition. The coordinates and color of the same image feature on the planar projection image are affected by a plurality of different parameters in the stereo head model and the rendering parameters, and different image features also affect each other, so it cannot be determined according to the The difference between the plane projection image and the face image directly restores the unique three-dimensional head model and the rendering parameters, and the three-dimensional head model and the rendering parameters need to be adjusted many times so that the The projected image tends to be the same as the face image. Due to the complex factors of the shooting environment, the computational resources and time resources consumed by completely restoring the stereoscopic head model and rendering parameters so that the planar projection image is exactly the same as the face image are very huge and unnecessary. When the similarity between the plane projection image and the face image of the target user reaches the expectation, the loop execution steps can be stopped. The preset condition specifically means that the difference between the coordinate vector and the color vector of one or more image features on the planar projection image and the face image of the target user is smaller than a preset threshold.

Further, in the above-mentioned virtual character generation system, the image matching module includes:

a feature extraction sub-module for extracting at least one first image feature of the planar projection image and the face image;

A vector conversion submodule, for converting the first image feature into two sets of coordinate vectors and two sets of color vectors corresponding to the plane projection image and the face image respectively;

A difference calculation submodule, configured to calculate the difference between the two sets of coordinate vectors and the difference between the two sets of color vectors.

Further, in the above virtual character generation system, the first image features include facial features images and facial contour images, and the parameter adjustment module includes:

A coordinate prediction submodule, configured to predict a plurality of vertex coordinates corresponding to the first image feature in the rendering parameters of the three-dimensional head model according to the difference between the two sets of coordinate vectors;

The coordinate modification submodule is used for modifying the coordinates of a plurality of vertices corresponding to the first image feature into predicted values.

Further, in the above virtual character generation system, the parameter acquisition module is further configured to acquire initialization texture parameters to generate an initialization plane texture space, and the virtual character generation system further includes:

a coordinate association module, configured to establish a coordinate correspondence between the face image and the initialization plane texture space according to the three-dimensional head model;

an image segmentation module, configured to segment the face image into partial images of multiple regions according to a preset color difference threshold;

A pixel mapping module, configured to map the partial images of the multiple regions to the plane texture space according to the coordinate mapping relationship.

Further, in the above-mentioned virtual character generation system, it also includes:

a feature extraction module for extracting at least one second image feature from the face image;

a parameter calculation module, configured to calculate ambient light parameters according to the second image feature;

The parameter adjustment module is further configured to adjust the rendering parameters of the three-dimensional head model and the texture parameters of the plane texture space according to the ambient light parameters.

The present invention provides a method and system for generating a virtual character. By acquiring a face image of a target user, a three-dimensional head model of a virtual character is generated according to a three-dimensional head model template, and initialization rendering parameters are obtained to generate the three-dimensional head model for the three-dimensional head model. Rendering, projecting the rendered three-dimensional head model to a designated plane to obtain a plane projection image, matching the plane projection image with the face image of the target user, and applying the stereo head according to the matching result. Adjust the model and rendering parameters, re-render the adjusted three-dimensional head model using the adjusted rendering parameters, and perform the above-mentioned projection, matching, model and rendering parameter adjustment and re-rendering steps in a loop until the plane projection image The matching map with the face image of the target user complies with the preset condition, so that the virtual character of the real image of the user can be quickly generated.

It should be noted that, in this document, relational terms such as first and second are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any relationship between these entities or operations. any such actual relationship or sequence exists. Moreover, the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device that includes a list of elements includes not only those elements, but also includes not explicitly listed or other elements inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

Embodiments in accordance with the present invention are described above, and these embodiments do not exhaust all the details, nor do they limit the invention to only the specific embodiments described. Obviously, many modifications and variations are possible in light of the above description. This specification selects and specifically describes these embodiments in order to better explain the principle and practical application of the present invention, so that those skilled in the art can make good use of the present invention and modifications based on the present invention. The present invention is to be limited only by the claims and their full scope and equivalents.

Claims (10)

1. A virtual character generation method is characterized by comprising the following steps:

acquiring a face image of a target user;

generating a three-dimensional head model of the virtual character according to the three-dimensional head model template;

acquiring an initialized rendering parameter to render the three-dimensional head model;

projecting the rendered stereoscopic head model to a designated plane to obtain a plane projection image;

matching the plane projection image with a face image of the target user;

adjusting the three-dimensional head model and the rendering parameters according to the matching result;

re-rendering the adjusted stereoscopic head model by using the adjusted rendering parameters;

and circularly executing the steps of projection, matching, model and rendering parameter adjustment and re-rendering until the matching graph of the plane projection image and the face image of the target user meets the preset condition.

2. The virtual character generation method according to claim 1, wherein the step of matching the planar projection image with the face image of the target user specifically comprises:

extracting at least one first image feature of the plane projection image and the face image;

converting the first image characteristics into two sets of coordinate vectors and two sets of color vectors corresponding to the plane projection image and the face image respectively;

calculating a difference between the two sets of coordinate vectors and a difference between the two sets of color vectors.

3. The virtual character generation method according to claim 2, wherein the first image feature includes a facial feature image and a facial contour image, and the step of adjusting the stereoscopic head model and the rendering parameters according to the matching result specifically includes:

predicting a plurality of vertex coordinates of the stereoscopic head model corresponding to the first image feature according to a difference value between the two sets of coordinate vectors;

modifying the plurality of vertex coordinates corresponding to the first image feature into predicted values.

4. The method for generating virtual characters according to claims 1-3, wherein before the step of obtaining initialization rendering parameters for rendering the stereoscopic head model, the method further comprises:

acquiring an initialization texture parameter to generate an initialization plane texture space;

after the steps of projection, matching, model and rendering parameter adjustment and re-rendering are executed in a circulating manner until the steps that the planar projection image and the matching image of the face image of the target user meet the preset conditions, the method further comprises the following steps:

establishing a coordinate corresponding relation between the face image and the initialized plane texture space according to the three-dimensional head model;

dividing the face image into local images of a plurality of regions according to a preset color difference threshold value;

and mapping the local images of the plurality of areas to the plane texture space according to the coordinate mapping relation.

5. The virtual character generation method of claim 4, wherein after the steps of projecting, matching, adjusting model and rendering parameters, and re-rendering are performed in a loop until the step of matching the planar projection image with the facial image of the target user meets a preset condition, the method further comprises:

extracting at least one second image feature from the face image;

calculating an ambient light parameter according to the second image characteristic;

and adjusting rendering parameters of the three-dimensional head model and texture parameters of the plane texture space according to the environment light parameters.

6. A virtual character generation system, comprising:

the image acquisition module is used for acquiring a face image of a target user;

the model generation module is used for generating a three-dimensional head model of the virtual character according to the three-dimensional head model template;

a model rendering module for obtaining an initialized rendering parameter to render the stereoscopic head model and re-rendering the adjusted stereoscopic head model using the adjusted rendering parameter after the rendering parameter and the stereoscopic head model are adjusted;

the plane projection module is used for projecting the rendered stereoscopic head model to a specified plane to obtain a plane projection image;

the image matching module is used for matching the plane projection image with a face image of the target user;

the parameter adjusting module is used for adjusting the three-dimensional head model and the rendering parameters according to the matching result;

and the circular execution module is used for circularly executing the steps of projection, matching and rendering parameter adjustment until the matching graph of the plane projection image and the face image of the target user meets the preset condition.

7. The virtual character generation system of claim 6, wherein the image matching module comprises:

the feature extraction submodule is used for extracting at least one first image feature of the plane projection image and the face image;

the vector conversion submodule is used for respectively converting the first image characteristics into two groups of coordinate vectors and two groups of color vectors corresponding to the plane projection image and the face image;

and the difference value calculation submodule is used for calculating the difference value between the two groups of coordinate vectors and the difference value between the two groups of color vectors.

8. The virtual character generation system of claim 7, wherein the first image feature comprises a facial feature image and a facial contour image, and the parameter adjustment module comprises:

a coordinate prediction sub-module, configured to predict, according to a difference between the two sets of coordinate vectors, a plurality of vertex coordinates corresponding to the first image feature in rendering parameters of the stereoscopic head model;

and the coordinate modification submodule is used for modifying the coordinates of the plurality of vertexes corresponding to the first image characteristic into predicted values.

9. The virtual character generation system according to claims 6-8, wherein the parameter obtaining module is further configured to obtain an initialization texture parameter to generate an initialization planar texture space, and the virtual character generation system further comprises:

the coordinate association module is used for establishing a coordinate corresponding relation between the face image and the initialized plane texture space according to the three-dimensional head model;

the image segmentation module is used for segmenting the face image into local images of a plurality of regions according to a preset color difference threshold value;

and the pixel mapping module is used for mapping the local images of the plurality of areas to the plane texture space according to the coordinate mapping relation.

10. The virtual character generation system according to claim 9, further comprising:

the feature extraction module is used for extracting at least one second image feature from the face image;

the parameter calculation module is used for calculating the ambient light parameter according to the second image characteristic;

the parameter adjusting module is further configured to adjust rendering parameters of the stereoscopic head model and texture parameters of the planar texture space according to the ambient light parameters.

Images