KR20080050284A - Method and device for real-time wrinkle expression of skin when character animation - Google Patents

Abstract

The present invention relates to a method for ensuring real time by providing an optimal speed and realistic expression in rendering skin wrinkles during character animation.

According to the present invention, the skin wrinkles of each facial expression are expressed as a normal map and a bump map, and the normalized skin wrinkle data and weight data are calculated by calculating the difference from the normal map and the bump map when these maps are in no expression. After generating the skin wrinkle data of the desired character using the generalized wrinkle data for each facial expression, calculate the normal map and bump map representing the final skin wrinkle using the weight of each facial expression at the current animation time (t) Render to the screen.

Description

An efficient real-time skin wrinkle rendering method and apparatus in character animation

The present invention relates to a method and apparatus for real-time expression of skin wrinkles during character animation. More specifically, the present invention relates to a method of expressing skin wrinkles of a character by being used for character expression in simulation, games, and education in the field of real-time three-dimensional graphic character rendering. The present invention relates to a method and apparatus for real-time expression of wrinkles on skin during character animation.

The present invention is derived from the research conducted as part of the IT new growth engine core technology development project of the Ministry of Information and Communication and the Ministry of Information and Communication Research and Development. [Task Management Number: 2006-S-044-01, Title: Multicore CPU and MPU-based Croat Platform Game Technology].

Currently, the real-time three-dimensional graphics field is developing at a very fast pace with the improvement of hardware and the increase of utilization. Character skin rendering is used not only for simulation but also for games and education. Various attempts have been made to render fast and realistic expressions in rendering such character skins.

Conventional technologies include muscle model based wrinkle expression from an anatomical perspective, facial animation based on an example model, skin expression using subsurface, and parameter based facial animation.

The simplest way is to create as many polygon meshes as necessary to represent the skin and then fine-tune the vertices of those meshes. However, this method uses a large number of polygons (a set of three vertices) for the details, and requires a very large number of polygons, and it takes a lot of time to render 3D and is difficult to model.

Another way is to use normal maps and bump maps.

The normal map is a texture representing the normal vector of the character model. The normal map is used to calculate light, environment map, etc. as if the character is coated with a diffuse color texture.

The bump map expresses the height and height of the character model in gray scale, and detailed expression of the face is possible by expressing the fine height of the face.

These normal maps and bump maps help to render fine curves of the skin easily and quickly by using texture.

Skin wrinkles based on anatomical muscle models have been used to express the wrinkles of the skin in facial animation. However, this method requires a lot of computation and uses a vertex-based model, making it difficult to use for real-time 3D character animation.

On the other hand, with respect to the muscle model-based wrinkle expression method from an anatomical perspective, the present applicant filed December 14, 2004, Republic of Korea Application No. 10-2004-106837 entitled "Face animation generation system using anatomical data and "Method" is disclosed as follows.

Anatomical data storage means for storing skull, skull geometry and muscle information corresponding to multiple face models in the prior art; Muscle placement means for retrieving a skull most similar to the received face model from the anatomical data storage means according to an input of a face model from outside to generate a face animation, and placing a predetermined number of muscles in the retrieved skull ; Skin generation means for combining the subcutaneous fat layer and skin to the skull where the muscle is disposed in the muscle placement means to generate a face mesh, and defining the movement of the skin according to the movement of the muscle; And an expression generating means for generating and storing a facial mesh having a specific expression by contracting or relaxing the corresponding muscle, the subcutaneous fat layer and the skin in response to the muscle control signal received from the outside.

However, such a prior art exerts an excellent effect in the expression of the facial expression, but the data to be processed is enormous for real time expression, and naturalness in the expression of the character represented by the processed data. Can be. However, in character animation, an exaggerated expression is required, but there is a problem such that an exaggerated expression cannot be performed.

The present invention synthesizes weights corresponding to the corresponding wrinkles on the normal map and the bump map to effectively express the skin wrinkles during facial animation, and applies skin wrinkle data extracted from one skin wrinkle example character model to the general character model. It is to provide a method and apparatus for real-time expression of skin wrinkles when character animation is possible.

The method of expressing the wrinkles of the skin in the character animation of the present invention comprises the first calculation step of extracting the facial expressions of the characters in order to synthesize the normal map at the current time and calculating the weight of the skin wrinkles on the normal map in which the wrinkles of these expressions are expressed; And a second calculation step of calculating the final normal map by giving weights calculated in the calculation step, and a rendering step of rendering the skin wrinkles by rendering the skin wrinkles using the final normal map calculated in the second calculation step. It is characterized by.

In addition, in the character animation of the present invention, the skin wrinkles real-time expression method, the first calculation to extract the facial expression of the character in order to synthesize the bump map at the current time and calculate the weight of the skin wrinkles on the bump map in which the wrinkles of these expressions are expressed And a second calculation step of calculating a final bump map by giving weights calculated in the calculation step, and an expression step of expressing wrinkles of the skin by rendering the skin wrinkles using the final bump map calculated in the second calculation step. It is characterized in that the configuration.

The apparatus for expressing wrinkles of the skin in the character animation according to the present invention includes a normal map module that normalizes an example of required expressions by interpolating them after pre-modeling an example of necessary expressions, and pre-models an example of necessary expressions and interpolates them. A bump map module for bump mapping by a method, a calculation module for obtaining a final normal map or a final bump map by assigning a weight to a normal map and a bump map corresponding to expressions for synthesis of weights, and a weighted normal Characterized by consisting of a rendering module that represents the wrinkles of the character by rendering using the map and bump map.

As described above, the present invention has the effect of expressing skin wrinkles easily by weight rather than muscle-based expression in character animation, which can be quickly, cheaply and naturally expressed.

In addition, the present invention has an effect that can be easily applied to all characters by using the normalized skin wrinkle normal map data to express the skin wrinkles.

In addition, according to the present invention, since the expression of the wrinkles may vary depending on which model is used, applying the face model of the old person to the face of the young person does not matter when it is expressionless, but when the facial expression changes, the wrinkles may be set to look like the old person. It has an effect.

All of the above methods are handled by the GPU's H / W Shader to enable real-time rendering.

Therefore, it is possible to express skin wrinkles faster and more realistic than ever before in the field that needs to deal with realistic character skin such as 3D game, simulation and education.

Hereinafter, with reference to the accompanying drawings will be described in more detail with respect to the configuration of the present invention.

1 is a block diagram showing a system configuration for expressing the wrinkles of the skin in real time during the character animation according to an embodiment of the present invention.

Referring to FIG. 1, first, a graphics processing unit (GPU) 10 of the present invention includes a normal map module 12, a bump member module 14, a calculation module 16, and a rendering module 18.

The GPU 10 is a processing device that receives data and processes graphics. Handles transform and lightening of graphics.

The normal map module 12 normalizes graphic data input through an interface device or the like. The normal map module 12 normalizes the input graphic data using an example-based model. In other words, models of required expressions are pre-modeled (polygon, normal map, bump map) and normalized by interpolating them.

The bump map module 14 also bumps the graphic data received through the interface device in the same manner as the normal map module 12 using an example-based model.

The calculation module 16 extracts the expressions for synthesizing the corresponding weights at the current time t, and weights and synthesizes the expressions corresponding to the expressions corresponding to the normal map or the bump map. Find the final normal map or final bump map. That is, when the normal maps are interpolated, weights are assigned to parts of each facial expression to interpolate according to the weights.

In the normal map synthesis and the bump map synthesis, the shader of the GPU 10 may be used to process the pixel values of the texture to achieve a more realistic wrinkle synthesis effect. In other words, you can synthesize them more effectively using shaders.

If the maximum number of simultaneous textures of the GPU 10 is smaller than the number of expressions to be synthesized at present, it may be modified by dividing the calculation several times. In this case, the sum of the composite normal map and the weight calculated in the previous step enters the input of the current step, and synthesizes the additional facial expressions.

In addition, the rendering module 18 applies the above algorithm to render using a normal map and a bump map employing weights to express the character's wrinkles and display the character animation expressing the wrinkles in real time through the display 20. Let's do it.

2 is a flowchart sequentially illustrating a process of expressing the wrinkles of the skin in real time when the character animation according to an embodiment of the present invention, Figures 3a and 3b is a rendering of the character's face in accordance with an embodiment of the present invention Each bump map and color map used in the present invention are illustrated. FIGS. 4A and 4B are bump maps and weight maps used to render a smiling face of a character according to an embodiment of the present invention.

2 to 4B, the character animation skin wrinkle rendering method for achieving the above object is to extract the facial expression of the character to synthesize at the current time (t) as shown in FIG. The final normal map or final bump map is obtained by synthesizing the weighted weight map to the expressed normal map or bump map as shown in FIG. The wrinkles of the skin are expressed by rendering the skin wrinkles. In this case, since FIG. 4B is a weight map used for the smiling face, the weighted character may be obtained by applying the weight using FIG. 4B.

First, the present invention will be described based on facial skin. However, the present invention is a method that can express the surface wrinkles of all parts of the character as well as the face.

In addition, in the present invention, since the normal map and the bump map are applied in the same manner, the normal map will be described based on the normal map. In the present invention, the normal map and bump map data required for skin wrinkle synthesis use an example-based model, and using the example-based model, the normal map module 12 produces a normal map. Similarly, the bump map module 14 also produces a bump map using an example-based model (S10). In other words, the normal map and the bump map are produced by pre-modeling examples of the required expressions (polygon, normal map and bump map) and interpolating them.

Each normal map and bump map generated by the normal map module 12 and the bump map module 14 are generated. The wrinkle data of each facial expression example model can be directly generated by a designer using a 3D scanner device or by viewing a 2D image. It can be made using other automated tools. The wrinkle data of the example model of each facial expression calculates a weight for each facial region when interpolating normal maps including skin wrinkles for each facial expression (S20).

The weight is a value between 0 and 1. The closer to 0, the smaller the effect of the part on the expression, and the closer to 1, the greater the effect of the part on the expression.

The weight data in the current facial expression uses (unexpression normal map-current expression normal map) as initial data. When the bump map is used, the bump map of FIG. 3A is used as initial data. The initial data manually contains wrinkles data where you don't want it, or there is a part you want to express.

The weight data may be stored in the alpha portion of the normal map. Recent textures can represent float values, which makes them more effective.

A normal map and a bump map example model in which skin wrinkles are expressed according to each expression are made, and these models are synthesized by interpolation using weights (S30). 4B shows a weight map used for laughing. Synthesis of the skin wrinkle normal map is represented by the following equation (1).

Normal (x, y) =

Sum ( Normal ( expr , x, y) * Weight ( expr , x, y)) / Sum ( Weight ( expr , x, y))

In Equation 1, expr represents an index when the expressions required for synthesis are listed up to 1 ... n. For example, there may be a number of modeled examples of smiley faces, angry faces, crying faces, and the like.

In addition, Normal (x, y) is a normal value employing a weight in the (x, y) coordinate with respect to the final wrinkled skin normal map.

Normal (expr, x, y) represents a normal value in normal map (x, y) coordinates, which is a skin wrinkle expression of the expression expr to be synthesized. Weight (expr, x, y) is the weight at the (x, y) coordinate of the expression expr to be synthesized at present.

When expressed as a facial animation, the skin wrinkle normal map synthesis at the time t is represented by the following equation.

Normal (t, x, y) =

Sum ( Normal ( expr , x, y) * Weight ( expr , x, y) * TimedWeight (t, expr)) / Sum ( Weight ( expr , x, y) * Timedweight (t, expr ))

Normal (t, x, y) is the normal value in the (x, y) coordinates for the skin wrinkle normal map that is the final result at the current time (t).

TimedWeight (t, expr) is a composite weight of the corresponding wrinkle expression expression at the current time t. Such normal map synthesis is more effective using a shader that is hardware (H / W) of the GPU 10.

As the input of the shader, the skin wrinkle normal map texture of each facial expression and the weight of each facial expression are input, and as the output, a normal map to which the weight is applied is obtained as a result of synthesis of each pixel using the above formulas (1) and (2).

If the maximum number of simultaneous textures of the GPU 10 is smaller than the number of facial expressions to be synthesized at present, the GPU 10 may be divided into several times and may be expressed according to an animation frame. In this case, the sum of the composite normal map and the weight calculated in the previous step is input to the current step and synthesized with an additional facial expression to obtain a smoother image during animation.

In order to generate a weighted expression of wrinkles by applying one example facial expression normal map data to another character's normal map, generalized facial expression normal map difference data and weights are needed. The generalized facial expression normal map difference data (general NormalMapDiff (expr, x, y)) is calculated from the normal map data of the current example, and the equation is as in Equation 3 below.

NormalMapDiff (expr, x, y) = Normal (expr, x, y)-Normal (expression, x, y)

By using Equation 3 above, the normal map difference data and the weight in each expression are calculated. The normal map difference data extracted in this way can be applied to expressions of other characters. Equation when applied to the expression of other characters is as shown in Equation 4 below.

Other Characters Normal (expr, x, y)

= Other character Normal (x, y) + normal NormalMapDiff (expr, x, y)

Like this, when the expressions of other characters are expressed, normal maps of various expressions can be generated only by the normal maps when no expression is present. The bump map of FIG. 4A is a bump map that generates a smiling expression by applying a weight. At this time, the geometry of the texture referenced by the polygon should be the same as shown in Figs. 3a to 4b. That is, make sure that the eye area on the texture in the general example model matches the eye area on the texture of another character. In addition, the character is coated with a color as shown in FIG. 3B to complete the character with wrinkles.

In the above, the normal map is mainly described, but the same algorithm is applied to the bump map. The algorithm is applied to render the rendering module 18 using the normal map and the bump map employing the weight (S40). The rendered data is displayed via the display 20.

In this way, the facial wrinkles can be expressed in any facial expression animation.

Other parts of the face can be used in the same way. By normalizing the normal maps according to the respective motions and synthesizing them, the skin wrinkles can be expressed.

1 is a block diagram showing a device configuration for expressing the wrinkles of the skin in real time during the character animation according to an embodiment of the present invention.

2 is a flowchart sequentially illustrating a process of expressing wrinkles of the skin in real time during character animation according to an embodiment of the present invention.

3A and 3B are bump maps and color maps (Albedo) respectively used for a character's face according to an embodiment of the present invention.

4A and 4B are bump maps and weight maps each showing a smiling face of a character according to an exemplary embodiment of the present invention.

<Brief description of symbols for the main parts of the drawings>

10: GPU 12: normal map module

14: bump member module 16: calculation module

18: rendering module 20: display

Claims (8)

A first calculation step of extracting facial expressions of characters to synthesize normal maps at a current time and calculating weights of skin wrinkles on normal maps in which wrinkles of these facial expressions are expressed; A second calculation step of calculating a final normal map by applying a weight calculated in the first calculation step; Rendering the skin wrinkles by using the final normal map calculated in the second calculation step Rendering step to represent the wrinkles of the skin, characterized in that the skin wrinkles real-time expression method. Extracting facial expressions of characters to synthesize bump maps at a current time and calculating weights of skin wrinkles on bump maps in which wrinkles of these facial expressions are expressed; A second calculation step of calculating a final bump map by applying a weight calculated in the first calculation step; The wrinkles real-time expression method of the character animation, characterized in that consisting of the expression step to express the wrinkles of the skin by rendering the skin wrinkles using the final bump map calculated in the second calculation step. The method according to claim 1 or 2, The bump map and the normal map data is produced using a modeled example-based model, characterized in that the wrinkles of the skin during the character animation. The method according to claim 1 or 2, The method of expressing the wrinkles of the skin during character animation, characterized in that to process the pixel value of the texture using a shader of a graphics processing unit (GPU) for the normal map synthesis or bump map synthesis. The method of claim 4, wherein If the maximum number of simultaneous textures of the GPU is smaller than the number of expressions to be synthesized at present, the sum of the composite normal map or the composite bump map and the weight calculated in the previous step is inputted to the current step and modified by combining with the additional expressions. How to express the wrinkles of the skin during character animation. The method according to claim 1 or 2, Characterized by calculating the normal map difference data and the weight by obtaining a generalized facial expression normal map difference data for applying the final normal map or the final bump map of another character to the normal map or the bump map, respectively Real-time expression of wrinkles on the skin. The method of claim 6, In order to apply the final normal map or the final bump map of another character to the normal map or the bump map, the wrinkles of the skin during character animation are applied in the same state of the geometry of the texture referenced by the polygon. Real time representation method. A normal map module for modeling examples of required expressions in advance and normalizing them by interpolating them; A bump map module for modeling examples of required expressions in advance and then bump-mapping them by interpolating them; A calculation module for obtaining a final normal map or a final bump map by assigning weights corresponding to facial expressions for synthesizing weights to the normal map and the bump map; A device for expressing wrinkles of skin during character animation, comprising a rendering module that renders by using a weighted normal map and a bump map to express a wrinkle of a character.

Images