CN109584358A - A kind of three-dimensional facial reconstruction method and device, equipment and storage medium - Google Patents

Abstract

Embodiments of the present application provide a three-dimensional face reconstruction method, device, device, and storage medium, wherein an RGB image and a depth image corresponding to the RGB image are acquired; an image after aligning the RGB image and the depth image is Perform face detection to determine face information; process the face information to generate a face model.

Description

A three-dimensional face reconstruction method and device, equipment and storage medium

technical field

The embodiments of the present application relate to the field of computer image processing, and relate to, but are not limited to, a three-dimensional face reconstruction method and apparatus, equipment, and storage medium.

Background technique

In the field of computer vision, 3D reconstruction techniques reconstruct 3D information from a single Red-Green-Blue (RGB) image or multiple RGB image inputs. For example, multi-view 3D reconstruction first calculates the correspondence between the captured image and the real-world coordinate system, and then reconstructs a 3D model that is highly similar to reality based on the information contained in each image. Nowadays, 3D reconstruction technology not only plays an important role in interactive stereo games, remote video conferencing, etc., but also has an extremely wide range of applications in the medical field. However, fast and accurate modeling effects cannot be achieved in mobile devices (such as smartphones, tablets, etc.) that are easily accessible to the general public.

SUMMARY OF THE INVENTION

In view of this, embodiments of the present application provide a three-dimensional face reconstruction method and apparatus, device, and storage medium.

The technical solutions of the embodiments of the present application are implemented as follows:

The embodiment of the present application provides a three-dimensional face reconstruction method, and the method includes:

acquiring an RGB image and a depth image corresponding to the RGB image;

performing face detection on the aligned image of the RGB image and the depth image to determine face information;

The face information is processed to generate a face model.

In this embodiment of the present application, the face model includes at least one of the following: a face mask and a three-dimensional face model; the face mask includes a face structure feature, and the three-dimensional face model includes a face structure feature and Face texture information.

In this embodiment of the present application, the acquiring an RGB image and a depth image corresponding to the RGB image includes:

Use a camera to obtain a single frame of RGB image, and use a depth camera to obtain a depth image corresponding to the single frame of RGB image;

or,

The acquiring an RGB image and a depth image corresponding to the RGB image includes: acquiring two single-frame RGB images with dual cameras;

Depth images corresponding to the two single-frame RGB images are determined according to the two single-frame RGB images.

In the embodiment of the present application, the processing of the face information to generate a face model includes:

Filtering the face information to obtain filtered face information;

A face mask is generated according to the spatial distribution information of the filtered face information.

In this embodiment of the present application, the acquiring an RGB image and a depth image corresponding to the RGB image includes:

The M frames of RGB images are acquired, and M frames of depth images corresponding to the M frames of RGB images are acquired respectively; wherein the M frames of RGB images include images of the same face from different angles; M is an integer greater than 1.

In the embodiment of the present application, the processing of the face information to generate a face model includes:

According to the face information of the M frames of depth images, screen out N frames of effective depth maps that meet preset conditions from the M frames of depth images, where N is an integer greater than 0, and N is less than or equal to M;

Fusing the N frames of effective depth maps to obtain a fused image;

According to the face information of the fusion image and the preset standard face model, the three-dimensional face model is obtained.

In the embodiment of the present application, obtaining the three-dimensional face model according to the face information of the fused image and the preset standard face model includes:

Adjusting the face information in the preset standard face model according to the face information of the fusion image to obtain a preliminary three-dimensional face model;

determining the face texture information corresponding to the fusion image;

Inputting the face texture information into the preliminary three-dimensional face model to obtain the three-dimensional face model.

In the embodiment of the present application, performing face detection on the aligned image of the RGB image and the depth image to determine face information includes:

Inputting the pixels of the RGB image and the pixel-aligned image of the depth image into a first preset deep neural network to obtain face information.

In the embodiment of the present application, performing face detection on the aligned image of the RGB image and the depth image to determine face information includes:

Inputting the aligned RGB image and the depth image to a second preset neural deep neural network to perform face key point detection, glasses detection and blur detection, and outputting face information.

In the embodiment of the present application, after the inputting the face texture information into the preliminary three-dimensional face model to obtain the three-dimensional face model, the method further includes:

save the three-dimensional face model;

The three-dimensional face model is edited according to the instructions input by the user.

An embodiment of the present application provides a three-dimensional face reconstruction device, the device includes: a first acquisition module, a first detection module and a first generation module, wherein:

The first obtaining module is used to obtain the depth image corresponding to the RGB image and the RGB image;

The first detection module is configured to perform face detection on the aligned image of the RGB image and the depth image to determine face information;

The first generation module is used for processing the face information to generate a face model.

In the embodiment of the present application, the face model includes at least one of the following: a face mask and a three-dimensional face model, the face mask includes face structure features, and the three-dimensional face model includes face structure features and Face texture information.

In this embodiment of the present application, when the RGB image is a single-frame image, the first acquisition module includes:

The first acquisition submodule is used to acquire a single frame of RGB image by using a camera, and acquire a depth image corresponding to the single frame of RGB image by using a depth camera;

or,

The second acquisition sub-module is used to acquire two single-frame RGB images using dual cameras;

The first determination submodule is configured to determine depth images corresponding to the two single-frame RGB images according to the two single-frame RGB images.

In this embodiment of the present application, the first generation module includes:

The first sub-module is used to filter the face information to obtain the filtered face information;

The first generating submodule is configured to generate a face mask according to the spatial distribution information of the filtered face information.

In the embodiment of the present application, when the RGB image is an M frame image, the first acquisition module includes:

The third acquisition sub-module is used for acquiring the M frames of RGB images, and respectively acquiring M frames of depth images corresponding to the M frames of RGB images; wherein the M frames of RGB images include images of the same face from different angles ; M is an integer greater than 1.

In this embodiment of the present application, the first generation module includes:

The second determination sub-module is configured to screen out N frames of effective depth maps that meet preset conditions from the M frames of depth images according to the face information of the M frames of depth images, where N is an integer greater than 0, and N less than or equal to M;

a first fusion submodule, configured to fuse the N frames of effective depth maps to obtain a fused image;

The third determination sub-module is configured to obtain the three-dimensional face model according to the face information of the fusion image and the preset standard face model.

In this embodiment of the present application, the root third determination submodule includes:

a first matching unit, configured to adjust the face information in the preset standard face model according to the face information of the fused image to obtain a preliminary three-dimensional face model;

a first determining unit, configured to determine the face texture information corresponding to the fusion image;

The first input unit is configured to input the face texture information into the preliminary three-dimensional face model to obtain the three-dimensional face model.

In this embodiment of the present application, the first detection module includes:

The first input sub-module is used for inputting the aligned image of the pixels of the RGB image and the pixels of the depth image into a first preset deep neural network to obtain face information.

In this embodiment of the present application, the first detection module includes:

The first detection sub-module is used for inputting the aligned image of the RGB image and the depth image to the second preset neural deep neural network to perform face key point detection, glasses detection and blur detection, and the output is obtained. information.

In the embodiment of the present application, the first detection module further includes:

a first saving submodule for saving the three-dimensional face model;

The first prompting sub-module is used to edit the three-dimensional face model according to the instruction input by the user.

The embodiments of the present application provide a computer program product, where the computer program product includes computer-executable instructions, which, after being executed, can implement the steps in the three-dimensional face reconstruction method provided by the embodiments of the present application.

An embodiment of the present application provides a computer device, the computer device includes a memory and a processor, the memory stores computer-executable instructions, and the processor can implement the present application when the processor runs the computer-executable instructions on the memory The steps in the three-dimensional face reconstruction method provided by the embodiment.

Embodiments of the present application provide a three-dimensional face reconstruction method, device, device, and storage medium, wherein, first, an RGB image and a depth image corresponding to the RGB image are acquired; then, the RGB image and the depth image are The aligned images are subjected to face detection to determine face information; finally, the face information is processed to generate a face model; in this way, through the RGB image and the depth image, the three-dimensional human with semantic information can be quickly and accurately reconstructed. face model, so that the three-dimensional face model can be applied to common mobile devices that are easy to obtain.

Description of drawings

The accompanying drawings, which are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present application, and together with the description, serve to explain the technical solutions of the present application.

FIG. 1A is a schematic structural diagram of the composition of a network architecture according to an embodiment of the present application;

FIG. 1B is a schematic diagram of an implementation flowchart of a three-dimensional face reconstruction method according to an embodiment of the present application;

FIG. 2A is a schematic flowchart of an implementation of generating a face mask by using a three-dimensional face reconstruction method according to an embodiment of the present application;

2B is a schematic diagram of the effect of a face mask generated by an embodiment of the application;

FIG. 3A is a schematic diagram of an implementation process of generating a 3D face model by using a three-dimensional face reconstruction method according to an embodiment of the present application;

3B is a schematic diagram of the effect of a 3D face model generated by an embodiment of the present application;

FIG. 4 is a schematic diagram of an implementation flowchart for quickly generating a 3D face mask using the 3D face reconstruction method provided by the embodiment of the present application according to an embodiment of the present application;

FIG. 5 is a schematic diagram of an implementation flowchart for quickly generating a 3D face model by using the 3D face reconstruction method provided by the embodiment of the present application according to an embodiment of the present application;

6 is a schematic diagram of the composition and structure of a three-dimensional face reconstruction apparatus according to an embodiment of the present application;

FIG. 7 is a schematic structural diagram of a computer device according to an embodiment of the present application.

Detailed ways

In order to make the purposes, technical solutions and advantages of the embodiments of the present application more clear, the specific technical solutions of the invention will be described in further detail below with reference to the accompanying drawings in the embodiments of the present application. The following examples are used to illustrate the present application, but are not intended to limit the scope of the present application.

This embodiment first provides a network architecture. FIG. 1A is a schematic structural diagram of the network architecture according to the embodiment of the present application. As shown in FIG. 1A , the network architecture includes two or more computer devices 11 to 1N and a server 31 , wherein the computer The devices 11 to 1N interact with the server 31 through the network 21 . The computer equipment can be various types of computer equipment with information processing capabilities during implementation, for example, the computer equipment can include mobile phones, tablet computers, desktop computers, personal digital assistants, navigators, digital phones, televisions, and so on.

This embodiment proposes a three-dimensional face reconstruction method, which can effectively solve the problem that in the process of three-dimensional face reconstruction, fast and accurate modeling effects cannot be achieved in common and easily obtained equipment, and the method is applied to computer equipment. The functions implemented by the method can be implemented by a processor in a computer device calling program codes, and of course the program codes can be stored in a computer storage medium. It can be seen that the computer device includes at least a processor and a storage medium.

FIG. 1B is a schematic diagram of an implementation flowchart of a three-dimensional face reconstruction method according to an embodiment of the present application. As shown in FIG. 1B , the method includes the following steps:

Step S101, acquiring an RGB image and a depth image corresponding to the RGB image.

Here, if the user wants to generate a face mask, he only needs to obtain a single-frame RGB image and a single-frame depth image; or use a dual-camera to obtain two single-frame RGB images, and then use the two single-frame RGB images. The corresponding depth image is estimated from the RGB image. If the user wishes to generate a three-dimensional (Three-Dimensional, 3D) face model, then acquire multiple frames of RGB images and multiple frames of depth images.

Step S102, performing face detection on the aligned image of the RGB image and the depth image to determine face information.

Here, if the user wants to generate a face mask, he only needs to detect the key points of the face, because the face mask does not have face skin color, skin texture, etc., which is equivalent to a white model containing only face features, so the When detecting, only the key points of the face need to be detected, for example, only the positions of the nose, glasses, mouth, etc. are detected. If the user wishes to generate a 3D face model, not only face key point detection, but also glasses detection, blur detection, etc. are required to make the obtained 3D face model closer to the real face.

Step S103, processing the face information to generate a face model.

Here, a preset processing rule is used to process the face information, and a face model corresponding to the processing rule is generated; the face model includes at least one of the following: a face mask and a three-dimensional face model, wherein, The face mask includes face structure features, and the three-dimensional face model includes face structure features and face texture information. The preset processing rules can be understood as the execution steps corresponding to generating a face mask and the execution steps corresponding to generating a 3D face model.

In the three-dimensional face reconstruction method provided in this embodiment, by aligning the RGB image and the depth image for detection, a face mask or a 3D face model can be quickly and accurately generated, so that the 3D face model and the face mask can be It can be applied to easily available electronic devices, such as mobile phone applications.

An embodiment of the present application provides a three-dimensional face reconstruction method, and FIG. 2A is a schematic diagram of an implementation flowchart for generating a face mask by using a three-dimensional face reconstruction method according to an embodiment of the present application. As shown in FIG. 2A , the method includes the following steps:

In this embodiment of the present application, if you want to generate a face mask, you can implement the following steps:

Step S201, using dual cameras to acquire two single-frame RGB images.

Step S202, determining depth images corresponding to the two single-frame RGB images according to the two single-frame RGB images.

Here, step S202 illustrates that, when generating a face mask in the embodiment of the present application, only a single frame of RGB image may be used, and multi-frame information is not required, nor is the depth map information directly output by the three-dimensional sensing hardware. The face mask can be generated only by using the depth image information estimated by the dual cameras based on the 2 RGB images. The obtaining of the RGB image and the depth image of a single frame may also be to use a camera to obtain the RGB image of the single frame, and activate the three-dimensional sensing function of the camera to obtain the depth image corresponding to the single frame of RGB image. The activation of the three-dimensional sensing function of the camera to obtain the depth image corresponding to the single-frame RGB image may be to activate the structured light sensor of the camera to obtain the depth image corresponding to the single-frame RGB image, or The depth image corresponding to the single frame of RGB image is acquired by using a time of flight (Time of Flight, ToF) sensing technology.

Step S203 , inputting the pixels of the single-frame RGB image and the image after the pixels of the depth image are aligned into the first preset deep neural network to obtain face information.

Here, the RGB image of a single frame and the aligned image of the depth image are input into the deep neural network to detect the key points of the face, and the detection result is also judged. If the detection result does not meet the preset conditions (for example, human The key points of the face are blurred or occluded, etc.), and a prompt message is sent to the user.

Step S204, filtering the face information to obtain filtered face information.

Here, the information in the face information that does not meet the preset conditions is filtered out, and only the information suitable for generating the face mask is retained, for example, the fuzzy points are filtered out, and only the clear and non-repetitive points are retained.

Step S205, generating a face mask according to the spatial distribution information of the filtered face information.

In the embodiment of the present application, only a single frame of RGB image is needed to generate a face mask quickly and accurately, and the face mask can be used in common electronic devices, such as a camera of a mobile phone, as shown in FIG. 2B , if If the user wants to take a face mask, he only needs to turn on the camera to capture the current face image, or select a photo 21 from the album, and input the photo into the module that generates the face mask, then the additional face can be obtained. The photo of the mask 22; or, when shooting a video, if the user selects the current model of the camera as the model for generating the face mask, then the face mask can be attached to the user's face screen, and will follow the screen. Changes vary, for example, if the user opens his mouth wide, then the screen with a face mask also opens his mouth wide, which increases the user's interest.

An embodiment of the present application provides a method for reconstructing a 3D face. FIG. 3A is a schematic diagram of an implementation flowchart for generating a 3D face model by using the method for reconstructing a 3D face according to an embodiment of the present application. As shown in FIG. 3A , the method includes the following steps:

In this embodiment of the present application, if you want to generate a 3D face model, you can implement the following steps:

Step S301, using a camera to acquire the M frames of RGB images, and using a depth camera to acquire a depth image corresponding to the single frame of RGB images.

Here, the depth camera is a camera that can emit a detection frame such as infrared light, laser, etc. to detect the distance of the image to obtain a depth image. The multi-frame RGB images are images of faces from different angles; M is an integer greater than 1. For example, images with the face turned 15 degrees to the left and 30 degrees to the left, etc.

Step S302 , perform face key point detection, glasses detection and blur detection on the image in which the pixels of the RGB image and the depth image pixels are aligned according to a preset time interval, to determine the face information.

Here, the step S302 can be understood as performing face key point detection, glasses detection and blur detection on the image after alignment of the RGB image and the depth image every few frames of images or every predetermined time.

Step S303, if the face information does not meet the preset conditions, send out prompt information.

Here, if the face information does not meet the preset conditions, prompt information is sent to the user. For example, if the user wears glasses, the reflection of the photo is not clear, or the user is too close to the camera and the photo is blurred, so a prompt message is sent, prompting the user to remove the glasses or move away, etc., or to terminate the face entry.

Step S304, according to the face information of the M frames of depth images, filter out N frames of effective depth maps that satisfy a preset condition from the M frames of depth images.

Here, N frames of effective depth maps with clear patterns and rich contents are selected from M frames of depth images.

Step S305, fuse the N frames of effective depth maps to obtain a fused image.

Step S306, adjusting the face information in the preset standard face model according to the face information of the fused image to obtain a preliminary three-dimensional face model.

Here, the preset standard face model is preset and consists of many key points and has complete point information. For example, there are 240 points. These points and the detected face information contain Point cloud fusion, find these 240 points in the final generated 3D face model, you can choose the 1st to 10th points to move forward or backward, so that the 3D face model is visualized and editable , which can be easily applied to camera applications such as mobile phones.

Step S307, determining the face texture information corresponding to the fusion image.

Step S308, inputting the face texture information into the preliminary three-dimensional face model to obtain the three-dimensional face model.

Step S309, save the three-dimensional face model.

Step S310, editing the three-dimensional face model according to the instruction input by the user.

Here, after the 3D face model is obtained, the model can be saved, and if the user wants to modify the model in a small range, he can input the modification instruction of the corresponding area, and the 3D face model can be modified according to the instruction. .

In the embodiment of the present application, multi-frame RGB images and depth images are used, and the multi-frame multi-image fusion image corresponds to the preset standard face model, so that the obtained 3D face model is editable , and the face is modeled, and the skins can be replaced with each other between the models built by different people. As shown in FIG. 3B, it can be seen in FIG. 3B that at the area 31, various parameters can be adjusted, so that the obtained 3D model can be re-edited according to the needs of the user, and the obtained 3D face model 32 is very Accurate.

Three-dimensional reconstruction transforms three-dimensional objects in real life into mathematical models that are conducive to computer representation and processing through steps such as data acquisition, feature extraction, and signal analysis. become possible. However, in the related art, a specific face modeling technology only supports one of face mask generation or 3D face model generation, and algorithms that support both are rare. The 3D face modeling technology in the related art requires a lot of computation in acquiring image texture features, etc., resulting in a slow model generation speed. At the same time, when the face moves greatly in the multi-frame images, the reconstruction results are unstable and the model accuracy is low. The modeling method of static geometric reconstruction only outputs an integrated face structure model, without the semantic information of facial features, and cannot distinguish different parts of the facial features. Therefore, the user cannot directly obtain the position information of "eyes", "nose", "mouth" and other parts through the model. It can be seen from the above that the face 3D modeling technology in the related art does not support subsequent fine-tuning or manual editing of the model, and its application is relatively limited.

The embodiment of the present application provides a three-dimensional face reconstruction method, and the method includes two cases, one is to generate a face mask, and the other is to generate a 3D face model.

The first case is to generate a face mask: the embodiment of the present application is developed based on a deep learning framework, and through a single frame of RGB image input, a mask with a face structure is quickly generated, and a real-time preview is performed according to actual needs, as shown in Figure 4. The application embodiment adopts the three-dimensional face reconstruction method provided by the embodiment of the application to quickly generate a schematic diagram of the implementation process of a 3D face mask, including the following steps:

In step S401, the image information after the alignment of the (single frame) RGB image and the depth map is acquired, and preprocessing is performed.

In step S402, the alignment image to be detected is transmitted to the deep neural network to detect the key points of the face.

Step S403, output the corresponding 3D face mask.

In the embodiment of the present application, firstly, a single frame of RGB image can be acquired as the data to be detected by shooting with a real-time camera or inputting an existing image file. Then, for a single-frame RGB image, a face mask with a face structure can be quickly generated, and real-time preview can be achieved according to actual needs.

In the second case, a 3D face model is generated: FIG. 5 is a schematic diagram of the implementation process of quickly generating a 3D face model by using the three-dimensional face reconstruction method provided by the embodiment of the present application, including the following steps:

Step S501 , obtain image information after the RGB image and the depth map are aligned, and perform preprocessing.

In step S502, the aligned image to be detected is transmitted to the deep neural network, and face key point detection, glasses detection, and blur detection are performed.

Here, in step S502, detection is performed every few frames of images (which can be adjusted according to specific time-consuming requirements). For face key point detection, the key points can be mapped to the 3D face model, so that the established face 3D model has the semantic information of facial features and other facial features. That is, the position of parts on the face model, such as eyes, nose, mouth, etc., can be found.

In step S503, multi-frame fusion is performed on the effective depth map and corresponding to the standard face model, and texture fitting of the face is performed.

Here, in the step S504, a standard face model is used, so that the final 3D face model can not only enable the user to obtain facial features from the 3D model, such as obtaining key point positions, measure distances, etc., but also support the user to reconstruct the reconstruction. 3D models for visual editing

Step S504, output the 3D face model.

In the generation of the 3D face model in the embodiment of the present application, the powerful description ability of the deep neural network is used for modeling, and the RGB image and the depth image are corresponded to generate a 3D model that is close to the real portrait, including the 3D structure of the face and the face. texture.

In this embodiment of the present application, multiple frames of RGB images and depth images may be acquired as data to be detected by shooting with a real-time camera or by passing in an existing image file. For multi-frame fusion images, a 3D face model with a face structure can be quickly generated according to the description in Figure 5, multi-angle rotation preview is supported, and specific parts can be visually edited according to actual needs. In a three-dimensional face reconstruction method provided in the embodiment of the present application, by merging multiple frames of images and corresponding to a preset standard face model, a face 3D structure mask or a face 3D model can be output, according to Use in actual scenarios to improve integration and update efficiency. Moreover, the deep neural network in the embodiment of the present application can be located in the terminal device to meet the face modeling requirements required by the mobile terminal application, and can also be located in the server to meet the face modeling requirements with a larger amount of information. The 3D face modeling deep neural network obtained in the embodiment of the present application can be iteratively updated in time, and can quickly respond to new user scenarios for optimization.

An embodiment of the present invention provides a three-dimensional face reconstruction apparatus. FIG. 6 is a schematic structural diagram of a three-dimensional face reconstruction apparatus according to an embodiment of the present application. As shown in FIG. 6 , the three-dimensional face reconstruction apparatus 600 includes: a first acquisition module 601, a first detection module 605 and a first generation module 603, wherein:

The first obtaining module 601 is configured to obtain a depth image corresponding to an RGB image and the RGB image;

The first detection module 602 is configured to perform face detection on the aligned image of the RGB image and the depth image to determine face information;

The first generating module 603 is configured to process the face information to generate a face model.

In the embodiment of the present application, the face model includes at least one of the following: a face mask and a three-dimensional face model, wherein the face mask includes a face structure feature, and the three-dimensional face model includes a face structure feature and face texture information.

In this embodiment of the present application, when the RGB image is a single-frame image, the first acquisition module 601 includes:

The first acquisition submodule is used to acquire a single frame of RGB image by using a camera, and acquire a depth image corresponding to the single frame of RGB image by using a depth camera;

or,

The second acquisition sub-module is used to acquire two single-frame RGB images using dual cameras;

The first determination submodule is configured to determine depth images corresponding to the two single-frame RGB images according to the two single-frame RGB images.

In this embodiment of the present application, the first generation module 603 includes:

The first sub-module is used to filter the face information to obtain the filtered face information;

The first generating submodule is configured to generate a face mask according to the spatial distribution information of the filtered face information.

In this embodiment of the present application, when the RGB images are M frames of images, the first acquisition module 601 includes:

The third acquisition sub-module is used for acquiring the M frames of RGB images, and respectively acquiring M frames of depth images corresponding to the M frames of RGB images; wherein the M frames of RGB images include images of the same face from different angles ; M is an integer greater than 1.

In this embodiment of the present application, the first generation module 603 includes:

The second determination sub-module is configured to screen out N frames of effective depth maps that meet preset conditions from the M frames of depth images according to the face information of the M frames of depth images, where N is an integer greater than 0, and N less than or equal to M;

a first fusion submodule, configured to fuse the N frames of effective depth maps to obtain a fused image;

The third determination sub-module is configured to obtain the three-dimensional face model according to the face information of the fusion image and the preset standard face model.

In this embodiment of the present application, the root third determination submodule includes:

a first matching unit, configured to adjust the face information in the preset standard face model according to the face information of the fused image to obtain a preliminary three-dimensional face model;

a first determining unit, configured to determine the face texture information corresponding to the fusion image;

The first input unit is configured to input the face texture information into the preliminary three-dimensional face model to obtain the three-dimensional face model.

In this embodiment of the present application, the first detection module 602 includes:

The first input sub-module is used for inputting the aligned image of the pixels of the RGB image and the pixels of the depth image into a first preset deep neural network to obtain face information.

In this embodiment of the present application, the first detection module 601 includes:

The first detection sub-module is used for inputting the aligned image of the RGB image and the depth image to the second preset neural deep neural network to perform face key point detection, glasses detection and blur detection, and the output is obtained. information.

In this embodiment of the present application, the first detection module 602 further includes:

a first saving submodule for saving the three-dimensional face model;

The first prompting sub-module is used to edit the three-dimensional face model according to the instruction input by the user.

It should be noted that the descriptions of the above apparatus embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects to the method embodiments. For technical details not disclosed in the device embodiments of the present application, please refer to the descriptions of the method embodiments of the present application for understanding.

It should be noted that, in the embodiments of the present application, if the above-mentioned three-dimensional face reconstruction method is implemented in the form of a software function module and sold or used as an independent product, it can also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence or in the parts that make contributions to the prior art. The computer software products are stored in a storage medium and include several instructions for An instant messaging device (which may be a terminal, a server, etc.) is caused to execute all or part of the methods described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a removable hard disk, a read only memory (Read Only Memory, ROM), a magnetic disk or an optical disk and other mediums that can store program codes. As such, the embodiments of the present application are not limited to any specific combination of hardware and software.

Correspondingly, the embodiments of the present application further provide a computer program product. The computer program product includes computer-executable instructions. After the computer-executable instructions are executed, the three-dimensional face reconstruction method provided by the embodiments of the present application can be implemented. step.

Correspondingly, the embodiments of the present application further provide a computer storage medium, where computer-executable instructions are stored thereon, and when the computer-executable instructions are executed by a processor, the three-dimensional face reconstruction provided by the above-mentioned embodiments is implemented. steps of the method.

Correspondingly, an embodiment of the present application provides a computer device, and FIG. 7 is a schematic structural diagram of a computer device according to an embodiment of the present application. As shown in FIG. 7 , the hardware entity of the computer device 700 includes: a processor 701 , a communication interface 702 and memory 703, where

The processor 701 generally controls the overall operation of the computer device 700 .

The communication interface 702 enables the computer device to communicate with other terminals or servers through a network.

The memory 703 is configured to store instructions and applications executable by the processor 701, and can also cache data to be processed or processed by the processor 701 and various modules in the computer device 700 (eg, image data, audio data, voice communication data and video communication data), which can be implemented by flash memory (FLASH) or random access memory (Random Access Memory, RAM).

The descriptions of the above instant computer device and storage medium embodiments are similar to the descriptions of the above method embodiments, and have similar beneficial effects to the method embodiments. For technical details not disclosed in the instant messaging device and storage medium embodiments of the present application, please refer to the description of the method embodiments of the present application for understanding.

It is to be understood that reference throughout the specification to "one embodiment" or "an embodiment" means that a particular feature, structure or characteristic associated with the embodiment is included in at least one embodiment of the present application. Thus, appearances of "in one embodiment" or "in an embodiment" in various places throughout this specification are not necessarily necessarily referring to the same embodiment. Furthermore, the particular features, structures or characteristics may be combined in any suitable manner in one or more embodiments. It should be understood that, in various embodiments of the present application, the size of the sequence numbers of the above-mentioned processes does not mean the sequence of execution, and the execution sequence of each process should be determined by its functions and internal logic, and should not be dealt with in the embodiments of the present application. implementation constitutes any limitation. The above-mentioned serial numbers of the embodiments of the present application are only for description, and do not represent the advantages or disadvantages of the embodiments.

It should be noted that, herein, the terms "comprising", "comprising" or any other variation thereof are intended to encompass non-exclusive inclusion, such that a process, method, article or device comprising a series of elements includes not only those elements, It also includes other elements not expressly listed or inherent to such a process, method, article or apparatus. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in a process, method, article or apparatus that includes the element.

In the several embodiments provided in this application, it should be understood that the disclosed apparatus and method may be implemented in other manners. The device embodiments described above are only illustrative. For example, the division of the units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components may be combined, or Can be integrated into another system, or some features can be ignored, or not implemented. In addition, the coupling, or direct coupling, or communication connection between the various components shown or discussed may be through some interfaces, and the indirect coupling or communication connection of devices or units may be electrical, mechanical or other forms. of.

The unit described above as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit; it may be located in one place or distributed to multiple network units; Some or all of the units may be selected according to actual needs to achieve the purpose of the solution in this embodiment.

In addition, each functional unit in each embodiment of the present application may all be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above integration The unit can be implemented either in the form of hardware or in the form of hardware plus software functional units.

Those of ordinary skill in the art can understand that all or part of the steps of implementing the above method embodiments can be completed by program instructions related to hardware, the aforementioned program can be stored in a computer-readable storage medium, and when the program is executed, the execution includes: The steps of the above method embodiments; and the aforementioned storage medium includes: a removable storage device, a read only memory (Read Only Memory, ROM), a magnetic disk or an optical disk and other media that can store program codes.

Alternatively, if the above-mentioned integrated units of the present application are implemented in the form of software function modules and sold or used as independent products, they may also be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the embodiments of the present application can be embodied in the form of software products in essence or in the parts that make contributions to the prior art. The computer software products are stored in a storage medium and include several instructions for A computer device (which may be a personal computer or a server, etc.) is caused to execute all or part of the methods described in the various embodiments of the present application. The aforementioned storage medium includes various media that can store program codes, such as a removable storage device, a ROM, a magnetic disk, or an optical disk.

The above are only specific embodiments of the present application, but the protection scope of the present application is not limited to this. should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be subject to the protection scope of the claims.

Claims (10)

1. a three-dimensional face reconstruction method, is characterized in that, comprises: acquiring an RGB image and a depth image corresponding to the RGB image; performing face detection on the aligned image of the RGB image and the depth image to determine face information; The face information is processed to generate a face model. 2. The method according to claim 1, wherein the human face model comprises at least one of the following: a human face mask and a three-dimensional human face model; the human face mask comprises human face structural features, and the three-dimensional human face The face model contains face structural features and face texture information. 3. The method according to claim 1, wherein the acquiring an RGB image and a depth image corresponding to the RGB image comprises: Use a camera to obtain a single frame of RGB image, and use a depth camera to obtain a depth image corresponding to the single frame of RGB image; or, The acquiring an RGB image and a depth image corresponding to the RGB image includes: acquiring two single-frame RGB images with dual cameras; Depth images corresponding to the two single-frame RGB images are determined according to the two single-frame RGB images. 4. The method according to claim 3, wherein the processing of the face information to generate a face model comprises: Filtering the face information to obtain filtered face information; A face mask is generated according to the spatial distribution information of the filtered face information. 5. The method according to claim 1, wherein the acquiring an RGB image and a depth image corresponding to the RGB image comprises: The M frames of RGB images are acquired, and M frames of depth images corresponding to the M frames of RGB images are acquired respectively; wherein the M frames of RGB images include images of the same face from different angles; M is an integer greater than 1. 6. The method according to claim 5, wherein the processing of the face information to generate a face model comprises: According to the face information of the M frames of depth images, screen out N frames of effective depth maps that meet preset conditions from the M frames of depth images, where N is an integer greater than 0, and N is less than or equal to M; Fusing the N frames of effective depth maps to obtain a fused image; According to the face information of the fusion image and the preset standard face model, the three-dimensional face model is obtained. 7. The method according to claim 6, wherein, obtaining the three-dimensional face model according to the face information of the fusion image and a preset standard face model, comprising: Adjusting the face information in the preset standard face model according to the face information of the fusion image to obtain a preliminary three-dimensional face model; determining the face texture information corresponding to the fusion image; Inputting the face texture information into the preliminary three-dimensional face model to obtain the three-dimensional face model. 8. A three-dimensional face reconstruction device, wherein the device comprises: a first acquisition module, a first detection module and a first generation module, wherein: The first obtaining module is used to obtain the depth image corresponding to the RGB image and the RGB image; The first detection module is configured to perform face detection on the aligned image of the RGB image and the depth image to determine face information; The first generation module is used for processing the face information to generate a face model. 9 . A computer storage medium, characterized in that the computer storage medium stores computer-executable instructions, which can implement the method steps of any one of claims 1 to 7 after the computer-executable instructions are executed. 10. A computer device, characterized in that the computer device comprises a memory and a processor, the memory stores computer-executable instructions, and the processor can implement rights when the processor executes the computer-executable instructions on the memory. The method steps of any one of claims 1 to 7.