WO2025011559A1 - Head size measurement method and apparatus, head grid generation method and apparatus, medium, and device - Google Patents

Abstract

The present disclosure relates to the technical field of artificial intelligence, and provides a head size measurement method and a 3D head grid generation method. According to the method, a face local area is determined by means of face feature points, a circular reference object is extracted from the face local area, and a head size of a target object is determined on the basis of the size of the circular reference object, thereby generating a 3D head grid. The accuracy is high.

Description

Head size measurement, head grid generation method and device, medium and equipment

CROSS-REFERENCE TO RELATED APPLICATIONS

The present disclosure claims priority to Chinese patent application numbered 2023108333083, filed on July 10, 2023, and entitled “Head size measurement, head grid generation method and device, medium and equipment”, the entire contents of which are incorporated herein by reference.

Technical Field

The present disclosure relates to the field of artificial intelligence technology, and in particular to a head size measurement, head mesh generation method and device, computer storage medium and electronic equipment.

Background Art

Currently, commercial 3D scanners are the most widely used method for obtaining 3D physical head models. However, they are expensive and inconvenient, which limits their popularity and accessibility. Therefore, reconstructing 3D models from 2D images is both cheap and convenient and more suitable for the public.

There are many mature 3D head/face reconstruction methods, which mainly focus on applications where physical size accuracy is not very important. On the contrary, size is very important in ergonomic design. Current methods cannot derive 3D head/face models with sufficient size accuracy.

It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field.

Summary of the invention

The purpose of the present disclosure is to provide a head size measurement method, a 3D head mesh generation method and a device thereof, which at least to some extent overcome the problem of inaccurate head size determination in the related art.

Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

According to one aspect of the present disclosure, a head size measurement method is provided, comprising: acquiring a facial image of a target object, the facial image having a circular reference object at the forehead position of the target object; determining facial feature points of the target object; determining a facial local area including the circular reference object based on the facial feature points; extracting the circular reference object from the facial local area; and determining the head size of the target object based on the size of the circular reference object.

In one embodiment, determining the head size of the target object based on the size of the circular reference object comprises: determining the diameter of the circular reference object; determining the head size of the target object based on the diameter of the circular reference object; inch.

In one embodiment, the facial feature points include eye feature points, and the facial partial image including the circular reference object is a rectangular area including the circular reference object.

In one embodiment, extracting the circular reference object from the local facial area includes: using an image segmentation artificial neural network to segment the local facial area to obtain the circular reference object; using an edge detector to detect the edge of the circular reference object; using an ellipse fitting method to perform ellipse measurement on the edge of the circular reference object, thereby extracting the circular reference object.

In one embodiment, the image segmentation artificial neural network is U-Net or a fully convolutional network FCN, and/or the edge detector is a canny edge detector.

In one embodiment, the method further includes: dedistorting the facial image of the target object based on camera parameters of an image acquisition device, wherein the camera parameters include distortion coefficients and an intrinsic matrix, and the distortion coefficients include radial distortion coefficients and tangential distortion coefficients.

In one embodiment, the method further comprises: acquiring multi-view images of a preset chessboard at different angles and distances by the image acquisition device, thereby determining camera parameters of the image acquisition device.

According to another aspect of the present disclosure, a 3D head mesh generation method is provided, comprising: acquiring multiple 2D face images of a target object, the 2D face images comprising a front face image, a right face side image, and a left face side image, the front face image having a circular reference object at a forehead position of the target object; reconstructing a 3D head shape of the target object based on the multiple 2D face images; determining a head size of the target object based on the front face image; and generating a 3D head mesh of the target object having the head size.

In one embodiment, determining the head size of the target object based on the frontal face image includes: determining the head size of the target object based on the frontal face image according to the above-mentioned head size measurement method.

In one embodiment, reconstructing the 3D head shape of the target object based on the multiple 2D face images includes:

The encoder of the 3DMM based on the head shape and texture is used to reconstruct the 3D head shape of the target object based on the multiple 2D face images to obtain the reconstructed 3D head shape.

In one embodiment, the method further comprises: pre-training the head shape and texture based 3DMM based on training data.

In one embodiment, the pre-training of the 3DMM based on head shape and texture based on training data includes: generating a parameterized head based on the training data using a non-rigid iterative closest point (NICP) algorithm; aligning the parameterized head with a uniform size using a generalized Procter & Gamble analysis (GPA); and extracting principal components (PCs) from all parameterized head meshes using PCA to obtain the 3DMM based on head shape and texture.

In one embodiment, a 3D head shape of the target object is reconstructed based on the multiple 2D face images using an encoder based on a 3DMM of head shape and texture, and obtaining the reconstructed 3D head shape comprises: inputting the multiple 2D face images into an artificial neural network encoder to predict 3D head mesh parameters, wherein the 3D head mesh parameters include 3DMM coefficients, lighting parameters, and camera parameters of the 3D head mesh, and minimizing the network loss function, wherein The network loss functions include: pixel-level loss _Limg (ξ), perceptual identity loss _Lide (ξ) of the reconstructed face image, facial edge loss _Ledge (ξ) and projection feature point loss _Llan (ξ).

In one embodiment, the method further comprises: extracting perceptual features used in perceptual identity loss calculation using a pre-trained face recognition network.

According to another aspect of the present disclosure, a 3D head mesh use method is provided, which is applied to a 3D head mesh generated according to the above-mentioned 3D head mesh generation method, and includes: using the 3D head mesh for headwear design.

In one embodiment, the headgear includes headphones, glasses, a helmet, a mask, or a hat.

According to another aspect of the present disclosure, a head size measurement device is provided, comprising: a facial image acquisition module, used to acquire a facial image of a target object, wherein the facial image has a circular reference object at the forehead position of the target object; a facial feature point determination module, used to determine the facial feature points of the target object; a local area determination module, used to determine a facial local area including the circular reference object based on the facial feature points; a reference object advance module, used to extract the circular reference object from the facial local area; and a head size determination module, used to determine a diameter of the circular reference object, and determine the head size of the target object based on the diameter of the circular reference object.

According to another aspect of the present disclosure, a 3D head mesh generation device is provided, including: a face image acquisition module, used to acquire multiple 2D face images of a target object, the 2D face images including a front face image, a right face side image, and a left face side image, the front face image having a circular reference object at the forehead position of the target object; a head shape reconstruction module, used to perform 3D head shape reconstruction on the target object based on the multiple 2D face images; a head size determination module, used to determine the head size of the target object based on the front face image; and a head mesh generation module, used to generate a 3D head mesh with the head size of the target object.

According to another aspect of the present disclosure, there is provided an electronic device, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-mentioned head size measurement method, or 3D head mesh generation method by executing the executable instructions.

According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the head size measurement method or the 3D head mesh generation method described above is implemented.

The head size measurement method and 3D head mesh generation method provided by the embodiments of the present disclosure determine the head size of the target object based on the size of a circular reference object, and have a high accuracy rate in size determination.

It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure.

BRIEF DESCRIPTION OF THE DRAWINGS

The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

FIG1 is a schematic diagram showing the structure of a computer system in one embodiment of the present disclosure;

FIG2 shows a flow chart of a method for measuring head size in one embodiment of the present disclosure;

FIG3 shows a schematic diagram of a square mark in one embodiment of the present disclosure;

FIG4 shows a schematic diagram of a circular mark in one embodiment of the present disclosure;

FIG5A shows a flow chart of a circular reference object extraction method according to an embodiment of the present disclosure;

FIG5B shows a schematic diagram of a circular reference object measurement channel in one embodiment of the present disclosure;

FIG6A shows a flow chart of a method for generating a D head mesh in one embodiment of the present disclosure;

FIG6B shows a main workflow diagram of a method for generating a 3D head mesh according to an embodiment of the present disclosure;

FIG7 is a schematic diagram of a 3DMM of a human head in one embodiment of the present disclosure;

FIG8 is a schematic diagram showing a 3D head shape reconstruction framework in one embodiment of the present disclosure;

FIG. 9 illustrates measurement evaluation of different circular reference objects in one embodiment;

FIG10 shows a comparison of qualitative results of two methods using circle or square markings in one embodiment;

Figure 11 shows the application scenario of head-related products;

FIG12 is a schematic diagram of a head size measuring device according to an embodiment of the present disclosure;

FIG13 is a schematic diagram of a 3D head mesh generation device in one embodiment of the present disclosure; and

FIG. 14 is a block diagram showing a structure of a computer device in one embodiment of the present disclosure.

DETAILED DESCRIPTION

Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and/or processor devices and/or microcontroller devices.

The solution provided by the present application uses a circular object as a measurement reference, and performs 3D head mesh reconstruction based on the head size determined by the circular reference object, and applies it to ergonomic design. For ease of understanding, several terms involved in the present application are first explained below.

Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain the best results. In other words, artificial intelligence is a comprehensive technology in computer science that attempts to understand the essence of intelligence and produce a new intelligent machine that can respond in a similar way to human intelligence. Artificial intelligence is to study the design principles and implementation methods of various intelligent machines, so that machines have the functions of perception, reasoning and decision-making.

Artificial intelligence technology is a comprehensive discipline that covers a wide range of fields, including both hardware-level technology and software-level technology. Technology. AI basic technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, big data processing technology, operation/interaction systems, mechatronics, etc. AI software technologies mainly include computer vision technology, speech processing technology, natural language processing technology, and machine learning/deep learning.

Computer Vision Technology (CV) Computer vision is a science that studies how to make machines "see". To put it more specifically, it refers to the use of cameras and computers to replace human eyes to identify, track and measure targets, and further perform graphic processing so that the computer processing becomes an image that is more suitable for human eye observation or transmission to instrument detection. As a scientific discipline, computer vision studies related theories and technologies, and attempts to establish an artificial intelligence system that can obtain information from images or multi-dimensional data. Computer vision technology usually includes image processing, image recognition, image semantic understanding, image retrieval, OCR (Optical Character Recognition), video processing, video semantic understanding, video content/behavior recognition, three-dimensional object reconstruction, 3D (3-Dimension) technology, virtual reality, augmented reality, simultaneous positioning and map construction, and other technologies, as well as common biometric recognition technologies such as face recognition and fingerprint recognition.

Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge structures to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence. Machine learning and deep learning usually include artificial neural networks, belief networks, reinforcement learning, transfer learning, inductive learning, and self-learning.

With the research and advancement of artificial intelligence technology, artificial intelligence technology has been studied and applied in many fields, such as common smart homes, smart wearable devices, virtual assistants, smart speakers, smart marketing, unmanned driving, automatic driving, drones, robots, smart medical care, smart customer service, etc. I believe that with the development of technology, artificial intelligence technology will be applied in more fields and play an increasingly important role.

The solution provided in the embodiments of the present application involves artificial intelligence computer vision, machine learning and other technologies, which are specifically described by the following embodiments:

FIG1 is a schematic diagram showing the structure of a computer system provided in one embodiment of the present disclosure. The system includes: a plurality of terminals 120 and a server (or server cluster) 140 .

The terminal 120 may be a mobile terminal such as a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a smart home device, an AR (Augmented Reality) device, a VR (Virtual Reality) device, or the terminal 120 may be a personal computer (PC), such as a laptop computer and a desktop computer.

The terminal 120 may be installed with an application program for providing and implementing the method disclosed herein.

The terminal 120 is connected to the server 140 via a communication network. Optionally, the communication network is a wired network or a wireless network. network.

The server 140 is a server, or is composed of several servers, or is a virtualization platform, or is a cloud computing service center. The server 140 is used to provide background services for the application program that provides .... Optionally, the server 140 undertakes the main computing work, and the terminal 120 undertakes the secondary computing work; or, the server 140 undertakes the secondary computing work, and the terminal 120 undertakes the main computing work; or, the terminal 120 and the server 140 adopt a distributed computing architecture for collaborative computing.

In some optional embodiments, the server 140 is used to store information such as the 3D head model established through training.

Optionally, the client of the application installed in different terminals 120 is the same, or the client of the application installed on the two terminals 120 is the client of the same type of application of different control system platforms. Based on the different terminal platforms, the specific form of the client of the application can also be different, for example, the application client can be a mobile client, a PC client or a World Wide Web (Web) client, etc.

Those skilled in the art will appreciate that the number of the terminals 120 may be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or a greater number. The embodiment of the present application does not limit the number and device type of the terminals.

Optionally, the system may further include a management device (not shown in FIG. 1 ), which is connected to the server 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

Optionally, the above-mentioned wireless network or wired network uses standard communication technology and/or protocol. The network is usually the Internet, but it can also be any network, including but not limited to Local Area Network (LAN), Metropolitan Area Network (MAN), Wide Area Network (WAN), mobile, wired or wireless network, any combination of private network or virtual private network). In some embodiments, technologies and/or formats including Hypertext Markup Language (HTML), Extensible Markup Language (XML), etc. are used to represent data exchanged through the network. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and/or dedicated data communication technologies may be used to replace or supplement the above-mentioned data communication technologies.

The following will describe in more detail the various steps of the head size measurement and reconstruction method in this example implementation with reference to the accompanying drawings and embodiments.

FIG2 shows a flow chart of a head size measurement method in an embodiment of the present disclosure. The method provided in the embodiment of the present disclosure can be executed by any electronic device with computing and processing capabilities, such as the terminal 120 and/or the server 140 in FIG1 . In the following example description, the server 140 is used as the execution subject for example description.

As shown in FIG. 2 , S202, a facial image of a target object is acquired, and the facial image has a circular reference object at the forehead position of the target object. The target object is a user. In one embodiment, the facial image uses a frontal facial image of the user. In one embodiment, during the facial image capture process, the user maintains a neutral expression (open eyes, close mouth) to avoid any expression effect in the 3D head reconstruction. The forehead position may include the middle position of the forehead. In one embodiment, the circular reference object is a printed circular mark, for example, a black circular mark printed on white paper. The circular reference object may also be a reference object similar to a coin. The color of the circular reference object is not limited to black, and other colors that are easily distinguished from the skin color may also be used.

S204, determining facial feature points of the target object from the facial image. In one embodiment, the facial feature points of the target object include eye feature points, eyebrow feature points, or both. Eye feature points may include eye corner feature points and eyeball feature points; eyebrow feature points may be the center of the eyebrow. In one embodiment, facial feature points may include nose feature points,

S206, determining a facial local area including a circular reference object based on facial feature points. Based on the determined facial feature points, the position range of the circular reference object on the target object's face can be determined, thereby determining a facial local area including the circular reference object, which can be a rectangle or a square.

S208, extracting a circular reference object from the local facial region. The circular reference object can be extracted from the local facial region by an image segmentation network based on an artificial neural network, for example, a U-Net network or an FCN network based on a CNN network. The local facial region can be determined more accurately by facial feature points, and extracting a circular reference object from the local facial region can improve the accuracy and speed of processing.

In one embodiment, the circular reference object segmentation based on U-Net includes the following steps: Conv+Pooling downsampling the local facial area image; then Deconv deconvolution upsampling, cropping the previous low-level feature map, and performing feature fusion; then upsampling again. Repeat the above process until a feature map of a predetermined output size is obtained, and finally the output segment map is obtained through softmax. The advantages of the U-Net network include that only a small sample training set is required for training, it is a relatively small segmentation network, and the structure is simple. Through the previous local facial area segmentation, it can better adapt to the situation of U-net, and the segmentation accuracy is high.

S210, determining the head size of the target object based on the size of the circular reference object. The head size of the target object can be determined based on the extracted size of the circular reference object. The size of the circular reference object can be the diameter, circumference, or area of the circular reference object.

In the above embodiment, the local facial area is determined by facial feature points, a circular reference object is extracted from the local facial area, and the head size of the target object is determined based on the size of the circular reference object, with high accuracy. Compared with reference objects of other shapes, the circular reference object is more suitable for the head area, has less distortion caused by deformation, and improves the measurement accuracy. Placing the circular reference object in the forehead area can reduce the deformation of the circular reference object.

It should be noted that the above figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

Before introducing face image capture, a method for measuring camera parameters is first introduced.

Before capturing the facial image, a checkerboard mark template (see, for example, FIG. 3 ) is printed for camera parameter measurement. For example, a black checkerboard mark can be printed on A4 paper.

Through these chessboard images, OpenCV is used to calibrate the camera and calculate the camera parameters (i.e. distortion coefficients and intrinsic matrix) and undistort the captured image.

Camera parameter measurement

Some cameras introduce significant distortion to the captured images, including radial distortion and tangential distortion. Camera calibration can be performed to determine the distortion coefficients (e.g., five distortion parameters), and then the distortion of the captured images is removed. In addition, the camera's intrinsic parameter matrix K is measured, which is used for image distortion removal and perspective transformation steps of 3D head reconstruction. The coefficients in this matrix include focal length (fx, fy) and optical center (cx, cy).

In one embodiment, a mobile phone is used as a representative tool to capture multi-view images (e.g., width and height: 3024×4025 pixels). A pre-designed chessboard (10×7 squares and 9×6 inner corners) with a total size of 210×297 mm is printed and then placed on a flat surface (e.g., a table). The size of all squares in the chessboard is the same (each square is 26.7 mm), and the size of the square is an important parameter for determining the actual distance in camera calibration.

Multiple images of the chessboard are taken at different angles and distances, for example more than 10 images, to provide various square angles with different viewing angles for camera calibration. Using these chessboard images, camera calibration is performed using, for example, OpenCV to calculate camera parameters (i.e., distortion coefficients and intrinsic matrices), and the captured images can be dedistorted according to the obtained camera parameters.

An example of preparation of a circular reference object is described below.

Use a black and white printer with A4 paper to print pre-designed circular markers for head size measurement. The printed template is shown in Figure 4. For example, 54 circular markers can be cut from one sheet of printed paper. The circular markers can be attached to the forehead of the participant, etc., where these areas have a high degree of flatness.

Here is a brief description of how to use the circular reference object.

3D head dimensions can be measured from 2D images using a black circle marker with a constant diameter (e.g., 15 mm) as a reference object. The target subject (participant) is asked to place this circle at a location on their forehead, such as the center of the forehead, which is relatively flatter than other facial areas. First, a frontal facial image with the marker attached to the participant's forehead is captured. Then, two face images without the marker are captured from the left and right (e.g., from 10°–50°). During the facial image capture process, the participants are instructed to maintain a neutral expression (eyes open, mouth closed) to avoid any expression effects in the 3D head reconstruction. After the facial image is captured, an image dedistortion method can be applied to it based on the camera parameters determined previously.

FIG. 5A shows a flow chart of a circular reference object extraction method in an embodiment of the present disclosure, and FIG. 5B shows a schematic diagram of a corresponding circular reference object measurement channel.

As shown in FIG5A , S502 , a rectangular region is cropped on the facial image of the target object based on the detected facial feature points.

In face image preprocessing, the face region is cropped from the captured face image of the target object using a pre-trained artificial neural network (e.g., FAN) to detect facial feature points (e.g., the white points shown in FIG. 5B(a)). Based on the facial feature points, a facial rectangular area (for example, the rectangular area shown in FIG. 5B (a)) is determined, thereby extracting a local facial area for cropping. The cropped image is shown in FIG. 5B (b), for example, with a size of 256×256 pixels.

S504 , using the pre-trained U-Net to perform accurate circular reference area segmentation to obtain a circular reference object, such as the black circular object shown in FIG. 5B(c) .

S506 , use an edge detector, such as a canny edge detector, to filter the edges of the circular reference object to obtain the edges of the circular reference object, as shown in FIG. 5B(d) .

S508 , performing ellipse fitting based on the edge of the circular reference object to obtain the diameter of the fitted ellipse, see the circular reference object shown in FIG. 5B(e) .

In the above embodiment, the facial local area can be determined relatively accurately by using facial feature points, and the circular reference object is extracted from the facial local area, which can improve the accuracy and speed of processing.

FIG. 6A shows a flowchart of a method for generating a 3D head mesh in one embodiment of the present disclosure.

As shown in FIG6A , S602 , multiple 2D face images of the target object are obtained, the 2D face images including a front face image, a right face side image, and a left face side image, the front face image having a circular reference object at the forehead position of the target object.

S604: reconstructing a 3D head shape of the target object based on multiple 2D face images.

S606, determining the head size of the target object based on the front face image. Determine the head size of the target object based on the size of the circular reference object in the face proof image.

S608: Generate a 3D head mesh with head size of the target object.

In the above embodiment, the head size of the target object is determined based on the circular reference object in the frontal face image, thereby generating a 3D head mesh with accurate head size, which can meet high-precision size requirements and is applied to applications such as headwear design that have high requirements on head model size.

FIG6B shows a main workflow diagram of a method for generating a 3D head mesh according to an embodiment of the present disclosure. As shown in FIG6B , the workflow consists of four parts:

(1) 2D face image capture;

(2) 3D head shape reconstruction (using the encoder and 3DMM of head shape and texture);

(3) 3D head size measurement (by measuring the diameter of a circular reference object using U-Net); and

(4) Application scenarios of reconstructing head mesh.

The 3D head shape reconstruction is described in detail below.

In one embodiment, 3D head shape reconstruction is implemented based on 3DMM. 3DMM is the basis for subsequent 3D head shape reconstruction. A large number of head shape 3DMMs are created based on the existing training data set. In addition, a head texture 3DMM model is also created. A specific example of creating a 3DMM model includes the following steps:

(1) Generate a parameterized head from a training dataset using a non-rigid iterative closest point (NICP) algorithm, which can be, for example, an optimal step-size non-rigid ICP algorithm for surface registration by Amberg B, Romdhani S, Vetter T. The training dataset can include a head shape training dataset and a head texture training dataset.

(2) aligning the parameterized heads to a uniform size using Generalized Platts Analysis (GPA);

(3) Principal components (PCs) were extracted from all parameterized head meshes using principal component analysis (PCA) to build the 3DMM model.

A schematic diagram of the generated 3DMM model of a human head is shown in FIG7 , which includes a head shape and a head texture model.

Fig. 8 shows a schematic diagram of a 3D head shape reconstruction framework in one embodiment of the present disclosure. As shown in Fig. 8, when multiple face images are input, a CNN-based encoder (e.g., ResNet50) is used to predict parameters, including a 3DMM coefficient of a head mesh (including head shape 801 and head texture 802 parameters), and a corresponding number of lighting and camera parameters 803, 804, and a fitting face image 806, a fitting face landmark 807, 808 are generated through a differentiable renderer 805, and the loss is determined in the ground-truth face images 809, the ground-truth face edge heatmap 810, and the ground-truth face landmark 811. The parameters ξ of the 3DMM-based shape reconstruction method include α _shp ,α _tex ,γ _light,k ,γ _view,k , where k = 1,2,..., _NI , _NI is the number of input images. The main network loss functions that need to be minimized include pixel-level loss _Limg (ξ), perceptual identity loss _Lide (ξ) of rendered face images, facial edge loss _Ledge (ξ) and projected landmark loss _Llan (ξ); a pre-trained face recognition network (Arcface) is used to extract perceptual features in the perceptual identity loss calculation.

FIG9 shows an embodiment of the measurement evaluation of different circular reference objects to evaluate the measurement of different circular reference objects. The measurement method can accurately detect coins of different sizes and colors, for example, 1 Hong Kong dollar (diameter: 25.5 mm); 0.5 Hong Kong dollar (diameter: 22.5 mm); 0.1 Hong Kong dollar (diameter: 17.5 mm). Experiments show that the measurement method of the coarse to fine circular reference objects disclosed in the present invention can accurately and reliably detect reference objects with different sizes and colors, such as coins, and the measurement method is accurate, robust, and convenient. As a result, the head reconstruction method disclosed in the present invention has a more consistent facial contour and better reconstruction accuracy.

Figure 10 shows a qualitative comparison of the two methods using circular or square markings in one embodiment. As can be seen from Figure 10, the method of the present disclosure using circular markings has higher accuracy than using square markings.

FIG11 shows an application scenario of head-related products, where the top portion represents a solid color reconstructed head and the bottom portion represents a reconstructed head with texture. The head model generated by the 3D head mesh reconstruction method disclosed in the present invention has higher dimensional accuracy and is more suitable for ergonomic design, such as headwear design, and can be used for personalized customization (personalization), virtual try-on, and size selection of face and head-related products. Application scenarios of head-related products include, for example: (a) headphones; (b) sunglasses; (c) VR glasses; (d) masks; (e) helmets.

FIG12 is a schematic diagram of a head size measuring device in one embodiment of the present disclosure. As shown in FIG12 , the head size measuring device includes:

A facial image acquisition module 1201 is used to acquire a facial image of a target object, wherein the facial image has a circular reference object at the forehead of the target object;

A facial feature point determination module 1202 is used to determine the facial feature points of the target object;

A local area determination module, configured to determine a facial local area including the circular reference object based on the facial feature points;

A reference object advance module 1203, configured to extract the circular reference object from the facial local area; and

The head size determination module 1204 is used to determine the diameter of the circular reference object, and determine the head size of the target object based on the diameter of the circular reference object.

FIG13 is a schematic diagram of a 3D head mesh generation device in one embodiment of the present disclosure. As shown in FIG13 , the 3D head mesh generation device includes:

A face image acquisition module 1301 is used to acquire multiple 2D face images of a target object, wherein the 2D face images include a front face image, a right face side image, and a left face side image, and the front face image has a circular reference object at the forehead position of the target object;

A head shape reconstruction module 1302 is used to reconstruct a 3D head shape of the target object based on the multiple 2D face images;

A head size determination module 1303, configured to determine the head size of the target object based on the front face image;

The head mesh generation module 1304 is used to generate a 3D head mesh having the head size of the target object.

In the solution of the disclosed embodiment, two high-resolution head shape and texture 3DMMs are created. The 3DMM-based 3D head shape reconstruction method uses a CNN-based encoder with higher accuracy. In the CNN-based encoder, a new feature point to edge loss is introduced to ensure the overall reconstructed shape accuracy. The experimental results determine the size and cost-effective number of input face images, which is very useful for practical applications.

In the solution of the embodiment of the present disclosure, a circular reference object is used instead of a square mark. The circular reference object is less sensitive to deformation when attached to the face, and is more stable and easy to measure the head size. In addition, the use of coin-shaped objects is also common in daily life. The accurate head size measurement method based on U-Net is more stable and convenient. The technical solution method of the present disclosure is more suitable for ergonomic design.

It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

The electronic device 1400 according to this embodiment of the present invention is described below with reference to Fig. 14. The electronic device 1400 shown in Fig. 14 is only an example and should not bring any limitation to the functions and application scope of the embodiment of the present invention.

As shown in Fig. 14, the electronic device 1400 is presented in the form of a general-purpose computing device. The components of the electronic device 1400 may include, but are not limited to: at least one processing unit 1410, at least one storage unit 1420, and a bus 1430 connecting different system components (including the storage unit 1420 and the processing unit 1410).

The storage unit stores a program code, which can be executed by the processing unit 1410, so that the processing unit 1410 performs the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of this specification. For example, the processing unit 1410 can perform the steps shown in Figures 2 and 6A.

The storage unit 1420 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 14201 and/or a cache storage unit 14202 , and may further include a read-only storage unit (ROM) 14203 .

The storage unit 1420 may also include a program/utility 14204 having a set (at least one) of program modules 14205, such program modules 14205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

Bus 1430 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

The electronic device 1400 may also communicate with one or more external devices 1500 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device 1500, and/or communicate with any device that enables the electronic device 1400 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input/output (I/O) interface 1450. Furthermore, the electronic device 1400 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) via a network adapter 1460. As shown, the network adapter 1460 communicates with other modules of the electronic device 1400 via a bus 1430. It should be understood that, although not shown in the figure, other hardware and/or software modules may be used in conjunction with the electronic device 1500, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

A program product for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

The program product may be in any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact A compact disk read only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java, C++, etc., and conventional procedural programming languages such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and/or one step may be decomposed into multiple steps, etc.

Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Industrial Applicability

The present disclosure is applicable to the field of artificial intelligence technology, and is used to solve the problem of inaccurate head size determination during 3D head/face reconstruction in related technologies, so as to achieve the effect of accurately determining the head size of a target object.

Claims (21)

A method for measuring head size, comprising: Acquire a facial image of a target object, wherein the facial image has a circular reference object at the forehead position of the target object; Determining facial feature points of the target object; Determine a facial local area including the circular reference object based on the facial feature points; Extracting the circular reference object from the local facial area; The size of the target subject's head is determined based on the size of the circular reference object. The method of claim 1, wherein determining the size of the target object's head based on the size of the circular reference object comprises: determining a diameter of the circular reference object; The target subject's head size is determined based on the diameter of the circular reference object. The method according to claim 1, wherein the facial feature points include eye feature points, and the facial partial image including the circular reference object is a rectangular area including the circular reference object. The method according to claim 1, 2 or 3, wherein extracting the circular reference object from the local facial area comprises: Using an image segmentation artificial neural network to segment the local facial area to obtain the circular reference object; detecting an edge of the circular reference object using an edge detector; The ellipse fitting method is used to perform ellipse measurement on the edge of the circular reference object, thereby extracting the circular reference object. The method according to claim 4, wherein the image segmentation artificial neural network is a U-Net or a fully convolutional network FCN, and/or the edge detector is a canny edge detector. The method according to claim 1, further comprising: The facial image of the target object is dedistorted based on camera parameters of an image acquisition device, wherein the camera parameters include distortion coefficients and an intrinsic matrix, and the distortion coefficients include radial distortion coefficients and tangential distortion coefficients. The method according to claim 6, further comprising: The image acquisition device is used to acquire multi-view images of a preset chessboard at different angles and distances, thereby determining the camera parameters of the image acquisition device. A 3D head mesh generation method, comprising: Acquire multiple 2D facial images of the target object, wherein the 2D facial images include a frontal face image, a right side face image, and a left side face image, and the frontal face image has a circular reference object at the forehead position of the target object; Reconstructing a 3D head shape of the target object based on the multiple 2D face images; Determining the head size of the target object based on the front face image; A 3D head mesh with head dimensions of the target object is generated. The method according to claim 8, wherein determining the head size of the target object based on the frontal face image comprises: The head size measurement method according to any one of claims 1 to 7 determines the head size of the target object based on the frontal face image. The method according to claim 8, wherein reconstructing the 3D head shape of the target object based on the multiple 2D face images comprises: The encoder of the 3DMM based on the head shape and texture is used to reconstruct the 3D head shape of the target object based on the multiple 2D face images to obtain the reconstructed 3D head shape. The method according to claim 10, further comprising: The head shape and texture based 3DMM is pre-trained based on training data. The method according to claim 11, wherein the pre-training of the 3DMM based on head shape and texture based on training data comprises: generating a parameterized head based on the training data using a non-rigid iterative closest point (NICP) algorithm; aligning the parameterized heads to a uniform size using Generic Platts Analysis (GPA); Principal components (PCs) are extracted from all parameterized head meshes using principal component analysis (PCA) to obtain the 3DMM based on head shape and texture. The method according to any one of claims 10 to 12, wherein reconstructing the 3D head shape of the target object based on the multiple 2D face images using an encoder of a 3DMM based on head shape and texture, and obtaining the reconstructed 3D head shape comprises: The multiple 2D face images are input into an artificial neural network encoder to predict 3D head mesh parameters, wherein the 3D head mesh parameters include 3DMM coefficients, lighting parameters and camera parameters of the 3D head mesh, so as to minimize a network loss function, wherein the network loss function includes: pixel-level loss L _img (ξ), perceptual identity loss L _ide (ξ) of the reconstructed face image, facial edge loss L _edge (ξ) and projection feature point loss L _lan (ξ). The method according to claim 13, further comprising: A pre-trained face recognition network is used to extract perceptual features used in the perceptual identity loss computation. A 3D head mesh using method, applied to a 3D head mesh generated by the 3D head mesh generating method according to any one of claims 8 to 14, comprising: The 3D head mesh is used for headwear design. The method according to claim 15, wherein the headwear comprises headphones, glasses, a helmet, a mask, or a hat. A head size measuring device, comprising: A facial image acquisition module, used to acquire a facial image of a target object, wherein the facial image has a circular reference object at the forehead position of the target object; A facial feature point determination module, used to determine the facial feature points of the target object; A local area determination module is used to determine the facial local area including the circular reference object based on the facial feature points. area; A reference object advance module, configured to extract the circular reference object from the facial local area; and The head size determination module is used to determine the diameter of the circular reference object, and determine the head size of the target object based on the diameter of the circular reference object. A 3D head mesh generation device, comprising: A face image acquisition module, used to acquire multiple 2D face images of a target object, wherein the 2D face images include a front face image, a right face side image, and a left face side image, and the front face image has a circular reference object at the forehead position of the target object; A head shape reconstruction module, used for reconstructing a 3D head shape of the target object based on the multiple 2D face images; A head size determination module, used to determine the head size of the target object based on the front face image; The head mesh generation module is used to generate a 3D head mesh with head size of the target object. An electronic device, comprising: Processor; and A memory, configured to store executable instructions of the processor; The processor is configured to execute the head size measurement method described in any one of claims 1 to 7, or the 3D head mesh generation method described in any one of claims 8 to 14 by executing the executable instructions. A computer-readable storage medium having a computer program stored thereon, wherein when the computer program is executed by a processor, the method for measuring head size described in any one of claims 1 to 7 or the method for generating a 3D head mesh described in any one of claims 8 to 14 is implemented. A computer program product, comprising a computer program/instruction, which, when executed by a processor, implements the head size measurement method described in any one of claims 1 to 7, or the 3D head mesh generation method described in any one of claims 8 to 14.