WO2021051562A1 - Facial feature point positioning method and apparatus, computing device, and storage medium - Google Patents

Abstract

A facial feature point positioning method and apparatus, a computing device and a storage medium. Said method comprises: inputting a target facial image of a feature point to be positioned into a first convolutional neural network model, so as to obtain a first shallow feature image which contains a plurality of candidate feature points and is outputted by a predetermined convolutional layer of the first convolutional neural network model (210); using a bilinear interpolation algorithm to perform bilinear interpolation on the candidate feature points in the first shallow feature image, so as to obtain a second shallow feature image (220); and inputting the second shallow feature image into a second convolutional neural network model cascaded with the first convolutional neural network model, so as to obtain a facial feature image which corresponds to the target facial image of the feature point to be positioned and is outputted by the second convolutional neural network model (230), the weight of a predetermined convolutional layer of the second convolutional model and the weight of each convolutional layer before same being consistent with the weight of the convolutional layer in a corresponding layer number of the first convolutional model, respectively. By means said method, the positioning precision of the facial feature point is improved, and since convolution layers of cascaded models share weights, the calculation amount and the parameter amount can be reduced, and the training speed and convergence speed of the models can be improved.

Description

Facial feature point positioning method, device, computing equipment and storage medium

This application is based on and claims the priority of the Chinese patent application filed on September 17, 2019 with the application number CN 201910877995. 2, titled "Face feature point positioning method, device, medium and electronic equipment", the entire content of which is here Incorporated as a reference.

Technical field

This application relates to the technical field of image algorithms, and in particular to a method, device, computing device, and computer non-volatile readable storage medium for locating facial feature points.

Background technique

The convolutional neural network model is one of the important and classic models in the field of image processing. And face image processing is an important subject in the field of image processing.

At present, many face feature point positioning models use convolutional neural network models. Normally, in order to locate the feature points in the face image, people usually input the face image directly into the convolutional neural network model, and use the convolutional layer of the convolutional neural network model to perform facial feature points. Positioning. The inventor of the present application realizes that the accuracy of the adopted facial feature point positioning method is not high enough in actual feature point positioning.

Summary of the invention

In the field of image algorithm technology, in order to solve the above technical problems, the purpose of this application is to provide a method, device, computing device, and computer non-volatile readable storage medium for locating facial feature points.

In the first aspect, a method for locating facial feature points is provided, including:

Input the target face image of the feature points to be located into the first convolutional neural network model, and obtain the first shallow feature containing multiple candidate feature points output by the predetermined layer convolutional layer of the first convolutional neural network model Figure;

Using a bilinear interpolation algorithm to perform bilinear interpolation on candidate feature points in the first shallow feature map to obtain a second shallow feature map;

Input the second shallow feature map to a second convolutional neural network model cascaded with the first convolutional neural network model to obtain the output of the second convolutional neural network model and the feature to be located The face feature map corresponding to the target face image of the point, where each of the predetermined convolutional layers of the second convolutional neural network model and all convolutional layers before the predetermined convolutional layer The weights of the first convolutional neural network model correspond to the weights of the convolutional layers corresponding to the number of layers, and the predetermined convolutional layers of the second convolutional neural network model are in all the second convolutional neural network models. The ordering in the convolutional layer of is consistent with the ordering of the predetermined convolutional layer of the first convolutional neural network model in all the convolutional layers of the first convolutional neural network model.

In the second aspect, a device for locating facial feature points is provided, including:

The first acquisition module is configured to input the target face image of the feature point to be located into the first convolutional neural network model, and obtain the output of the predetermined layer convolutional layer of the first convolutional neural network model, which contains multiple candidates The first shallow feature map of feature points;

An interpolation module, configured to perform bilinear interpolation on candidate feature points in the first shallow feature map by using a bilinear interpolation algorithm to obtain a second shallow feature map;

The second acquisition module is configured to input the second shallow feature map to a second convolutional neural network model cascaded with the first convolutional neural network model to obtain the second convolutional neural network model The output face feature map corresponding to the target face image of the feature point to be located, wherein the predetermined convolutional layer of the second convolutional neural network model and all the volumes before the predetermined convolutional layer The weight of each convolutional layer in the buildup layer is consistent with the weight of the convolutional layer corresponding to the number of layers of the first convolutional neural network model. The predetermined convolutional layer of the second convolutional neural network model is in all the convolutional layers. The ordering in the convolutional layer of the second convolutional neural network model is consistent with the ordering of the predetermined convolutional layer of the first convolutional neural network model in all convolutional layers of the first convolutional neural network model .

In a third aspect, a computing device is provided, including a memory and a processor, the memory is configured to store a program for locating facial feature points of the processor, and the processor is configured to execute the facial feature points via The positioning program performs the following processing: input the target face image of the feature points to be located into the first convolutional neural network model, and obtain the predetermined layer convolutional layer output of the first convolutional neural network model containing multiple candidates A first shallow feature map of feature points; bilinear interpolation is performed on candidate feature points in the first shallow feature map using a bilinear interpolation algorithm to obtain a second shallow feature map; The layer feature map is input to the second convolutional neural network model cascaded with the first convolutional neural network model to obtain the target face image output by the second convolutional neural network model and the feature points to be located Corresponding face feature map, wherein the weight of each convolutional layer in the predetermined layer of the second convolutional neural network model and all convolutional layers before the predetermined layer of convolutional layer is the same as the weight of each convolutional layer. The weights of the convolutional layers corresponding to the number of layers of the first convolutional neural network model are the same, and the predetermined convolutional layers of the second convolutional neural network model are among all the convolutional layers of the second convolutional neural network model. The ordering is consistent with the ordering of the predetermined convolutional layers of the first convolutional neural network model in all convolutional layers of the first convolutional neural network model.

In a fourth aspect, a computer non-volatile readable storage medium storing computer readable instructions is provided, and a program for locating facial feature points is stored thereon. When the program for locating facial feature points is executed by a processor, there is provided The following processing is implemented: input the target face image of the feature points to be located into the first convolutional neural network model, and obtain the first convolutional layer output of the predetermined layer of the first convolutional neural network model that contains multiple candidate feature points A shallow feature map; using a bilinear interpolation algorithm to perform bilinear interpolation on candidate feature points in the first shallow feature map to obtain a second shallow feature map; input the second shallow feature map To the second convolutional neural network model cascaded with the first convolutional neural network model, to obtain the face corresponding to the target face image of the feature point to be located, which is output by the second convolutional neural network model A feature map, wherein the weight of each convolutional layer in the predetermined convolutional layer of the second convolutional neural network model and all convolutional layers before the predetermined convolutional layer is the same as that of the first convolutional layer. The weights of the convolutional layers corresponding to the number of layers of the neural network model are the same, and the order of the predetermined convolutional layers of the second convolutional neural network model in all convolutional layers of the second convolutional neural network model is the same as that of the The order of the predetermined convolutional layers of the first convolutional neural network model in all the convolutional layers of the first convolutional neural network model is consistent.

The above-mentioned facial feature point positioning method, device, computing device and computer non-volatile readable storage medium, firstly locate the facial feature point through the cascade of two convolutional neural network models, and use the first convolution The neural network model removes the background and other interference factors, and then inputs the output of the first convolutional neural network model to the second convolutional neural network model, which can improve the final positioning accuracy of facial feature points. On this basis, Before the output of the first convolutional neural network model is input to the second convolutional neural network model, bilinear interpolation is performed on the feature map output by the first convolutional neural network model by using a bilinear interpolation algorithm, so that the input is input to the second convolutional neural network model. The feature map of the convolutional neural network model can accurately reflect the location of the feature points in the original image, making the positioning accuracy of the facial feature points more accurate. In addition, because the convolutional layers of the two convolutional neural network models realize Weight sharing. During model training, the second convolutional neural network model does not need to relearn from the original input picture according to the input, which reduces the amount of parameter calculation and accelerates the training speed and convergence speed of the overall model.

It should be understood that the above general description and the following detailed description are only exemplary and cannot limit the application.

Description of the drawings

Fig. 1 is a schematic diagram showing an application scenario of a method for locating facial feature points according to an exemplary embodiment;

Fig. 2 is a flow chart showing a method for locating facial feature points according to an exemplary embodiment;

Fig. 3 is a schematic structural diagram of a cascaded convolutional neural network model used in a method for locating facial feature points according to an exemplary embodiment;

FIG. 4 is a detailed flowchart of step 220 of an embodiment shown according to the embodiment corresponding to FIG. 2;

FIG. 5 is a detailed flowchart of step 223 of an embodiment shown according to the embodiment corresponding to FIG. 4;

Fig. 6 is a block diagram showing a device for locating facial feature points according to an exemplary embodiment;

Fig. 7 is an exemplary block diagram of a computing device that implements the aforementioned method for locating facial feature points according to an exemplary embodiment;

Fig. 8 is a computer non-volatile readable storage medium for realizing the above-mentioned method for locating facial feature points according to an exemplary embodiment.

detailed description

The exemplary embodiments will be described in detail here, and examples thereof are shown in the accompanying drawings. When the following description refers to the drawings, unless otherwise indicated, the same numbers in different drawings indicate the same or similar elements. The implementation manners described in the following exemplary embodiments do not represent all implementation manners consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the application as detailed in the appended claims.

In addition, the drawings are only schematic illustrations of the application, and are not necessarily drawn to scale. The same reference numerals in the figures denote the same or similar parts, and thus their repeated description will be omitted. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities.

This application first provides a method for locating facial feature points. A human face refers to a human face in an electronic image. The electronic image here can be either a photo or a picture, or a frame in a video. Generally, an electronic image containing a human face includes multiple pixels. Feature points refer to pixels or areas containing pixel points that can represent the feature information or characteristics of a human face. Because facial feature points are the most representative of human faces, facial feature points can also be called face key points. Facial feature point positioning refers to the process of determining the feature points or key points of a human face in an electronic image containing a human face. By using the facial feature point positioning method provided by the present application, the feature points of the face in the image containing the face can be efficiently and accurately located, and the training process of the entire model is more efficient.

The implementation terminal of this application can be any device with computing and processing functions. The device can be connected to an external device to receive or send data. Specifically, it can be a portable mobile device, such as a smart phone, a tablet computer, a notebook computer, or a PDA ( Personal Digital Assistant), etc., can also be fixed devices, such as computer equipment, field terminals, desktop computers, servers, workstations, etc., or a collection of multiple devices, such as the physical infrastructure of cloud computing.

Optionally, the implementation terminal of the present application may be a server, a physical infrastructure of cloud computing, or a computer device with a high-performance graphics card.

Fig. 1 is a schematic diagram showing an application scenario of a method for locating facial feature points according to an exemplary embodiment. As shown in FIG. 1, it includes a server 110, a database 120, and a user terminal 130. The database 120 and the user terminal 130 are respectively connected to the server 110 through a communication link. In this embodiment, the server 110 is the implementation terminal of the application. The user terminal 130 can also be any device with computing and processing functions. It can be the same type of terminal as the implementation terminal of the application, or a different type of terminal, and can be the same terminal as the implementation terminal of the application, or it can be the same terminal as the implementation terminal of the application. For different terminals. It is necessary to use the cascaded convolutional neural network model used in this application to locate the facial feature points. The untrained cascaded convolutional neural network model will first be embedded in the server 110, and a large number of them are stored in the database 120 in advance. The image samples accurately labeled with facial feature points are input into the cascaded convolutional neural network model embedded in the server 110, and the cascaded convolutional neural network model can be trained. When the convolutional neural network model is trained, it can be used to locate the facial feature points. When the user needs to locate the facial feature points of a face in an electronic image, the user terminal 130 can use the user terminal 130 to send the electronic image for facial feature point positioning to the server 110, and the server 110 uses the trained cascade The convolutional neural network model can output an electronic image containing the located facial feature points and return it to the user terminal 130.

It is worth mentioning that although in this embodiment, the implementation terminal of this application is a server, and the cascaded convolutional neural network model is fixed in the implementation terminal of this application-the server, in other embodiments or In specific applications, various terminals can be selected as the implementation terminals of this application as required, and the cascaded convolutional neural network model can be fixed on any two identical or different terminals including the implementation terminals of this application. This application does not make any limitation on this, and the protection scope of this application should not be restricted in any way.

Fig. 2 is a flow chart showing a method for locating facial feature points according to an exemplary embodiment. As shown in Figure 2, the following steps can be included:

Step 210: Input the target face image of the feature points to be located into the first convolutional neural network model, and obtain the first convolutional layer output of the predetermined layer of the first convolutional neural network model that contains multiple candidate feature points. Shallow feature map.

The target face image is the image that requires feature point positioning. It can be an electronic image in various forms or formats. For example, it can be a picture, photo, or a frame in a video file generated in any way, which can be .jpg,. Various formats such as jpeg, .png, .bmp, etc.

The first convolutional neural network model includes multiple convolutional layers, and each convolutional layer is constructed in a stacked manner. The predetermined convolutional layer may be any convolutional layer except the first convolutional layer in the multi-layer convolutional layer of the first convolutional neural network model. The convolutional layer is used to extract the features in the target face image. The more convolutional layers a convolutional neural network model contains, the deeper the feature map can be extracted. Each convolutional layer can contain a convolution kernel. The size of the convolution kernel can be arbitrary, for example, it can be 3×3, 5×5, and so on.

In one embodiment, the first convolutional neural network model includes a pooling layer in addition to a convolutional layer. The pooling layer can be used to compress the feature map. The method of processing the feature map by the pooling layer can be average pooling or maximum pooling. Average pooling is to use the average of the pixel values of each pixel in an area to represent the entire area, and Maximum pooling is to use the maximum value of each pixel in an area to represent the entire area.

In one embodiment, before each convolutional layer of the first convolutional neural network model, a pooling layer is included.

In one embodiment, after each convolutional layer before the last convolutional layer of the first convolutional neural network model, a pooling layer is included.

The candidate feature points are the points in the first shallow feature map output by the first convolutional neural network model, and are feature points obtained by roughly extracting the target face image.

Step 220: Perform bilinear interpolation on candidate feature points in the first shallow feature map by using a bilinear interpolation algorithm to obtain a second shallow feature map.

The bilinear interpolation algorithm can realize the transformation of the size of the image by performing linear interpolation in the two directions of the abscissa and the ordinate respectively. For example, the first shallow feature map of the first size can be transformed into the second size. The second shallow feature map. The bilinear interpolation algorithm can convert the first shallow feature map to the second shallow feature map, and at the same time, the second shallow feature map obtained by interpolation can accurately reflect the candidate feature points in the first shallow feature map The corresponding relationship between the position and the pixel value.

In an embodiment, the specific implementation steps of step 220 may be as shown in FIG. 4. FIG. 4 is a detailed flowchart of step 220 of an embodiment shown according to the embodiment corresponding to FIG. 2. As shown in FIG. 4, step 220 may specifically include the following sub-steps:

Step 221: For each candidate feature point in the first shallow feature map, obtain the target side length of the square area to be determined with the candidate feature point as the coordinate center.

In an embodiment, the target side length obtained for each candidate feature point in the first shallow feature map is the same. The advantage of this is that each point in the second shallow feature map obtained by interpolation is obtained according to the same bilinear interpolation method, which ensures the smoothness of interpolation.

In an embodiment, the target side length obtained for each candidate feature point in the first shallow feature map is obtained in a predetermined order.

For example, a target side length sequence table is set in advance, and each target side length to be acquired for each candidate feature point in the first shallow feature map is stored, and each target side length is sorted in a predetermined order in the sequence table , First sort each candidate feature point in the first shallow feature map in the order from left to right and top to bottom; and then obtain the target side length for each candidate feature point according to the sort. Specifically, starting from the first candidate feature point in the first shallow feature map, for each candidate feature point, according to the order of each candidate feature point, according to the predetermined target side lengths that are not marked as acquired. Obtain a target side length in sequence, use the obtained target side length as the target side length for the corresponding candidate feature point, and mark the target side length as selected, until each target side length is selected in the first shallow feature map. The candidate feature points have all acquired the target side length, and if all the target side lengths are marked as selected when the target side length is acquired for the candidate feature points, the marking of all target side lengths will be cancelled.

In an embodiment, for each candidate feature point in the first shallow feature map, obtaining the target side length of a square area to be determined with the candidate feature point as a coordinate center includes:

Taking each candidate feature point in the first shallow feature map as a coordinate center, acquiring a circular area with a preset first side length in the first shallow feature map;

For each candidate feature point in the first shallow feature map, determine the variance of each pixel point in the circular area corresponding to the candidate feature point;

In the case that the variance is greater than or equal to the preset variance threshold, use the preset first side length as the target side length of the square area to be determined with the candidate feature point as the coordinate center;

In the case that the variance is less than the preset variance threshold, the preset second side length is used as the target side length of the square area to be determined with the candidate feature point as the coordinate center, wherein the preset second side length The side length is smaller than the preset first side length.

In general, the closer the pixel point is, the smaller the pixel value difference is. Therefore, if the pixel value variance of the pixel point in the circular area with a candidate feature point as the coordinate center is larger, it means that the pixel value in the circular area is The pixel value changes more drastically, so the advantage of this embodiment is that when the variance of the pixel values of the pixel points around the candidate feature point is large enough, the smaller side length is selected as the coordinate center of the candidate feature point. Determine the target side length of the square area, so that the pixel value obtained by interpolation can be more similar to the pixel value of the corresponding feature point, and to a certain extent, the obtained second shallow feature map can more accurately reflect the target face image. feature.

Step 222: For each candidate feature point in the first shallow feature map, a square area is determined in the first shallow feature map with the candidate feature point as a coordinate center.

Wherein, the side length of the square area is the target side length obtained for the candidate feature point.

In an embodiment, if the coordinates of the candidate feature point in the first shallow feature map are (x, y), and the target side length corresponding to the candidate feature point is d, then the candidate feature point is The square area where the side length of the center is the target side length corresponding to the candidate feature point is a square surrounded by the images of the following function group:

Among them, f(x) _x indicates that the dependent variable of the function is the abscissa, and f(y) _y indicates that the dependent variable of the image of the function is the ordinate.

Step 223: Obtain the coordinate values of the four vertices of the square region determined for each candidate feature point and the pixel value at each vertex.

In an embodiment, the obtaining the coordinate values of the four vertices of the square region determined for each candidate feature point and the pixel value at each vertex includes:

Acquiring the coordinate values of the four vertices of the square area determined for each candidate feature point;

Determine whether each vertex is located in the first shallow feature map according to the acquired coordinate value of each vertex;

When at least one of the four vertices of the square area determined for each candidate feature point is not located in the first shallow feature map, the four vertices of the square area determined for the candidate feature point are not located The coordinate value of the vertex in the first shallow feature map is replaced with the coordinate value of any one of the four vertices of the square region determined for each candidate feature point located in the first shallow feature map;

For the coordinate values of the four vertices of the square region determined by each candidate feature point, the pixel value of each vertex is obtained.

Step 224: For each candidate feature point, based on the coordinates of the vertex corresponding to the candidate feature point and the pixel value at each vertex, use the following formula to obtain the pixel value of each pixel that constitutes the second shallow feature map, To get the second shallow feature map:

Among them, (x, y) is the coordinate value of the pixel point corresponding to the feature point in the second shallow feature map obtained for the candidate feature point, (x ₁ , y ₁ ), (x ₂ , y ₁ ), (x ₁ ,y ₂ ) and (x ₂ ,y ₂ ) are the coordinate values of the four vertices of the square area corresponding to the candidate feature point, f(x ₁ ,y ₁ ), f(x ₂ ,y ₁ ) , F(x ₁ , y ₂ ) and f(x ₂ , y ₂ ) are the pixel values at the four vertices of the square region corresponding to the candidate feature point in the first shallow feature map.

In one embodiment, the pixel value of each point is a gray value.

In one embodiment, the pixel value of each point is an RGB value.

Step 230: Input the second shallow feature map to a second convolutional neural network model cascaded with the first convolutional neural network model to obtain the output of the second convolutional neural network model and the The face feature map corresponding to the target face image of the feature point to be located.

Wherein, the weight of each convolutional layer in the predetermined convolutional layer of the second convolutional neural network model and all convolutional layers before the predetermined convolutional layer is the same as that of the first convolutional neural network model. The weights of the convolutional layers corresponding to the number of layers are the same, and the order of the predetermined convolutional layers of the second convolutional neural network model in all convolutional layers of the second convolutional neural network model is the same as that of the first convolutional layer. The order of the predetermined convolutional layers of the convolutional neural network model in all the convolutional layers of the first convolutional neural network model is consistent.

The number of convolutional layers included in the second convolutional neural network model is greater than or equal to the number of all convolutional layers before the predetermined convolutional layer of the first convolutional neural network model plus 1, that is, the second The number of convolutional layers included in the convolutional neural network model is greater than or equal to the sum of the number of convolutional layers of the predetermined convolutional layer of the first convolutional neural network model and all convolutional layers before the predetermined convolutional layer , The second convolutional neural network model contains the convolutional layers of each layer in a stacked arrangement.

Since the number of convolutional layers contained in the second convolutional neural network model is greater than or equal to the sum of the number of convolutional layers of the predetermined layer and all convolutional layers before the predetermined layer of convolutional layer, and the second convolutional layer The order of the predetermined convolutional layer of the convolutional neural network model in all the convolutional layers of the second convolutional neural network model and the predetermined convolutional layer of the first convolutional neural network model in all the first convolutional layers The ordering in the convolutional layers of the convolutional neural network model is consistent, so for the predetermined convolutional layer of the second convolutional neural network model and each convolutional layer before the predetermined convolutional layer, in the The first convolutional neural network model has convolutional layers with the same number of layers as the convolutional layer, so that the weights of the convolutional layers corresponding to the number of layers in the two convolutional neural network models can be consistent.

In one embodiment, in addition to the convolutional layer, the second convolutional neural network also includes at least one pooling layer and at least one fully connected layer.

The cascading of the second convolutional neural network model and the first convolutional neural network model refers to directly using the output of the first convolutional neural network model as the input of the second convolutional neural network model.

Since the second convolutional neural network model is the same as the first convolutional neural network model, it contains multiple convolutional layers, so the second convolutional neural network model can output the output of the first convolutional neural network model. The feature map is further extracted to obtain a more refined facial feature map.

The weight of the convolutional layer refers to the parameters used when the convolutional layer performs arithmetic processing on the feature map.

In one embodiment, when the cascaded convolutional neural network model is established, the back-propagation algorithm is used to train the model, and the training process is the process of determining the parameters of the model including the weight of the convolutional layer.

In one embodiment, the weight of the convolution layer is the weight matrix on the convolution kernel used when extracting the features of the feature map.

By sharing the weights of the convolutional layers of the first convolutional neural network model and the second convolutional neural network model, efficient training of the cascaded convolutional neural network model used in the facial feature point positioning method can be realized: on the one hand, By sharing the weights of the convolutional layers of the two cascaded convolutional neural network models, the amount of parameters and calculations can be reduced; on the other hand, when the second convolutional neural network model is to be used to further the second shallow feature map When extracting features, you can directly use the abstract semantic features that the first convolutional neural network model has learned, and the second convolutional model does not need to relearn semantic features from the original input image level, which speeds up the training speed and convergence of the overall model speed.

Fig. 3 is a schematic structural diagram of a cascaded convolutional neural network model used in a method for locating facial feature points according to an exemplary embodiment. As shown in FIG. 3, it includes an input target face image 310, a first convolutional neural network model 320, and a second convolutional neural network model 340 cascaded with the first convolutional neural network model, where the first convolutional neural network model The network model 320 includes a multi-layer convolutional layer 330, and the second convolutional neural network model 340 includes a multi-layer convolutional layer 350 and a multi-layer fully connected layer 360, the first convolutional neural network model 320 and the second convolutional neural network The models 340 together form a cascaded convolutional neural network model. When using the cascaded convolutional neural network model to locate the facial feature points, the specific process is as follows: after the target face image 310 is input to the first convolutional neural network model 320, the first convolutional neural network model The multi-layer convolutional layer 330 in 320 can realize the preliminary rough extraction of facial feature points in the target face image 310, and output the first shallow feature map; the implementation terminal of this application can double the first shallow feature map. Linear interpolation is used to obtain the second shallow feature map, and then feed the second shallow feature map to the second convolutional neural network model 340; after the second shallow feature map is input to the second convolutional neural network model 340, The second convolutional neural network model 340 uses the multi-layer convolutional layer 350 and the multi-layer fully connected layer 360 that it includes to perform further fine feature extraction on the second shallow feature map, and finally obtains the corresponding target face image Face feature map.

It is worth mentioning that Fig. 3 is only an embodiment of the present application. Although in the embodiment of FIG. 3, the first convolutional neural network model only includes the convolutional layer, and the second convolutional neural network model only includes the convolutional layer and the fully connected layer, in practical applications, they can be separately One or more pooling layers are added to the first convolutional neural network model and the second convolutional neural network model. Therefore, the various embodiments of this application compare the levels used in the facial feature point positioning method provided by this application. The specific structure of the convolutional neural network model is not limited in any way, and the scope of protection of this application should not be restricted in any way.

In summary, according to the facial feature point locating method provided in the embodiment in FIG. 2, by using two cascaded convolutional neural network models to extract feature maps, more accurate facial feature points are located. On this basis, by using the bilinear interpolation algorithm to perform bilinear interpolation on the first shallow feature map to be input to the second convolutional neural network model, the accuracy of the acquired feature map can be further improved. At the same time, due to the two The convolutional layer of the convolutional neural network model shares weights, which can reduce the amount of calculation and the amount of parameters. When the second convolutional neural network model receives the second shallow feature map input to the model, the second convolutional neural network model does not need Learning the semantic features of the image from the input level of the original image can speed up the training speed of the model and make the loss function converge quickly.

FIG. 5 is a detailed flowchart of step 223 of an embodiment shown according to the embodiment corresponding to FIG. 4. In the embodiment of FIG. 5, the target side length obtained for each candidate feature point is the same preset side length. As shown in Figure 5, the following steps can be included:

Step 2231: For each candidate feature point, use the following expressions to obtain the coordinate values of the four vertices of the square area determined for the candidate feature point:

(x ₃ -r,y ₃ -r),(x ₃ -r,y ₃ +r),(x ₃ +r,y ₃ -r),(x ₃ +r,y ₃ +r),

Wherein, (x ₃ , y ₃ ) is the coordinate value of the candidate feature point, and r is one half of the preset side length.

Step 2232: For each candidate feature point, determine whether the coordinate value of each of the four vertices of the square region determined for the candidate feature point is located in the first shallow feature map.

In an embodiment, it is achieved by comparing the coordinate value of each of the four vertices of the square region determined for the candidate feature point with the coordinate values of all the pixels in the first shallow feature map. Determine whether the coordinate value of each of the four vertices of the square area determined for the candidate feature point is located in the first shallow feature map.

Step 2233, if yes, obtain the pixel value of the corresponding vertex in the first shallow feature map according to the coordinate value.

When the coordinate value of each of the four vertices of the square area determined for a candidate feature point is located in the first shallow feature map, it means that the pixel value at the coordinate value of each vertex can be obtained.

Step 2234, if not, for the vertices located in the first shallow feature map among the four vertices of the square region determined for the candidate feature point, the vertices located in the first shallow layer feature map according to the coordinate value corresponding to each vertex are set in the first shallow layer. Get the pixel value of the corresponding vertex in the feature map.

When the coordinate value of each of the four vertices of the square region determined for the candidate feature point is not all located in the first shallow feature map, it is also possible that the four vertices of the square region determined for the candidate feature point There is at least one vertex located in the first shallow feature map.

Step 2235: Obtain the vertices outside the first shallow feature map among the four vertices of the square region determined for the candidate feature point as auxiliary vertices.

Step 2236: For each auxiliary vertex, obtain the pixel value of the pixel point closest to the auxiliary vertex in the first shallow feature map as the pixel value of the auxiliary vertex.

The advantage of this embodiment is that it provides a solution when the four vertices in the square area are not located in the first shallow feature map. At the same time, because the distance between the two pixels in the same feature map is greater Nearly, the pixel values corresponding to the two pixels are more likely to be similar, so this embodiment can also ensure the accuracy of the obtained pixel values of the vertices, thereby improving the accuracy of the obtained second shallow feature map.

In an embodiment, for each auxiliary vertex, acquiring the pixel value at the pixel point closest to the auxiliary vertex in the first shallow feature map as the pixel value of the auxiliary vertex includes:

For each auxiliary vertex, for each pixel in the first shallow feature map, use the following formula to determine the distance between the pixel and the auxiliary vertex:

Where x ₄ and y ₄ are the abscissa and ordinate of the auxiliary vertex, respectively, x, y are the abscissa and ordinate of the pixel in the first shallow feature map, and D is the auxiliary vertex and the first The distance between pixels in the shallow feature map; for each auxiliary vertex, the smallest distance between each pixel in the first shallow feature map and the auxiliary vertex acquired for the auxiliary vertex is obtained The distance; the pixel value at the pixel point corresponding to the smallest distance is used as the pixel value of the auxiliary vertex.

In one embodiment, for each auxiliary vertex, acquiring the smallest distance among the distances between each pixel point in the first shallow feature map and the auxiliary vertex acquired for the auxiliary vertex includes: For each auxiliary vertex, any one of the distances between each pixel in the first shallow feature map obtained for the auxiliary vertex and the auxiliary vertex is selected and marked as the candidate minimum distance; in addition to the candidate minimum In the distance between each pixel point outside the distance and the auxiliary vertex, determine whether the distance between the pixel point and the auxiliary vertex is less than the candidate minimum distance according to the distance between each pixel point and the auxiliary vertex in a predetermined order; if Yes, cancel the mark of all candidate minimum distances, and mark the distance between the pixel and the auxiliary vertex as the candidate minimum distance; in the distance between the unobtained pixel and the auxiliary vertex, again for each pixel in a predetermined order Determine whether the distance between the pixel and the auxiliary vertex is less than the candidate minimum distance, until there is no pixel and the auxiliary vertex is less than the candidate minimum distance; take the candidate minimum distance as the minimum the distance.

In one embodiment, for each auxiliary vertex, for each pixel in the first shallow feature map, the following formula is used to determine the distance between the pixel and the auxiliary vertex, the method further include:

For each auxiliary vertex, from the pixel points of the first shallow feature map, obtain a pixel whose abscissa difference from the auxiliary vertex is less than a predetermined coordinate difference and whose ordinate difference is less than a predetermined coordinate difference, as a candidate pixel;

For each auxiliary vertex, for each pixel in the first shallow feature map, using the following formula to determine the distance between the pixel and the auxiliary vertex includes:

For each auxiliary vertex, for each candidate pixel in the first shallow feature map, the following formula is used to determine the distance between the candidate pixel and the auxiliary vertex:

Where x ₄ and y ₄ are the abscissa and ordinate of the auxiliary vertex, respectively, x, y are the abscissa and ordinate of the candidate pixel in the first shallow feature map, and D is the auxiliary vertex and the first The distance between candidate pixels in a shallow feature map.

Since the calculation complexity of obtaining the distance between the pixel and the auxiliary vertex is relatively high and consumes a lot of resources, the advantage of this embodiment is that by first obtaining the candidate pixel, and then obtaining the distance between the candidate pixel and the auxiliary vertex The distance between the pixel point and the auxiliary vertex greatly reduces the calculation task when obtaining the distance between the pixel point and the auxiliary vertex, so that the efficiency of obtaining the minimum distance can be improved.

The present application also provides a device for locating facial feature points. The following are device embodiments of the present application.

Fig. 6 is a block diagram showing a device for locating facial feature points according to an exemplary embodiment. As shown in FIG. 6, the apparatus 600 includes:

The first acquisition module 610 is configured to input the target face image of the feature points to be located into the first convolutional neural network model, and obtain the predetermined layer convolutional layer output of the first convolutional neural network model. The first shallow feature map of candidate feature points.

The interpolation module 620 is configured to perform bilinear interpolation on candidate feature points in the first shallow feature map by using a bilinear interpolation algorithm to obtain a second shallow feature map.

The second acquisition module 630 is configured to input the second shallow feature map to a second convolutional neural network model cascaded with the first convolutional neural network model to obtain the second convolutional neural network The face feature map output by the model corresponding to the target face image of the feature point to be located, wherein the predetermined convolutional layer of the second convolutional neural network model and all preceding convolutional layers of the predetermined convolutional layer The weight of each convolutional layer in the convolutional layer is the same as the weight of the corresponding convolutional layer of the first convolutional neural network model. The predetermined convolutional layer of the second convolutional neural network model is in all the convolutional layers. The ordering in the convolutional layer of the second convolutional neural network model and the ordering of the predetermined convolutional layer of the first convolutional neural network model in all the convolutional layers of the first convolutional neural network model Unanimous.

In an embodiment, the interpolation module is further configured to:

For each candidate feature point in the first shallow feature map, obtain the target side length of a square area to be determined with the candidate feature point as a coordinate center;

For each candidate feature point in the first shallow feature map, a square area is determined in the first shallow feature map with the candidate feature point as the coordinate center, wherein the side length of the square area is for The target side length obtained by the candidate feature point;

Acquiring the coordinate values of the four vertices of the square area determined for each candidate feature point and the pixel value at each vertex;

For each candidate feature point, based on the coordinates of the vertex corresponding to the candidate feature point and the pixel value at each vertex, the following formula is used to obtain the pixel value of each pixel that constitutes the second shallow feature map to obtain the first Two shallow feature maps:

Among them, (x, y) is the coordinate value of the pixel point corresponding to the feature point in the second shallow feature map obtained for the candidate feature point, (x ₁ , y ₁ ), (x ₂ , y ₁ ), (x ₁ ,y ₂ ) and (x ₂ ,y ₂ ) are the coordinate values of the four vertices of the square area corresponding to the candidate feature point, f(x ₁ ,y ₁ ), f(x ₂ ,y ₁ ) , F(x ₁ , y ₂ ) and f(x ₂ , y ₂ ) are the pixel values at the four vertices of the square region corresponding to the candidate feature point in the first shallow feature map.

In one embodiment, the target side length obtained for each candidate feature point is the same preset side length, and the obtained coordinate values of the four vertices of the square area determined for each candidate feature point and each The pixel value at the vertex, including:

For each candidate feature point, the following expressions are used to obtain the coordinate values of the four vertices of the square area determined for the candidate feature point:

(x ₃ -r,y ₃ -r),(x ₃ -r,y ₃ +r),(x ₃ +r,y ₃ -r),(x ₃ +r,y ₃ +r),

Wherein, (x ₃ , y ₃ ) is the coordinate value of the candidate feature point, and r is one half of the preset side length;

For each candidate feature point, determine whether the coordinate value of each of the four vertices of the square area determined for the candidate feature point is located in the first shallow feature map;

If yes, obtain the pixel value of the corresponding vertex in the first shallow feature map according to the coordinate value;

If not, for the vertices located in the first shallow feature map among the four vertices of the square area determined for the candidate feature point, in the first shallow feature map according to the coordinate value corresponding to each vertex Obtain the pixel value of the corresponding vertex;

Acquiring a vertex outside the first shallow feature map among the four vertices of the square region determined for the candidate feature point as an auxiliary vertex;

For each auxiliary vertex, the pixel value at the pixel point closest to the auxiliary vertex is acquired in the first shallow feature map, as the pixel value of the auxiliary vertex.

In an embodiment, for each auxiliary vertex, obtaining the pixel value at the pixel point closest to the auxiliary vertex in the first shallow feature map as the pixel value of the auxiliary vertex includes:

For each auxiliary vertex, for each pixel in the first shallow feature map, use the following formula to determine the distance between the pixel and the auxiliary vertex:

Where x ₄ and y ₄ are the abscissa and ordinate of the auxiliary vertex, respectively, x, y are the abscissa and ordinate of the pixel in the first shallow feature map, and D is the auxiliary vertex and the first The distance between pixels in the shallow feature map;

For each auxiliary vertex, obtain the smallest distance among the distances between each pixel point in the first shallow feature map obtained for the auxiliary vertex and the auxiliary vertex;

The pixel value at the pixel point corresponding to the smallest distance is used as the pixel value of the auxiliary vertex.

In one embodiment, for each auxiliary vertex, acquiring the smallest distance among the distances between each pixel point in the first shallow feature map and the auxiliary vertex acquired for the auxiliary vertex includes:

For each auxiliary vertex, any one of the distances between each pixel in the first shallow feature map obtained for the auxiliary vertex and the auxiliary vertex is selected, and it is marked as a candidate minimum distance;

In the distance between each pixel point and the auxiliary vertex except the candidate minimum distance, determine whether the distance between the pixel point and the auxiliary vertex is less than the distance between each pixel point and the auxiliary vertex according to a predetermined sequence. The candidate minimum distance;

If yes, cancel the mark of all candidate minimum distances, and mark the distance between the pixel and the auxiliary vertex as the candidate minimum distance;

In the distance between the unacquired pixel and the auxiliary vertex, the distance between each pixel and the auxiliary vertex is again determined in a predetermined order to determine whether the distance between the pixel and the auxiliary vertex is less than the candidate minimum distance, until The distance between no pixel and the auxiliary vertex is less than the candidate minimum distance;

The candidate minimum distance is taken as the minimum distance.

In one embodiment, for each auxiliary vertex, for each pixel in the first shallow feature map, before determining the distance between the pixel and the auxiliary vertex using the following formula, the method further includes:

For each auxiliary vertex, from the pixel points of the first shallow feature map, obtain a pixel whose abscissa difference from the auxiliary vertex is less than a predetermined coordinate difference and whose ordinate difference is less than a predetermined coordinate difference, as a candidate pixel;

For each auxiliary vertex, for each pixel in the first shallow feature map, using the following formula to determine the distance between the pixel and the auxiliary vertex includes:

For each auxiliary vertex, for each candidate pixel in the first shallow feature map, the following formula is used to determine the distance between the candidate pixel and the auxiliary vertex.

In an embodiment, for each candidate feature point in the first shallow feature map, obtaining the target side length of a square area to be determined with the candidate feature point as a coordinate center includes:

Taking each candidate feature point in the first shallow feature map as a coordinate center, acquiring a circular area with a preset first side length in the first shallow feature map;

For each candidate feature point in the first shallow feature map, determine the variance of each pixel point in the circular area corresponding to the candidate feature point;

In the case that the variance is greater than or equal to the preset variance threshold, use the preset first side length as the target side length of the square area to be determined with the candidate feature point as the coordinate center;

In the case that the variance is less than the preset variance threshold, the preset second side length is used as the target side length of the square area to be determined with the candidate feature point as the coordinate center, wherein the preset second side length The side length is smaller than the preset first side length.

According to the third aspect of the present application, there is also provided a computing device that executes all or part of the steps of any one of the above-mentioned facial feature point positioning methods. The computing equipment includes:

At least one processor; and

A memory communicatively connected with the at least one processor; wherein,

The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor, so that the at least one processor can execute as shown in any of the above exemplary embodiments. The method of locating facial feature points.

Those skilled in the art can understand that various aspects of the present application can be implemented as a system, a method, or a program product. Therefore, each aspect of the present application can be specifically implemented in the following forms, namely: complete hardware implementation, complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software implementations, which can be collectively referred to herein as "Circuit", "Module" or "System".

The computing device 700 according to this embodiment of the present application will be described below with reference to FIG. 7. The computing device 700 shown in FIG. 7 is only an example, and should not bring any limitation to the function and use scope of the embodiments of the present application.

As shown in FIG. 7, the computing device 700 is in the form of a general-purpose computing device. The components of the computing device 700 may include, but are not limited to: the aforementioned at least one processing unit 710, the aforementioned at least one storage unit 720, and a bus 730 connecting different system components (including the storage unit 720 and the processing unit 710).

Wherein, the storage unit stores a program code, and the program code can be executed by the processing unit 710, so that the processing unit 710 executes the various exemplary methods described in the above-mentioned "Embodiment Method" section of this specification. Steps of implementation.

The storage unit 720 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 721 and/or a cache storage unit 722, and may further include a read-only storage unit (ROM) 723.

The storage unit 720 may also include a program/utility tool 724 having a set of (at least one) program module 725. Such program module 725 includes but is not limited to: an operating system, one or more application programs, other program modules, and program data, Each of these examples or some combination may include the implementation of a network environment.

The bus 730 may represent one or more of several types of bus structures, including a storage unit bus or a storage unit controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local area using any bus structure among multiple bus structures. bus.

The computing device 700 may also communicate with one or more external devices 900 (such as keyboards, pointing devices, Bluetooth devices, etc.), and may also communicate with one or more devices that enable a user to interact with the computing device 700, and/or communicate with Any device (eg, router, modem, etc.) that enables the computing device 700 to communicate with one or more other computing devices. This communication can be performed through an input/output (I/O) interface 750. In addition, the computing device 700 may also communicate with one or more networks (for example, a local area network (LAN), a wide area network (WAN), and/or a public network, such as the Internet) through the network adapter 760. As shown in the figure, the network adapter 760 communicates with other modules of the computing device 700 through the bus 730. It should be understood that although not shown in the figure, other hardware and/or software modules can be used in conjunction with the computing device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives And data backup storage system, etc.

Through the description of the above embodiments, those skilled in the art can easily understand that the example embodiments described here can be implemented by software, or can be implemented by combining software with necessary hardware. Therefore, the technical solution according to the embodiments of the present application can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, U disk, mobile hard disk, etc.) or on the network , Including several instructions to make a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) execute the method according to the embodiment of the present application.

According to the fourth aspect of the present application, there is also provided a computer non-volatile readable storage medium on which is stored a program product capable of implementing the above-mentioned method in this specification. In some possible implementation manners, various aspects of the present application can also be implemented in the form of a program product, which includes program code. When the program product runs on a terminal device, the program code is used to make the The terminal device executes the steps according to various exemplary embodiments of the present application described in the above-mentioned "Exemplary Method" section of this specification.

Referring to FIG. 8, a computer non-volatile readable storage medium 800 for implementing the above method according to an embodiment of the present application is described, which may adopt a portable compact disk read-only memory (CD-ROM) and includes program code , And can run on terminal devices, such as personal computers. However, the program product of this application is not limited to this. In this document, the computer non-volatile readable storage medium can be any tangible medium that contains or stores a program. The program can be used by or in combination with an instruction execution system, device, or device. In conjunction with.

The program product can use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or a combination of any of the above. More specific examples (non-exhaustive list) of readable storage media include: electrical connections with one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable Type programmable read only memory (EPROM or flash memory), optical fiber, portable compact disk read only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above.

The computer-readable signal medium may include a data signal propagated in baseband or as a part of a carrier wave, and readable program code is carried therein. This propagated data signal can take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. The readable signal medium may also be any readable medium other than a readable storage medium, and the readable medium may send, propagate, or transmit a program for use by or in combination with the instruction execution system, apparatus, or device.

The program code contained on the readable medium can be transmitted by any suitable medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

The program code used to perform the operations of the present application can be written in any combination of one or more programming languages. The programming languages include object-oriented programming languages—such as Java, C++, etc., as well as conventional procedural programming languages. Programming language-such as "C" language or similar programming language. The program code can be executed entirely on the user's computing device, partly on the user's device, executed as an independent software package, partly on the user's computing device and partly executed on the remote computing device, or entirely on the remote computing device or server Executed on. In the case of a remote computing device, the remote computing device can be connected to a user computing device through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (for example, using Internet service providers). Business to connect via the Internet).

In addition, the above-mentioned drawings are merely schematic illustrations of the processing included in the method according to the exemplary embodiments of the present application, and are not intended for limitation. It is easy to understand that the processing shown in the above drawings does not indicate or limit the time sequence of these processings. In addition, it is easy to understand that these processes can be executed synchronously or asynchronously in multiple modules, for example.

It should be understood that the present application is not limited to the precise structure that has been described above and shown in the drawings, and various modifications and changes can be performed without departing from its scope. The scope of the application is only limited by the appended claims.

Claims (22)

A method for locating facial feature points, including: Input the target face image of the feature points to be located into the first convolutional neural network model, and obtain the first shallow feature containing multiple candidate feature points output by the predetermined layer convolutional layer of the first convolutional neural network model Figure; Using a bilinear interpolation algorithm to perform bilinear interpolation on candidate feature points in the first shallow feature map to obtain a second shallow feature map; Input the second shallow feature map to a second convolutional neural network model cascaded with the first convolutional neural network model to obtain the output of the second convolutional neural network model and the feature to be located The face feature map corresponding to the target face image of the point, where each of the predetermined convolutional layers of the second convolutional neural network model and all convolutional layers before the predetermined convolutional layer The weights of the first convolutional neural network model correspond to the weights of the convolutional layers corresponding to the number of layers, and the predetermined convolutional layers of the second convolutional neural network model are in all the second convolutional neural network models. The ordering in the convolutional layer of is consistent with the ordering of the predetermined convolutional layer of the first convolutional neural network model in all the convolutional layers of the first convolutional neural network model. The method according to claim 1, wherein the using a bilinear interpolation algorithm to perform bilinear interpolation on candidate feature points in the first shallow feature map to obtain a second shallow feature map comprises: For each candidate feature point in the first shallow feature map, obtain the target side length of a square area to be determined with the candidate feature point as a coordinate center; For each candidate feature point in the first shallow feature map, a square area is determined in the first shallow feature map with the candidate feature point as the coordinate center, wherein the side length of the square area is for The target side length obtained by the candidate feature point; Acquiring the coordinate values of the four vertices of the square area determined for each candidate feature point and the pixel value at each vertex; For each candidate feature point, based on the coordinates of the vertex corresponding to the candidate feature point and the pixel value at each vertex, the following formula is used to obtain the pixel value of each pixel that constitutes the second shallow feature map to obtain the first Two shallow feature maps:

Among them, (x, y) is the coordinate value of the pixel point corresponding to the feature point in the second shallow feature map obtained for the candidate feature point, (x ₁ , y ₁ ), (x ₂ , y ₁ ), (x ₁ ,y ₂ ) and (x ₂ ,y ₂ ) are the coordinate values of the four vertices of the square area corresponding to the candidate feature point, f(x ₁ ,y ₁ ), f(x ₂ ,y ₁ ) , F(x ₁ , y ₂ ) and f(x ₂ , y ₂ ) are the pixel values at the four vertices of the square region corresponding to the candidate feature point in the first shallow feature map. The method according to claim 2, wherein the target side length acquired for each candidate feature point is the same preset side length, and the acquisition of the four vertices of the square area determined for each candidate feature point The coordinate value and the pixel value at each vertex include: For each candidate feature point, the following expressions are used to obtain the coordinate values of the four vertices of the square area determined for the candidate feature point: (x ₃ -r,y ₃ -r),(x ₃ -r,y ₃ +r),(x ₃ +r,y ₃ -r),(x ₃ +r,y ₃ +r), Wherein, (x ₃ , y ₃ ) is the coordinate value of the candidate feature point, and r is one half of the preset side length; For each candidate feature point, determine whether the coordinate value of each of the four vertices of the square area determined for the candidate feature point is located in the first shallow feature map; If yes, obtain the pixel value of the corresponding vertex in the first shallow feature map according to the coordinate value; If not, for the vertices located in the first shallow feature map among the four vertices of the square area determined for the candidate feature point, in the first shallow feature map according to the coordinate value corresponding to each vertex Obtain the pixel value of the corresponding vertex; Acquiring a vertex outside the first shallow feature map among the four vertices of the square region determined for the candidate feature point as an auxiliary vertex; For each auxiliary vertex, the pixel value at the pixel point closest to the auxiliary vertex is acquired in the first shallow feature map, as the pixel value of the auxiliary vertex. The method according to claim 3, wherein, for each auxiliary vertex, the pixel value at the pixel point closest to the auxiliary vertex is obtained in the first shallow feature map as the pixel of the auxiliary vertex Values include: For each auxiliary vertex, for each pixel in the first shallow feature map, use the following formula to determine the distance between the pixel and the auxiliary vertex:

Where x ₄ and y ₄ are the abscissa and ordinate of the auxiliary vertex, respectively, x, y are the abscissa and ordinate of the pixel in the first shallow feature map, and D is the auxiliary vertex and the first The distance between pixels in the shallow feature map; For each auxiliary vertex, obtain the smallest distance among the distances between each pixel point in the first shallow feature map obtained for the auxiliary vertex and the auxiliary vertex; The pixel value at the pixel point corresponding to the smallest distance is used as the pixel value of the auxiliary vertex. The method according to claim 4, wherein, for each auxiliary vertex, the smallest distance between each pixel in the first shallow feature map and the auxiliary vertex obtained for the auxiliary vertex is obtained The distance includes: For each auxiliary vertex, any one of the distances between each pixel in the first shallow feature map obtained for the auxiliary vertex and the auxiliary vertex is selected, and it is marked as a candidate minimum distance; In the distance between each pixel point and the auxiliary vertex except the candidate minimum distance, determine whether the distance between the pixel point and the auxiliary vertex is less than the distance between each pixel point and the auxiliary vertex according to a predetermined sequence. The candidate minimum distance; If yes, cancel the mark of all candidate minimum distances, and mark the distance between the pixel and the auxiliary vertex as the candidate minimum distance; In the distance between the unacquired pixel and the auxiliary vertex, the distance between each pixel and the auxiliary vertex is again determined in a predetermined order to determine whether the distance between the pixel and the auxiliary vertex is less than the candidate minimum distance, until The distance between no pixel and the auxiliary vertex is less than the candidate minimum distance; The candidate minimum distance is taken as the minimum distance. The method according to claim 4, wherein, for each auxiliary vertex, for each pixel in the first shallow feature map, the following formula is used to determine the distance between the pixel and the auxiliary vertex , The method further includes: For each auxiliary vertex, from the pixel points of the first shallow feature map, obtain a pixel whose abscissa difference from the auxiliary vertex is less than a predetermined coordinate difference and whose ordinate difference is less than a predetermined coordinate difference, as a candidate pixel; For each auxiliary vertex, for each pixel in the first shallow feature map, using the following formula to determine the distance between the pixel and the auxiliary vertex includes: For each auxiliary vertex, for each candidate pixel in the first shallow feature map, the following formula is used to determine the distance between the candidate pixel and the auxiliary vertex. The method according to claim 2, wherein, for each candidate feature point in the first shallow feature map, acquiring the target side length of a square area to be determined with the candidate feature point as a coordinate center comprises : Taking each candidate feature point in the first shallow feature map as a coordinate center, acquiring a circular area with a preset first side length in the first shallow feature map; For each candidate feature point in the first shallow feature map, determine the variance of each pixel point in the circular area corresponding to the candidate feature point; In the case that the variance is greater than or equal to the preset variance threshold, use the preset first side length as the target side length of the square area to be determined with the candidate feature point as the coordinate center; In the case that the variance is less than the preset variance threshold, the preset second side length is used as the target side length of the square area to be determined with the candidate feature point as the coordinate center, wherein the preset second side length The side length is smaller than the preset first side length. A device for locating facial feature points, including: The first acquisition module is configured to input the target face image of the feature point to be located into the first convolutional neural network model, and obtain the output of the predetermined layer convolutional layer of the first convolutional neural network model, which contains multiple candidates The first shallow feature map of feature points; An interpolation module, configured to perform bilinear interpolation on candidate feature points in the first shallow feature map by using a bilinear interpolation algorithm to obtain a second shallow feature map; The second acquisition module is configured to input the second shallow feature map to a second convolutional neural network model cascaded with the first convolutional neural network model to obtain the second convolutional neural network model The output face feature map corresponding to the target face image of the feature point to be located, wherein the predetermined convolutional layer of the second convolutional neural network model and all the volumes before the predetermined convolutional layer The weight of each convolutional layer in the buildup layer is consistent with the weight of the convolutional layer corresponding to the number of layers of the first convolutional neural network model. The predetermined convolutional layer of the second convolutional neural network model is in all the convolutional layers. The ordering in the convolutional layer of the second convolutional neural network model is consistent with the ordering of the predetermined convolutional layer of the first convolutional neural network model in all convolutional layers of the first convolutional neural network model . The apparatus according to claim 8, wherein the interpolation module is further configured to: For each candidate feature point in the first shallow feature map, obtain the target side length of a square area to be determined with the candidate feature point as a coordinate center; For each candidate feature point in the first shallow feature map, a square area is determined in the first shallow feature map with the candidate feature point as the coordinate center, wherein the side length of the square area is for The target side length obtained by the candidate feature point; Acquiring the coordinate values of the four vertices of the square area determined for each candidate feature point and the pixel value at each vertex; For each candidate feature point, based on the coordinates of the vertex corresponding to the candidate feature point and the pixel value at each vertex, the following formula is used to obtain the pixel value of each pixel that constitutes the second shallow feature map to obtain the first Two shallow feature maps:

Among them, (x, y) is the coordinate value of the pixel point corresponding to the feature point in the second shallow feature map obtained for the candidate feature point, (x ₁ , y ₁ ), (x ₂ , y ₁ ), (x ₁ ,y ₂ ) and (x ₂ ,y ₂ ) are the coordinate values of the four vertices of the square area corresponding to the candidate feature point, f(x ₁ ,y ₁ ), f(x ₂ ,y ₁ ) , F(x ₁ , y ₂ ) and f(x ₂ , y ₂ ) are the pixel values at the four vertices of the square region corresponding to the candidate feature point in the first shallow feature map. The device according to claim 9, wherein the target side length acquired for each candidate feature point is the same preset side length, and the acquisition of the four vertices of the square region determined for each candidate feature point The coordinate value and the pixel value at each vertex include: For each candidate feature point, the following expressions are used to obtain the coordinate values of the four vertices of the square area determined for the candidate feature point: (x ₃ -r,y ₃ -r),(x ₃ -r,y ₃ +r),(x ₃ +r,y ₃ -r),(x ₃ +r,y ₃ +r), Wherein, (x ₃ , y ₃ ) is the coordinate value of the candidate feature point, and r is one half of the preset side length; For each candidate feature point, determine whether the coordinate value of each of the four vertices of the square area determined for the candidate feature point is located in the first shallow feature map; If yes, obtain the pixel value of the corresponding vertex in the first shallow feature map according to the coordinate value; If not, for the vertices located in the first shallow feature map among the four vertices of the square area determined for the candidate feature point, in the first shallow feature map according to the coordinate value corresponding to each vertex Obtain the pixel value of the corresponding vertex; Acquiring a vertex outside the first shallow feature map among the four vertices of the square region determined for the candidate feature point as an auxiliary vertex; For each auxiliary vertex, the pixel value at the pixel point closest to the auxiliary vertex is acquired in the first shallow feature map, as the pixel value of the auxiliary vertex. 11. The device according to claim 10, wherein, for each auxiliary vertex, the pixel value at the pixel point closest to the auxiliary vertex is obtained in the first shallow feature map as the pixel of the auxiliary vertex Values include: For each auxiliary vertex, for each pixel in the first shallow feature map, use the following formula to determine the distance between the pixel and the auxiliary vertex:

Where x ₄ and y ₄ are the abscissa and ordinate of the auxiliary vertex, respectively, x, y are the abscissa and ordinate of the pixel in the first shallow feature map, and D is the auxiliary vertex and the first The distance between pixels in the shallow feature map; For each auxiliary vertex, obtain the smallest distance among the distances between each pixel point in the first shallow feature map obtained for the auxiliary vertex and the auxiliary vertex; The pixel value at the pixel point corresponding to the smallest distance is used as the pixel value of the auxiliary vertex. 11. The device according to claim 11, wherein for each auxiliary vertex, the smallest distance between each pixel in the first shallow feature map and the auxiliary vertex obtained for the auxiliary vertex is obtained The distance includes: For each auxiliary vertex, any one of the distances between each pixel in the first shallow feature map obtained for the auxiliary vertex and the auxiliary vertex is selected, and it is marked as a candidate minimum distance; In the distance between each pixel point and the auxiliary vertex except the candidate minimum distance, determine whether the distance between the pixel point and the auxiliary vertex is less than the distance between each pixel point and the auxiliary vertex according to a predetermined sequence. The candidate minimum distance; If yes, cancel the mark of all candidate minimum distances, and mark the distance between the pixel and the auxiliary vertex as the candidate minimum distance; In the distance between the unacquired pixel and the auxiliary vertex, the distance between each pixel and the auxiliary vertex is again determined in a predetermined order to determine whether the distance between the pixel and the auxiliary vertex is less than the candidate minimum distance, until The distance between no pixel and the auxiliary vertex is less than the candidate minimum distance; The candidate minimum distance is taken as the minimum distance. 11. The device of claim 11, wherein for each auxiliary vertex, for each pixel in the first shallow feature map, before determining the distance between the pixel and the auxiliary vertex using the following formula ,Also includes: For each auxiliary vertex, from the pixel points of the first shallow feature map, obtain a pixel whose abscissa difference from the auxiliary vertex is less than a predetermined coordinate difference and whose ordinate difference is less than a predetermined coordinate difference, as a candidate pixel; For each auxiliary vertex, for each pixel in the first shallow feature map, using the following formula to determine the distance between the pixel and the auxiliary vertex includes: For each auxiliary vertex, for each candidate pixel in the first shallow feature map, the following formula is used to determine the distance between the candidate pixel and the auxiliary vertex. The device according to claim 9, wherein, for each candidate feature point in the first shallow feature map, acquiring the target side length of a square area to be determined with the candidate feature point as a coordinate center comprises : Taking each candidate feature point in the first shallow feature map as a coordinate center, acquiring a circular area with a preset first side length in the first shallow feature map; For each candidate feature point in the first shallow feature map, determine the variance of each pixel point in the circular area corresponding to the candidate feature point; In the case that the variance is greater than or equal to the preset variance threshold, use the preset first side length as the target side length of the square area to be determined with the candidate feature point as the coordinate center; In the case that the variance is less than the preset variance threshold, the preset second side length is used as the target side length of the square area to be determined with the candidate feature point as the coordinate center, wherein the preset second side length The side length is smaller than the preset first side length. A computing device includes a memory and a processor. The memory stores computer-readable instructions, and when the computer-readable instructions are executed by the processor, the processor executes: Input the target face image of the feature points to be located into the first convolutional neural network model, and obtain the first shallow feature containing multiple candidate feature points output by the predetermined layer convolutional layer of the first convolutional neural network model Figure; Using a bilinear interpolation algorithm to perform bilinear interpolation on candidate feature points in the first shallow feature map to obtain a second shallow feature map; Input the second shallow feature map to a second convolutional neural network model cascaded with the first convolutional neural network model to obtain the output of the second convolutional neural network model and the feature to be located The face feature map corresponding to the target face image of the point, where each of the predetermined convolutional layers of the second convolutional neural network model and all convolutional layers before the predetermined convolutional layer The weights of the first convolutional neural network model correspond to the weights of the convolutional layers corresponding to the number of layers, and the predetermined convolutional layers of the second convolutional neural network model are in all the second convolutional neural network models. The ordering in the convolutional layer of is consistent with the ordering of the predetermined convolutional layer of the first convolutional neural network model in all the convolutional layers of the first convolutional neural network model. The computing device according to claim 15, wherein said using a bilinear interpolation algorithm to perform bilinear interpolation on candidate feature points in the first shallow feature map to obtain a second shallow feature map comprises: For each candidate feature point in the first shallow feature map, obtain the target side length of a square area to be determined with the candidate feature point as a coordinate center; For each candidate feature point in the first shallow feature map, a square area is determined in the first shallow feature map with the candidate feature point as the coordinate center, wherein the side length of the square area is for The target side length obtained by the candidate feature point; Acquiring the coordinate values of the four vertices of the square area determined for each candidate feature point and the pixel value at each vertex; For each candidate feature point, based on the coordinates of the vertex corresponding to the candidate feature point and the pixel value at each vertex, the following formula is used to obtain the pixel value of each pixel that constitutes the second shallow feature map to obtain the first Two shallow feature maps:

Among them, (x, y) is the coordinate value of the pixel point corresponding to the feature point in the second shallow feature map obtained for the candidate feature point, (x ₁ , y ₁ ), (x ₂ , y ₁ ), (x ₁ ,y ₂ ) and (x ₂ ,y ₂ ) are the coordinate values of the four vertices of the square area corresponding to the candidate feature point, f(x ₁ ,y ₁ ), f(x ₂ ,y ₁ ) , F(x ₁ , y ₂ ) and f(x ₂ , y ₂ ) are the pixel values at the four vertices of the square region corresponding to the candidate feature point in the first shallow feature map. The computing device according to claim 16, wherein the target side length obtained for each candidate feature point is the same preset side length, and the four vertices of the square area determined for each candidate feature point are obtained The coordinate value of and the pixel value at each vertex, including: For each candidate feature point, the following expressions are used to obtain the coordinate values of the four vertices of the square area determined for the candidate feature point: (x ₃ -r,y ₃ -r),(x ₃ -r,y ₃ +r),(x ₃ +r,y ₃ -r),(x ₃ +r,y ₃ +r), Wherein, (x ₃ , y ₃ ) is the coordinate value of the candidate feature point, and r is one half of the preset side length; For each candidate feature point, determine whether the coordinate value of each of the four vertices of the square area determined for the candidate feature point is located in the first shallow feature map; If yes, obtain the pixel value of the corresponding vertex in the first shallow feature map according to the coordinate value; If not, for the vertices located in the first shallow feature map among the four vertices of the square area determined for the candidate feature point, in the first shallow feature map according to the coordinate value corresponding to each vertex Obtain the pixel value of the corresponding vertex; Acquiring a vertex outside the first shallow feature map among the four vertices of the square region determined for the candidate feature point as an auxiliary vertex; For each auxiliary vertex, the pixel value at the pixel point closest to the auxiliary vertex is obtained in the first shallow feature map, as the pixel value of the auxiliary vertex. The computing device according to claim 17, wherein for each auxiliary vertex, the pixel value at the pixel point closest to the auxiliary vertex is obtained in the first shallow feature map as the value of the auxiliary vertex Pixel value, including: For each auxiliary vertex, for each pixel in the first shallow feature map, use the following formula to determine the distance between the pixel and the auxiliary vertex:

Where x ₄ and y ₄ are the abscissa and ordinate of the auxiliary vertex, respectively, x, y are the abscissa and ordinate of the pixel in the first shallow feature map, and D is the auxiliary vertex and the first The distance between pixels in the shallow feature map; For each auxiliary vertex, obtain the smallest distance among the distances between each pixel point in the first shallow feature map obtained for the auxiliary vertex and the auxiliary vertex; The pixel value at the pixel point corresponding to the smallest distance is used as the pixel value of the auxiliary vertex. The computing device according to claim 18, wherein, for each auxiliary vertex, in the distance between each pixel in the first shallow feature map obtained for the auxiliary vertex and the auxiliary vertex, the The minimum distance includes: For each auxiliary vertex, any one of the distances between each pixel in the first shallow feature map obtained for the auxiliary vertex and the auxiliary vertex is selected, and it is marked as a candidate minimum distance; In the distance between each pixel point and the auxiliary vertex except the candidate minimum distance, determine whether the distance between the pixel point and the auxiliary vertex is less than the distance between each pixel point and the auxiliary vertex according to a predetermined sequence. The candidate minimum distance; If yes, cancel the mark of all candidate minimum distances, and mark the distance between the pixel and the auxiliary vertex as the candidate minimum distance; In the distance between the unacquired pixel and the auxiliary vertex, the distance between each pixel and the auxiliary vertex is again determined in a predetermined order to determine whether the distance between the pixel and the auxiliary vertex is less than the candidate minimum distance, until The distance between no pixel and the auxiliary vertex is less than the candidate minimum distance; The candidate minimum distance is taken as the minimum distance. 18. The computing device according to claim 18, wherein, for each auxiliary vertex, for each pixel in the first shallow feature map, the distance between the pixel and the auxiliary vertex is determined using the following formula Previously, when the computer-readable instructions were executed by the processor, the processor was caused to also execute: For each auxiliary vertex, from the pixel points of the first shallow feature map, obtain a pixel whose abscissa difference from the auxiliary vertex is less than a predetermined coordinate difference and whose ordinate difference is less than a predetermined coordinate difference, as a candidate pixel; For each auxiliary vertex, for each pixel in the first shallow feature map, using the following formula to determine the distance between the pixel and the auxiliary vertex includes: For each auxiliary vertex, for each candidate pixel in the first shallow feature map, the following formula is used to determine the distance between the candidate pixel and the auxiliary vertex. The computing device according to claim 16, wherein, for each candidate feature point in the first shallow feature map, acquiring the target side length of a square area to be determined with the candidate feature point as a coordinate center, include: Taking each candidate feature point in the first shallow feature map as a coordinate center, acquiring a circular area with a preset first side length in the first shallow feature map; For each candidate feature point in the first shallow feature map, determine the variance of each pixel point in the circular area corresponding to the candidate feature point; In the case that the variance is greater than or equal to the preset variance threshold, use the preset first side length as the target side length of the square area to be determined with the candidate feature point as the coordinate center; In the case that the variance is less than the preset variance threshold, the preset second side length is used as the target side length of the square area to be determined with the candidate feature point as the coordinate center, wherein the preset second side length The side length is smaller than the preset first side length. A computer non-volatile readable storage medium storing computer readable instructions, which when executed by one or more processors, cause one or more processors to execute any one of claims 1 to 7 The method described in the item.

Images