WO2019192121A1 - Dual-channel neural network model training and human face comparison method, and terminal and medium - Google Patents

Abstract

A dual-channel neural network model training method, comprising: inputting each human facial picture in human face sample data into a first deep neural network to extract a first human facial feature, and simultaneously inputting the corresponding human facial picture having been through histogram equalization processing into a second deep neural network model to extract a second human facial feature; normalizing the first human facial feature and the second human facial feature and connecting same to serve as a final human facial feature representation; and normalizing the final human facial feature representation and inputting same into a pre-set loss function to calculate a loss function value, and when the loss function value is smaller than or equal to a pre-set loss function threshold, then ending the dual-channel neural network model training and updating a weight and bias in the dual-channel neural network model. Also provided are a human face comparison method, a terminal and a medium. By means of the method, a dual-channel neural network model applicable for human face comparison can be trained, and a better human face comparison result can be obtained.

Description

Dual channel neural network model training and face matching method, terminal and medium

This application claims to be filed on April 4, 2018, the Chinese Patent Office, the application number is 201101829180.6. The invention is entitled "Double-channel neural network model training and face comparison method, terminal and medium" priority of Chinese patent application, all of which The content is incorporated herein by reference.

Technical field

The present application relates to the field of image recognition technologies, and in particular, to a two-channel neural network model training and a face comparison method, a terminal, and a medium.

Background technique

In recent years, with the development of machine learning and deep learning, biometrics have been widely used in identity authentication because of their portability, loss, no forgetting, no borrowing, and no misappropriation. As one of the biometric recognition technologies, face recognition technology is the most direct, friendly and convenient, and is our ideal choice. Face matching is a sub-area of face recognition. Face comparison is to judge whether two faces are the same person. The most common scene is to judge whether the certificate is the person or not. Face recognition is given a face. The picture, and then judge who this person is, its essence is equivalent to multiple face comparisons.

Because the face image in the dynamic environment has many effects such as insufficient illumination, occlusion, insufficient resolution, and incorrect posture, the face matching in the dynamic environment is very difficult, resulting in a decrease in the accuracy of face comparison.

Summary of the invention

In view of the above, it is necessary to propose a two-channel neural network model training method, a face matching method, a terminal and a medium, which can train a classification model suitable for face matching, and the trained features have strong distinguishing ability. Get better face matching effect.

A first aspect of the present application provides a two-channel neural network model training method, the method comprising:

a. obtaining a primitive face image of different people, each original face image has a size of 182*182;

b. randomly cutting each original face image into a preset number of face images to obtain a face sample data set;

c. performing histogram equalization processing on each face image in the face sample data set to obtain a histogram equalization face image;

d. inputting each face image in the face sample data into the first depth neural network to extract the first face feature, and inputting the corresponding histogram equalized face image into the second depth neural network for extracting a second face feature;

e. normalizing the first facial feature to obtain a first normalized facial feature, and normalizing the second facial feature to obtain a second normalized facial feature;

f. connecting the first normalized face feature and the second normalized face feature according to a preset connection rule to obtain a new normalized face feature as a final face feature representation;

g. normalizing the final facial feature representation and inputting it into a preset loss function to calculate a loss function value. When the loss function value is less than or equal to a preset loss function threshold, the dual channel neural network model training End and update the weights and offsets in the two-channel neural network model.

A second aspect of the present application provides a method for face alignment using the two-channel neural network model, the method comprising:

After receiving the face image of the target user to be compared, the histogram of the target user is subjected to histogram equalization processing to obtain a histogram equalized face image of the target user;

Importing a face image of the target user into a first deep neural network in the trained two-channel neural network model to extract a third facial feature, and simultaneously mapping the target user's histogram equalized face image into the training The fourth facial feature is extracted from the second deep neural network in the good two-channel neural network model;

Normalizing the third facial feature to obtain a third normalized facial feature, and normalizing the fourth facial feature to obtain a fourth normalized facial feature;

And connecting the third normalized face feature and the fourth normalized face feature according to the preset connection rule to obtain a new normalized face feature as a final feature representation of the target user;

Calculating a similarity between a final feature representation of the target user and a final feature representation of a registered user in the database; and

The face comparison result is determined according to the relationship between the calculated similarity and the preset confidence threshold.

A third aspect of the present application provides a terminal, the terminal comprising a processor, the processor is configured to implement the dual channel neural network model training method or implement the face ratio when executing computer readable instructions stored in a memory The method.

A fourth aspect of the present application provides a non-volatile readable storage medium having stored thereon computer readable instructions, the computer readable instructions being implemented by a processor to implement the The two-channel neural network model training method or the face matching method is implemented.

The two-channel neural network model training method, the face comparison method, the terminal and the medium described in the present application apply the trained two-channel neural network model to the face comparison, and can solve the existence of the face image in the dynamic environment. Insufficient illumination, occlusion, insufficient resolution, incorrect posture, etc.; only a small number of samples are needed in the training phase of the two-channel neural network model, thus solving the problem of the data volume requirement of the algorithm and increasing the practicability of the algorithm. Improve the accuracy and efficiency of witness verification.

DRAWINGS

FIG. 1 is a flowchart of a two-channel neural network model training method according to Embodiment 1 of the present application.

2 is a flow chart of a two-channel neural network model training method provided by Embodiment 2 of the present application.

FIG. 3 is a flowchart of a face matching method according to Embodiment 3 of the present application.

4 is a flowchart of a face matching method provided in Embodiment 4 of the present application.

FIG. 5 is a structural diagram of a two-channel neural network model training apparatus according to Embodiment 5 of the present application.

FIG. 6 is a structural diagram of a face matching device according to Embodiment 6 of the present application.

FIG. 7 is a schematic diagram of a terminal provided in Embodiment 7 of the present application.

detailed description

The face matching method of the present application is applied to one or more terminals. The two-channel neural network model training method and the face comparison method can also be applied to a hardware environment composed of a terminal and a server connected to the terminal through a network. The two-channel neural network model training method and the face comparison method of the embodiment of the present application may be performed by the server at the same time, or may be performed by the terminal at the same time; or may be performed by the server and the terminal together, for example, the dual channel nerve The network model training method is performed by a server, and the face matching method is executed by a terminal, or the face matching method is executed by a server, and the two-channel neural network model training method is executed by a terminal. This application is not limited herein.

Embodiment 1

FIG. 1 is a flowchart of a two-channel neural network model training method according to Embodiment 1 of the present application.

101: Obtain a raw face picture of different people, and the size of each original face picture is 182*182.

In the preferred embodiment, the obtaining of the original face image may include the following two methods:

(1) If the resolution of the photographing device (for example, camera, camera, etc.) is set to 182*182 in advance, the face of different people can be photographed in a dynamic environment to obtain the original face image of size 182*182;

(2) Obtain the original face image of different people from the face data set, and set the original face picture to the size of 182*182.

The LFW data set was created to investigate the face recognition problem in an unrestricted environment, which contains more than 13,000 face images, all of which are from the Internet, not the lab environment.

Setting the original face image resolution to 182*182 not only provides more facial detail information, but also adapts to existing general transmission, storage, and management systems and devices.

102: Randomly crop each original face image into a preset number of first face images to obtain a face sample data set.

The preset number can be adjusted or modified according to actual needs. In the preferred embodiment, each original face image is randomly cut into five first face images of the same size, and each first face image is Both contain face areas.

Randomly crop each original face image to obtain the first face image under arbitrary angle and arbitrary illumination conditions. The random process is the process of collecting the original face image in the simulated dynamic environment; secondly, by cutting In this way, multiple first face images can be obtained, thereby increasing the number of face sample data sets, and more face sample data sets can improve the robustness of the two-channel neural network model.

103: Perform a histogram equalization process on each first face image in the face sample data set to obtain a histogram equalization face image.

In this embodiment, the specific steps of performing histogram equalization processing on the first face image in the face sample data set include:

1) performing pixel grayscale statistics on the first face image;

Counting the number of occurrences of each pixel gray level in the first face image, and accumulating the same number of gray level pixels.

2) Calculate the pixel gray distribution density;

Calculating the probability Pr(r _k ) of the kth gray level appearing in the entire first face picture according to the formula (1-1),

Where n is the sum of the number of pixels in the first face picture, n _k is the number of pixels of the gray level r _k , and L is the total number of gray levels in the first face picture.

3) transforming the calculated grayscale distribution density of the pixel according to the cumulative distribution function;

The calculated pixel grayscale distribution density is transformed according to formula (1-2).

4) calculating the converted new pixel gray scale;

The gray value is mapped to obtain a new pixel gray level, and the new pixel gray level is rounded to obtain a transformed new pixel gray level.

Performing a histogram equalization process on each of the first face images in the face sample data set, which can further enhance the contrast of the first face image in the face sample data set, especially for exposure The first face image that is excessive or underexposed can better maintain the detail information of the face region in the first face image after the histogram equalization processing.

Preferably, if the first face image in the face sample data set is a color picture, since the histogram equalization processing of the three components R, G, and B in the color picture cannot be directly performed, the color picture needs to be The color space is converted into an HSV (Hue Chroma, Value Saturation) color space from the RGB color space, and then the histogram equalization process is performed on the V component.

104: Input each first face image in the face sample data into the first depth neural network to extract the first face feature, and input the corresponding histogram equalized face image into the second depth neural network. The second face feature is extracted.

In the preferred embodiment, the first deep neural network and the second deep neural network are neural networks having the same number of layers (for example, 200 layers). Optionally, the first depth neural network and the second deep neural network adopt a perceptron-based Inception-Resnet-V2 (Inception-V2 and Resnet combined neural network) neural network. The perceptron-based Inception-Resnet-V2 neural network is a prior art, and the present application does not repeat here.

Preferably, the step 104 may further include: the dimension of the first facial feature extracted by the first depth neural network is 64 dimensions, and the second person extracted by the second deep neural network The dimension of the face feature is 64 dimensions.

105: normalize the first facial feature to obtain a first normalized facial feature, and normalize the second facial feature to obtain a second normalized facial feature.

In the preferred embodiment, the first facial feature and the second facial feature may be normalized by a normalization function, and the normalization function may be a Euclidean distance function or a Manhattan Distance function, minimum absolute error. Optionally, the first normalized face feature is obtained by using the Euclidean distance function on the first facial feature, and the second facial feature is normalized to obtain a second normalized facial feature.

Normalization can compress light, shadows, and edges so that the first normalized face feature and the second normalized face feature are robust to illumination, shadow, and edge changes, further enhancing the two-channel neural network. Model robustness. In addition, normalization using the Euclidean distance function can avoid over-fitting of the two-channel neural network model, thereby improving the generalization ability of the two-channel neural network model, and optimizing the weight and offset of the subsequent two-channel neural network model. Become more stable and fast.

106: Connect the first normalized face feature and the second normalized face feature according to a preset connection rule to obtain a new normalized face feature as a final face feature representation.

In the preferred embodiment, the preset connection rule may be a cross connection. For example, the first normalized face feature is (x ₁ , x ₂ , x ₃ , . . . , x _m ), and the second normalized face feature is (y ₁ , y ₂ , y ₃ , . . . , y _m ), the final face features obtained by the cross-connection connection are expressed as (x ₁ , y ₁ , x ₂ , y ₂ , x ₃ , y ₃ , . . . , x _m , y _m ).

Alternatively, the preset connection rules may be sequential connections. For example, the first normalized face feature is (x ₁ , x ₂ , x ₃ , . . . , x _m ), and the second normalized face feature is (y ₁ , y ₂ , y ₃ , . . . , y _m ), the final face features obtained by sequential connection are expressed as (x ₁ , x ₂ , x ₃ , . . . , x _m , y ₁ , y ₂ , y ₃ , . . . , y _m ).

107: normalize the final facial feature representation and input to a preset loss function to calculate a loss function value. When the loss function value is less than or equal to a preset loss function threshold, the dual channel neural network model training Ending and updating the weights and offsets in the two-channel neural network model; when the loss function value is greater than a preset loss function threshold, acquiring more original face images and re-executing based on the increased original face images The above steps 101-107.

In the preferred embodiment, the preset loss function L is represented by the following formula (1-3):

代表的是第yi个人的实际输出，L _C表示的是中心损失函数(center loss)的结果，cyi表示第yi个人的特征中心，xi表示全连接层之前的特征。预先设置的损失函数L越小，代表模型越好。 Where m is the size of the mini-batch. L _S represents the result of the cross-entropy loss function (softmax loss).

Represents the actual output of the yi person, L _C represents the result of the center loss function, cyi represents the feature center of the yi person, and xi represents the feature before the fully connected layer. The smaller the loss function L set in advance, the better the representative model.

Normalizing the final facial feature representation again, in order to make the normalized final facial feature representation distributed between 0-1, with a unified critical standard, which not only facilitates subsequent data processing, but also The convergence of the loss function can be accelerated. Secondly, the preset loss function can make the distance between different face images larger, and the distance between different face images of the same person is reduced, thereby improving the classification of the two-channel neural network model. Ability to improve the accuracy of face recognition.

In summary, the dual-channel neural network model trained in the present application can significantly enhance the feature extraction capability of the face image in the input face sample data set and the face image processed by the histogram equalization process. It can reduce the risk of performance degradation when the network level is deepened, and improve the robustness of the two-channel neural network model, which is convenient for improving the recognition rate of face recognition or face comparison.

Embodiment 2

2 is a flow chart of a two-channel neural network model training method provided by Embodiment 2 of the present application.

201: Obtain a raw face picture of different people, and the size of each original face picture is 182*182.

The step 201 in this embodiment is the same as the step 101 in the first embodiment, and the application is not described in detail herein.

202: Randomly crop each original face image into a preset number of second face images to obtain a face sample data set, where a size of each second face image in the face sample data set is 171*160.

In the preferred embodiment, the second face image size obtained after random cropping is 171*160, which not only can maintain the facial detail information in the original face image to the greatest extent, but also utilizes the original face image. When the face sample data set is involved in training, the training time of the two-channel neural network model can be shortened, and the training speed of the two-channel neural network model can be improved.

203: Detect each second face image that is cropped by using a pre-stored face detection algorithm, and determine whether each second face image in the cropped second face image includes a face region. .

The pre-stored face detection algorithm may be one of the following algorithms: a template-based face detection method, an artificial neural network-based face detection method, a model-based face detection method, and a skin color-based face detection method. The method or the face detection method based on the feature sub-face, and the like.

In this embodiment, the pre-stored face detection algorithm is not limited to the above enumeration, and any algorithm capable of detecting whether the face image is included in the cropped face image may be referred to herein. In addition, the face detection algorithm pre-stored in this embodiment is a prior art, and will not be described in detail herein.

When it is determined that each of the second face images in the cropped second face image includes a face region, then performing the following step 204; when determining each of the cropped second face images or When the face area is not included in the partial face picture, the following step 205 is performed.

The face region that is not included in the cropped second face image includes the following two situations: there is no face region at all; only a part of the face region exists.

204: Perform a histogram equalization process on each second face image in the face sample data set to obtain a histogram equalization face image.

The step 204 in this embodiment is the same as the step 103 in the first embodiment, and the application is not described in detail herein.

205: Continue to perform random cropping on the original face image corresponding to the second face image that does not include the face region, until the preset number of second face images are obtained.

206: Input each second face image in the face sample data into the first depth neural network to extract the first face feature, and input the corresponding histogram equalized face image into the second depth neural network. The second face feature is extracted.

The step 206 in this embodiment is the same as the step 104 in the first embodiment, and the application is not described in detail herein.

207: normalize the first facial feature to obtain a first normalized facial feature, and normalize the second facial feature to obtain a second normalized facial feature.

The step 207 in this embodiment is the same as the step 105 in the first embodiment, and the application is not described in detail herein.

208: Connect the first normalized face feature and the second normalized face feature according to a preset connection rule to obtain a new normalized face feature as a final face feature representation.

The step 208 in this embodiment is the same as the step 106 in the first embodiment, and the application is not described in detail herein.

209: normalize the final facial feature representation and input to a preset loss function to calculate a loss function value. When the loss function value is less than or equal to a preset loss function threshold, the dual channel neural network model training Ending and updating the weights and offsets in the two-channel neural network model; when the loss function value is greater than a preset loss function threshold, acquiring more original face images and re-executing based on the increased original face images The above steps 201-209.

The step 209 in this embodiment is the same as the step 107 in the first embodiment, and the present application will not be described in detail herein.

In summary, the two-channel neural network model trained in the present application detects each of the cropped second face images by a pre-stored face detection algorithm to ensure that each second face image is included. All the face regions can ensure the validity of the extracted face features, which can further improve the robustness of the two-channel neural network model, and facilitate the subsequent recognition of face recognition or face comparison.

Embodiment 3

FIG. 3 is a flowchart of a face matching method according to Embodiment 3 of the present application.

301: After receiving the face image of the target user to be compared, perform a histogram equalization process on the face image of the target user to obtain a histogram equalized face image of the target user.

302: Enter a face image of the target user into a first deep neural network in the trained two-channel neural network model to extract a third facial feature, and simultaneously equalize the face image of the target user The fourth facial feature is extracted from the second deep neural network in the trained two-channel neural network model.

303: normalize the third facial feature to obtain a third normalized facial feature, and normalize the fourth facial feature to obtain a fourth normalized facial feature.

304: Connect the third normalized face feature and the fourth normalized face feature according to the preset connection rule to obtain a new normalized face feature, as a final feature representation of the target user. .

305: Calculate a similarity between the final feature representation of the target user and the final feature representation of the registered user in the database.

The similarity between the final feature representation of the target user and the final feature representation of the registered user in the database is calculated according to equations (2-1) and (2-2).

SIM=0.5+0.5*COSθ (2-2)

Wherein, the feature vector A represents the final feature representation of the target user, and the feature vector B represents the final feature representation of the registered user. The cosine value between the feature vector A and each of the feature vectors B in the database is calculated according to the equation (2-1), and the range of the calculated cosine value is [-1, 1]. The SIM calculated according to equation (2-2) is the similarity, and the value of the similarity is normalized to [0, 1].

306: Determine a face comparison result according to a relationship between the calculated similarity and a preset confidence threshold.

The results of the face alignment include alignment results and comparison failure results. When the calculated similarity is greater than or equal to the preset confidence threshold, it is determined that the user matches, that is, the registered user who is the same person as the target user is matched in the database. When the calculated similarity is less than the preset confidence threshold, it is determined that the user comparison fails, that is, the registered user who is the same person as the target user is not matched in the database.

Preferably, the preset confidence threshold is 0.5.

In summary, the dual-channel neural network model is used for face comparison, which can improve the accuracy of face comparison, shorten the time of face comparison, and improve the efficiency of face comparison.

Embodiment 4

4 is a flowchart of a face matching method provided in Embodiment 4 of the present application.

401: After receiving the face image of the target user to be compared with the face, pre-processing the face image of the target user.

In the preferred embodiment, the pre-processing may include, but is not limited to, image denoising, illumination normalization, pose calibration, grayscale normalization, and the like.

After receiving the face image of the target user to be compared, the face image of the target user is pre-processed to further improve the face image of the target user, so as to extract more discriminative power. Face features.

402: Perform a histogram equalization process on the face image of the target user after the pre-processing, and obtain a histogram equalization face image of the target user.

403: Input the pre-processed target user's face image into the first deep neural network in the trained two-channel neural network model to extract the third facial feature, and simultaneously equalize the target user's histogram. The fourth face feature is extracted from the second deep neural network in the trained two-channel neural network model.

404: normalize the third facial feature to obtain a third normalized facial feature, and normalize the fourth facial feature to obtain a fourth normalized facial feature.

405: Connect the third normalized face feature and the fourth normalized face feature according to the preset connection rule to obtain a new normalized face feature, as a final feature representation of the target user. .

406: Calculate a similarity between the final feature representation of the target user and the final feature representation of the registered user in the database.

407: Sort the calculated similarities according to a preset sorting method.

In the preferred embodiment, the pre-set sorting method includes, but is not limited to, sorting from large to small, and sorting from small to large.

408: Compare the relationship between the maximum similarity and the preset confidence threshold to determine the face comparison result.

The results of the face alignment include alignment results and comparison failure results. When the maximum similarity is greater than or equal to the preset confidence threshold, it is determined that the user matches, that is, the registered user who is the same person as the target user is matched in the database. When the maximum similarity is less than the preset confidence threshold, it is determined that the user comparison fails, that is, the registered user who is the same person as the target user is not matched in the database.

Preferably, the preset confidence threshold is 0.5.

By sorting the calculated similarities and directly comparing the relationship between the maximum similarity and the preset confidence threshold, when the maximum similarity is greater than or equal to the preset confidence threshold, the user comparison is determined. by. When the maximum similarity is less than the preset confidence threshold, it is determined that the user comparison fails. This can shorten the comparison time, and only need to compare once to determine whether the target user is a registered user.

In summary, the face matching method described in the present application can extract a better facial feature representation by using a two-channel neural network model, and a better face can be obtained by taking a shorter face comparison time. Compare effects.

The above description of the steps in the flowcharts of FIG. 1, FIG. 2, FIG. 3 and FIG. 4 may be performed according to different requirements, and the order of execution in the flowchart may be changed, and some steps may be omitted.

The above description is only a specific embodiment of the present application, but the scope of protection of the present application is not limited thereto, and those skilled in the art can also make without departing from the concept of the present application. Improvements, but these are all within the scope of this application.

The function modules and hardware structures of the terminal for realizing the above-described two-channel neural network model training method and face comparison method are respectively described below with reference to the fifth to seventh figures.

Embodiment 5

FIG. 5 is a functional block diagram of a preferred embodiment of the dual channel neural network model training device of the present application.

In some embodiments, the two-channel neural network model training device 50 operates in the terminal 7. The dual channel neural network model training device 50 can include a plurality of functional modules comprised of computer readable instruction code segments. Each of the computer readable instruction code segments in the dual channel neural network model training device 50 may be stored in the memory 71 and executed by the at least one processor 72 for execution (see Figures 1 and 2 for details). Description) Training for a two-channel neural network model.

In this embodiment, the dual channel neural network model training device 50 can be divided into multiple functional modules according to the functions performed by the dual channel neural network model. The function module may include: an obtaining module 501, a cropping module 502, a first histogram processing module 503, a first feature extraction module 504, a first normalization module 505, a first feature connection module 506, and a model update module 507. And a face detection module 508.

The obtaining module 501 is configured to obtain a raw face image of different people, and the size of each original face image is 182*182.

The cropping module 502 is configured to randomly cut each original face image into a preset number of first face images to obtain a face sample data set.

The preset number can be adjusted or modified according to actual needs. In the preferred embodiment, the cropping module 502 randomly cuts each original face image into five first face images of the same size, each of which The first face image contains a face area.

Randomly crop each original face image to obtain the first face image under arbitrary angle and arbitrary illumination conditions. The random process is the process of collecting the original face image in the simulated dynamic environment; secondly, by cutting In this way, multiple first face images can be obtained, thereby increasing the number of face sample data sets, and more face sample data sets can improve the robustness of the two-channel neural network model.

The histogram processing module 503 is configured to perform histogram equalization processing on each first face image in the face sample data set to obtain a histogram equalized face image.

The first feature extraction module 504 is configured to input each first face image in the face sample data into the first depth neural network to extract the first facial feature, and simultaneously equalize the corresponding histogram. The picture input second depth neural network extracts the second face feature.

In the preferred embodiment, the first deep neural network and the second deep neural network are neural networks having the same number of layers (for example, 200 layers). Optionally, the first depth neural network and the second deep neural network adopt a perceptron-based Inception-Resnet-V2 (Inception-V2 and Resnet combined neural network) neural network. The perceptron-based Inception-Resnet-V2 neural network is a prior art, and the present application does not repeat here.

Preferably, the first feature extraction module 504 is further configured to: the dimension of the first facial feature extracted by the first depth neural network is 64 dimensions, and the second depth neural network extracts The dimension of the second face feature is 64 dimensions.

a first normalization module 505, configured to normalize the first facial feature to obtain a first normalized facial feature, and normalize the second facial feature to obtain a second normalized Face features.

a first feature connection module 506, configured to connect the first normalized face feature and the second normalized face feature according to a preset connection rule to obtain a new normalized face feature, as a final face Feature representation.

The model update module 507 is configured to normalize the final facial feature representation and input the value to the preset loss function to calculate a loss function value. When the loss function value is less than or equal to a preset loss function threshold, the double The channel neural network model training ends and updates the weights and offsets in the two-channel neural network model; when the loss function value is greater than the preset loss function threshold, more original face images are acquired, and based on the increased original The face picture re-executes the above modules 501-507.

Further, the cropping module 502 is further configured to randomly cut each original face image into a preset number of second face images to obtain a face sample data set, where each of the face sample data sets The size of the second face image is 171*160.

In the preferred embodiment, the second face image size obtained after random cropping is 171*160, which not only can maintain the facial detail information in the original face image to the greatest extent, but also utilizes the original face image. When the face sample data set is involved in training, the training time of the two-channel neural network model can be shortened, and the training speed of the two-channel neural network model can be improved.

Further, the dual-channel neural network model training device 50 further includes: a face detection module 508, configured to detect each of the cropped second face images by using a pre-stored face detection algorithm, and determine the cropping Whether or not the face area is included in each second face picture in all the second face pictures.

The pre-stored face detection algorithm may be one of the following algorithms: a template-based face detection method, an artificial neural network-based face detection method, a model-based face detection method, and a skin color-based face detection method. The method or the face detection method based on the feature sub-face, and the like.

In this embodiment, the pre-stored face detection algorithm is not limited to the above enumeration, and any algorithm capable of detecting whether the face image is included in the cropped face image may be referred to herein. In addition, the face detection algorithm pre-stored in this embodiment is a prior art, and will not be described in detail herein.

Further, the first histogram processing module 503 is further configured to: when the face detection module 508 determines that each of the second face images in the cropped second face image includes a face region, Histogram equalization processing is performed on each second face image in the face sample data set to obtain a histogram equalization face image.

Further, the cropping module 502 is further configured to: when the face detection module 508 determines that each of the second face images that are cropped includes no face region in the face image, the pair does not include The original face image corresponding to the second face image of the face region is randomly cropped until the preset number of second face images are obtained.

The face region that is not included in the cropped second face image includes the following two situations: there is no face region at all; only a part of the face region exists.

Further, the first normalization module 505 is further configured to input each second face image in the face sample data into the first depth neural network to extract the first facial feature, and correspondingly The histogram equalization face image input second depth neural network extracts the second face feature.

In summary, the dual-channel neural network model trained in the present application can significantly enhance the feature extraction capability of the face image in the input face sample data set and the face image processed by the histogram equalization process. It mitigates the risk of performance degradation when the network level is deepened, and improves the robustness of the two-channel neural network model. Secondly, each second face image that is cropped is detected by a pre-stored face detection algorithm to ensure each piece. The second face image contains all the face regions, which can ensure the validity of the extracted face features, and can further improve the robustness of the two-channel neural network model, and facilitate the subsequent improvement of face recognition or face comparison. Recognition rate.

Embodiment 6

FIG. 6 is a functional block diagram of a preferred embodiment of the applicant's face matching device.

In some embodiments, the face matching device 60 operates in the terminal 7. The face matching device 60 can include a plurality of functional modules comprised of computer readable instruction code segments. The instruction codes of the respective computer readable instruction segments in the face matching device 60 may be stored in the memory 71 and executed by the at least one processor 72 for execution (see Figures 3 and 4 for details). Description) The comparison of faces.

In this embodiment, the face matching device 60 of the terminal 7 can be divided into a plurality of functional modules according to the functions performed by the terminal 7. The function module may include: a second histogram processing module 601, a second feature extraction module 602, a second normalization module 603, a second feature connection module 604, a calculation module 605, a result determination module 606, and a pre-processing module 607. And sorting module 608.

The second histogram processing module 601 is configured to perform histogram equalization processing on the face image of the target user after receiving the face image of the target user to be compared by the face, and obtain a histogram equalization of the target user. Face picture.

a second feature extraction module 602, configured to input a face image of the target user into a first depth neural network in the trained two-channel neural network model, and extract a third facial feature, and simultaneously The fourth facial feature is extracted from the second deep neural network in the two-channel neural network model trained by the histogram equalization face image input;

The second normalization module 603 is configured to normalize the third facial feature to obtain a third normalized facial feature, and normalize the fourth facial feature to obtain a fourth normalized facial feature. ;

The second feature connection module 604 is configured to connect the third normalized face feature and the fourth normalized face feature according to the preset connection rule to obtain a new normalized face feature. The final feature representation of the target user.

The calculating module 605 is configured to calculate a similarity between the final feature representation of the target user and the final feature representation of the registered user in the database.

The result determining module 606 is configured to determine a face comparison result according to a relationship between the calculated similarity and a preset confidence threshold.

The results of the face alignment include alignment results and comparison failure results. When the calculated similarity is greater than or equal to the preset confidence threshold, the result determination module 606 determines that the user matches, that is, the registered user who is the same person as the target user is matched in the database. When the calculated similarity is less than the preset confidence threshold, the result determination module 606 determines that the user comparison fails, that is, the registered user who is the same person as the target user is not matched in the database.

Preferably, the preset confidence threshold is 0.5.

Preferably, the face matching device 60 further includes: a pre-processing module 607, configured to pre-process the face image of the target user after receiving the face image of the target user to be compared by the face deal with. In the preferred embodiment, the pre-processing may include, but is not limited to, image denoising, illumination normalization, pose calibration, grayscale normalization, and the like.

After receiving the face image of the target user to be compared, the face image of the target user is pre-processed to further improve the face image of the target user, so as to extract more discriminative power. Face features.

Further, the second histogram processing module 601 further performs histogram equalization processing on the pre-processed target user's face image to obtain a histogram equalized face image of the target user.

Further, the second feature extraction module 602 is further configured to input the face image of the pre-processed target user into the first deep neural network in the trained two-channel neural network model to extract the third facial feature. And extracting the fourth facial feature from the second deep neural network in the trained two-channel neural network model by inputting the histogram equalized face image of the target user;

Preferably, the face matching device 60 further includes: a sorting module 608, configured to sort the calculated similarities according to a preset sorting method. In the preferred embodiment, the pre-set sorting method includes, but is not limited to, sorting from large to small, and sorting from small to large.

Further, the result determining module 606 is further configured to compare a relationship between the maximum similarity and a preset confidence threshold to determine a face comparison result.

The results of the face alignment include alignment results and comparison failure results. When the maximum similarity is greater than or equal to the preset confidence threshold, the result determination module 606 determines that the user has passed the comparison, ie, the registered user who is the same person as the target user is matched in the database. When the maximum similarity is less than the preset confidence threshold, the result determination module 606 determines that the user comparison failed, ie, the registered user who is the same person as the target user is not matched in the database.

In summary, the face matching method described in the present application can extract a better facial feature representation by using a two-channel neural network model, and a better face can be obtained by taking a shorter face comparison time. Compare effects.

Example 7

As shown in FIG. 7 , it is a hardware structure diagram of a terminal for implementing the dual channel neural network model training method and/or the face matching method described in the present application.

In the preferred embodiment of the present application, the terminal 7 includes a memory 71, at least one processor 72, and at least one communication bus 73.

It should be understood by those skilled in the art that the structure of the terminal 7 shown in FIG. 7 does not constitute a limitation of the embodiment of the present application, and may be a bus type structure or a star structure, and the terminal 7 may further include a ratio More or less other hardware or software, or different component arrangements.

In some embodiments, the memory 71 is configured to store computer readable instruction code and various data, such as the dual channel neural network model training device 50 and the face matching device 60 installed in the terminal 7, and The high speed, automatic completion of computer readable instructions or data access during the operation of the terminal 7.

In some embodiments, the at least one processor 72 may be composed of an integrated circuit, for example, may be composed of a single packaged integrated circuit, or may be composed of multiple integrated circuits of the same function or different functions, including one. Or a combination of a plurality of central processing units, microprocessors, digital processing chips, graphics processors, and various control chips. The at least one processor 72 is a control core of the terminal 7, connecting various components of the entire terminal 7 using various interfaces and lines, by running or executing computer readable instructions or modules stored in the memory 71, and The data stored in the memory 71 is invoked to perform various functions and processing data of the terminal 7, such as executing the two-channel neural network model training device 50 and/or the face matching device 60.

In some embodiments, the at least one communication bus 73 is arranged to effect connection communication between the memory 71, the at least one processor 72, and the like.

For a specific implementation of the above-mentioned instructions by the at least one processor 72, reference may be made to the description of related steps in the corresponding embodiments in FIG. 1 to FIG. 4, and details are not described herein.

The functional modules in the various embodiments of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above integrated unit can be implemented in the form of hardware or in the form of hardware plus software function modules.

It is obvious to those skilled in the art that the present application is not limited to the details of the above-described exemplary embodiments, and the present invention can be implemented in other specific forms without departing from the spirit or essential characteristics of the present application. In addition, it is to be understood that the term "comprising" does not exclude other elements or the singular does not exclude the plural. The plurality of units or devices recited in the system claims can also be implemented by a unit or device by software or hardware. The first, second, etc. words are used to denote names and do not denote any particular order.

It should be noted that the above embodiments are only used to explain the technical solutions of the present application, and are not limited thereto. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be applied. Modifications or equivalents are made without departing from the spirit and scope of the technical solutions of the present application.

Claims (20)

A two-channel neural network model training method, characterized in that the method comprises: a. obtaining a primitive face image of different people, each original face image has a size of 182*182; b. randomly cutting each original face image into a preset number of face images to obtain a face sample data set; c. performing histogram equalization processing on each face image in the face sample data set to obtain a histogram equalization face image; d. inputting each face image in the face sample data into the first depth neural network to extract the first face feature, and inputting the corresponding histogram equalized face image into the second depth neural network for extracting a second face feature; e. normalizing the first facial feature to obtain a first normalized facial feature, and normalizing the second facial feature to obtain a second normalized facial feature; f. connecting the first normalized face feature and the second normalized face feature according to a preset connection rule to obtain a new normalized face feature as a final face feature representation; g. normalizing the final facial feature representation and inputting it into a preset loss function to calculate a loss function value. When the loss function value is less than or equal to a preset loss function threshold, the dual channel neural network model training End and update the weights and offsets in the two-channel neural network model. The method of claim 1 wherein the method further comprises: When the loss function value is greater than the preset loss function threshold, more original face pictures are acquired, and the above steps a-g are re-executed based on the added original face picture. The method according to any one of claims 1 to 2, wherein the method further comprises: Each face image that is cropped is detected by using a pre-stored face detection algorithm to determine whether each face image in the cropped face image includes a face region; and When it is determined that each of the face images in the cropped face image includes a face region, each face image in the face sample data set is subjected to histogram equalization processing to obtain a histogram equalization. Face picture; or When it is determined that each of the face images in the cropped face image does not include a face region, the original face image corresponding to the face image not including the face region is continuously cropped until it is obtained. The preset number of face images. A method for performing face alignment using the two-channel neural network model according to any one of claims 1 to 3, wherein the method comprises: After receiving the face image of the target user to be compared, the histogram of the target user is subjected to histogram equalization processing to obtain a histogram equalized face image of the target user; Importing a face image of the target user into a first deep neural network in the trained two-channel neural network model to extract a third facial feature, and simultaneously mapping the target user's histogram equalized face image into the training The fourth facial feature is extracted from the second deep neural network in the good two-channel neural network model; Normalizing the third facial feature to obtain a third normalized facial feature, and normalizing the fourth facial feature to obtain a fourth normalized facial feature; And connecting the third normalized face feature and the fourth normalized face feature according to the preset connection rule to obtain a new normalized face feature as a final feature representation of the target user; Calculating a similarity between a final feature representation of the target user and a final feature representation of a registered user in the database; and The face comparison result is determined according to the relationship between the calculated similarity and the preset confidence threshold. The method of claim 4, wherein the calculating the similarity between the final feature representation of the target user and the final feature representation of the registered user in the database is SIM = 0.5 + 0.5 * COS θ, and

Wherein, COS θ is a cosine value between the feature vector A and the feature vector B, and the feature vector A is a final feature representation of the target user, and the feature vector B is a final feature representation of the registered user. The method of claim 5, wherein after the computing the similarity between the final feature representation of the target user and the final feature representation of the registered user in the database, the method further comprises: Sorting the calculated similarities according to a preset sorting method; The relationship between the largest similarity and the preset confidence threshold is determined to determine the face comparison result. The method of claim 6 wherein said determining a face alignment result comprises: When the maximum similarity is greater than or equal to the preset confidence threshold, determining that the user is compared; When the maximum similarity is less than the preset confidence threshold, it is determined that the user comparison fails. A terminal, characterized in that the terminal comprises a processor for executing computer readable instructions stored in at least one memory to implement the following steps: a. obtaining a primitive face image of different people, each original face image has a size of 182*182; b. randomly cutting each original face image into a preset number of face images to obtain a face sample data set; c. performing histogram equalization processing on each face image in the face sample data set to obtain a histogram equalization face image; d. inputting each face image in the face sample data into the first depth neural network to extract the first face feature, and inputting the corresponding histogram equalized face image into the second depth neural network for extracting a second face feature; e. normalizing the first facial feature to obtain a first normalized facial feature, and normalizing the second facial feature to obtain a second normalized facial feature; f. connecting the first normalized face feature and the second normalized face feature according to a preset connection rule to obtain a new normalized face feature as a final face feature representation; g. normalizing the final facial feature representation and inputting it into a preset loss function to calculate a loss function value. When the loss function value is less than or equal to a preset loss function threshold, the dual channel neural network model training End and update the weights and offsets in the two-channel neural network model. The terminal of claim 8 wherein said processor is further operative to execute computer readable instructions to: When the loss function value is greater than a preset loss function threshold, more original face pictures are acquired, and the above steps a-g are re-executed based on the added original face picture. The terminal according to any one of claims 8-9, wherein the processor is further configured to execute computer readable instructions to implement the following steps: Each face image that is cropped is detected by using a pre-stored face detection algorithm to determine whether each face image in the cropped face image includes a face region; and When it is determined that each of the face images in the cropped face image includes a face region, each face image in the face sample data set is subjected to histogram equalization processing to obtain a histogram equalization. Face picture; or When it is determined that each of the face images in the cropped face image does not include a face region, the original face image corresponding to the face image not including the face region is continuously cropped until it is obtained. The preset number of face images. A terminal, characterized in that the terminal comprises a processor for executing computer readable instructions stored in at least one memory to implement the following steps: After receiving the face image of the target user to be compared, the histogram of the target user is subjected to histogram equalization processing to obtain a histogram equalized face image of the target user; Importing a face image of the target user into a first deep neural network in the trained two-channel neural network model to extract a third facial feature, and simultaneously mapping the target user's histogram equalized face image into the training The fourth facial feature is extracted from the second deep neural network in the good two-channel neural network model; Normalizing the third facial feature to obtain a third normalized facial feature, and normalizing the fourth facial feature to obtain a fourth normalized facial feature; And connecting the third normalized face feature and the fourth normalized face feature according to the preset connection rule to obtain a new normalized face feature as a final feature representation of the target user; Calculating a similarity between a final feature representation of the target user and a final feature representation of a registered user in the database; and The face comparison result is determined according to the relationship between the calculated similarity and the preset confidence threshold. The terminal according to claim 11, wherein the similarity between the final feature representation of the target user and the final feature representation of the registered user in the database is SIM=0.5+0.5*COSθ, and

Wherein, COS θ is a cosine value between the feature vector A and the feature vector B, and the feature vector A is a final feature representation of the target user, and the feature vector B is a final feature representation of the registered user. The terminal of claim 12, wherein said processor executes at least one computer after said calculating a similarity between said final user's final feature representation and a final feature representation of a registered user in the database The read command is also used to implement the following steps: Sorting the calculated similarities according to a preset sorting method; The relationship between the largest similarity and the preset confidence threshold is determined to determine the face comparison result. The terminal according to claim 13, wherein the determining the face comparison result comprises: When the maximum similarity is greater than or equal to the preset confidence threshold, determining that the user is compared; When the maximum similarity is less than the preset confidence threshold, it is determined that the user comparison fails. A non-volatile readable storage medium having stored thereon computer readable instructions, wherein the computer readable instructions, when executed by a processor, implement the following steps: a. obtaining a primitive face image of different people, each original face image has a size of 182*182; b. randomly cutting each original face image into a preset number of face images to obtain a face sample data set; c. performing histogram equalization processing on each face image in the face sample data set to obtain a histogram equalization face image; d. inputting each face image in the face sample data into the first depth neural network to extract the first face feature, and inputting the corresponding histogram equalized face image into the second depth neural network for extracting a second face feature; e. normalizing the first facial feature to obtain a first normalized facial feature, and normalizing the second facial feature to obtain a second normalized facial feature; f. connecting the first normalized face feature and the second normalized face feature according to a preset connection rule to obtain a new normalized face feature as a final face feature representation; g. normalizing the final facial feature representation and inputting it into a preset loss function to calculate a loss function value. When the loss function value is less than or equal to a preset loss function threshold, the dual channel neural network model training End and update the weights and offsets in the two-channel neural network model. The storage medium of claim 15 wherein said computer readable instructions are executed by a processor to further implement the following steps: When the loss function value is greater than a preset loss function threshold, more original face pictures are acquired, and the above steps a-g are re-executed based on the added original face picture. The storage medium of any of claims 15-16, wherein the computer readable instructions are further executed by the processor to: Each face image that is cropped is detected by using a pre-stored face detection algorithm to determine whether each face image in the cropped face image includes a face region; and When it is determined that each of the face images in the cropped face image includes a face region, each face image in the face sample data set is subjected to histogram equalization processing to obtain a histogram equalization. Face picture; or When it is determined that each of the face images in the cropped face image does not include a face region, the original face image corresponding to the face image not including the face region is continuously cropped until it is obtained. The preset number of face images. A non-volatile readable storage medium having stored thereon computer readable instructions, wherein the computer readable instructions, when executed by a processor, implement the following steps: After receiving the face image of the target user to be compared, the histogram of the target user is subjected to histogram equalization processing to obtain a histogram equalized face image of the target user; Importing a face image of the target user into a first deep neural network in the trained two-channel neural network model to extract a third facial feature, and simultaneously mapping the target user's histogram equalized face image into the training The fourth facial feature is extracted from the second deep neural network in the good two-channel neural network model; Normalizing the third facial feature to obtain a third normalized facial feature, and normalizing the fourth facial feature to obtain a fourth normalized facial feature; And connecting the third normalized face feature and the fourth normalized face feature according to the preset connection rule to obtain a new normalized face feature as a final feature representation of the target user; Calculating a similarity between a final feature representation of the target user and a final feature representation of a registered user in the database; and The face comparison result is determined according to the relationship between the calculated similarity and the preset confidence threshold. The storage medium of claim 18, wherein said computer readable instructions are processed after said calculating a similarity between said final user's final feature representation and a final feature representation of a registered user in the database The following steps are also implemented when the device is executed: Sorting the calculated similarities according to a preset sorting method; The relationship between the largest similarity and the preset confidence threshold is determined to determine the face comparison result. The storage medium according to claim 19, wherein said determining a face comparison result comprises: When the maximum similarity is greater than or equal to the preset confidence threshold, determining that the user is compared; When the maximum similarity is less than the preset confidence threshold, it is determined that the user comparison fails.

Images