CN108427921A - A kind of face identification method based on convolutional neural networks - Google Patents

Abstract

The present invention provides a face recognition method based on convolutional neural network. The face recognition method based on deep learning overcomes the shortcomings of traditional face recognition methods. By building a new convolutional neural network model in CASIA-WebFace face Large-scale training is carried out on the data set, wherein the convolutional layer of the network model of the present invention uses the Mlp convolutional layer to improve the feature extraction ability of the face, uses the MFM activation function to increase the nonlinearity of the model, and adds the Center Loss loss function to Improve the network's ability to classify faces. Finally, the trained model is applied to face classification prediction and face verification. Among them, the recognition rate of 90.3% has been achieved in the face classification prediction, and the accuracy rate of 92.5% has been achieved in the face verification experiment.

Description

A Face Recognition Method Based on Convolutional Neural Network

technical field

The invention relates to the technical field of face recognition methods, in particular to a face recognition method based on a convolutional neural network.

Background technique

Since various countries have increased their supervision on national security and social security, and identification is one of the important means, biometric identification technology has gradually come into people's sight. Among them, face recognition technology provides a simple, easy and reliable way for human identity verification. Due to the rise of deep learning methods in recent years, the recognition effect of face recognition methods based on deep learning has improved a lot, and face recognition technology has made great progress.

Face recognition is a major research hotspot in pattern recognition classification. The biggest difficulty in face recognition is how to distinguish between intra-class changes due to factors such as lighting, actions, and expressions, and inter-class changes due to different individuals. Feature extraction from face images is a key issue in face recognition, and the effectiveness of face features determines the performance of face recognition. Existing face recognition methods include geometric feature face recognition methods, elastic graph matching face recognition methods, line Hausdorff distance (LHD) face recognition methods, support vector machine (SVM) face recognition methods, and face recognition methods based on Face recognition method based on eigenface (PCA). The calculations of these face recognition methods are relatively complicated, and manual operations are required to complete them in actual use. In addition, the above-mentioned methods are difficult to meet the needs of face recognition with a large amount of data, so the recognition effect obtained is not satisfactory and needs to be further improved.

As a new machine learning theory, deep learning is motivated by establishing and simulating a neural network that simulates the human brain for analysis and learning. It imitates the mechanism of the human brain to interpret data such as images, sounds, and texts. The essence of deep learning is to discover the distributed feature representation of data by combining low-level features to form more abstract high-level representation attribute features. So deep learning is also called unsupervised feature learning. The convolutional neural network in deep learning has good performance for image recognition. It has become a representative deep learning technology and is widely used in image processing and computer vision.

Contents of the invention

In order to solve the problems described in the background technology, the present invention provides a face recognition method based on a convolutional neural network. , the purpose is to overcome the shortcomings of the above-mentioned face recognition technology, provide a face recognition method based on deep learning, and use the strong unsupervised feature learning ability of deep learning to further improve the performance of face recognition.

In order to achieve the above object, the present invention adopts the following technical solutions to realize:

A face recognition method based on convolutional neural network, comprising the following steps:

Step 1: Select a database of face images and preprocess the face images;

Step 2: Build the convolutional neural network model zy_net;

Step 3: training of convolutional neural network model zy_net;

Step 4: Perform face classification prediction and face verification on the trained network model;

In the described step 1, the face database adopted is CASIA-WebFace;

In said step 2, building the convolutional neural network includes the design and implementation of the convolutional neural network zy_net, which includes the design of the convolutional layer, the design of the activation function and the design of the loss function; wherein the convolutional layer uses the Mlp convolutional layer to Improve the feature extraction ability of the face, the MFM excitation function used in the excitation function increases the nonlinearity, and effectively separates the noise signal from the information signal, and adds a center loss loss function to improve the classification ability of the network for faces;

The training of the convolutional neural network zy_net in the step 3 includes two steps of forward training and reverse tuning:

Step 301, the forward training of the convolutional neural network zy_net, including the forward training of the bottom-up unsupervised learning method; the specific process of the forward training of the convolutional neural network is as follows:

(a) Convolution layer operation: the formula is expressed as follows:

in, Represents the jth feature map of the l-th layer, f( ) represents the activation function, _Mj represents the set of selected input maps, Indicates the bias item corresponding to the jth feature map, Represents the convolution kernel between the j-th feature map of the l-th layer and the i-th feature map connection of the l-1 layer, "*" indicates the convolution operation;

(b) Downsampling layer operation; for the downsampling layer, if there are N input images, there are N output images, and the formula is expressed as:

Among them, down( ) represents the downsampling function, and the typical operation is to sum all the pixels in different N×N size regions of the input image; Indicates the coefficient corresponding to the jth feature map of the l-th layer, Its corresponding bias term;

Step 302, the reverse tuning of the convolutional neural network is to use a top-down supervised learning method for tuning. After tuning, the network weight value of each hidden layer in the convolutional neural network model can be Reach the optimal value; the convolutional neural network tuning process is as follows:

(a) Gradient calculation of the convolutional layer;

This process is described by the formula:

Among them, * represents the dot multiplication operation, that is, multiplication of each pixel, and up(.) represents an upsampling operation. If the sampling factor of the downsampling is n, each pixel is copied n times in the horizontal and vertical directions. This function can be multiplied by the Kronecker product to fulfill:

In this way, for a given feature map of the convolutional layer, its residual map can be calculated, and through this residual map, the gradient of the bias item corresponding to the feature map and the gradient of the corresponding convolution kernel can be obtained, see The following formula;

Among them, J is the cost function, item During convolution with An area that is multiplied element by element, u, v represent the image coordinates in the feature map;

(b) Gradient calculation of the downsampling layer;

The parameters involved in the forward process of downsampling are a multiplicative factor β and a bias term b corresponding to each feature map. If the residual map of this layer is obtained, the gradient of these two parameters can be easily obtained; This process can be expressed in formulas as:

After obtaining the residual graph, the gradient calculation corresponding to the multiplicative factor β and the paranoid term b satisfies the formula:

in,

Described step four is specifically, the output method of face classification prediction result and face verification result is as follows:

Step 401, select 10575 face images of different categories in the test sample and input them into the network model, and finally output the classification prediction results of the face images;

Step 402, the face verification database adopts the LFW face database. During training, each pair of images is input into the trained network model in turn, and finally a 1×512×1×1 dimensional feature vector of each image is obtained. It represents the features of the face image and calculates the difference of the cosine distance of each face pair feature vector, and selects an appropriate threshold to determine whether the face pair belongs to the same identity.

Compared with prior art, the beneficial effect of the present invention is:

1. The face recognition method based on deep learning of the present invention overcomes the deficiencies of traditional face recognition methods, and conducts large-scale training on the CASIA-WebFace face data set by building a new convolutional neural network model, wherein the present invention The convolutional layer of the network model uses the Mlp convolutional layer to improve the feature extraction ability of faces, uses the MFM activation function to add nonlinearity to the model, and adds the Center Loss loss function to improve the network's ability to classify faces. Finally, the trained model is applied to face classification prediction and face verification. Among them, the recognition rate of 90.3% has been achieved in the face classification prediction, and the accuracy rate of 92.5% has been achieved in the face verification experiment.

2. For the extraction of facial features, the method of image convolution is used to extract facial image features without any manual operation.

3. The inventive method provides a lightweight face recognition network model, which achieves a better recognition effect.

4. The trained network model can be used for face classification prediction and face verification.

Description of drawings

Fig. 1 is a block diagram of the face recognition system based on deep learning of the present invention;

Fig. 2 is the method network model structure of the present invention;

Fig. 3 is the partial human face image after the preprocessing of the present invention;

Fig. 4 is a comparison between the method of the present invention and other methods on face classification prediction;

Fig. 5 is a comparison of the face verification results between the method of the present invention and other methods.

Detailed ways

The specific embodiments provided by the present invention will be described in detail below in conjunction with the accompanying drawings.

Figure 1 is a block diagram of the system, including:

Step 1: Select the database of face images and preprocess the face images;

Step 2: Build a convolutional neural network model;

Step 3: training of convolutional neural network model zy_net;

Step 4: Output classification results.

Fig. 2 is a network model structure of the present invention, which includes 5 layers of Mlp convolutional layers, 4 layers of downsampling layers, two fully connected layers and a Softmax classification layer.

Figure 3 is the face image after preprocessing. The original images in the face data set have various forms, so in order to improve the recognition effect, it is necessary to preprocess the face images in the face database (face feature point detection, human face alignment, face cropping), and the preprocessed face image is a face image with a size of 128×128.

Fig. 4 is a comparison of the face recognition method of the present invention and the Alexnet network model on the effect of face classification prediction. It can be seen from Fig. 4 that the face recognition method of the present invention is 22.8% higher than the Alexnet model in face recognition rate.

Fig. 5 is a comparison between the face recognition method of the present invention and the Alexnet network model on the face verification results. It can be seen from Fig. 5 that the correct rate of the network model of the present invention in face verification is 23.6% higher than that of the Alexnet model in face verification.

Specific implementation steps:

(1) Select the database of face images and preprocess the face images

Choose Li Ziqing's CASIA-WebFace face database, which contains more than 500,000 face pictures, a total of 10,575 faces, and the number of face images in each category ranges from dozens to hundreds, which is a very suitable A database for training face recognition networks. An example of a CASIA-WebFace sample image is shown in Figure 3. Image preprocessing first detects facial feature points on the original images of different sizes in the database, and classifies and detects the positions of the two eyes, the nose and the two corners of the mouth of the face image, and detects a total of 5 feature points. According to the detected feature points, we can perform a face alignment operation to correct the biased face images in the face dataset. The final face clipping operation is to clip the face according to the position of the feature points, and the final captured face image is an image sample with a size of 128×128.

(2) Construction of convolutional neural network zy_net

The network model of the present invention includes 5 Mlp convolutional layers, each convolutional layer is followed by an activation function layer and a downsampling layer (Max pooling), and finally 2 fully connected layers and a Softmax layer. The following is a detailed introduction to the construction principle of the network model:

(2-1) The convolutional layer uses the Mlp convolutional layer to improve the feature extraction ability of the face. The Mlp convolutional layer is equivalent to adding multiple 1×1 convolutional layers after the traditional convolutional layer, and performing another convolution on each feature map output by the traditional convolutional layer. The formula is as follows:

...

Where (i, j) represents the position index of the image pixel, x _{i, j} represents the image block in the convolution window, and k represents the index of the feature map to be extracted.

(2-2) The MFM activation function used by the activation function to increase nonlinearity, the maximum feature map (MFM) operation, which is an operation that extends the MAXOUT activation function, can effectively segment the noise signal and the information signal to improve recognition accuracy. Unlike the MAXOUT activation function, MFM makes the convolutional neural network model simpler and more robust.

(2-3) Add the Center Loss loss function to improve the network's ability to classify faces. The addition of the Center Loss loss function makes the inter-class element spacing smaller, which facilitates the generalization of unrecognized face features, and avoids recognition errors due to too small inter-class distances between different classes. The Center Loss loss function maintains a class center c for each class, and then in the feature layer, if the feature of the class center in the sample is too far away, it will be punished. The formula is: x _i is the feature value of the i-th picture, and c _yi is the center of the category to which the picture belongs (the center of the feature value of the category).

(3) Training of convolutional neural network zy_net

It includes two steps of forward propagation training and backward propagation tuning. The image samples used in the training process come from the training samples in the face database. Convolutional neural network structure parameter setting: the number of convolution kernels of the Mlp convolutional layer is set to 96, 192, 384, 256, and 256 respectively, and the output of the first fully connected layer is a 1×512×1×1-dimensional feature vector . The last fully connected layer outputs 10575 categories. During the training process, the number of cycles of the convolutional neural network is 5,000,000 in order to achieve a better convergence effect.

(3-1) The specific process of the forward training of the convolutional neural network zy_net is as follows:

(a) Convolutional layer operation: In a convolutional layer, the feature map of the previous layer is convoluted by a learnable convolution kernel, these convolution operations are combined, and then an output feature map can be obtained through an activation function , the formula is expressed as follows:

in, Represents the jth feature map of the l-th layer, f( ) represents the activation function, _Mj represents the set of selected input maps, Indicates the bias item corresponding to the jth feature map, Represents the convolution kernel between the j-th feature map of the l-th layer and the i-th feature map connection of the l-1 layer, "*" indicates the convolution operation. where the activation function Is the MFM activation function, that is, the nonlinear action function of neurons.

(b) Downsampling layer operation. For the downsampling layer, there are N input images, and there are N output images, but each output image becomes smaller. The formula is expressed as:

Among them, down(·) represents the downsampling function, and the typical operation is to sum all the pixels in different N×N size regions of the input image. Indicates the coefficient corresponding to the jth feature map of the l-th layer, Its corresponding bias term.

(3-2) The reverse tuning of the convolutional neural network is to use a top-down supervised learning method for tuning, that is, to use labeled sample data for training, and the error is transmitted from top to bottom to tune the network. excellent. After tuning, the network weight value of each hidden layer in the convolutional neural network model can reach the optimal value. The convolutional neural network tuning process is as follows:

(a) Gradient calculation for convolutional layers.

Assuming that each convolutional layer l will be connected to a downsampling layer l+1, according to the backpropagation algorithm of the classical neural network, in order to obtain the gradient of the weight corresponding to each neuron in the lth layer, it is necessary to first obtain the lth The residual δ of each neural node of . In order to find this residual, we must first sum the residuals of the nodes in the next layer (nodes of layer l+1 connected to the node of interest in current l) (to obtain δ ^l+1 ), and then multiply these connections by the corresponding The weight (the weight of the connection layer node l and the l+1 layer node) W, and then multiplied by the derivative value of the activation function f of the input u of the neuron node of the current layer l, so that we can get The residual corresponding to each neural node in the current layer l.

Due to the existence of downsampling, the residual δ corresponding to a neuron node in the sampling layer corresponds to an area of the output feature map of the previous layer (downsampling window size). Therefore each node of each feature map in layer l is connected to only one node in layer l+1. The above residual is actually the partial derivative of the cost function to each neuron, so each pixel in the feature map has a corresponding residual, which constitutes the residual map corresponding to the feature map. In order to efficiently calculate the residual of layer l, it is necessary to upsample the residual map corresponding to the downsampling layer, and then multiply the residual map obtained by this upsampling with the partial derivative of the excitation value of the feature map of the current layer l. The weights of the feature maps of the downsampling layer all take the same value β, so multiplying the result obtained in the previous step by β completes the calculation of the residual of the first layer l. This process is described by the formula:

Among them, * represents the dot multiplication operation, that is, multiplication of each pixel, and up(.) represents an up-sampling operation. The process is that if the sampling factor of the down-sampling is n, each pixel is copied horizontally and vertically by n Second, in fact, this function can be multiplied by the Kronecker product to fulfill:

In this way, for a given feature map of the convolutional layer, its residual map can be calculated, and through this residual map, the gradient of the bias item corresponding to the feature map and the gradient of the corresponding convolution kernel can be obtained, see The following formula.

Among them, J is the cost function, item During convolution with An area multiplied element by element, u, v represent the image coordinates in the feature map.

(b) Gradient calculation for the downsampling layer.

The parameters involved in the forward process of downsampling are a multiplicative factor β and a bias term b corresponding to each feature map. If the residual map of this layer is obtained, the gradient of these two parameters can be easily obtained. When calculating the gradient of the convolution kernel, it is necessary to find which area in the input image corresponds to which pixel in the output image. Here, it is necessary to find which area in the residual image of the current layer corresponds to a given pixel in the residual image of the next layer, so that The residual can be backpropagated back. In addition, it needs to be multiplied by the weight between the input area and the output pixel. The formula is expressed as:

After obtaining the residual map, the gradient calculation process of the multiplicative factor β and the bias term b is as follows.

in,

(4) Output of face classification prediction results and face verification results.

(4-1) Face classification prediction is to apply the trained convolutional neural network model to face recognition classification, select 10575 face images of different categories in the test sample, input them into the network model, and then output the face image classification prediction results. The network model outputs the recognition rate of 10575 face images, and the average recognition rate of 10575 images is taken as the recognition result of the network model for face image classification. Through 10575 experiments, the calculated model recognition rate is 90.3%.

(4-2) The face verification data set uses the LFW face database. There are 3000 pairs of face images of the same identity and 3000 pairs of face images of different identities in LFW. During training, input each image into the trained network model in turn, and finally get a 1×512×1×1 dimensional feature vector of each image, which represents the features of the face image and calculates the The difference between the cosine distances of the pairs, and select an appropriate threshold to determine whether the face pair belongs to the same identity. Finally, the ratio of the number of correct face verifications to 6000 is taken as the result of face verification of the model, which is the accuracy rate of face verification, and the accuracy rate of face verification of the final model is 92.5%.

The face recognition method based on deep learning of the present invention overcomes the deficiencies of traditional face recognition methods, and conducts large-scale training on the CASIA-WebFace face data set by building a new convolutional neural network model, wherein the network model of the present invention The convolutional layer uses the Mlp convolutional layer to improve the feature extraction ability of faces, uses the MFM activation function to add nonlinearity to the model, and adds the Center Loss loss function to improve the network's ability to classify faces. Finally, the trained model is applied to face classification prediction and face verification. Among them, the recognition rate of 90.3% has been achieved in the face classification prediction, and the accuracy rate of 92.5% has been achieved in the face verification experiment. The present invention provides a high-performance face recognition method based on deep learning for face recognition.

The above embodiments are implemented on the premise of the technical solutions of the present invention, and detailed implementation methods and specific operation processes are given, but the protection scope of the present invention is not limited to the above embodiments. The methods used in the above examples are conventional methods unless otherwise specified.

Claims (2)

1. A face recognition method based on a convolutional neural network is characterized by comprising the following steps:

step 1: selecting a database of the face image and preprocessing the face image;

step 2: building a convolutional neural network model zy _ net;

and step 3: training a convolutional neural network model zy _ net;

and 4, step 4: carrying out face classification prediction and face verification on the trained network model;

in the step 1, the adopted face database is CASIA-Webface;

in the step 2, the construction of the convolutional neural network comprises the design and implementation of the convolutional neural network zy _ net, wherein the convolutional layer is designed, the excitation function is designed and the loss function is designed; the convolutional layer adopts an Mlp convolutional layer to improve the feature extraction capability of the human face, an MFM excitation function used by an excitation function is used for increasing nonlinearity, a noise signal and an information signal are effectively separated, and a center loss function is added to improve the classification capability of the network on the human face;

the training of the convolutional neural network zy _ net in the step 3 includes: forward training and backward tuning:

301, performing forward training of the convolutional neural network zy _ net, wherein the forward training is performed in a bottom-up unsupervised learning mode; the forward training of the convolutional neural network specifically comprises the following steps:

(a) and (3) convolutional layer operation: the formula is expressed as follows:

wherein,j-th feature map representing the l-th layer, f (.) representing the excitation function, M_jRepresents a collection of selected input maps that are,representing the bias term corresponding to the jth characteristic diagram,represents a convolution kernel between the j-th characteristic diagram of the l-th layer and the i-th characteristic diagram of the l-1 layer, and the 'x' represents convolution operation;

(b) a downsampling layer operation; for the down-sampled layer, there are N input maps, and there are N output maps, the formula is expressed as:

where down (.) represents a down-sampling function, the typical operation is to sum all pixels in different nxn sized regions of the input image;representing the coefficient corresponding to the jth characteristic diagram of the ith layer,the bias term for which it corresponds;

step 302, performing reverse tuning of the convolutional neural network by using a top-down supervised learning mode to perform tuning and tuning, so that the network weight value of each hidden layer in the convolutional neural network model can reach an optimal value; the tuning process of the convolutional neural network is as follows:

(a) calculating the gradient of the convolution layer;

this process is described by the formula:

wherein, the dot multiplication operation is represented, i.e. each pixel is multiplied, up represents an up-sampling operation, if the down-sampling factor is n, each pixel is copied n times in horizontal and vertical directions, and the function can be represented by Kronecker productTo realize that:

thus, for a given characteristic diagram of the convolution layer, a residual diagram can be calculated, and the gradient of the characteristic diagram corresponding to the bias term and the gradient of the convolution kernel can be obtained through the residual diagram, which is shown in the following formula;

wherein J is a cost function,item(s)In the convolution withA block of areas multiplied element by element, u, v representing image coordinates in the feature map;

(b) calculating the gradient of the down-sampling layer;

the parameters involved in the down-sampling forward process are a multiplicative factor β and a bias term b corresponding to each feature map, and if a residual map of the layer is obtained, the gradient of the two parameters can be easily obtained, and the process can be expressed by the following formula:

after obtaining the residual map, the multiplicative factor β and the gradient calculation corresponding to the paranoia term b satisfy the formula:

wherein,

2. the face recognition method based on the convolutional neural network as claimed in claim 1, wherein the fourth step is specifically that the output method of the face classification prediction result and the face verification result is as follows:

step 401, selecting 10575 face images of different categories from a test sample, inputting the face images into a network model, and finally outputting a classification prediction result of the face images;

step 402, the face verification database is an LFW face database, each pair of images is sequentially input into a trained network model during training, finally, a feature vector of 1 × 512 × 1 × 1 dimension of each image is obtained, which represents the feature of the face image and calculates the difference between cosine distances of feature vectors of each face pair, and a proper threshold is selected to determine whether the face pair belongs to the same identity.