CN106503687A - The monitor video system for identifying figures of fusion face multi-angle feature and its method - Google Patents

Abstract

The invention discloses a surveillance video person identity recognition system and a method thereof which integrate multi-angle features of faces, comprising a target detection module, a multi-angle face recognition module and an identity matching module. The target detection module is used to convert a surveillance video into a key frame set containing people, the multi-angle face recognition module is used to convert the key frame set containing people into a sequence of face images with angle value labels, and the identity matching module is used to To complete the similarity matching between the face image sequence with the angle value label and the feature vector of the image sequence in the identity database, and find the closest identity output as the recognition result. The invention can synthesize the feature information of multiple angles of the human face in the monitoring video, thereby improving the accuracy of identity recognition when the posture of the person in the monitoring video is relatively random.

Description

Surveillance video person identification system and method based on fusion of multi-angle features of human face

technical field

The invention belongs to the field of intelligent video monitoring, relates to technologies such as pattern recognition and artificial intelligence, and in particular relates to a monitoring video character identification system and method thereof which integrates multi-angle features.

Background technique

In modern society, there are more and more places that require identity authentication. Face recognition is a technology that uses the biological characteristics of human beings for identity authentication. With the development needs of video surveillance, information security, access control and other application fields, video face recognition systems have great application prospects in these fields.

At present, face recognition is mainly used in fields such as attendance, access control and identity verification, and there is no relatively complete face recognition monitoring equipment used in monitoring scenarios. The main reason is that face recognition in video sequences is more complex than face recognition in static images. For example, video surveillance cameras are far away from the target, making it difficult to collect good-quality face images; In the state of motion, the probability of side faces and facing away from the camera is greatly increased; in addition, surveillance scenes usually have body occlusions, etc., which bring considerable difficulties to face detection and face comparison recognition.

This also requires overcoming the problems of low probability of people appearing in the surveillance video, fuzzy pictures, and small number of samples, and identifying the identity information of the person by combining images from various angles of the face. The traditional two-dimensional face recognition technology is based on the key feature extraction and comparison of the face to complete the recognition. The current mainstream feature extraction algorithms include the LBP feature to express the image texture features, the eigenface method (PCA) to express the statistical features of the face image, and An extraction method based on image local geometric features. The usual feature extraction method is based on static frontal face images. This method usually relies on the accurate detection of facial features. The integrity of the features is an extremely critical factor for the success of the algorithm. The extracted features are used for face matching. The recognition performance is better. Its defect is that it is subject to a lot of external interference. Once the face is rotated, occluded or partially blurred, causing some features to disappear and resulting in incomplete face image features, this algorithm will fail, resulting in the inability to communicate with the face information in the database. match. Therefore, this method is difficult to adapt to complex problems such as low-quality face images, multiple angles, and long distances brought about by video surveillance.

Contents of the invention

The present invention overcomes the deficiencies of the prior art, and proposes a monitoring video person identification system and method thereof that integrates the multi-angle features of the human face, in order to comprehensively monitor the feature information of multiple angles of the human face in the monitoring video, thereby improving the quality of life in the surveillance video. The accuracy of the identification of the person in the surveillance video is relatively random.

The present invention adopts following technical scheme in order to achieve the above-mentioned purpose of the invention:

The characteristics of a monitoring video person identification system integrating multi-angle features of the present invention are that it consists of: a target detection module, a multi-angle face recognition module and an identity matching module;

The target detection module collects single-frame human pictures and unmanned pictures, and assigns category labels after performing size normalization processing on each picture, thereby obtaining a training sample data set composed of positive samples and negative samples; extracting the The SIFT feature of each sample in the training sample data set, thereby converting each sample into a feature vector; then using the support vector machine SVM to train the feature vector to obtain a target detection model;

The target detection module converts the monitoring video into a series of single-frame pictures as a test set; utilizes the target detection model to identify whether each frame of pictures in the test set contains people, and if it contains people, then retains the corresponding single-frame pictures, And as a key frame, otherwise, discard the corresponding single frame picture, so as to obtain a key frame set;

The multi-angle face recognition module takes the front face of the face at 0° as the starting collection point, and collects a face image every k degrees clockwise, wherein, the face images of 90° and 270° are not collected, each The identity collects a total of m=360/k-2 face images from different angles, thus forming a face image sequence composed of a human face and a non-face, and then obtaining a multi-angle face image sequence composed of n different identities. Face database; each face image is classified into its own category set according to the angle value, and the multi-angle face is expanded by partially occluding the upper left, upper right, lower left, lower right, and middle parts of each face image database, thus forming a multi-angle face training set; extracting the SIFT feature of each face image in the multi-angle face training set, so that each face image is converted into a multi-angle feature vector; and then using the support vector machine SVM to The multi-angle feature vector is trained to obtain a multi-angle face detection model;

The multi-angle face recognition module uses the multi-angle face detection model to detect the set of key frames to obtain the face angle value of each key frame, and judge whether each key frame is Contains a human face; if it contains a human face, then retain the corresponding key frame, otherwise discard the corresponding key frame, thereby converting the key frame set into a human face image sequence with an angle value label;

The identity matching module constructs an identity library using the face image sequences of faces in the multi-angle face database, uses the identity library as the input of the neural network, and performs training to obtain a neural network model for identity recognition ; The face image sequence with the angle value label is used as a test sample again, and the test sample is input in the described neural network model for identity recognition, and any layer of output in the middle of the neural network is extracted as a feature, thereby Convert the test sample into a multi-dimensional identity feature vector to be identified;

The identity matching module inputs n face image sequences of different identities in the multi-angle face database into the neural network model for identity recognition, thereby obtaining n multidimensional identity feature vectors for matching; Then the multi-dimensional identity feature vector to be identified is compared with the n multi-dimensional identity feature vectors for matching by cosine distance similarity, and the multi-dimensional identity feature vector for matching corresponding to the maximum cosine value is found as the corresponding multi-dimensional identity feature vector to be identified The identity matching result of the multi-dimensional identity feature vector, the identity tag corresponding to the identity matching result is used as the identity recognition result of the multi-dimensional identity feature vector to be identified.

A kind of surveillance video character identification method of fusion multi-angle feature of the present invention is characterized in following steps:

Step 1. Collect single-frame pictures of people and people, and assign category labels to each picture after normalizing the size, so as to obtain a training sample data set composed of positive samples and negative samples;

Step 2, extracting the SIFT feature of each sample in the training sample data set, thereby converting each sample into a feature vector;

Step 3, using a support vector machine (SVM) to train the feature vectors to obtain a target detection model;

Step 4, convert the monitoring video into a series of single-frame pictures, and use it as a test set;

Step 5. Use the target detection model to identify whether each frame of the test set contains a person, and if it contains a person, keep the corresponding single-frame picture and use it as a key frame; otherwise, discard the corresponding single-frame picture to obtain a key frame set ;

Step 6. Start with the front face of the face at 0°, and collect a face image every k degrees clockwise, where the 90° and 270° face images are not collected, and each identity collects a total of m= 360/k-2 face images from different angles to form a face image sequence composed of human faces and non-faces, and then obtain a multi-angle face database composed of n face image sequences with different identities;

Step 7. Classify each face image into the corresponding category set according to the angle value, and expand the multi-angle face database by partially blocking the upper left, upper right, lower left, lower right, and middle parts of each face image , thus forming a multi-angle face training set;

Step 8, extracting the SIFT feature of each face image in the multi-angle face training set, thereby converting each face image into a multi-angle feature vector;

Step 9, using a support vector machine (SVM) to train the multi-angle feature vector to obtain a multi-angle face detection model;

Step 10, using the multi-angle face detection model to detect the set of key frames to obtain the face angle value of each key frame;

Step 11, judging whether each key frame contains a human face by the angle value of the human face; if it contains a human face, then keep the corresponding key frame, otherwise discard the corresponding key frame, thereby converting the key frame set into one with an angle value Labeled face image sequence;

Step 12, using the human face image sequence in the multi-angle human face database to construct an identity library, using the identity library as the input of the neural network, and training, thereby obtaining a neural network model for identity recognition;

Step 13, using the face image sequence with the angle value label as a test sample, and inputting the test sample into the neural network model for identity recognition, extracting the output of any layer in the middle of the neural network as a feature, Thereby converting the test sample into a multi-dimensional identity feature vector to be identified;

Step 14, input the face image sequences of n different identities in the identity library into the neural network model for identity recognition respectively, so as to obtain n multidimensional identity feature vectors for matching;

Step 15: Comparing the multi-dimensional identity feature vector to be identified with the n multi-dimensional identity feature vectors for matching by cosine distance similarity, and finding the multi-dimensional identity feature vector for matching corresponding to the maximum cosine value as a corresponding For the identity matching result of the multi-dimensional identity feature vector to be identified, the identity tag corresponding to the identity matching result is used as the identity identification result of the corresponding multi-dimensional identity feature vector to be identified.

Compared with the prior art, the beneficial effects of the present invention are reflected in:

1. The present invention proposes a face detection that is suitable for monitoring video, and the face posture of the collected object is relatively random, and the posture angle value is recognized. The convolutional neural network is used to extract features from a sequence of face images from different angles within the acquisition cycle, which overcomes the difficulty of extracting features from a single frame of frontal face images in the prior art for recognition, and realizes the fusion of features from multiple angles. The deep learning algorithm reduces the feature dimension and improves the accuracy of face recognition in surveillance video. It can realize the early warning of dangerous people, protect people's life and property safety, and play an important role in assisting the maintenance of public security in our country.

2. The present invention uses the scale-invariant feature transformation (SIFT) algorithm of detecting local features of images in the image feature extraction stage in the target detection module and the multi-angle face recognition module to monitor video single-frame pictures and key frame sets Perform feature extraction. The SIFT feature is not only scale invariant, it can still achieve good detection results even if the rotation angle, image brightness or shooting vision is changed. In the case that the movement posture of the person in the surveillance video is relatively random, more image invariant features can be learned, which improves the accuracy of target recognition in complex environments.

3. In the multi-angle face recognition module of the present invention, the collected multi-angle face database adopts a partial occlusion method to expand the data set. Making the training samples contain a large number of occluded samples can improve the distribution of samples on the one hand, and solve the limitations of face detection caused by face occlusion in the prior art. On the other hand, it also effectively solves the problem of small number of samples. The problem of weak generalization ability. This kind of data enhances the method of expanding the database, improves the accuracy of the algorithm and the feature dimension, and then can improve the accuracy of recognition.

4. In the identity matching module, the present invention uses the face image sequence in the identity library to train the convolutional neural network, and the CNN model of construction is regarded as a feature extractor, which is used to learn the identity feature of a different angle face image sequence. The frontal features and side features of a person's face are fused together and converted into an advanced multi-dimensional identity feature vector, which improves the accuracy of identity expression, matches the target with the identity feature vector in the library, and searches for the identity information of the target person , which overcomes the complexity of extracting these image features from different angles by traditional methods and then training classifiers for matching. Deep learning is a new field in machine learning research. Its motivation is to establish and simulate the neural network of human brain for analysis and learning. Compared with general machine learning algorithms, deep learning interprets data by imitating the mechanism of human brain. The eigenvalues are well optimized for dimensionality reduction, and at the same time, identity eigenvalues with better feature levels can be obtained.

Description of drawings

Fig. 1 is a schematic diagram of the identification system of the present invention.

detailed description

In this embodiment, as shown in Figure 1, a monitoring video person identification system that integrates multi-angle features is composed of: a target detection module: used to convert a piece of monitoring video into a key frame set containing people; multi-angle people Face recognition module: used to convert the key frame set containing people into a face image sequence with angle value labels; identity matching module: used to complete the face image sequence with angle value labels and the image sequence in the identity library identity match.

The target detection module includes two stages: training human image target detection model and video preprocessing. Training the human picture object detection model first collects single frame human pictures and unmanned pictures, and assigns category labels after normalizing the size of each picture, so as to obtain a training sample data set composed of positive samples and negative samples. The positive samples are pictures of people in the data set, the category label is set to 1, and the negative samples are pictures of no one with a label of 0, which are placed in a folder respectively; the SIFT features of each sample in the training sample data set are extracted, and each sample is Extract a fixed number of feature points to convert each sample into a feature vector; then use the support vector machine (SVM) to train the feature vectors of the training sample data set to obtain a human image target detection model, which is used to complete the surveillance video preprocessing;

In the video preprocessing stage of the target detection module, the FFmpeg software package is used to convert the surveillance video into a series of single-frame pictures, and as a test set, each picture extracts the same number of SIFT features; Whether each frame contains people, the output result of SVM corresponds to the identification label of each test set piece, if the category is 1, it means that it contains people, then keep the corresponding single frame picture, cut out the part containing people, and use it as a key frame, otherwise, Discard the corresponding single-frame picture to obtain a set of key frames;

The multi-angle face recognition module includes two stages: training a multi-angle face detection model and key frame set face angle recognition. To train the multi-angle face detection model, first build a multi-angle face database, start with the frontal 0° of the face as the collection point, and collect a face image every k degrees clockwise, of which 90° and 270° are not collected face images. In this embodiment, k=10 is taken, and m=360/k-2=34 face images from different angles are collected for each identity, thereby forming a face image sequence composed of a human face and a non-face, in which there are human faces The face image sequence of the face contains m/2 images in the range of [0°,80°]∪[280°,350°], and the face image sequence of no face contains the range of [100°,260°] m/2 images, and then obtain a multi-angle face database composed of n face image sequences with different identities; assign a category label to each face image according to the angle value, and classify each face image according to the angle value in the corresponding category set. At the same time, in order to expand the training samples and solve the occluded face detection, some data enhancement methods can be applied to expand the multi-angle face database by partially occluding the upper left, upper right, lower left, lower right, and middle parts of each face image. Thus forming a multi-angle face training set; extracting the SIFT feature of each face image in the multi-angle face training set, thereby converting each face image into a multi-angle feature vector; and then using the support vector machine SVM to perform multi-angle feature vector Perform training to obtain a multi-angle face detection model, which can be used to identify the angle of the face in the key frame set;

In the face angle recognition stage of the key frame set of the multi-angle face recognition module, first extract the SIFT feature of each picture in the key frame set, convert the picture into a vector, and use the multi-angle face detection model to detect the key frame set, according to SVM Output the result and get the face angle value of each key frame; judge whether each key frame contains a face by the face angle value, if it contains a face, then keep the corresponding key frame, otherwise discard the corresponding key frame, and the reserved key frame The frame detects and crops the face part in the image, and the cropped image is unified into a fixed size. Thereby converting the set of keyframes into a sequence of face images labeled with angle values;

The identity matching module includes three stages: training a neural network model for identity recognition, learning identity features of all identity databases and face image sequences to be recognized, and identity similarity matching. Use the face image sequences in the multi-angle face database to construct the identity database. Each identity corresponds to an integer and real number category label. Each face image sequence in the identity database is used as a training sample. The pictures are arranged in order of angles. Each sample is converted into a feature matrix represented by pixels, which is used as the input of the convolutional neural network and trained to obtain a neural network model for identity recognition;

Then, the face image sequence with the angle value label is used as a test sample, and the test sample is input into the neural network model for identity recognition. Instead of using the model to output the result, the neural network model is used for feature extraction. In the embodiment, the output of the penultimate fully connected layer of the convolutional neural network is used as the high-level feature learned by the test sample, so that the test sample is converted into a multi-dimensional identity feature vector to be recognized. According to the same method, the face image sequences of n different identities in the identity library are respectively input into the neural network model for identity recognition, and the results of the same intermediate layer are extracted as features, so as to obtain n multi-dimensional identity feature vectors for matching ;

In the identity similarity matching stage, the multi-dimensional identity feature vector to be identified is compared with n multi-dimensional identity feature vectors for matching by cosine distance similarity, and the multi-dimensional identity feature vector for matching corresponding to the maximum cosine value is found as For the identity matching result corresponding to the multi-dimensional identity feature vector to be identified, the identity label corresponding to the identity matching result is used as the identity recognition result corresponding to the multi-dimensional identity feature vector to be identified.

In this embodiment, a surveillance video person identification method that combines multi-angle features includes the following steps:

Step 1. Collect a single frame of pictures of people and pictures of people without people, build a sample database of pictures of people and people without people, and assign a category label after normalizing the size of each picture. In this embodiment, positive samples are pictures of people in the data set, and human body parts with a size of 128*128 pixels are intercepted, and all positive samples are marked as 1; negative samples are randomly intercepted from pictures that do not contain human bodies, and the size is also 128*128, all negative samples are labeled as 0, and the positive and negative samples are placed in a folder respectively, so as to obtain a training sample data set composed of positive samples and negative samples;

Step 2. Extract the SIFT features of each sample in the training sample data set, and each SIFT key point descriptor is a 4*4*8=128-dimensional vector. After feature extraction, each sample is converted into an n*128-dimensional feature matrix, where n is the number of feature points extracted. In this example, it is stipulated that each sample extracts 100 feature points, and the feature vectors of each feature point are sequentially connected, so that each sample is converted into a 100*128-dimensional feature vector;

Step 3. Use the support vector machine (SVM) to train the feature vectors of the training sample data set to obtain a human picture object detection model, which is used to detect whether there is a person in a single frame picture. Arrange the SIFT feature vectors of the positive and negative samples and the labels corresponding to the samples according to the format required by libsvm (Label 1:value 2:value...). Use svmscale to scale the sample and normalize the data to the [-1,1] range. Use grid.py cross-validation to select the best parameters c and g, and then svmtrain uses the obtained best parameters c and g, linear classifier, and RBF kernel function to train the feature vectors of the entire training data set to obtain support vector machine model parameters , after training, the result is saved as a .model model file, and a classifier is obtained, which is a human image target detection model.

Step 4, preprocessing the surveillance video, using the ffmeg software package to convert a surveillance video to be recognized into a series of single-frame pictures, and use it as a test set, the size of the test set images is also 128*128. Extract the SIFT feature of each picture, and convert each test set picture into a 100*128-dimensional feature vector;

Step 5. Using the human picture object detection model to identify whether each frame of the test set contains a person. Load the trained model file, call the svmpredict function in libsvm to identify the test data picture, generate a .predict file, and get the category label (1 or 0) of each test picture correspondingly, that is, identify whether each frame picture contains people. For a test instance x, classify according to the following function, where ( _xi , y _i ) is the training sample, y _i ∈ {1,0},

If the SVM output result is 1, indicating that there are people, then keep the corresponding single frame picture, remove the surrounding information of the image, intercept the human body part, normalize the picture size to 64*64 size, and use it as a key frame, otherwise, discard the corresponding A single frame picture, so as to get a set of key frames;

Step 6. Build a multi-angle face database. The database contains human faces and no faces. The frontal 0° of the face is used as the starting collection point, and a face image is evenly collected every k degrees clockwise. Each image Capture the face part with a size of 64*64 pixels. In this embodiment, take k=10, rotate from 0 degrees to 360 degrees from the front, and collect an image evenly every 10 degrees. Among them, the face images of 90° and 270° are not collected, and each identity collects a total of m=360/k-2=34 face images from different angles, thus forming a face image sequence composed of a human face and a non-face, wherein the human face image sequence with a human face contains [0°,80°]∪[ 17 images in the range of 280°, 350°], and the face image sequence without a face contains 17 images in the range of [100°, 260°], and then obtain the face image sequence composed of n different identities Multi-angle face database.

Step 7. Assign a category label to each face image in the collected multi-angle face database according to the angle value, for example, the image with a face angle of 0° is assigned a label 0, and the image with a face angle of 280° is assigned a label 28, and so on. Put each face image into its own category collection according to the angle value, a total of 34 category folders. In face detection, an image to be detected may have a face blocked by other people or objects, or a face covered by glasses and masks. If the training samples all use multi-angle unoccluded faces, when the faces in the test samples are partially occluded, it is easy to be screened as non-human faces and cause missed detection. At the same time, due to the small number of samples collected in the multi-angle face database, data enhancement methods can be used to expand the database and improve the algorithm accuracy and feature dimension. Therefore, the multi-angle face database is expanded by partially occluding the upper left, upper right, lower left, lower right, and middle parts of each face image. The size of the occlusion window is 4*4, and each complete face corresponds to five occluded persons. Face, assign the same category label to each occluded face, thus forming a multi-angle face training set;

Step 8, extract the SIFT features of each face image in the multi-angle face training set. In this embodiment, set each image to extract 100 feature points, and each feature point vector is connected in turn, so that each face The image is converted into a multi-angle feature vector, and the dimension of each multi-angle feature vector is 100*128;

Step 9, using the support vector machine (SVM) to train the multi-angle feature vectors in the entire multi-angle face training set to obtain a multi-angle face detection model. For n classification problems, svm has two multi-class division methods, one is "one-to-one" method, this method constructs a classifier for any two classes, and there are n(n-1)/2 classes in total. The other is the "one-to-many" method, which treats n training samples of a certain class as one class, and all other classes as another class, so there are n classifiers in total. In this embodiment, the "one-to-one" method is used, so samples of n=34 categories need to train 561 two-class classifiers in total. Make judgments and take the form of voting, and the category with the most votes in the end is the category of the test sample. The multi-angle face detection model obtained after the training can be used to detect the angle of the face in the key frame set, so as to determine whether there is a face in the image.

Step 10, using the multi-angle face detection model to detect the key frame set, and obtain the face angle value of each key frame. First extract the SIFT feature of each picture in the key frame set, convert each test set picture into a 100*128-dimensional feature vector, and then train 561 second-class classifiers in turn, if the test case is obtained by the classification function If x belongs to class i, the vote of class i will be increased by 1; if it belongs to class j, the vote of class j will be increased by 1. Accumulate the scores of each category and select the category corresponding to the one with the highest score as the category of the test picture x. If the category is 10, it means that the face angle value of the test picture x is 10°. Similarly, the face angle value of the key frame set is obtained.

Step 11. Determine whether each key frame contains a human face according to the face angle value; if it contains a human face, keep the corresponding key frame, otherwise discard the corresponding key frame. If the svm output face angle category result is [0,8]∪[28,35], it means there is a face, and the output result is [10,34], it means no face. For the reserved face key frame, remove the surrounding information of the image, use the Haar classifier in opencv to detect the face, and only keep the face part of the cropped image, and the size is unified to 64*64 pixels. Thereby converting the set of keyframes into a sequence of face images labeled with angle values;

Step 12. Construct an identity library using human face image sequences in the multi-angle face database, use the identity library as the input of the neural network, and perform training to obtain a neural network model for identity recognition. The identity database uses 17 human face image sequences of each identity in the multi-angle face database, and removes 17 unidentified human face image sequences to construct an identity database containing n identities. Each image sequence is used as a training sample, representing an identity category, and assigning identity labels from 0 to n. Divide the samples in the identity library into the training set and the verification set according to the ratio of 8:2, and use the theano-based keras library to train the convolutional neural network, including three convolutional layers, two pooling layers, two fully connected layers and A softmax layer. Take the pixel values of each point of each face image as input data, each image is a 64*64-dimensional vector, and the human face image sequence of each identity is according to (0,...,8,28,...,35) The face angles of are arranged in sequence, that is, each training sample is converted into a 17*64*64 feature matrix, which is used as the input of the convolutional neural network. The training process includes forward training and backward training. The forward training process is bottom-up unsupervised learning, that is, starting from the bottom layer and training to the top layer layer by layer. During the training process, after training and learning to obtain the parameters of the n-1th layer, the output of the n-1 layer As the input of the nth layer, the nth layer is trained to obtain the parameters of each layer respectively; the backward training process is a top-down supervised learning, that is, the training error is transmitted from top to bottom, and the parameters are fine-tuned. After the training is completed, a model.pkl model file is obtained.

Step 13, use the face image sequence with angle value label as the test sample, also arrange in sequence according to the face angle sequence of (0, ..., 8, 28, ..., 35), if there is no corresponding face angle value , then the image is represented by a zero matrix with a dimension of 64*64, and the test sample is converted into a feature matrix represented by the size of pixel values. The test sample is input into the neural network model for identity recognition, and the output of the penultimate fully connected layer of the convolutional neural network is extracted as the advanced feature learned by the test sample. The output result is a k-dimensional feature vector, where the size of k is determined by the number of nodes in the fully connected layer. This feature vector combines image features from multiple angles of the face and can be used to uniquely represent an identity. . Thus, the complex test sample is converted into a k-dimensional identity feature vector to be recognized.

Step 14, input the face image sequences of n different identities in the identity database into the neural network model for identity recognition respectively, each face image sequence according to (0, ..., 8, 28, ..., 35) people The face angles are arranged in order as a sample, converted into a feature matrix, and the output of the first fully connected layer of the penultimate layer of the convolutional neural network is also extracted as a result, so as to learn n k-dimensional identity feature vectors for matching;

Step 15. In order to match the identity feature vector to be identified with the identity in the identity database, the k-dimensional identity feature vector to be identified is compared with n k-dimensional identity feature vectors for matching by cosine distance similarity, namely The cosine of the angle between two vectors can be used as a measure of similarity. For example, the m-dimensional identity feature vector to be identified is expressed as An m-dimensional identity feature vector for matching is expressed as The cosine of the angle between two vectors is: The larger the value obtained, the more similar the two vectors are. Therefore, the matching multi-dimensional identity feature vector corresponding to the maximum cosine value can be found as the identity matching result of the corresponding multi-dimensional identity feature vector to be identified. The identity tag corresponding to the identity matching result is used as the identity recognition result of the corresponding multi-dimensional identity feature vector to be recognized. If the identity feature vector to be identified does not find a successful identity matching in the identity database, the face image sequence will be used as a new user, and the sample image information will be entered into the identity database to improve the identity database and improve the identity matching of the system. efficiency.

Claims (2)

1. A monitoring video person identification system that integrates multi-angle features is characterized in that it consists of: a target detection module, a multi-angle face recognition module and an identity matching module; The target detection module collects single-frame human pictures and unmanned pictures, and assigns category labels after performing size normalization processing on each picture, thereby obtaining a training sample data set composed of positive samples and negative samples; extracting the The SIFT feature of each sample in the training sample data set, thereby converting each sample into a feature vector; then using the support vector machine SVM to train the feature vector to obtain a target detection model; The target detection module converts the monitoring video into a series of single-frame pictures as a test set; utilizes the target detection model to identify whether each frame of pictures in the test set contains people, and if it contains people, then retains the corresponding single-frame pictures, And as a key frame, otherwise, discard the corresponding single frame picture, so as to obtain a key frame set; The multi-angle face recognition module takes the front face of the face at 0° as the starting collection point, and collects a face image every k degrees clockwise, wherein, the face images of 90° and 270° are not collected, each The identity collects a total of m=360/k-2 face images from different angles, thus forming a face image sequence composed of a human face and a non-face, and then obtaining a multi-angle face image sequence composed of n different identities. Face database; each face image is classified into its own category set according to the angle value, and the multi-angle face is expanded by partially occluding the upper left, upper right, lower left, lower right, and middle parts of each face image database, thus forming a multi-angle face training set; extracting the SIFT feature of each face image in the multi-angle face training set, so that each face image is converted into a multi-angle feature vector; and then using the support vector machine SVM to The multi-angle feature vector is trained to obtain a multi-angle face detection model; The multi-angle face recognition module uses the multi-angle face detection model to detect the set of key frames to obtain the face angle value of each key frame, and judge whether each key frame is Contains a human face; if it contains a human face, then retain the corresponding key frame, otherwise discard the corresponding key frame, thereby converting the key frame set into a human face image sequence with an angle value label; The identity matching module constructs an identity library using the face image sequences of faces in the multi-angle face database, uses the identity library as the input of the neural network, and performs training to obtain a neural network model for identity recognition ; The face image sequence with the angle value label is used as a test sample again, and the test sample is input in the described neural network model for identity recognition, and any layer of output in the middle of the neural network is extracted as a feature, thereby Convert the test sample into a multi-dimensional identity feature vector to be identified; The identity matching module inputs n face image sequences of different identities in the multi-angle face database into the neural network model for identity recognition, thereby obtaining n multidimensional identity feature vectors for matching; Then the multi-dimensional identity feature vector to be identified is compared with the n multi-dimensional identity feature vectors for matching by cosine distance similarity, and the multi-dimensional identity feature vector for matching corresponding to the maximum cosine value is found as the corresponding multi-dimensional identity feature vector to be identified The identity matching result of the multi-dimensional identity feature vector, the identity tag corresponding to the identity matching result is used as the identity recognition result of the multi-dimensional identity feature vector to be identified. 2. A monitoring video person identification method that merges multi-angle features, is characterized in that it is carried out as follows: Step 1. Collect single-frame pictures of people and people, and assign category labels to each picture after normalizing the size, so as to obtain a training sample data set composed of positive samples and negative samples; Step 2, extracting the SIFT feature of each sample in the training sample data set, thereby converting each sample into a feature vector; Step 3, using a support vector machine (SVM) to train the feature vectors to obtain a target detection model; Step 4, convert the monitoring video into a series of single-frame pictures, and use it as a test set; Step 5. Use the target detection model to identify whether each frame of the test set contains a person, and if it contains a person, keep the corresponding single-frame picture and use it as a key frame; otherwise, discard the corresponding single-frame picture to obtain a key frame set ; Step 6. Start with the front face of the face at 0°, and collect a face image every k degrees clockwise, where the 90° and 270° face images are not collected, and each identity collects a total of m= 360/k-2 face images from different angles to form a face image sequence composed of human faces and non-faces, and then obtain a multi-angle face database composed of n face image sequences with different identities; Step 7. Classify each face image into the corresponding category set according to the angle value, and expand the multi-angle face database by partially blocking the upper left, upper right, lower left, lower right, and middle parts of each face image , thus forming a multi-angle face training set; Step 8, extracting the SIFT feature of each face image in the multi-angle face training set, thereby converting each face image into a multi-angle feature vector; Step 9, using a support vector machine (SVM) to train the multi-angle feature vector to obtain a multi-angle face detection model; Step 10, using the multi-angle face detection model to detect the set of key frames to obtain the face angle value of each key frame; Step 11, judging whether each key frame contains a human face by the angle value of the human face; if it contains a human face, then keep the corresponding key frame, otherwise discard the corresponding key frame, thereby converting the key frame set into one with an angle value Labeled face image sequence; Step 12, using the human face image sequence in the multi-angle human face database to construct an identity library, using the identity library as the input of the neural network, and training, thereby obtaining a neural network model for identity recognition; Step 13, using the face image sequence with the angle value label as a test sample, and inputting the test sample into the neural network model for identity recognition, extracting the output of any layer in the middle of the neural network as a feature, Thereby converting the test sample into a multi-dimensional identity feature vector to be identified; Step 14, input the face image sequences of n different identities in the identity library into the neural network model for identity recognition respectively, so as to obtain n multidimensional identity feature vectors for matching; Step 15: Comparing the multi-dimensional identity feature vector to be identified with the n multi-dimensional identity feature vectors for matching by cosine distance similarity, and finding the multi-dimensional identity feature vector for matching corresponding to the maximum cosine value as a corresponding For the identity matching result of the multi-dimensional identity feature vector to be identified, the identity tag corresponding to the identity matching result is used as the identity identification result of the corresponding multi-dimensional identity feature vector to be identified.