CN116092167A - Human face living body detection method based on reading - Google Patents

Abstract

The invention provides a face detection method based on readings, and relates to the field of medical equipment. The face detection method based on readings specifically includes the following steps: S1. face detection, S2. reading prompt, S3. reading data collection, S4. reading judgment, S5. living body detection. The present invention judges whether the action and the sound are consistent by identifying the lip movement and the sound content. After meeting the requirements, the motion video liveness detection and the sound liveness detection are carried out. Information, with more information, the upper limit of liveness detection is raised, and the ability of liveness detection is greatly improved.

Description

A face detection method based on readings

technical field

The invention relates to the technical field of face liveness detection, in particular to a reading-based face liveness detection method.

Background technique

With the widespread use of face recognition technology in various identity authentication scenarios, illegal production and impersonation of others are becoming more and more rampant, and the requirements for face liveness detection are also getting higher and higher.

At first, face images were used in face liveness detection; subsequently, motion-based dynamic face liveness detection methods were also widely used. At the same time, with the development of technologies such as voice recognition and voiceprint recognition, human voices are also used for identity verification, which also leads to the need for voice liveness detection.

However, the accuracy of existing detection methods is low, and the ability of live detection is not high, which affects the detection results.

Contents of the invention

(1) Solved technical problems

Aiming at the deficiencies of the prior art, the present invention provides a face detection method based on readings, which solves the problem of low accuracy of the existing detection method.

(2) Technical solutions

In order to achieve the above object, the present invention is achieved through the following technical solutions: a reading-based face detection method, specifically comprising the following steps:

S1. Face detection

After entering the face authentication process, collect the picture captured by the camera, and use the neural network face detection model to detect whether there is a human face in the picture;

S2. Reading prompt

After detecting the face, enter the reading stage;

S3. Reading data acquisition

After the prompt is over, enter the reading stage;

S4. Reading judgment

Because there is no guarantee that the user will read the specified number audibly according to the requirements.

The device side needs to judge whether the user has completed the action of "voice reading". If it is not detected that the user has read out the number by voice, it will prompt the user that the operation failed and guide the collection again;

S5. Liveness detection

The device collects reading video data and transmits it to the backend for liveness detection.

Preferably, the step S1 further includes not proceeding to the next stage if there is no human face, or the image quality of the human face is low, and entering the next stage if there is a human face.

Preferably, the step S2 also includes teaching the user how to complete the data collection at this stage through a certain interactive method.

Preferably, the step S3 also includes that the user needs to read out the prompted number within a period of time, and at the same time, the device saves the video-sound data collected by the camera. Regardless of whether the user takes the reading as requested, the data will be cached.

Preferably, in the step S4, a voice activity detection algorithm is further used to detect whether someone is making a sound, and the VAD algorithm detects a change in sound energy to determine whether there is a change in the sound. Under normal circumstances, the energy of the sound signal will change in the reading of the sound, and the device can judge whether there is sound by detecting the change of the sound signal energy.

Preferably, the step S5 further uses "multimodal Transformer-based video-sound content recognition" and "video-based action liveness detection" for liveness detection, and the verification is passed only when the two modules pass at the same time .

Preferably, the step S5 specifically includes the following steps:

a. Video-sound content recognition based on multi-modal Transformer

Forged video-sound data, it is more likely that the sound and video do not match, and the data that does not match the sound and video will be rejected. The structure of the Transformer can record the location information of the data. Using this ability of the model, the mismatched data will be rejected. The data is identified, if the sound and video do not match, it will not pass, only the sound and video match will pass;

b. Video-based liveness detection

Human voice readings, the human mouth moves, the optical flow can represent the size and direction of the pixel change of the two frames of images before and after the moving object, when the human mouth moves, the position moves, and the optical flow can be used to represent the distance between the images. movement characteristics.

(3) Beneficial effects

The invention provides a face detection method based on readings. Has the following beneficial effects:

The present invention provides a face detection method based on readings. The present invention prompts the person to read the number by interacting with the person, and the person's lips will make a corresponding movement and make a sound. By identifying the lip movement and the sound content, the judgment Whether the action and sound are consistent, and after meeting the requirements, perform action liveness detection and sound liveness detection. This method uses image and voice information at the same time for face liveness detection, which improves the accuracy of liveness detection. This method is used in practical industrial applications. , the effect is significantly improved, and has been widely promoted and used.

Description of drawings

Fig. 1 is the flow chart of human face biopsy detection of the present invention;

Fig. 2 is the structural diagram of the video-sound content recognition model based on the multimodal Transformer of the present invention;

FIG. 3 is a structural diagram of a video-based motion detection model of the present invention.

Detailed ways

The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some, not all, embodiments of the present invention. Based on the embodiments of the present invention, all other embodiments obtained by persons of ordinary skill in the art without making creative efforts belong to the protection scope of the present invention.

Example:

As shown in Figures 1-3, the embodiment of the present invention provides a face detection method based on readings, which specifically includes the following steps:

S1. Face detection

After entering the face authentication process, collect the picture captured by the camera, and use the neural network face detection model to detect whether there is a human face in the picture. If there is no face, or the image quality of the face is low, do not proceed to the next stage; when there is a face, enter the next stage;

S2. Reading prompt

After the face is detected, enter the reading stage. Through certain interactive methods, teach users how to complete the data collection at this stage. For example: on the screen of the device, an animation prompt of the reading is displayed, and a sound prompt is played at the same time to show the user how to operate correctly.

Then, the user audibly reads the specified number according to the prompt. This scheme uses Arabic numerals, and for different languages, the pronunciation of the same numeral is different. However, for the same number, the types of pronunciation are limited;

S3. Reading data acquisition

After the prompt ends, enter the reading stage. The user needs to read out the number prompted within a period of time. At the same time, the device saves the video-sound data collected by the camera. Regardless of whether the user takes the reading as requested, the data will be cached.

When the device saves sound and video data, it is required that the sound and video must be aligned in time and kept in sync;

S4. Reading judgment

Because there is no guarantee that the user will read the specified number audibly according to the requirements.

The device side needs to judge whether the user has completed the action of "voice reading". If it is not detected that the user has read out the number, it will prompt the user that the operation failed and guide the collection again.

In this solution, a voice activity detection (Voice Activity Detection, VAD) algorithm is used to detect whether someone is speaking. The VAD algorithm judges whether the sound has changed by detecting the change of sound energy. Under normal circumstances, the energy of the sound signal will change in the reading of the sound, and the device can judge whether there is sound by detecting the change of the sound signal energy.

In this solution, an action detection model based on deep learning is used to detect whether the human mouth has a reading action. Usually, when a person reads normally, the mouth will open and close, or the shape of the mouth will change. By detecting whether there is a mouth movement in the video, the device can determine whether the user is taking a normal reading.

In order to ensure that the use of normal users will not be disturbed, if one of the two conditions is met, it means that the user has taken a correct action. The device uploads the collected reading video-sound data.

In order to ensure the robustness of the device for judging the reading behavior, the threshold and threshold should be lowered as much as possible. The device side makes judgments, filters out some data that does not meet the requirements, and provides feedback in a timely manner without waiting for the feedback results from the backend, mainly to ensure the experience of novice users in the early stage;

S5. Liveness detection

The device collects reading video data and transmits it to the backend for liveness detection. This solution uses "Video-Sound Content Recognition Based on Multimodal Transformer" and "Video-Based Action Liveness Detection" for liveness detection. Only when the two modules pass at the same time can the verification be passed.

Step S5 specifically includes the following steps:

a. Video-sound content recognition based on multi-modal Transformer

Forged video-sound data, it is more likely that the sound and video do not match, and the data that does not match the sound and video will be rejected. The structure of the Transformer can record the location information of the data. Using this ability of the model, the mismatched data will be rejected. The data is identified, if the sound and video do not match, it will not pass, only if the sound and video match will pass.

At the same time, the model can recognize actions in sound and video, and judge whether the reading content of the person is consistent with the required reading. If they are consistent, the verification is passed; otherwise, it is not passed.

In this solution, the model results of video-sound content recognition based on multi-modal Transformer are shown in Figure 2. The input of the model is sound data and video data, the output of the model is the category of the living body, and the recognition model is obtained by training the Transformer with the data;

b. Video-based liveness detection

Human voice readings, the human mouth moves, the optical flow can represent the size and direction of the pixel change of the two frames of images before and after the moving object, when the human mouth moves, the position moves, and the optical flow can be used to represent the distance between the images. movement characteristics.

For video, there are many frames of images [I1,I2,...,In], calculate the optical flow between each frame of images to get the optical flow sequence [F1,F2,...,Fn-1], merge the two to get A sequence [I1,F1,I2,F2,...,In-1,Fn-1,In] of moving face images and optical flow between every two consecutive images.

Set the threshold, if the living body score is greater than the threshold and the model predicts that it is living, the verification is passed; otherwise it is not passed.

In this solution, the results of the video-based liveness detection model are shown in Figure 3. The model inputs the video frame and the optical flow sequence, and the model output is the category of the living body, and uses the data to train the neural network to obtain the living body detection model.

The present invention further is a flow chart of face detection based on readings as shown in Figure 1. After entering the face scanning process, the device captures camera data and performs face detection on images collected by the camera. If a face is detected, enter the next step. One stage, otherwise face detection is performed in a continuous cycle; after a face is detected, the user is prompted to read the number, and the voice and video of the user's reading are cached; the saved voice and video are detected for motion and voice activity, when there is a mouth Enter the next stage when there is an action or sound, otherwise start a new round of collection; upload the data collected above to the background, use the liveness detection model to judge whether the face to be verified is a real person, if it is a real person, return to liveness detection and pass, Otherwise, return liveness detection failed.

Although the embodiments of the present invention have been shown and described, those skilled in the art can understand that various changes, modifications and substitutions can be made to these embodiments without departing from the principle and spirit of the present invention. and modifications, the scope of the invention is defined by the appended claims and their equivalents.

Claims (7)

1. A human face living body detection method based on readings, is characterized in that, specifically comprises the following steps: S1. Face detection After entering the face authentication process, collect the picture captured by the camera, and use the neural network face detection model to detect whether there is a human face in the picture; S2. Reading prompt After detecting the face, enter the reading stage; S3. Reading data acquisition After the prompt is over, enter the reading stage; S4. Reading judgment Because there is no guarantee that the user will read the specified number audibly according to the requirements. The device side needs to judge whether the user has completed the action of "voice reading". If it is not detected that the user has read out the number by voice, it will prompt the user that the operation failed and guide the collection again; S5. Liveness detection The device collects reading video data and transmits it to the backend for liveness detection. 2. A kind of human face living body detection method based on reading according to claim 1, it is characterized in that: if there is no human face in the described step S1, or the image quality of human face is low, then do not carry out the next step. stage, when there is a face, enter the next stage. 3. A face detection method based on readings according to claim 1, characterized in that: the step S2 also includes teaching the user how to complete the data collection at this stage through a certain interactive mode. 4. A reading-based human face living body detection method according to claim 1, characterized in that: said step S3 also includes that the user needs to read out the number of prompts within a period of time, and at the same time, the device collects the number with the camera The video-sound data is saved. Regardless of whether the user takes the reading as requested, the data will be cached. 5. a kind of human face living body detection method based on reading according to claim 1, it is characterized in that: in described step S4, further be to use voice activity detection algorithm to detect whether people's utterance, VAD algorithm detects sound energy to determine whether the sound has changed. Under normal circumstances, the energy of the sound signal will change in the reading of the sound, and the device can judge whether there is sound by detecting the change of the sound signal energy. 6. A kind of human face living body detection method based on reading according to claim 1, it is characterized in that: in described step S5, further be to use " video-sound content recognition based on multimodal Tansformer " and " based on Video motion liveness detection" is used for liveness detection, and the verification is passed only when the two modules pass at the same time. 7. a kind of face detection method based on reading according to claim 1, is characterized in that: specifically comprise the following steps in the described step S5: a. Video-sound content recognition based on multi-modal Transformer Forged video-sound data, it is more likely that the sound and video do not match, and the data that does not match the sound and video will be rejected. The structure of the Transformer can record the location information of the data. Using this ability of the model, the mismatched data will be rejected. The data is identified, if the sound and video do not match, it will not pass, only the sound and video match will pass; b. Video-based liveness detection Human voice readings, the human mouth moves, the optical flow can represent the size and direction of the pixel change of the two frames of images before and after the moving object, when the human mouth moves, the position moves, and the optical flow can be used to represent the distance between the images. movement characteristics.

Images