CN108900700A - Authentication method and system based on the double verification that recognition of face and sight position - Google Patents

Abstract

The present invention discloses an authentication method and system based on a double verification mechanism of face recognition and line of sight positioning. On the basis of face recognition and liveness verification by the user, the password information given by the user is further verified to achieve double verification. Gaze positioning and tracking technology, combined with the mobile phone lock screen interface, minimizes additional user operations, and further improves the security of the face verification mechanism without affecting the user experience, thereby better ensuring the security of the user's mobile phone. Utilizing the technical scheme of the present invention, there is no need to modify the existing methods of face detection and liveness inspection, but on the basis of the existing technology, the line-of-sight positioning monitoring is used to further improve the face recognition and authentication, and further improve the face detection and detection method. The security of identification can resist 3D headgear and even twin attacks.

Description

Authentication method and system based on double verification of face recognition and line of sight positioning

technical field

The invention belongs to the technical field of smart phone (system) security authentication, relates to image processing and deep learning technology, and in particular to an authentication method and system based on a double verification mechanism of face recognition and line of sight positioning.

Background technique

With the increasing popularity of smart phones and the continuous enhancement of their functions, coupled with the popularization of mobile payment in recent years, smart phones have become an indispensable item in people's daily life; Family members keep in touch, do online shopping, make offline payments, record life, entertain, etc. Therefore, smartphones play an increasingly important role in daily life, and the security of smartphones is also very important. Once a smartphone is lost or stolen , the loss is not only the value of the device itself, but also may lead to privacy leaks and property security issues.

In order to prevent the smart phone from being used by illegal users after it is lost or stolen, a sufficiently strong authentication mechanism/system will play a vital role in the security of the mobile phone. At present, the authentication system of smartphones is also constantly developing and improving. The existing user authentication methods mainly include the following:

(1) Swipe the screen to the right to unlock, the disadvantage of this method is that it is not safe;

(2) Text password unlocking, disadvantages: not easy to remember, long unlocking time, need to tap the screen multiple times, poor user experience;

(3) Jiugongge unlocking, disadvantages: easy to be stolen, not safe;

(4) Voice unlocking, disadvantages: low success rate, vulnerable to recording attacks;

(5) Fingerprint unlocking, disadvantages: the success rate is low, the fingerprint is easily affected, and the fingerprint is easily obtained by a third party, and the security is not high.

(6) Face recognition unlocking, disadvantages: vulnerable to attacks in the form of photos, videos, and 3D headgears, etc., and verification is required, and face recognition is easily cracked by twins.

In recent years, with the development of artificial intelligence technology, face recognition technology has become more and more advanced and mature, and due to the convenience of face recognition, face recognition technology has gradually been applied to mobile phone identity verification, such as Apple's FaceID and Samsung Galaxy S8's face recognition, etc. However, due to the particularity of the face, face recognition verification often suffers from some picture attacks, video replay attacks, 3D headgear attacks, and even twin attacks. These face recognition technologies all have a problem, that is, they cannot solve the problems of twin cracking and 3D headgear attacks. For example, at the end of 2017, a Vietnamese security company called Bkav used 3D printed face masks to easily fool Apple's face recognition system Face ID and successfully unlocked the iPhone X. Another example is the Galaxy S8 produced by Samsung. Users only need a high-definition photo to successfully deceive its face recognition system and complete unlocking. Moreover, most face recognition systems at this stage cannot defend against twin attacks at all. Therefore, major companies have adopted various verification technologies, such as 3D modeling, texture detection, and interactive verification. However, as more and more attacks are launched against face recognition, some attack methods can successfully deceive existing face recognition systems.

Contents of the invention

Aiming at the problems existing in the face authentication process of existing smart phones, the present invention proposes an authentication method and system based on a double verification mechanism of face recognition and line of sight positioning. On the basis of face recognition and verification of users, Further verify the password information given by the user to achieve double verification, use the sight positioning and tracking technology, combined with the mobile phone lock screen interface, to reduce the user's extra operations as much as possible, and further improve the security of the face verification mechanism without affecting the user experience Therefore, it can better ensure the security of the user's mobile phone.

The core idea of the present invention is: combining the existing face recognition and life verification technology and adding a sight positioning module on this basis, judging whether the user's sight line in the face recognition and life verification stage is positioned within the sight range set by the user before, if so, It proves that the face recognition result is the user himself, not someone else, and the verification is passed; otherwise, the verification fails. Specifically, firstly, through information collection, the user’s basic information for face recognition (including the information required for the user’s face recognition and verification stage) and the coordinate range of the user’s line of sight selected by the user are collected. stage, face recognition and biopsy detection are performed first, and if the face recognition and biopsy detection both pass, then the line of sight positioning detection is performed to detect whether the position determined by the line of sight is within the coordinate range of the line of sight position selected by the user, if it belongs to If it does not belong to, then the verification fails.

The technical scheme provided by the invention is:

An authentication method based on double verification of face recognition and line of sight positioning, comprising the following steps:

1) Collect and enter the information required for face recognition and life inspection, and the information required for location passwords; the information required for location passwords includes the designated sight area;

11) For the information required for face recognition and life verification, if 3D modeling is used, the picture information of face recognition and life verification needs to be collected and entered; if interactive verification is used, videos containing interactive information need to be collected and entered;

12) The location password information can be selected in combination with the screen saver of the mobile phone, which is convenient for the user to remember the password information. 13) The input of location password information, according to the positioning accuracy of the line of sight, by selecting an appropriate size range area as the password information, the higher the positioning accuracy, the smaller the range area, the larger the password space, and the larger the screen, the larger the password space.

2) Calibrate the gaze positioning, including the calibration of the gaze tracking accuracy and the calibration of the gaze positioning difference caused by the user's distance from the screen;

21) Line of sight positioning calibration can allow the mobile phone to randomly generate different position coordinate information points, allowing users to view the coordinate information points in the mobile phone at different distances (middle, far, near, etc.) and different postures (front view, side view, etc.) to complete the line of sight Calibration of tracking positioning accuracy.

3) Real-time collection of information for mobile phone verification; information includes information points required for line of sight positioning;

4) Perform face recognition detection: By comparing the information collected in step 3) with the information entered in step 1), perform face detection to determine whether they belong to the same face;

5) Perform a live test to determine whether the collected information belongs to a living body, and ensure that the information does not come from the same photo or video;

At this stage, most of the image and video attacks aimed at face recognition can be blocked, but it cannot prevent attacks similar to 3D headgear or further twin attacks;

6) Perform line of sight positioning detection: according to the information points (such as pupil position) required for line of sight positioning of the information collected in step 3), use the corresponding algorithm to perform line of sight positioning detection to determine whether the line of sight of the eyes belongs to the previously selected line of sight range, If it belongs to, the authentication is successful; otherwise, the authentication fails.

The present invention also provides an authentication method and system based on double verification of face recognition and line of sight positioning, which mainly includes the following six modules: information entry module, line of sight positioning calibration module, information collection module, face recognition detection module, and live inspection module. Module, line of sight positioning monitoring module. Here are the details:

1. Information input module: This module is mainly used to input information about face recognition and life verification and the selection of location passwords. The information collected for face recognition and life verification needs to be based on the specific face recognition used by the system. For example, if the system uses the 3D modeling information of FaceID, in the information collection stage, it is necessary to collect the feature points required for 3D modeling, and then build a 3D model. After the information, the next step is to collect the location information of the screen. For example, it can be some feature points of the screen saver photo, which is convenient for the user to remember. Of course, it can also be combined with your mobile phone to select a specific viewing area.

2. Line of sight positioning calibration module: This module is mainly used to calibrate line of sight positioning. At present, there are many technologies that can realize line of sight positioning and tracking, such as pupil simulation positioning, infrared eye tracking positioning, and other methods. The verification method and accuracy of each method They are all different. Therefore, this module is mainly for the calibration of eye-tracking accuracy, and the calibration of the difference in eye-sight positioning caused by the user's distance from the screen.

3. Information collection module: This module is mainly used to collect mobile phone verification information. For example, when using 3D modeling, the information entry stage will collect pictures of face recognition and verification information; if interactive verification is used, It is necessary to capture video containing interactive information.

4. Face recognition detection module: This module compares the information collected by the previous module with the information entered in the first stage, performs face detection, and judges whether the information collected by the information entry module and the information collected in the previous information collection stage belong to The same face.

5. Liveness verification module: This module mainly performs liveness verification, judging whether the information collected by the information collection module comes from a living body, or from a photo or video, etc. At this stage, most of the pictures for face recognition can be blocked And video attacks, but it can't stop similar to 3D headgear or further twin attacks.

6. Line of sight positioning detection module: This module is mainly for line of sight positioning monitoring. The information points required for line of sight positioning are collected from the information collection module, such as the pupil position, and then using the corresponding algorithm (such as the difference between the moving point and the reference point) To calculate the gaze direction and position of the human eye, the center of the pupil is used as the moving point, because it can accurately reflect the change of the eyeball, and the corner of the eye is used as the reference point, because the corner of the eye is a very stable point on the human face.) to detect the location of the line of sight , to judge whether the line of sight of the eyes belongs to the previously selected line of sight range, if so, the authentication is successful, otherwise, the authentication fails.

Compared with prior art, the beneficial effect of the present invention is:

The present invention provides an authentication method and system based on a double verification mechanism of face recognition and line of sight positioning. On the basis of face recognition and life verification by the user, the password information given by the user is further verified to achieve double verification. The positioning and tracking technology, combined with the mobile phone lock screen interface, minimizes additional user operations, and further improves the security of the face verification mechanism without affecting the user experience, thereby better ensuring the security of the user's mobile phone.

The technical advantage of the present invention is that it is not necessary to modify the existing methods of face detection and life verification, but on the basis of the existing technology, the line-of-sight positioning monitoring method/module is used to further improve face recognition and authentication , to make up for the shortcomings of the existing face detection technology, and further improve the security of face recognition, so as to be able to resist 3D headgear and even twin attacks.

Description of drawings

Fig. 1 is a structural block diagram of the components of the authentication system provided by the present invention.

Fig. 2 is a flowchart of the authentication method provided by the present invention.

Detailed ways

Below in conjunction with accompanying drawing, further describe the present invention through embodiment, but do not limit the scope of the present invention in any form.

The present invention provides an authentication method and system based on a double verification mechanism of face recognition and line of sight positioning. On the basis of face recognition and life verification by the user, the password information given by the user is further verified to achieve double verification. The positioning and tracking technology, combined with the mobile phone lock screen interface, minimizes additional user operations, and further improves the security of the face verification mechanism without affecting the user experience, thereby better ensuring the security of the user's mobile phone.

Fig. 1 is a structural block diagram of the modules of the authentication system provided by the present invention; the system includes six modules: an information input module, a line of sight positioning and calibration module, an information collection module, a face recognition detection module, a live inspection module, and a line of sight positioning monitoring module.

Fig. 2 is the block flow diagram of the authentication method that the present invention provides; The specific embodiment of the present invention is as follows:

1) First use the mobile phone to judge whether the information required for face recognition and life verification has been entered and stored; if it has been entered, the verification information collection will be carried out directly, and after the information collection is completed, it will enter the verification stage; if it has not been entered, the information will be collected first Input, after the input is completed, perform the above operations.

2) Input information, including the information points and sight positioning area required for face recognition and life inspection; the method of inputting the sight positioning area can specifically select the screen saver photo, check the sight positioning area through the touch screen, and then set the coordinate range of the checked area Passed into the database as password information for storage;

The user enters information successively according to the prompts of the mobile phone. First, enter the information points required for face recognition and verification. For example, the system needs to enter the 3D data of the user's face area and the corresponding feature points at this stage; Check the line of sight positioning area on the screen, the size of the area is fixed, and the area range is determined by the line of sight positioning accuracy. For example, this system uses the face++Detect API to estimate the line of sight. The gap between the estimated line of sight and the real position of the line of sight is about 3.5mm. Therefore, The size of the selected line of sight positioning area is 5mm*5mm, as long as the line of sight is located within this range, the verification is considered successful. (The reason for choosing the screen saver photo is that it is more convenient for the user to remember the location information in combination with the picture).

3) After the information entry is completed, the mobile phone prompts the user to look at different positions of the mobile phone at different distances and different postures to perform alignment on the user's line of sight. The viewing distance (15cm), the short viewing distance (10cm), the long viewing distance (20cm) and other different distances to watch the mobile phone, at the same time, the mobile phone randomly displays different location information points, allowing the user to lock the line of sight in this area, the mobile phone according to the user's line of sight In this case, adjust and modify the line of sight estimation parameters, and iterate in this way until the gap between the estimated line of sight position and the real line of sight position is about 3.5mm, stop the iteration, and the calibration is completed.

4) In the verification stage, the specific operations are as follows: the user uses the mobile phone to perform face recognition, the camera is facing the face and maintains an appropriate distance, the eyes are kept open, and the user checks the line of sight positioning area for verification.

5) The mobile phone collects the required information according to the selected face recognition and life verification methods, and at the same time obtains the information required to verify the location of the line of sight, such as pupil coordinates. For example, the face recognition method used in this system is feature point detection, which collects 128 feature point data on the user's face, and the verification method is 3D modeling. The face is captured by a 3D camera to obtain a 3D image of the corresponding face area. Data, and further analysis based on these data, and finally judge whether the face is from a living body or a non-living body. The line of sight estimation and positioning uses the face++Detect API, which needs to collect the position coordinate information of the user's pupil center and eye corner points.

6) The mobile phone performs face recognition feature point comparison according to the selected face recognition method, and judges whether it belongs to the user's feature point. If it belongs, it will enter the next stage of verification, otherwise it will directly return the result of verification failure. In this system, the mobile phone compares the collected 128 facial feature points with the feature point information entered in the first stage to determine whether they belong to the same user’s feature information. If they belong, then enter the next stage of verification, otherwise return the result of verification failure .

7) The mobile phone performs liveness detection according to the selected liveness verification method to determine whether the entered information comes from real people or videos and pictures. If it belongs to real people, it will proceed to the next stage of verification. If it does not belong to real people, it will directly return the verification failure result. In this system, the principle of life verification is based on the 3D face data of living and non-living bodies, using the machine learning classifier model, selecting the most distinguishing features (such as EGI features) to train the classifier, and finally using the trained classification Organs to distinguish living from non-living. According to the 3D data obtained by the face area captured by the 3D camera, the mobile phone enters the trained machine learning model, obtains the classification result, and judges whether it belongs to a living body.

8) The mobile phone estimates the coordinates of the line of sight position on the screen according to the input line of sight positioning information point, and then judges whether the line of sight position belongs to the line of sight range selected by the user before. If it belongs, the verification is successful; otherwise, the verification failure result is returned. This system uses the line of sight estimation method of the face++Detect API. Pass our collected data into the API to get the corresponding left and right eyeball position and line of sight direction information. After we get the line of sight direction information, we can analyze the line of sight Whether the coordinates belong to the line of sight selected by our input information in the first stage, if so, the verification is successful, otherwise, the verification fails.

9) If the number of verification failures exceeds the specified number of times, only text password input can be used to unlock.

Through the above method, only need to add a line of sight positioning detection module to the existing face recognition and life verification mechanism, and the security of face recognition can be greatly improved.

It should be noted that the purpose of the disclosed embodiments is to help further understand the present invention, but those skilled in the art can understand that various replacements and modifications are possible without departing from the spirit and scope of the present invention and the appended claims of. Therefore, the present invention should not be limited to the content disclosed in the examples, and the protection scope of the present invention is subject to the scope defined in the claims.

Claims (7)

1. a kind of authentication method based on recognition of face and sight positioning double verification, includes the following steps：

1) acquisition and typing recognition of face and information needed living, positioning password information needed are tested；

11) recognition of face and information needed living is tested：According to 3D modeling, needs to acquire and typing recognition of face and test picture living Information；It is verified according to interactive mode, needs to acquire and typing includes the video of interactive information；

12) positioning password information needed includes specified visibility region；

2) sight positioning is calibrated, it is fixed including sight caused by the calibration and user's off screen curtain distance to Eye-controlling focus precision The different calibration of potential difference；Specifically：Different location coordinate information point is randomly generated in mobile phone, and user is with different distances, different appearances State watches the coordinate information point in mobile phone, is achieved in the calibration to Eye-controlling focus positioning accuracy；

3) information that acquisition is verified for mobile phone in real time；Information includes information point needed for sight positioning；

4) recognition of face detection is carried out：It is compared, is carried out by the information of the information and step 1) typing that acquire step 3) Face datection judges whether to belong to same face；

5) it carries out testing work, judges whether collected information belongs to a living body, exclude collected information from identical photograph Piece or video；

6) sight detection and localization is carried out：Information point needed for information sight positioning according to step 3) acquisition, is positioned using sight Detection algorithm carries out the detection of sight positioning, judges whether the sight of eyes belongs to selected range of visibility；

Success is authenticated if the sight of eyes belongs to selected range of visibility；Otherwise, authentification failure；

It is achieved in the certification based on recognition of face and sight positioning double verification.

2. the authentication method based on recognition of face and sight positioning double verification as described in claim 1, characterized in that step 1) It acquires and typing positioning password information needed is especially by mobile phone screen protection selection visibility region, be convenient for user's memory cipher information； Positioning accuracy is higher, and visibility region range is smaller, and password space is bigger；Screen is bigger, and password space is bigger.

3. the authentication method based on recognition of face and sight positioning double verification as described in claim 1, characterized in that step 2) To user's sight positioning carry out it is calibrated, specifically, by prompt user from mobile phone with normal viewing distance, compared with close up see away from From, farther out viewing distance watch mobile phone；Meanwhile mobile phone random display different location information point, allow user that sight is locked in this Region；Mobile phone estimates parameter according to user's sight situation, redjustment and modification sight；Such iteration, until sight estimated location and view The gap size of line actual position stops iteration in 3.5mm or so, and calibration is completed.

4. the authentication method based on recognition of face and sight positioning double verification as described in claim 1, characterized in that step 3) Information point needed for the information sight positioning of acquisition includes pupil position and eye corner location.

5. the authentication method based on recognition of face and sight positioning double verification as described in claim 1, characterized in that step 6) Sight detection and localization algorithm is specifically direction of gaze and the position that human eye is calculated with the difference of dynamic point and reference point；Wherein move point Using pupil center, reference point uses canthus point.

6. the authentication method based on recognition of face and sight positioning double verification as described in claim 1, characterized in that step 6) The specific gaze estimation method for using face++Detect API, inputs sight location information point, obtains corresponding right and left eyes eyeball Position and direction of visual lines information；Obtain whether LOS coordinate belongs to the selected sight of typing information further according to direction of visual lines information In range.

7. a kind of Verification System of the double verification positioned based on recognition of face and sight, including：Data input module, sight are fixed Position calibration module, recognition of face detection module, tests flexible module, sight position monitor module at information acquisition module；

The data input module is for typing recognition of face and tests work, realizes the selection of positioning encrypted message；Wherein face is known It does not include for the characteristic point of 3D modeling, for the view comprising interactive information of interactive mode verifying with the information for testing required acquisition living Frequently；Positioning encrypted message includes characteristic point or the specific line of sight region of screen protection photo；

Sight positioning calibration module is used to calibrate sight positioning, including to Eye-controlling focus precision calibration and user The calibration of sight position disparity caused by off screen curtain distance；

The information acquisition module is used to acquire the information of mobile phone verifying；

The information of information and the typing of data input module that the recognition of face detection module acquires information acquisition module carries out Comparison carries out Face datection, judges whether the information of data input module acquisition and information acquisition module acquisition information belong to together One face；

The flexible module of testing for carrying out testing work, judge the collected information of information acquisition module from a living body, still From a photo or video；

The sight detection and localization module is for carrying out sight position monitor, the sight positioning institute acquired from information acquisition module The information point needed carries out the detection of sight positioning using sight detection and localization algorithm, before judging whether the sight of eyes belongs to Selected range of visibility；Success, otherwise, authentification failure are authenticated if belonging to.