RU2844774C1 - Method for non-biometric verification of persons - Google Patents

Abstract

FIELD: information security.

SUBSTANCE: method includes steps of: obtaining two facial images for verification, constructing a single image, applying a convolutional neural network to the single image, the output of this network is the similarity of faces on the two original images score, wherein neural network training is performed on data set, where X is plurality of face images, Y is a plurality of marks that one-to-one correspond to images from plurality X, where mark is a class – unique for each person identifier, further generating, using said set of data, positive and negative pairs of images, further, for these pairs, marking is automatically created, which indicates whether this pair is negative or positive.

EFFECT: high information security of the face verification process on two source images due to that verification takes place without calculation, transmission or storage of biometric data of the user.

1 cl, 4 dwg, 1 tbl

Description

The claimed technical solution relates to the field of information security, namely to the verification of a face on two original images without calculating, transmitting or storing the user's biometric data.

Face verification is a widely used technology for human identity verification, which, based on two facial images, makes a verdict on whether the faces belong to the same person. Patents for face verification methods, as well as patents for systems using face verification, propose to implement verification by independently calculating biometric data from the images of two compared faces, for example, in CN110555450B (published on 28.06.2022), CN107194341B (published on 21.04.2020), JP7165746B2 (published on 04.11.2022), WO2016/131083A1 (published on 25.08.2016). Also, many patents (for example, CN105407042 (published 04.01.2019), RU2249248C2 (published 27.03.2005), RU2638775C2 (published 15.12.2017)) suggest a scheme in which the calculation of biometric data for one of the images is performed in advance, and this data is stored in the database as a standard for subsequent comparison.

Thus, the standard and widely accepted method for comparing faces in two images is to independently calculate the feature vectors of each face using one method or another and then compare the two vectors. Patents CN110414305A (published 05.11.2019), CN106096538B (published 23.08.2019), US10747989B2 (published 18.08.2020) propose using an artificial neural network as a method for calculating the feature vector. As a rule, such networks are trained using a triplet loss function or a contrastive loss function. For training using the triplet loss function, so-called triplet generators are used - generators of image triplets, where one image is called an "anchor" and has class A, another image has the same class A and forms a positive pair with the anchor, the third image has class B and forms a negative pair with the anchor. When training using the pair loss function, pair generators are used that create positive and negative pairs from the original sample. This method of face verification poses significant threats to the information security of the person being checked. The threat lies in the potential leakage of biometric data of his face. Modern generative models allow, based on biometric features of faces, to reconstruct the image of the victim's face, which allows deceiving face recognition systems [1, 2].

The objective of the claimed invention is to eliminate the disadvantages of the known level of technology. The technical result consists in providing a method for non-biometric verification of faces, which makes it possible to verify faces on two original images without calculating, transmitting or storing the user's biometric data, and also ensures increased information security of the verification process.

The stated problem is solved and the claimed technical result is achieved by means of the claimed method of non-biometric verification of faces. The claimed method of non-biometric verification of faces consists in obtaining two images of a face for verification, constructing a single image from the two images of faces, then applying a convolutional neural network to the constructed single image, the output of this network is an assessment of the similarity of faces in two original images, and the neural network is trained on a data set, where X is a set of face images, Y is a set of labels that mutually uniquely correspond to images from set X, where the label is a class - a unique identifier for each face, then positive and negative pairs of images are generated using this data set, then a markup is automatically created for these pairs, showing whether the given pair is negative or positive, and for use in the training process, the face images are preliminarily normalized, and for each pair the network predicts the probability that two faces in the image belong to the same class, binary cross-entropy is used as a loss function.

The figures show:

Fig. 1: Examples of images of one face (one line - one face).

Fig. 2: Example of images of different faces.

Fig. 3: Example of face normalization.

Fig. 4: Algorithm of the claimed method.

The implementation of the claimed method is as follows.

At the input we have two images of a face for verification. First, a single image is constructed from two images of faces (for example, by increasing the channelization of the image, i.e., from two images of a face, each measuring W×H×1, one image measuring W×H×2 is constructed). Construction of a single image is necessary, since when two separate images of faces are fed to the network sequentially or in parallel, biometric features of faces are inevitably constructed (since these features are the response of the neural network). Next comes the main stage of the proposed method - application of a convolutional neural network to the constructed single image. No layer divides the processed information into two sets of features. The output of this network is an assessment of the similarity of faces in two original images.

Data for training a neural network

Let X be a set of face images. Y be a set of labels that correspond one-to-one to images from the set X. A label represents a class - a unique identifier for each face. This means that different images of the same face belong to the same class, and images of different faces belong to different classes (see Fig. 1, 2).

Using this dataset, pairs of images are created: positive and negative. Positive pairs are pairs of different images of the same class. Negative pairs are pairs of images of different classes. For these pairs, a markup is automatically created showing whether the pair is negative or positive.

For use in the training process, the images of faces are first normalized (see Fig. 3), that is, they are presented in full face, where the face occupies most of the image.

Also, various augmentations can be used in the learning process, for example:

1. Blur an image with a Gaussian filter

2. Imposition of impulse noise.

3. Overlaying several thin lines of black or white color.

4. Applying highlights.

5. Rotation around the vertical axis.

6. Image rotations by several degrees (up to three).

7. Slightly scaling the image

Training a neural network

Two images in a pair are combined into one (as described earlier). The neural network used is a sequence of layers with activation functions. For each pair, the network predicts the probability that two faces in the image belong to the same class, i.e. it classifies pairs into two classes (see Fig. 4). Binary cross-entropy is used as a loss function.

Features of neural network training

When training such a neural network, it is important to have information not about the belonging of each face image to a certain class, but about the belonging of faces in each pair to the same class. This information can be obtained from standard data sets, including open ones, where each face has an identifier of the class to which it belongs. The network itself can be trained using a triplet generator for the triplet loss function. Each triplet is made up of a pair of faces belonging to the same class (anchor and positive element) and a pair of faces belonging to different classes (anchor and negative element). The network is then trained using standard cross entropy instead of the triplet loss function, namely as a classifying network, not as a metric one.

Equipment disclosure:

The implementation of such verification is possible on different equipment capable of launching the neural network used. Thus, the network of the proposed architecture can be launched on the central processor or GPU of a personal computer, the central processor or GPU of a smartphone, the central processor or GPU of a tablet computer.

Example of implementation:

The creation of a combined image can be carried out by the operation of channel-by-channel gluing along one of the dimensions, for example, if gluing along channels, then from two images of a face, each of size W×H×C, one image of size W×H×2C is constructed.

It is proposed to use a convolutional network consisting of several successive convolution layers as the network that builds the assessment, an example of which is given in Table 1.

In other implementations, networks of various architectures can be used, such as networks with residual connections (ResNet), networks with attention layers, recurrent networks, including LSTM and other architectures. The output of a neural network can be a single number on a fixed interval, such as on a continuous interval [0, 1]. In another implementation, the output of the network can be two classes with ratings, where one class corresponds to identical faces, and the second to different faces.

For training, images of ~1 million different people of different races, genders, ages, images of people with accessories that interfere with recognition, such as beards and glasses, are used.

List of sources:

1 D. Deb, J. Zhang and A.K. Jain, "AdvFaces: Adversarial Face Synthesis," 2020 IEEE International Joint Conference on Biometrics (IJCB), Houston, TX, USA, 2020, pp. 1-10, doi: 10.1109/IJCB48548.2020.9304898

2 Yujun Shen, Ping Luo, Junjie Yan, Xiaogang Wang, Xiaoou Tang; Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2018, pp. 821-830

Claims (1)

A method for non-biometric verification of faces, which consists in obtaining two images of a face for verification, constructing a single image from the two images of faces, then applying a convolutional neural network to the constructed single image, the output of this network is an assessment of the similarity of faces in two original images, wherein the neural network is trained on a data set, where X is a set of images of faces, Y is a set of labels that mutually one-to-one correspond to images from set X, where the label is a class - an identifier unique to each face, then positive and negative pairs of images are generated using this data set, then a markup is automatically created for these pairs, showing whether the given pair is negative or positive, wherein the images of faces are pre-normalized for use in the training process, wherein for each pair the network predicts the probability that two faces in the image belong to the same class, binary cross-entropy is used as a loss function.