RU2840316C1 - Method and system for authenticating face on image - Google Patents

Abstract

FIELD: computer engineering.

SUBSTANCE: at stages of method: initial image of user is obtained; determining coordinates of the position of the user’s face; forming a face image in accordance with the coordinates of the position of the user’s face; determining the position of the face key points for the face image and the position of the user key points for the user image; generating a first face key point mask for the face image and a second user key point mask for the user image; calculating a first vector representation of the face features based on the face image and the first mask of the face keypoints and the second vector representation of the user features based on the user image and the second keypoint mask of the user; based on the obtained feature vectors, determining the final assessment of the probability of the authenticity of the user on the original image; based on the value of the final assessment of the probability of authenticity of the user on the image, the class of the image is determined, which indicates that the face presented on the image is authentic or fake.

EFFECT: high accuracy when determining the authenticity of a face.

15 cl, 5 dwg

Description

AREA OF TECHNOLOGY

[001] The present invention relates generally to computing technology and, in particular, to a method and system for determining the authenticity of a face in an image for the purpose of protecting against fraud.

LEVEL OF TECHNOLOGY

[002] Currently, there are several known solutions used in facial biometrics tasks to protect against fraud. Fraud is understood as situations in which an intruder in a facial identification system attempts to impersonate another person by replacing the registered photograph or video recording from the user terminal. Examples of forgery include, for example, a color printout of a person's photograph on paper, displaying a photograph or video recording on a phone screen, a paper or volumetric mask made of special materials.

[003] Facial identification protection technologies are capable of determining whether a photograph or video recording received from a user terminal is fake or real. This is done by performing a computational analysis of facial features, usually based on the use of machine learning algorithms, in particular, using artificial neural networks (ANN), to form the final authentication solution.

[004] A method is known for detecting counterfeits in biometric systems by providing it with a photograph of a person registered in the system (Method and device for recognizing facial relief, patent No. RU 2431190 C2, 22.06.2009), which is based on an analysis of facial relief. This approach includes performing a sequence of operations aimed at identifying the features of the volume of the obtained photograph of the object. In particular, two images of the target object are formed: one with the backlight on, the other with the backlight off. In the formed images, areas containing the face are highlighted, after which a comparison of the areas between the pair of images is performed and an intensity distribution map is constructed. An analysis of changes in the intensity of the compared areas allows a decision to be made on the relief of the face presented in the images and an attempt to deceive the biometric system to be detected.

[005] A solution is known for detecting attempts to deceive biometric facial recognition systems, based on the analysis of an infrared image (Living body detection method and device, patent No. CN112883758B, 08/25/2023), from which features of the area of interest containing key facial features are extracted, then the data is fed to a neural network classifier. The classifier determines the presence or absence of information about the distribution of veins in the image, after which a decision is made on the authenticity of the original data.

[006] A known method for detecting counterfeits in biometric user identification systems is based on extracting face depth information from successive images (Anti-spoofing, Patent No. GB2579583B, 06.01.2021). This approach uses a machine learning algorithm that learns to restore pseudo-depth using reference three-dimensional images. A special processing module is also used that adapts the parameters to compare the results of the analysis of the obtained images with the real depth. The trained system checks the calculated depth and the expected values for a real face for compliance. If a discrepancy is detected, the image is determined to be counterfeit. The disadvantage of this approach is the need to use several successive frames from the user's terminal, which leads to a time delay in data processing.

[007] The prior art also includes a method for analyzing the authenticity of a face in an image using a trained artificial intelligence model (Method and device for determining face liveness, patent No. KR102509134B1, 14.03.2023). This system first receives a first image with a face, on the basis of which training data is generated. To do this, the received image is processed in various ways, such as rotation, enlargement, shift, etc., to create multiple variations of the original data. Next, a second image is fed to the trained model to extract a feature vector, on the basis of which a decision is made about the presence of a real face in the image.

[008] A method for protecting biometric identification in an image using a neural network is also known (Biometric task network, patent No. US11776323B2, 03.10.2023). In this case, several separate neural networks are used - a common neural network extracts key features of the image, then the features are transferred to specialized neural networks that determine the area of the face, texture, body pose and other parameters. The obtained results are transferred to the neural network to combine the features, a decision is made on whether the input photograph is genuine.

[009] A method for protecting biometric systems from intruders is known based on the analysis of texture characteristics of an image (Facial liveness detection, patent No. US11244150B2, 08.02.2022). The system selects the area of the face in the received image, then the mirror and texture features of the face are determined. Mirror features are extracted from the Y-channel of the image after its processing in the YUV color space. Then the features are combined into a final vector, on the basis of which the final solution is formed.

[010] A method for determining the authenticity of a person in an image is also known (Method and apparatus with liveness detection, patent No. US11804070B2, 10/31/2023), in which the face area is determined using detection algorithms and various image parameters are extracted, such as hue, face tilt, color component balance, brightness and contrast. The parameters are compared with predefined ranges of values that were calculated on the training sample. If it is detected that the parameters are outside the permissible values, the system can correct them. After correcting the parameters, the system applies a neural network-based model to determine the authenticity of the image. It is known that this approach is characterized by instability to external lighting conditions, changes in which are difficult to compensate for even taking into account the correction system.

[011] The closest analogue of the claimed invention is a method for determining the authenticity of a human face (Liveness test method and apparatus, patent No. US10121059B2, 06.11.2018), which is based on image processing using a neural network. A subregion of the face is extracted from the original image and fed to the first input layer of the neural network model, the original face image is fed to the second input layer. The model processes two images, extracts texture data from the first image and combines them with features obtained from the second image. The combined features are processed by subsequent layers of the neural network, after which a decision is formed at the output on whether the person in the image is authentic. This approach uses only a face image, but the original, full-size photograph from the user terminal can be used to assess authenticity. The original image contains general texture information and may contain artifacts, the signs of which are especially important in the case of detecting a number of spoofing attacks, for example, showing a person from a mobile phone, when the edges or frames of the screen are visible, which serves as an important sign of the presence of an attack.

[012] At the same time, known solutions for determining the authenticity of an image do not take into account information about the position of the user's key points, characterizing, for example, the contours of the face (eyes, nose, mouth), the location of the arms, shoulders, etc. Using such data in image analysis allows for the explicit identification of areas of interest, which ensures an increase in the resulting accuracy of authenticity determination models.

[013] In order to reliably identify counterfeit features, it is also necessary to perform joint processing of the face and user images. In particular, the face image allows one to find unnatural changes in the skin structure when using a counterfeit mask, and the user image allows one to identify artifacts of the general texture and background breaks, which most often appear, for example, when showing a printed photograph. Thus, the analysis of the combined features of the face and user images increases the reliability of the biometric system when an intruder uses various types of counterfeits.

DISCLOSURE OF INVENTION

[014] The technical problem or task posed by the present invention is to create a new efficient, simple and reliable solution for determining the authenticity of a face in an image.

[015] The technical result achieved by performing the above task is an increase in the accuracy in determining the authenticity of a face.

[016] The specified technical result is achieved by implementing a method for determining the authenticity of a face in an image, performed by at least one computing device, comprising the steps of:

a. obtain the user's original image;

b. determine the coordinates of the user's face position;

c. generate an image of the face in accordance with the coordinates of the position of the user's face;

d. determine the position of the facial key points for the facial image and the position of the user key points for the user image;

e. forming a first mask of facial key points for a face image and a second mask of user key points for a user image;

f. calculating a first vector representation of features of a face based on an image of a face and a first mask of key points of a face, and a second vector representation of features of a user based on an image of a user and a second mask of key points of a user;

g. based on the first and second feature vectors, the final assessment of the probability of the user's authenticity in the original image is determined;

h. based on the value of the final assessment of the probability of authenticity of the user in the image, a class of the image is determined indicating whether the person presented in the image is genuine or fake.

[017] In one particular example of implementing the method, steps f-g are performed by a machine learning model or an ensemble of models, wherein the machine learning model or ensemble of models is trained on a data set containing genuine and fake images of users' faces.

[018] In another particular example of implementing the method, the ensemble of models consists of individual models, each of which is trained to identify a type of counterfeit determined by the developer.

[019] In another particular example of implementing the method, the stage of forming masks of key points of the face and the user is carried out using a neural network trained for the purpose of increasing the accuracy of determining the final assessment of the probability of the user's authenticity.

[020] In another particular example of implementing the method, the stage of forming masks of key points of the face and the user comprises stages in which:

- form closed contours in accordance with the positions of the coordinates of key points;

- increase the line width of closed contours by adding a specified number of adjacent pixels to the said lines, or increase the line width of closed contours by using blur filters with specified parameters, such as a Gaussian filter.

[021] In another particular example of implementing the method, an additional step is performed in which the images are normalized by bringing the pixel values to specified levels of mathematical expectation and distribution variance or by means of a linear transformation to a specified range of values.

[022] In another particular example of implementing the method, an additional step is performed in which the sizes of the images are changed to match the inputs of the image feature determination models by means of two-dimensional image interpolation.

[023] In another particular example of implementing the method, the step of forming an image of a face comprises the steps of:

- in accordance with the coordinates of the face position, a square area is selected on the user's image;

- enlarge the square area by a specified number of pixels, and select the image located in the enlarged square area as the face image.

[024] In another particular example of implementing the method, additional steps are performed in which it is determined that the selected enlarged square area extends beyond the user's image and replication of the boundary pixels of the user's image is performed to fill said selected area.

[025] In another particular example of implementing the method, additional steps are performed in which the following are formed:

- input data for the first model by matrix addition or multiplication of the pixel values of the first keypoint mask and the processed face image;

- input data for the second model by means of matrix addition or multiplication of the pixel values of the second key point mask and the processed user image, wherein when performing the addition or multiplication operation, the elements of the key point mask matrix are multiplied by a predetermined numerical coefficient.

[026] In another particular example of implementing the method, additional steps are performed in which the following are formed:

- input data for the first model by concatenating the matrices of the first keypoint mask and the processed face image; and

- input data for the second model by concatenating the matrices of the second keypoint mask and the processed user image.

[027] In another particular example of implementing the method, additional steps are performed in which the following are formed:

- input data for the first model by weighted addition of three matrices corresponding to the pixels of the first keypoint mask, the inverted first keypoint mask and the processed face image;

- input data for the second model by weighted addition of three matrices corresponding to the pixels of the second keypoint mask, the inverted second keypoint mask and the processed user image.

[028] In another particular example of implementing the method for determining the class of an image, the first and second feature vectors are processed by independent layers of a neural network.

[029] In another particular example of implementing the method for determining the class of an image, the first and second feature vectors are concatenated and then processed by a common neural network.

[030] In another preferred embodiment of the claimed solution, a system for determining the authenticity of a face in an image is presented, comprising at least one computing device and at least one memory containing machine-readable instructions which, when executed by at least one computing device, perform the above-mentioned method.

BRIEF DESCRIPTION OF DRAWINGS

[031] The features and advantages of the present invention will become apparent from the following detailed description of the invention and the accompanying drawings, in which:

[032] Fig. 1 shows a general diagram of the interaction of elements of the face authenticity determination system.

[033] Fig. 2 shows an example of an original user image and a face image.

[034] Fig. 3 shows an example of a processed image of a user.

[035] Fig. 4 shows an example of forming a closed contour with an increased line width for a mask of key points of a user's face image.

[036] Fig. 5 shows an example of a general view of a computing device.

IMPLEMENTATION OF THE INVENTION

[037] Below we will describe the concepts and terms necessary for understanding this technical solution.

[038] In this technical solution, the term “system” means, among other things, a computer system, a computer (electronic computer), a numerical control (CNC), a PLC (programmable logic controller), computerized control systems and any other devices capable of performing a given, clearly defined sequence of operations (actions, instructions).

[039] A command processing unit is an electronic unit or integrated circuit (microprocessor) that executes machine instructions (programs).

[040] The command processing unit reads and executes machine instructions (programs) from one or more data storage devices. The data storage devices may include, but are not limited to, hard disk drives (HDD), flash memory, ROM (read-only memory), solid state drives (SSD), and optical drives.

[041] A program is a sequence of instructions intended for execution by a computer control unit or command processing device.

[042] Database (DB) - a collection of data organized according to a conceptual structure that describes the characteristics of this data and the relationships between them, and such a collection of data that supports one or more application areas (ISO/IEC 2382:2015, 2121423 "database").

[043] Signal - a material embodiment of a message for use in the transmission, processing and storage of information.

[044] Logical element - an element implementing certain logical dependencies between input and output signals. Logical elements are usually used to construct logical circuits of computers, discrete circuits of automatic control and management. All types of logical elements, regardless of their physical nature, are characterized by discrete values of input and output signals.

[045] In accordance with the diagram shown in Fig. 1, the claimed system for determining the authenticity of a face in an image comprises: an image forming device 10; an image processing device 20; devices 31, 32 for generating a mask of key points; devices 41, 42 for forming input data; models 51, 52 for determining image features; a feature classification device 60.

[046] The imaging device 10 can be implemented on the basis of at least one photo/video camera of various types, in particular a PTZ camera, IP camera, stationary vandal-proof cameras, all-round cameras, mobile device or terminal cameras, etc., equipped with a face detector, for example, disclosed in the article by P. Viola and M. P. Jones "Robust real-time face detection*. Also, the following approaches can be used as an algorithm for detecting people's faces: adapted enhancement and the Viola-Jones method based on it, MTCNN, the elastic graph matching method, DeepFace Facebook, the YOLO object detection method, the R-CNN family of neural network architectures, Fast R-CNN, Faster R-CNN.

[047] The image processing device 20 can be implemented on the basis of a computing device and equipped with logical elements implemented on transistors and various signal converters (DPA, APC, etc.) arranged in such a way as to ensure the receipt and processing of signals from the device 10 in the manner described below.

[048] The devices 31, 32 for generating a mask of key points can be implemented on the basis of a computing device or neural networks containing input, output and other layers consisting of a plurality of neurons with corresponding weight coefficients, configured by training the neural networks on a training data set, in particular, characterizing the user's key points in the image for forming an attention matrix. In this case, for the primary formation of key points, known algorithms for recognizing facial points are used, disclosed, for example, in the manual "Face landmark detection guide", published on the Internet at: https://developers.google.com/mediapipe/solutions/vision/face_landmarker or such algorithms as Active Appearance Models (AAM), Active Shape Models (ASM), SURF, NeoFace, SHORE, ROl, Template Matching Methods, DPM (deformable part model).

[049] The devices 41, 42 for generating input data for the model can be implemented on the basis of a computing device equipped with known software and hardware for performing the operation of combining data of generated masks of key points and processed images received from the device 20.

[050] Models 51, 52 for determining image features can be implemented on the basis of neural networks containing input, output and other layers consisting of a plurality of neurons with corresponding weight coefficients, configured by training on the posted images and the corresponding data (masks) characterizing the structure of the key areas, wherein model 51 is trained on images of user faces, in contrast to model 52, which is trained on user images. Thus, model 51 allows for more efficient analysis of characteristic facial features, in particular, skin texture, changes in facial colors, the presence of small-scale artifacts, such as glare or reflections. Such analysis is extremely important for detecting complex types of counterfeits, such as silicone masks or volumetric masks printed on a 3D printer. In turn, model 52 takes into account the presence of large details in the image to determine the unnatural position of body parts, performs a general analysis of the image texture and identifies signs of a break or change in the background, which may occur, for example, when showing a printed photograph of a face or user.

[051] Data augmentation for training one or more machine learning models may be performed using at least one of the following approaches: image scaling (enlargement, reduction); image cropping; darkening the entire image, individual image channels; lightening the entire image, individual image channels; contrast enhancement; color transformations: changing the places (mixing) of color channels, amplifying, decreasing one or more color channels, obtaining an image in grayscale, obtaining a monochrome image, removing a color channel; image shifts and decentering; image rotations at different angles in different directions, rotation of the image or part thereof; image tilts, skews; mirroring along an arbitrary axis, line; additional lines or geometric objects in the image: with transparency of their color, without transparency, colored objects; gray objects (from white to black), including the removal of part of the image (placing a black object on the image) in geometric or semantic positions of the image; adding any background to the image; highlights and darkening of parts of the image; defocusing (blurring) of an image or its parts; increasing the graininess, sharpness (sharpness) of an image; compression and stretching along axes, lines; adding noise to an image over the entire image or its part, placing white or other noise; adding one or more elements of Gaussian noise, spotty noise; combining (overlaying) two or more images from a training sample (parts of images) with different weights; elastic transformation of an image (Elastic Transform); grid distortion of an image (GridDistortion); compression of image data by various image processing algorithms with a certain quality (for example, compressing the original bmp image according to the JPEG standard of a certain quality, and then obtaining a bmp image from it again); isotropic, affine and other transformations.

[052] One or more of the following algorithms can be used as a training algorithm for a machine learning model: Adagrad (Adaptive gradient algorithm), RMS (Root mean square), RMSProp (Root mean square propagation), Rprop (Resilient backpropagation algorithm), SGD (Stochastic Gradient Descent), BGD (Batch Gradient Descent), MBGD (Mini-batch Gradient Descent), Momentum, Nesterov Momentum, NAG (Nesterov Accelerated Gradient), FussySGD, SGDNesterov (SGD+Nesterov Momentum), AdaDelta, Adam (Adaptive Moment Estimation), AMSGrad, AdamW, ASGD (Averaged Stochastic Gradient Descent), LBFGS (L-BFGS algorithm - the Broyden-Fletcher-Goldfarb-Shanno algorithm with limited memory use), as well as second-order optimizers, such as: Method Newton, Quasi-Newton method, Gauss-Newton algorithm, Conjugate gradient method, Levenberg-Marquardt algorithm.

[053] At least one of the following functions is used as the objective function when training the machine learning model: L1Loss, MSELoss, CrossEntropyLoss, CTCLoss, NLLLoss, PoissonNLLLoss, GaussianNLLLoss, KLDivLoss, BCELoss, BCEWithLogitsLoss, MarginRankingLoss, HingeEmbeddingLoss, MultiLabelMarginLoss, HuberLoss, SmoothL1Loss, SoftMarginLoss, MultiLabelSoftMarginLoss, CosineEmbeddingLoss, MultiMarginLoss, TripletMarginLoss, TripletMarginWithDistanceLoss.

[054] The feature classification device 60 may be implemented on the basis of a computing device or neural networks containing input, output and other layers consisting of a plurality of neurons with corresponding weight coefficients, configured by training neural networks on a marked training data set containing feature vectors characteristic of genuine and counterfeit images, in particular, for calculating the final probability of user authenticity based on the obtained features of the user's images. In an alternative embodiment of the presented solution, the feature vectors contained in the marked training data set are concatenated, after which they are used for training by a common neural network. The device 60 is additionally equipped with means for reading lines and means for comparing elements of said lines, implemented on the basis of logical elements on transistors.

[055] Accordingly, the original image of the user (see Fig. 2, left image), placed opposite the imaging device 10, is captured by said device 10, after which the device 10, using known methods, determines the presence and position of the face in the image and transmits the original image of the user with the coordinates of the face area to the image processing device 20.

[056] The device 20, based on the original image of the user and the coordinates of the face region, forms an image of the face (see Fig. 2, right image) as follows. In the first stage, the device 20 selects a square region on the image of the user in accordance with the coordinates of the face region, after which the device 20 increases the selected face region by a specified number of pixels, for example, by 5-20 percent of the size of the square region. Such an increase makes it possible to compensate for possible inaccuracies in determining the boundaries of the face region during the operation of the detector and provides an increase in the resulting accuracy of determining the authenticity of the face due to the addition of additional information context. Then, the device 20 extracts the image that fell into the enlarged selected face region and forms an image of the face.

[057] Additionally, when enlarging the selected area of the face, the device 20 can be configured to determine that said selected area extends beyond the user image. Accordingly, when determining that said selected area extends beyond the user image, the device 20 similarly extracts an image that falls within the enlarged selected area of the face and forms an image of the face, wherein the missing part of the image (i.e. the part that extends beyond the user image) is filled, for example, with pixels obtained by replicating the boundary pixels of the image. Then, the device 20 changes the size of the face image using known methods to match the size of the input of the model 51, for example, by applying two-dimensional image interpolation methods.

[058] The device 20 also performs a change in the size of the original image of the user in order for the size of said image of the user to correspond to the size of the input of the model 52 using two-dimensional image interpolation methods (see Fig. 3). Also, the device 20 can be configured to determine the shape of the image of the user, and if the device 20 determines that the image of the user has a shape other than square (for example, has a rectangular shape), then the device 20 fills the missing part of the image to obtain a square shape of the image, for example, with pixels obtained by replicating the boundary pixels of the image of the user.

[059] In order to match the sizes of the models 51 and 52 of the said face image and the user image, known algorithms such as the nearest neighbor method, bilinear interpolation, bicubic interpolation, spline-based interpolation, discrete Fourier transform-based resampling, or image super-resolution methods may be used as two-dimensional interpolation methods.

[060] If necessary, the device 20 can perform additional image processing using known methods, such as brightness and contrast adjustment, linear and nonlinear filtering, noise compensation, RGB/HSV/YUV/LAB/YCbCr color format image conversion, histogram equalization, pixel normalization, for example, by bringing the range of values to specified levels of mathematical expectation and distribution variance or by linear transformation to a specified range of values, and many others. The use of additional processing improves the quality of training and the ability of the models 51, 52 to more effectively extract user authenticity features.

[061] In the third step, the device 20 sends the face image and the user image to the devices 31 and 32, respectively, which determine the positions of the key points for each image, in particular, for the face image - the face key points, and for the user image - the user key points. Then, based on the received points, the device 31 forms a first mask of the face key points for the face image, and the device 32 forms a second mask of the user key points for the user image.

[062] Several methods can be used to form a keypoint mask: either using a specialized neural network or using a closed-loop formation algorithm. Thus, the formed keypoint masks allow for explicitly indicating areas of increased interest during image analysis, such as characteristic areas of the face (nose, eyes, mouth) and body (arms, neck, shoulders).

[063] For the first method of forming a mask, devices 31 and 32 can be equipped with neural networks, each of which is trained on binary images of key points and the corresponding output masks. A binary image of key points of a face contains zeros and ones, where "1" indicates that the coordinates of the pixel of the image coincide with the coordinates of the key points of the face, and "0" indicates that the coordinates of the pixel of the image do not coincide with the coordinates of the key points of the face. Accordingly, binary images of key points are fed to the input of the neural networks, and at the output of devices 31 and 32, the first and second masks of key points of the face and the user are obtained, respectively.

Thus, these neural networks adaptively generate attention matrices depending on the position of the user's key points. At the same time, their parameters are trained together with other models, such as feature detection models, in such a way as to maximize the resulting accuracy of determining the probability of user authenticity.

[064] For the second method of forming a mask, devices 31 and 32, using known methods, form closed contours based on key points of the face and the user, respectively (see, for example, Fig. 4, left image), after which devices 31 and 32 increase the width of the line of the closed contours by adding a specified number of neighboring pixels to said lines, for example, by 2-10 percent of the image size, or by using blur filters, for example, a Gaussian filter, with which said devices can be equipped (see, for example, Fig. 4, right image).

[065] Accordingly, the obtained keypoint mask may be a matrix containing zeros and ones, where “1” indicates that the coordinates of the image pixel coincide with the coordinates of the extended contour of the face mask or user, and “0” indicates that the coordinates of the image pixel do not coincide with the coordinates of the extended contour of the face mask or user.

[066] Next, the first mask of key points obtained by device 31 for the face image and the face image from device 20 are sent to device 41, and the second mask of key points obtained by device 32 for the user image and the user image from device 20 are sent to device 42. Accordingly, if the original user image has undergone additional processing, then instead of the user image, the processed user image is sent to device 42.

[067] The device 41, upon receiving the said data of the first mask of key points and the image of the face, performs the formation of input data for the model 51, for example, by means of matrix addition, wherein when performing the addition operation, the elements of the matrix of the mask of key points can be multiplied by a predetermined numerical coefficient:

input=coef*mask+img

where mask is a mask of key points, img is a face image, coef is a given numerical coefficient.

[068] In another embodiment, the input data by device 41 can be formed by element-by-element multiplication of matrices corresponding to the pixels of the obtained key point mask and the face image, wherein when performing the multiplication operation, the elements of the key point mask matrix can be multiplied by a predetermined numerical coefficient:

input=coef*mask*img

[069] As another option, the input data by device 41 can be formed by a concatenation operation of matrices corresponding to the pixels of the obtained keypoint mask and the face image:

input=[mask, img]

[070] Alternatively, the input data by device 41 may be generated by weighted addition of three matrices corresponding to the pixels of the obtained keypoint mask, the inverted keypoint mask, and the face image:

input=coef1*mask+coef2*inv_mask+img

where inv_mask is the inverted mask of key points (the values “0” and “1” are replaced with each other), coef1 and coef2 are pre-set coefficients.

[071] In a similar manner, the device 42 generates input data for the model 52 based on the received data of the key point mask and the user image.

[072] Then, devices 41 and 42 send input data to the input of models 51 and 52, respectively, the outputs of which are transmitted to device 60 in the form of feature vectors characterizing that the user's image is genuine. Based on the first and second feature vectors received from models 51 and 52, respectively, device 60 determines the final value of the probability of the user's authenticity, after which the device compares the calculated final value of the probability with a specified threshold value specified by the developer of device 60, to determine the class of the image indicating that the face presented in the image is genuine or fake.

[073] To determine the probability value of user authenticity based on the feature vectors of images in device 60, a trained neural network may be used, for example, based on a multilayer perceptron or other machine learning algorithms, such as gradient boosting and binary decision trees.

[074] In one implementation of device 60, a variant is possible in which the first and second feature vectors are processed by independent layers of the neural network, after which intermediate estimates of the probability of authenticity are formed, on the basis of which the final estimate of the probability of authenticity of the user is then calculated by weighted summation with a predetermined numerical coefficient:

prob=coef*prob1+(1-coef)*prob2

where prob and prob2 are intermediate estimates of the probability of authenticity calculated on the basis of the first and second feature vectors, coef is a predetermined coefficient.

[075] In another implementation of device 60, a concatenation operation is performed on the first and second feature vectors, after which the combined vector is processed by a common neural network to form a final assessment of the probability of user authenticity.

[076] The image class information may be forwarded by the device 60 to various user authorization systems to make appropriate decisions depending on whether the face in the image is genuine.

[077] Thus, due to the fact that the final value of the probability of authenticity of the user is determined taking into account the location of the key points determined for the face image and the user image, the resulting accuracy in determining the authenticity of the face is increased.

[078] In general form (see Fig. 5), the computing device comprises one or more processors (101), memory means such as RAM (102) and ROM (103), input/output interfaces (104), input/output devices (105), and a device for network interaction (106), united by a common information exchange bus.

[079] The processor (101) (or several processors, a multi-core processor, etc.) can be selected from a range of devices that are widely used at present, for example, from manufacturers such as: Intel™, AMD™, Apple™, Samsung Exynos™, MediaTEK™, Qualcomm Snapdragon™, etc. The processor or one of the processors used in the device (100) must also include a graphics processor, for example, an NVIDIA or Graphcore GPU, the type of which is also suitable for the full or partial implementation of the method, and can also be used for training and applying machine learning models in various information systems.

[080] RAM (102) is a random access memory and is intended for storing machine-readable instructions executable by the processor (101) for performing the necessary operations for logical data processing. RAM (102), as a rule, contains executable instructions of the operating system and the corresponding software components (applications, software modules, etc.). In this case, the available memory capacity of the graphic card or graphic processor may act as RAM (102).

[081] The ROM (103) is one or more permanent data storage devices, such as a hard disk drive (HDD), a solid-state drive (SSD), flash memory (EEPROM, NAND, etc.), optical storage media (CD-R/RW, DVD-R/RW, BlueRay Disc, MD), etc.

[082] To organize the operation of the components of the device (100) and to organize the operation of external connected devices, various types of I/O interfaces (104) are used. The choice of the corresponding interfaces depends on the specific design of the computing device, which may be, but are not limited to: PCI, AGP, PS/2, IrDa, FireWire, LPT, COM, SATA, IDE, Lightning, USB (2.0, 3.0, 3.1, micro, mini, type C), TRS/Audio jack (2.5, 3.5, 6.35), HDMI, DVI, VGA, Display Port, RJ45, RS232, etc.

[083] To ensure user interaction with the device (100), various means (105) of I/O information are used, for example, a keyboard, a display (monitor), a touch display, a touchpad, a joystick, a mouse, a light pen, a stylus, a touch panel, a trackball, speakers, a microphone, augmented reality means, optical sensors, a tablet, light indicators, a projector, a camera, biometric identification means (a retina scanner, a fingerprint scanner, a voice recognition module), etc.

[084] The network interaction means (106) provides data transmission via an internal or external computer network, for example, an Intranet, the Internet, a LAN, etc. One or more means (206) may be, but are not limited to: an Ethernet card, a GSM modem, a GPRS modem, an LTE modem, a 5G modem, a satellite communication module, an NFC module, a Bluetooth and/or BLE module, a Wi-Fi module, etc.

[085] Additionally, satellite navigation means may also be used as part of the device (100), for example, GPS, GLONASS, BeiDou, Galileo. The specific selection of elements of the device (100) for the implementation of various software and hardware architectural solutions may vary while maintaining the required functionality provided.

[086] Modifications and improvements of the above-described embodiments of the present technical solution will be clear to those skilled in the art. The preceding description is provided only as an example and does not carry any limitations. Therefore, the scope of the present technical solution is limited only by the scope of the appended claims.

Claims (33)

1. A method for determining the authenticity of a face in an image, performed by at least one computing device, comprising the steps of: a. obtain the user's original image; b. determine the coordinates of the user's face position; c. generate an image of the face in accordance with the coordinates of the position of the user's face; d. determine the position of the facial key points for the facial image and the position of the user key points for the user image; e. forming a first mask of facial key points for a face image and a second mask of user key points for a user image; f. calculating a first vector representation of features of a face based on the face image and the first mask of key points of the face and a second vector representation of features of a user based on the user image and the second mask of key points of the user; g. based on the first and second feature vectors, the final assessment of the probability of the user's authenticity in the original image is determined; h. based on the value of the final assessment of the probability of authenticity of the user in the image, a class of the image is determined indicating whether the person presented in the image is genuine or fake. 2. The method according to claim 1, characterized in that steps f-g are performed by a machine learning model or an ensemble of models, wherein the machine learning model or ensemble of models is trained on a data set containing genuine and fake images of users’ faces. 3. The method according to paragraph 2, characterized in that the ensemble of models consists of individual models, each of which is trained to identify a type of counterfeit determined by the developer. 4. The method according to item 1, characterized in that the stage of forming masks of key points of the face and the user is carried out using a neural network trained for the purpose of increasing the accuracy of determining the final assessment of the probability of the user's authenticity. 5. The method according to item 1, characterized in that the stage of forming masks of key points of the face and the user comprises stages in which: - form closed contours in accordance with the positions of the coordinates of key points; - increase the line width of closed contours by adding a specified number of adjacent pixels to the said lines, or increase the line width of closed contours by using blur filters with specified parameters, such as a Gaussian filter. 6. The method according to item 1, characterized in that an additional step is performed in which images are normalized by bringing the pixel values to specified levels of mathematical expectation and distribution variance or by means of a linear transformation to a specified range of values. 7. The method according to claim 1, characterized in that a step is additionally performed in which the sizes of the images are changed to match the inputs of the models for determining image features by means of two-dimensional interpolation of the images. 8. The method according to item 1, characterized in that the stage of forming a face image comprises stages in which: - in accordance with the coordinates of the face position, a square area is selected on the user's image; - enlarge the square area by a specified number of pixels, and select the image located in the enlarged square area as the face image. 9. The method according to claim 4, characterized in that additional steps are performed in which it is determined that the selected enlarged square area extends beyond the user's image and replication of the boundary pixels of the user's image is performed to fill said selected area. 10. The method according to paragraph 1, characterized in that additional steps are performed at which the following is formed: - input data for the first model by matrix addition or multiplication of the pixel values of the first keypoint mask and the processed face image; - input data for the second model by means of matrix addition or multiplication of the pixel values of the second key point mask and the processed user image, wherein when performing the addition or multiplication operation, the elements of the key point mask matrix are multiplied by a predetermined numerical coefficient. 11. The method according to paragraph 1, characterized in that additional steps are performed at which the following are formed: - input data for the first model by concatenating the matrices of the first keypoint mask and the processed face image; and - input data for the second model by concatenating the matrices of the second keypoint mask and the processed user image. 12. The method according to paragraph 1, characterized in that additional steps are performed at which the following is formed: - input data for the first model by weighted addition of three matrices corresponding to the pixels of the first keypoint mask, the inverted first keypoint mask and the processed face image; - input data for the second model by weighted addition of three matrices corresponding to the pixels of the second keypoint mask, the inverted second keypoint mask and the processed user image. 13. The method according to item 1, characterized in that in order to determine the class of the image, the first and second feature vectors are processed by independent layers of the neural network. 14. The method according to item 1, characterized in that in order to determine the class of the image, the first and second feature vectors are concatenated, after which they are processed by a common neural network. 15. A system for determining the authenticity of a face in an image, comprising at least one computing device and at least one memory containing machine-readable instructions which, when executed by at least one computing device, perform the method according to any one of paragraphs 1-14.