WO2021131029A1 - Filter generation device, estimation device, facial authentication system, filter generation method, and recording medium - Google Patents

Abstract

A filter generation device comprises: an adversarial-perturbation acquisition means that acquires a plurality of adversarial perturbations due to the difference between a face image and an adversarial sample based on the face image; and a filter generation means that generates a filter on the basis of the plurality of adversarial perturbations.

Description

Filter generator, estimation device, face recognition system, filter generation method and recording medium

The present invention relates to a filter generation device, an estimation device, a face recognition system, a filter generation method, and a recording medium.

In relation to the prevention of spoofing in face recognition, Patent Document 1 discloses a face recognition system that executes multitasking including a face recognition task and a biometric detection task.

Japan Special Table 2018-508801 Gazette

When performing face recognition using artificial intelligence, it is preferable to be able to estimate the possibility that the face image to be authenticated is a hostile sample.

An example of an object of the present invention is to provide a filter generation device, an estimation device, a face recognition system, a filter generation method, and a recording medium capable of solving the above problems.

According to the first aspect of the present invention, the filter generation device includes a hostile perturbation acquisition means for acquiring a plurality of hostile perturbations due to a difference between a face image and a hostile sample based on the face image, and a plurality of the hostiles. A filter generation means for generating a filter based on a target perturbation is provided.

According to the second aspect of the present invention, the filter generation method is based on a step of acquiring a plurality of hostile perturbations due to a difference between a face image and a hostile sample based on the face image, and a plurality of the hostile perturbations. The process of generating a filter and the like.

According to a third aspect of the present invention, the recording medium includes a step of acquiring a plurality of hostile perturbations due to a difference between a face image and a hostile sample based on the face image, and a plurality of the hostile perturbations. It is a recording medium for recording a process for generating a filter based on the above and a program for executing the filter.

According to the above aspect, it is possible to estimate the possibility that the face image to be authenticated is a hostile sample.

It is a schematic block diagram which shows the example of the apparatus configuration of the face recognition system which concerns on embodiment. It is a schematic block diagram which shows the example of the functional structure of the face recognition apparatus which concerns on embodiment. It is a schematic block diagram which shows the example of the functional structure of the filter generation apparatus which concerns on embodiment. It is a schematic block diagram which shows the example of the functional structure of the estimation apparatus which concerns on embodiment. It is a flowchart which shows the example of the processing procedure which the filter generation apparatus which concerns on embodiment generates a filter for hostile sample determination. FIG. 5 is a flowchart showing an example of a processing procedure in which the estimation device according to the embodiment determines whether or not the face image to be authenticated is a hostile sample. It is a figure which shows the example of the structure of the filter generation apparatus which concerns on embodiment. It is a figure which shows the example of the processing procedure in the filter generation method which concerns on embodiment. It is a schematic block diagram which shows the example of the structure of the computer which concerns on at least one Embodiment.

Hereinafter, embodiments of the present invention will be described, but the following embodiments do not limit the inventions claimed. Also, not all combinations of features described in the embodiments are essential to the means of solving the invention.
FIG. 1 is a schematic configuration diagram showing an example of a device configuration of a face recognition system according to an embodiment. With the configuration shown in FIG. 1, the face recognition system 1 includes a face recognition device 100, a filter generation device 200, and an estimation device 300.

Face recognition system 1 performs face recognition using an artificial intelligence model. Specifically, the face recognition system 1 inputs the face image of the authentication target person into the artificial intelligence model, acquires the feature vector of the face image, and authenticates the authentication target person using the obtained feature vector. The artificial intelligence model referred to here is a model that has been trained by machine learning. In addition, the face recognition system 1 estimates the possibility of a hostile sample (Adversarial Example) for the face image of the person to be authenticated.

By processing the original image, the hostile sample here is classified into the same class as the original image in the classification by humans, whereas the class classification by artificial intelligence is different from the original image. It is an image after processing classified into. Also, processing the original image to generate a hostile sample is also referred to as a hostile sample.

Processing in hostile samples is referred to as adding "noise" because human classification does not cause a significant change in different classes.
As an example of a hostile sample, in particular, an example of intentionally deceiving a neural network model by deep learning is known. However, the artificial intelligence model used by the face recognition system 1 is not limited to deep learning, and can be various artificial intelligence models in which classification errors due to hostile samples can occur.

If a hostile sample is used for face recognition by the face recognition system 1, the face recognition system 1 may erroneously authenticate. Therefore, it is expected that the face recognition system 1 can prevent or reduce the damage caused by erroneous authentication when the hostile sample is used by estimating the possibility of the hostile sample.

For example, when the face recognition system 1 notifies a person such as an administrator of the result of face recognition, the face recognition system 1 may notify the possibility of a hostile sample together with the authentication result. In this case, the face recognition system 1 may determine whether or not the sample is hostile and notify the determination result. Alternatively, the face recognition system 1 may calculate a score (index value) indicating the magnitude of the possibility of the hostile sample and notify the calculated score.

Alternatively, when the face recognition system 1 performs login authentication of a certain system such as a computer system, it may be determined whether or not it is a hostile sample in addition to the face recognition. Then, when the face recognition system 1 determines that the sample is hostile, login may not be permitted regardless of the result of face recognition.

When the face recognition system 1 is provided with a gate opening / closing mechanism and the entrance is permitted by opening the gate, it may be determined whether or not the sample is hostile in addition to the face recognition. Then, when the face recognition system 1 determines that the sample is hostile, the entrance may not be permitted by not opening the gate regardless of the result of the face recognition.

An example of a hostile sample in face recognition is the use of eyeglasses. If you are wearing glasses with hostile noise, the image taken by the camera may be a hostile sample.
In this case, the face recognition system 1 can prevent or reduce the damage caused by the erroneous authentication as described above by estimating the possibility that the captured image is a hostile sample.

Also, it is possible that the registered image for face recognition is a hostile sample. For example, in the case of an operation in which the person submits a face photo for registration, it is conceivable that a target person who wants to manipulate the authentication result illegally submits a photo of a hostile sample. Alternatively, in the case of an operation of taking a face photograph for registration, it is conceivable that a target person who wants to manipulate the authentication result illegally wears glasses with hostile noise and is photographed.

In this case, the face recognition system 1 may estimate the possibility that the registered image is a hostile sample. For example, when registering an image, the face recognition system 1 determines whether or not the image to be registered is a hostile sample, and if it determines that the image is a hostile sample, the registration is stopped and an alarm message is output. You may. Alternatively, the face recognition system 1 may determine whether or not the registered image is a hostile sample at the time of executing face recognition or periodically by background processing or the like.

The face recognition device 100 performs the face recognition described above for the face recognition system 1.
The face recognition device 100 may be an existing one as long as it can output the result of class classification in face recognition and the feature vector of the face image calculated in the process of face recognition. That is, the face recognition system 1 may be configured by using the existing face recognition device as the face recognition device 100.

The result of the classification in the face recognition referred to here is whether the person to be authenticated is judged to be one or more of the pre-registered people, or is judged to be different from any person. This is the judgment result.
The feature vector of the image referred to here is a vector indicating the feature amount of the image. In the face recognition device 100, the machine learning model receives the input of the face image and outputs the feature vector of the face image.

The estimation device 300 estimates the possibility of the hostile sample described above for the face recognition system 1.
The filter generation device 200 generates a filter (image filter) for the estimation device 300 to use for estimating the possibility of a hostile sample. The filter generated by the filter generation device 200 is also referred to as a hostile sample determination filter.

Each of the face recognition device 100, the filter generation device 200, and the estimation device 300 may be configured by using a computer such as a personal computer (PC) or a workstation (WS). Alternatively, one or more of the face recognition device 100, the filter generation device 200, and the estimation device 300 may be configured by using dedicated hardware.

For example, when the face recognition device 100 uses a neural network model as an artificial intelligence model for face recognition, the computer constituting the face recognition device 100 may simulate the function of the neural network. Alternatively, the face recognition device 100 may include a neural network configured in hardware.

The face recognition device 100, the filter generation device 200, and the estimation device 300 may be configured as separate devices. Alternatively, any two or more of the face recognition device 100, the filter generation device 200, and the estimation device 300 may be configured as an integrated device.
For example, the face recognition device 100 and the estimation device 300 may be configured as an integrated device using one computer. Then, as described above for the face recognition system 1, this device may perform processing based on the estimation result of the possibility of the hostile sample in addition to the result of the face recognition.

FIG. 2 is a schematic block diagram showing an example of the functional configuration of the face recognition device 100. With the configuration shown in FIG. 2, the face recognition device 100 includes a first communication unit 110, an authentication image acquisition unit 120, an authentication result processing unit 130, a first storage unit 180, and a first control unit 190. .. The first storage unit 180 includes an authentication database unit 181. The first control unit 190 includes a feature vector calculation unit 191 and a classification unit 192.

The first communication unit 110 communicates with another device. In particular, the first communication unit 110 receives the face image generated by the filter generation device 200 as a candidate for the hostile sample and the original face image thereof. Then, the first communication unit 110 transmits the result of the classification in the face recognition to the filter generation device 200 for each of the hostile sample candidate and the original face image.
The result of the classification in the face recognition is that the filter generation device 200 determines whether or not the candidate for the hostile sample is a hostile sample and collects the hostile sample when the filter for determining the hostile sample is generated. Used for.

Further, the first communication unit 110 transmits the face image to be authenticated to the estimation device 300, and the estimation device 300 receives the face image obtained by applying the hostile sample determination filter to the face image to be authenticated. The face image to be authenticated is also referred to as a first image. A face image obtained by applying a hostile sample determination filter to the first image is also referred to as a second image.
Then, the first communication unit 110 transmits the feature vector of the first image and the feature vector of the second image to the estimation device 300. The feature vector of the first image is also referred to as a first feature vector. The feature vector of the second image is also referred to as a second feature vector.
The first feature vector and the second feature vector transmitted by the first communication unit 110 to the estimation device 300 are used by the estimation device 300 to estimate the possibility that the face image to be authenticated is a hostile sample.

The authentication image acquisition unit 120 acquires a face image to be authenticated. The authentication image acquisition unit 120 may be configured to include a camera and capture a face image of the person to be authenticated. Alternatively, the authentication image acquisition unit 120 may be configured to include a camera or a scanner, and may read a face photograph of a person to be authenticated, for example, a face photograph of a passport or a face photograph of a driver's license. Alternatively, the first communication unit 110 may be used as the authentication image acquisition unit 120 to receive the face image from another device, such as receiving the face image transmitted by the client device on the Internet.

The authentication result processing unit 130 performs processing according to the result of face authentication by the face authentication device 100. The processing performed by the authentication result processing unit 130 can be various processing depending on the usage pattern of the face recognition system 1.
For example, when the face recognition system 1 performs login authentication to a certain system, when the face recognition system 1 succeeds in face recognition, the authentication result processing unit 130 informs the user identified as the person to be authenticated by the face recognition. The login process may be performed accordingly. Successful face recognition in this case means that the person to be authenticated is identified as one of the users registered in the authentication database.

On the other hand, when the face recognition system 1 fails in face recognition, the authentication result processing unit 130 may control the display device of the system to display a message indicating that the authentication has failed. If the authentication fails in this case, it is determined that the authentication target person is different from any user registered in the authentication database.

Alternatively, when the face recognition system 1 is provided with a gate opening / closing mechanism and the entrance is permitted by opening the gate, the authentication result processing unit 130 may control the opening / closing of the gate according to the result of face recognition. In this case, when the face recognition system 1 succeeds in face recognition, the authentication result processing unit 130 may open the gate. Successful face recognition in this case means that the person to be authenticated is identified as one of the admission persons registered in the authentication database.
On the other hand, when the face recognition system 1 fails in face recognition, the authentication result processing unit 130 may keep the gate closed. If the authentication fails in this case, it is determined that the person to be authenticated is different from any admission person registered in the authentication database.

The first storage unit 180 stores various data. The function of the first storage unit 180 is executed by using the storage device included in the face recognition device 100.
The authentication database unit 181 stores the authentication database. The authentication database referred to here is data to be compared with the data of the authentication target person in face authentication. The authentication database contains data of candidates whose authentication target is identified by face recognition. The data of each candidate includes a feature vector of the candidate's facial image. In the classification in face recognition, each candidate and no corresponding candidate correspond to the example of the class. Then, determining which candidate the authentication target person is, or determining that the person to be authenticated is different from any candidate corresponds to the example of classification.

For example, when the face authentication device 100 performs login authentication by face authentication, the authentication database includes data for each user. The data for each user in this case includes a feature vector of the user's face image, the user name of the user, and setting information when the user logs in.

The first control unit 190 controls each unit of the face recognition device 100 to execute various processes. The function of the first control unit 190 is executed, for example, by the CPU (Central Processing Unit) included in the face authentication device 100 reading a program from the first storage unit 180 and executing the program.

The feature vector calculation unit 191 is configured to include the above-mentioned artificial intelligence model, and calculates the feature vector of the face image. Specifically, the feature vector calculation unit 191 calculates the feature vector (first feature vector) of the face image to be authenticated. In addition, the feature vector calculation unit 191 calculates the feature vector of the hostile sampling candidate generated and transmitted by the filter generation device 200. In addition, the feature vector calculation unit 191 calculates the feature vector (second feature vector) of the second image generated and transmitted by the estimation device 300.

The classification unit 192 classifies the face in face recognition using the feature vector of the face image to be authenticated. For example, the classification unit 192 calculates a score that quantitatively indicates the degree of similarity between the feature vector of the face image of the candidate and the feature vector of the face image to be authenticated for each candidate shown in the authentication database. The score calculated by the classification unit 192 is also referred to as an authentication score.

In the following, an example will be described in which the larger the score calculated by the classification unit 192, the greater the degree of similarity of the feature vectors. However, the classification unit 192 may calculate the authentication score indicating that the smaller the score, the greater the degree of similarity of the feature vectors.

When all the calculated authentication scores are less than the predetermined authentication threshold value, the classification unit 192 determines that the authentication target person is different from any candidate. On the other hand, when any one or more of the calculated authentication scores is equal to or higher than the authentication threshold value, the classification unit 192 determines that the authentication target person is the candidate whose authentication score is calculated to be the largest.

FIG. 3 is a schematic block diagram showing an example of the functional configuration of the filter generation device 200. With the configuration shown in FIG. 3, the filter generation device 200 includes a second communication unit 210, a second storage unit 280, and a second control unit 290. The second control unit 290 includes a hostile perturbation acquisition unit 291, a filter generation image acquisition unit 292, a noise addition unit 293, a hostile sample selection unit 294, an acquisition processing unit 295, and a filter generation unit 296. Be prepared.

The second communication unit 210 communicates with other devices. In particular, the second communication unit 210 transmits the face image generated by the filter generation device 200 as a candidate for the hostile sample and the original face image thereof to the face recognition device 100. Then, the second communication unit 210 receives the result of the classification in the face recognition from the face recognition device 100 for each of the hostile sample candidate and the original face image.

The second storage unit 280 stores various data. The function of the second storage unit 280 is executed by using the storage device included in the filter generation device 200.
The second control unit 290 controls each unit of the filter generation device 200 to execute various processes. The function of the second control unit 290 is executed, for example, by the CPU included in the filter generation device 200 reading a program from the second storage unit 280 and executing the program.

The hostile perturbation acquisition unit 291 acquires a plurality of adversarial perturbations due to the difference between the face image and the hostile sample based on the face image. The hostile sample based on the face image referred to here is a face image as a hostile sample in which noise is added to the face image. The hostile perturbation here is the difference obtained by subtracting the original facial image from the hostile sample. Therefore, a hostile perturbation is an image that corresponds to the noise added to the original facial image to generate a hostile sample.
The hostile perturbation acquisition unit 291 corresponds to an example of a hostile perturbation acquisition means.

The method by which the hostile perturbation acquisition unit 291 acquires a hostile perturbation is not limited to a specific method.
When the hostile perturbation acquisition unit 291 adds noise to the face image to generate a hostile sample, the noise may be used as the hostile perturbation.

Alternatively, when the hostile perturbation acquisition unit 291 acquires an existing hostile sample and its original face image, the difference obtained by subtracting the value of the corresponding pixel in the original face image from the value of each pixel of the hostile sample. The image of may be calculated as a hostile perturbation. The corresponding pixels referred to here are pixels having the same relative position in the image. Therefore, the pixels that overlap when a plurality of images of the same size are overlapped with their edges aligned correspond to the corresponding pixels. Equal image size means that both the number of vertical pixels and the number of horizontal pixels of a plurality of images are the same.

Alternatively, when the hostile perturbation acquisition unit 291 acquires an existing hostile sample, the hostile perturbation may be acquired by using a known hostile perturbation calculation method.
Alternatively, the hostile perturbation acquisition unit 291 may acquire an existing hostile perturbation.

The plurality of hostile perturbations acquired by the hostile perturbation acquisition unit 291 may all be calculated using the same facial image.
Alternatively, the hostile perturbation acquisition unit 291 may acquire the hostile perturbation calculated by using the face image for each of the plurality of face images. When the hostile perturbation acquisition unit 291 acquires the hostile perturbations calculated by using the face images for each of the plurality of face images, the plurality of face images may be the face images of the same person. However, facial images of a plurality of persons may be included.
In the following, a case where the hostile perturbation acquisition unit 291 adds noise to the face image to generate a hostile sample will be described as an example.

The filter generation image acquisition unit 292 acquires a face image that is the source of the hostile sample. The face image acquired by the filter generation image acquisition unit 292 is also referred to as a filter generation image.
The method by which the filter generation image acquisition unit 292 acquires the filter generation image is not limited to a specific method. For example, the filter generation device 200 may include a camera, and the filter generation image acquisition unit 292 may acquire the face image captured by the camera as the filter generation image. Alternatively, the authentication image acquisition unit 120 of the face recognition device 100 may include a camera, and the filter generation image acquisition unit 292 may acquire the face image captured by the camera as a filter generation image. Alternatively, the filter generation image acquisition unit 292 may acquire the existing face image, such as the second communication unit 210 receiving the existing face image from another device.

The noise addition unit 293 acquires candidates for hostile samples by adding noise to the filter generation image. For example, the noise addition unit 293 may randomly generate a noise image in which the amount of increase / decrease in the brightness value of each pixel is within a predetermined range, and add the generated noise to the filter generation image.

The hostile sample selection unit 294 selects a hostile sample from the hostile sample candidates generated by the noise addition unit 293. The method by which the hostile sample selection unit 294 selects a hostile sample is not limited to a specific method.
For example, the face recognition device 100 may classify each of the hostile sample candidate and the original face image (filter generation image) in face recognition, and transmit the classification result to the filter generation device 200. Then, the hostile sample selection unit 294 may select, among the candidates for the hostile sample, a candidate classified into a class different from the filter generation image as the hostile sample.

Alternatively, a person such as a user of the face recognition system 1 may visually check the candidates for the hostile sample and determine whether or not the sample is hostile. In this case, the filter generation device 200 may include a display device and an input device, display hostile sample candidates on the display device, and accept the user operation of the determination result on the input device. Then, the hostile sample selection unit 294 may manually select an image determined to be a hostile sample from the candidates for the hostile sample.

Acquisition processing unit 295 acquires hostile perturbations. Specifically, the acquisition processing unit 295 acquires the noise added to the filter generation image by the noise addition unit 293 as a hostile perturbation in order to generate the hostile sample selected by the hostile sample selection unit 294. ..

The filter generation unit 296 generates a hostile sample determination filter based on a plurality of hostile perturbations. Therefore, the hostile perturbation acquisition unit 291 may acquire a plurality of hostile perturbations. The filter generation unit 296 generates a filter for determining a hostile sample by synthesizing a plurality of hostile perturbations, for example, obtaining an average or a weighted sum of a plurality of hostile perturbations.
The filter generation unit 296 corresponds to an example of a filter generation means.

The average of a plurality of hostile perturbations is one image obtained by calculating the average of pixel values for each corresponding pixel for a plurality of hostile perturbations as a plurality of images having the same size. The weighted total of multiple hostile perturbations is a weighting coefficient set for each hostile perturbation for multiple hostile perturbations as multiple images of the same size, and the weighted total is calculated for each corresponding pixel. It is one image obtained.

For example, the filter generator 296 multiplies each of the plurality of hostile perturbations by a weighting factor that increases the difference between the original image and the hostile sample based on the difference between the original image and the hostile sample. May be done. As the degree of difference between the original image and the hostile sample, for example, the distance between the feature vector of the original image and the feature vector of the hostile sample can be used. As the distance in this case, for example, L1 norm, L2 norm, L∞ norm, or cosine distance can be used.
However, the weight used by the filter generation unit 296 for the total weighting is not limited to a specific one.

FIG. 4 is a schematic block diagram showing an example of the functional configuration of the estimation device 300. With the configuration shown in FIG. 4, the estimation device 300 includes a third communication unit 310, a determination result processing unit 320, a third storage unit 380, and a third control unit 390. The third control unit 390 includes a first image acquisition unit 391, a filter application unit 392, a score calculation unit 393, and a determination unit 394.

The third communication unit 310 communicates with other devices. In particular, the third communication unit 310 receives the face image (first image) to be authenticated from the face recognition device 100, and the estimation device 300 applies the hostile sample determination filter to the face image to be authenticated. The face image (second image) is transmitted to the face recognition device 100. Then, the third communication unit 310 receives the feature vector of the first image and the feature vector of the second image from the face recognition device 100.

When the estimation device 300 determines that the face image to be authenticated is a hostile sample, the determination result processing unit 320 performs processing according to the determination result. The processing performed by the determination result processing unit 320 can be various processing depending on the usage pattern of the face authentication system 1. The process performed by the determination result processing unit 320 is also referred to as a hostile sample response process.

For example, when the face recognition system 1 notifies a person such as an administrator of the result of face recognition, the judgment result processing unit 320 displays a judgment result that the face image to be authenticated is a hostile sample. The result may be notified to the person. Alternatively, the determination result processing unit 320 notifies the person of the score, such as displaying a score indicating the possibility that the face image to be authenticated is a hostile sample, regardless of the determination result of whether or not the sample is hostile. You may do so.

Alternatively, when the face recognition system 1 performs login authentication to a certain system, the third communication unit 310 functions as the determination result processing unit 320 to notify the face that the face image to be authenticated is a hostile sample. It may be transmitted to the authentication device 100. In the face recognition device 100 that has been notified that the face image to be authenticated is a hostile sample, the authentication result processing unit 130 performs processing when face recognition fails regardless of the result of face recognition. You may.

Alternatively, when the face recognition system 1 is provided with a gate opening / closing mechanism and the entrance is permitted by opening the gate, the third communication unit 310 functions as the determination result processing unit 320, and the face image to be authenticated is a hostile sample. The notification to that effect may be transmitted to the face recognition device 100. In the face recognition device 100 that has been notified that the face image to be authenticated is a hostile sample, the authentication result processing unit 130 performs processing when face recognition fails regardless of the result of face recognition. You may.

The third storage unit 380 stores various data. The function of the third storage unit 380 is executed by using the storage device included in the estimation device 300.
The third control unit 390 controls each unit of the estimation device 300 to execute various processes. The function of the third control unit 390 is executed, for example, by the CPU included in the estimation device 300 reading a program from the third storage unit 380 and executing the program.

The first image acquisition unit 391 acquires the first image. Specifically, the face recognition device 100 transmits a face image (first image) to be authenticated acquired when performing face recognition to the estimation device 300. In the estimation device 300, the first image acquisition unit 391 extracts the first signal from the signal received from the face recognition device 100 by the third communication unit 310.

The filter application unit 392 applies a hostile sample determination filter to the first image acquired by the first image acquisition unit 391 to acquire the second image. For example, the filter application unit 392 adds the value of each pixel of the first image to the value of the corresponding pixel in the hostile sample determination filter as the application of the hostile sample determination filter to the first image.
The filter application unit 392 corresponds to an example of the filter application means.

The score calculation unit 393 calculates a score that quantitatively indicates the degree of difference between the first feature vector (feature vector of the first image) and the second feature vector (feature vector of the second image). The score calculated by the score calculation unit 393 is also referred to as a hostile sample score. The method in which the score calculation unit 393 calculates the hostile sample score and the method in which the classification unit 192 calculates the authentication score may be the same method or different methods.
The score calculation unit 393 corresponds to an example of the score calculation means.

The method by which the score calculation unit 393 calculates the hostile sample score is not limited to a specific method. The score calculation unit 393 may calculate a hostile sample score indicating that the larger the score, the greater the degree of difference between the first feature vector and the second feature vector. Alternatively, the score calculation unit 393 may calculate a hostile sample score indicating that the smaller the score, the greater the degree of difference between the first feature vector and the second feature vector.

For example, the score calculation unit 393 may calculate any of the L1 norm, L2 norm, L∞ norm, or cosine distance between the first feature vector and the second feature vector as a hostile sample score. The cosine distance is 1 minus the cosine similarity. These hostile sample scores correspond to the example of the hostile sample scores showing that the larger the score, the greater the degree of difference between the first feature vector and the second feature vector.

Alternatively, the score calculation unit 393 may calculate either the cosine similarity between the first feature vector and the second feature vector or the inner product as a hostile sample score. These hostile sample scores correspond to examples of hostile sample scores indicating that the smaller the score, the greater the degree of difference between the first feature vector and the second feature vector.

The hostile sample score calculated by the score calculation unit 393 is used as a score indicating the degree of possibility that the face image to be authenticated is a hostile sample. It is presumed that the greater the degree of difference between the first feature vector and the second feature vector indicated by the hostile sample score, the higher the possibility that the face image to be authenticated is a hostile sample.

If the larger the hostile sample score indicates that the degree of difference between the first feature vector and the second feature vector is greater, the larger the hostile sample score, the more likely the face image to be authenticated is a hostile sample. Indicates that is high. On the other hand, when the smaller the hostile sample score indicates that the degree of difference between the first feature vector and the second feature vector is larger, the smaller the hostile sample score, the more the face image to be authenticated is the hostile sample. Indicates a high probability.

Using the hostile sample score as a score indicating the magnitude of the possibility that the face image to be authenticated is a hostile sample is described as "the first image and the first image to which the hostile sample determination filter is applied. The greater the degree of difference between the two images, the more likely the first image is a hostile sample. " In the experiment, "The greater the degree of difference between the first image and the second image to which the hostile sample determination filter is applied to the first image, the higher the possibility that the first image is a hostile sample." , Positive results have been obtained.

The determination unit 394 determines whether or not the face image to be authenticated is a hostile sample based on the hostile sample score calculated by the score calculation unit 393. For example, the determination unit 394 compares the hostile sample score with a predetermined determination threshold.
When the larger the hostile sample score indicates that the face image to be authenticated is more likely to be a hostile sample, the determination unit 394 determines that the face to be authenticated is equal to or greater than the determination threshold value. Determine that the image is a hostile sample. In this case, the determination unit 394 determines that the face image to be authenticated is not a hostile sample when the hostile sample score is less than the determination threshold value.

When the smaller the hostile sample score indicates that the face image to be authenticated is more likely to be a hostile sample, the determination unit 394 determines the face to be authenticated when the hostile sample score is equal to or less than the determination threshold value. Determine that the image is a hostile sample. In this case, the determination unit 394 determines that the face image to be authenticated is not a hostile sample when the hostile sample score is larger than the determination threshold value.
The determination result processing unit 320 described above performs processing according to the determination result of the determination unit 394.

However, the estimation device 300 may output a score indicating the degree of possibility that the face image to be authenticated is a hostile sample instead of the determination result of whether or not the face image to be authenticated is a hostile sample. In this case, the estimation device 300 does not have to include the determination unit 394.

The estimation device 300 may output the hostile sample score as it is as a score indicating the magnitude of the possibility that the face image to be authenticated is a hostile sample. Alternatively, the estimation device 300 may process and output the hostile sample score. For example, the estimation device 300 may convert the hostile sample score into a score indicating the magnitude of the possibility that the face image to be authenticated is a hostile sample and output it.
Further, as described above for the face recognition system 1, the estimation device 300 estimates and estimates the possibility that the registered image or the image to be registered is a hostile sample in addition to or instead of the face image to be authenticated. The process may be performed according to the result.

Next, the operation of the face recognition system 1 will be described with reference to FIGS. 5 and 6.
FIG. 5 is a flowchart showing an example of a processing procedure in which the filter generation device 200 generates a filter for hostile sample determination. The filter generation device 200 performs the processing of FIG. 5, for example, when the generation of the hostile sample determination filter is instructed by a user operation.

In the process of FIG. 5, the filter generation image acquisition unit 292 acquires the filter generation image (step S101). The noise addition unit 293 adds noise to the filter generation image acquired by the filter generation image acquisition unit 292 to generate hostile sample candidates (step S102).
Next, the hostile sample selection unit 294 acquires the result of the classification in the face recognition of the face recognition device 100 for each of the filter generation image and the hostile sample candidate (step S103).

For example, the second communication unit 210 transmits a filter generation image and a candidate for a hostile sample to the face recognition device 100. The face recognition device 100 classifies each of the filter generation image and the hostile sample candidate into classes, and transmits the classification result to the filter generation device 200. In the filter generation device 200, the hostile sample selection unit 294 uses the signal received by the second communication unit 210 from the filter generation device 200 to classify the filter generation image and the hostile sample candidate. Is extracted.

Then, the hostile sample selection unit 294 determines whether or not the candidate for the hostile sample is a hostile sample (step S104). Specifically, when the filter generation image and the hostile sample candidate are classified into different classes, the hostile sample selection unit 294 determines that the hostile sample candidate is a hostile sample. On the other hand, when the filter generation image and the hostile sample candidate are classified into the same class, the hostile sample selection unit 294 determines that the hostile sample candidate is not a hostile sample.

When the hostile sample selection unit 294 determines that the hostile sample candidate is a hostile sample (step S104: YES), the acquisition processing unit 295 acquires the hostile perturbation and stores it in the second storage unit 280. (Step S111). For example, the acquisition processing unit 295 acquires the noise added to the filter generation image by the noise addition unit 293 in step S102 as a hostile perturbation.

Then, the acquisition processing unit 295 determines whether or not the number of acquired hostile perturbations has reached a predetermined number (step S112).
If the hostile sample selection unit 294 determines in step S104 that the candidate for the hostile sample is not a hostile sample (step S104: NO), the process proceeds to step S112.

If the acquisition processing unit 295 determines in step S112 that the number of acquired hostile perturbations has not reached a predetermined number (step S112: NO), the processing returns to step S102.
On the other hand, in step S112, when the acquisition processing unit 295 determines that the number of acquired hostile perturbations has reached a predetermined number (step S112: YES), the filter generation unit 296 determines that the number of acquired hostile perturbations has reached a predetermined number (step S112: YES). To generate a hostile sample determination filter (step S121).
After step S121, the filter generator 200 ends the process of FIG.

FIG. 6 is a flowchart showing an example of a processing procedure in which the estimation device 300 determines whether or not the face image to be authenticated is a hostile sample. For example, when the face recognition device 100 starts face recognition, it transmits a signal instructing the start of the process of FIG. 6 to the estimation device 300. The estimation device 300 starts the process of FIG. 6 in response to an instruction from the face recognition device 100.

In the process of FIG. 6, the first image acquisition unit 391 acquires the first image (face image to be authenticated) (step S211). Specifically, the face recognition device 100 transmits the first image to the estimation device 300. The first image acquisition unit 391 extracts the first image from the signal received from the face recognition device 100 by the third communication unit 310. The first image acquisition unit 391 may acquire the registered image or the image to be registered as the first image in addition to or instead of the face image to be authenticated.
The filter application unit 392 applies a hostile sample determination filter to the first image acquired by the first image acquisition unit 391 to generate a second image (step S212).

Then, the score calculation unit 393 acquires the second feature vector (step S213). For example, the third communication unit 310 transmits the second image to the face recognition device 100. In the face recognition device 100, the feature vector calculation unit 191 calculates the feature vector (second feature vector) of the second image, and the first communication unit 110 transmits the obtained second feature vector to the estimation device 300. .. The score calculation unit 393 extracts the second feature vector from the signal received from the face recognition device 100 by the third communication unit 310.

Further, the score calculation unit 393 acquires the first feature vector (step S221). For example, in the face recognition device 100, the feature vector calculation unit 191 calculates the feature vector (first feature vector) of the face image (first image) to be authenticated, and the first communication unit 110 obtains the first feature vector. The feature vector is transmitted to the estimation device 300. The score calculation unit 393 extracts the first feature vector from the signal received from the face recognition device 100 by the third communication unit 310.
The estimation device 300 may perform the processes of steps S211 to S213 and the processes of steps S221 in parallel, or may sequentially perform the processes.

After step S213 and step S221, the score calculation unit 393 calculates a hostile sample score (step S231). As described above, the method by which the score calculation unit 393 calculates the hostile sample score is not limited to a specific method.
Then, the score calculation unit 393 determines whether or not the face image to be authenticated is a hostile sample (step S232). For example, as described above, the score calculation unit 393 compares the hostile sample score with the determination threshold value to determine whether or not the face image to be authenticated is a hostile sample.

When it is determined that the face image to be authenticated is a hostile sample (step S232: YES), the determination result processing unit 320 executes the hostile sample correspondence process (step S233). As described above, the processing performed by the determination result processing unit 320 can be various processing depending on the usage pattern of the face authentication system 1.
After step S233, the estimation device 300 ends the process of FIG.
On the other hand, when it is determined in step S232 that the face image to be authenticated is not a hostile sample (step S232: NO), the estimation device 300 ends the process of FIG.

As described above, the hostile perturbation acquisition unit 291 acquires a plurality of hostile perturbations. The filter generation unit 296 generates a filter for determining a hostile sample based on a plurality of hostile perturbations acquired by the hostile perturbation acquisition unit 291.
According to the filter generation device 200, a filter for determining a hostile sample can be obtained. Using the hostile sample determination filter, it is possible to estimate the possibility that the face image to be authenticated is a hostile sample.

In addition, the filter generation unit 296 calculates the hostile sample determination filter by averaging a plurality of hostile perturbations.
The filter generation unit 296 can calculate the hostile sample determination filter by a relatively simple process such as averaging the pixel values for each pixel of the hostile perturbation. According to the filter generation device 200, the processing load of the filter generation unit 296 can be lightened in this respect.

In addition, the filter application unit 392 acquires a second image to which a hostile sample determination filter is applied to the first image, which is an image used for face recognition. The score calculation unit 393 calculates a hostile sample score that quantitatively indicates the degree of difference between the feature vector of the first image and the feature vector of the second image.
According to the estimation device 300, it is possible to estimate the possibility that the face image to be authenticated is a hostile sample. Specifically, the hostile sample score indicates that the face image to be authenticated may be a hostile sample. In addition, the score calculation unit 393 can compare the hostile sample score with the determination threshold value to determine whether or not the face image to be authenticated is a hostile sample.

FIG. 7 is a diagram showing an example of the configuration of the filter generation device according to the embodiment. The filter generation device 500 shown in FIG. 7 includes a hostile perturbation acquisition unit 501 and a filter generation unit 502.
With this configuration, the hostile perturbation acquisition unit 501 acquires a plurality of hostile perturbations due to the difference between the face image and the hostile sample based on the face image. The filter generation unit 502 generates a filter based on a plurality of hostile perturbations.
The hostile perturbation acquisition unit 501 corresponds to an example of a hostile perturbation acquisition means. The filter generation unit 502 corresponds to an example of a filter generation means.
According to the filter generation device 500, it is possible to estimate the possibility that the face image to be authenticated is a hostile sample by using this filter.

FIG. 8 is a diagram showing an example of a processing procedure in the filter generation method according to the embodiment. The filter generation method shown in FIG. 8 includes a hostile perturbation acquisition step (step S501) and a filter generation step (step S502).
In the hostile perturbation acquisition step (step S501), a plurality of hostile perturbations due to the difference between the face image and the hostile sample based on the face image are acquired. In the filter generation step (step S502), a filter is generated based on a plurality of hostile perturbations.
According to the filter generation method shown in FIG. 8, it is possible to estimate the possibility that the face image to be authenticated is a hostile sample by using this filter.

FIG. 9 is a schematic block diagram showing an example of a computer configuration according to at least one embodiment.
With the configuration shown in FIG. 9, the computer 700 includes a CPU 710, a main storage device 720, an auxiliary storage device 730, and an interface 740.
Any one or more of the face recognition device 100, the filter generation device 200, the estimation device 300, and the filter generation device 500 may be mounted on the computer 700. In that case, the operation of each of the above-mentioned processing units is stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it to the main storage device 720, and executes the above processing according to the program. Further, the CPU 710 secures a storage area corresponding to each of the above-mentioned storage units in the main storage device 720 according to the program. Communication between each device and other devices is executed by having the interface 740 have a communication function and performing communication according to the control of the CPU 710.

When the face recognition device 100 is mounted on the computer 700, the operations of the first control unit 190 and each unit thereof are stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it to the main storage device 720, and executes the above processing according to the program.
Further, the CPU 710 secures the first storage unit 180 and the storage area corresponding to each unit in the main storage device 720 according to the program. The communication performed by the first communication unit 110 is executed by the interface 740 including a communication device and performing communication according to the control of the CPU 710. The process of acquiring the face image for authentication by the authentication image acquisition unit 120 is executed by, for example, the interface 740 including an image acquisition device such as a camera or a scanner and operating according to the control of the CPU 710. The processing performed by the authentication result processing unit 130 is executed by, for example, the interface 740 including a device such as a display device or a communication device according to the processing mode of the authentication result processing unit 130 and operating according to the control of the CPU 710.

When the filter generation device 200 is mounted on the computer 700, the operations of the second control unit 290 and each unit thereof are stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it to the main storage device 720, and executes the above processing according to the program.
Further, the CPU 710 secures a storage area corresponding to the second storage unit 280 in the main storage device 720 according to the program. The communication performed by the second communication unit 210 is executed by the interface 740 including a communication device and performing communication according to the control of the CPU 710.

When the estimation device 300 is mounted on the computer 700, the operations of the third control unit 390 and each unit thereof are stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it to the main storage device 720, and executes the above processing according to the program.
Further, the CPU 710 secures a storage area corresponding to the third storage unit 380 in the main storage device 720 according to the program. The communication performed by the third communication unit 310 is executed when the interface 740 includes a communication device and performs communication according to the control of the CPU 710. The processing performed by the determination result processing unit 320 is executed by, for example, the interface 740 including a device such as a display device or a communication device according to the processing mode of the determination result processing unit 320 and operating according to the control of the CPU 710.

When the filter generation device 500 is mounted on the computer 700, the operations of the hostile perturbation acquisition unit 501 and the filter generation unit 502 are stored in the auxiliary storage device 730 in the form of a program. The CPU 710 reads the program from the auxiliary storage device 730, expands it to the main storage device 720, and executes the above processing according to the program.
Further, the CPU 710 secures a storage area required for processing of the filter generation device 500 in the main storage device 720 according to the program. The input / output process performed by the filter generation device 500 is executed when the interface 740 is provided with a device such as a communication device according to the mode of processing and operates according to the control of the CPU 710.

A computer-readable recording medium is used to record a program for realizing all or part of the functions of the face recognition device 100, the filter generation device 200, the estimation device 300, and the filter generation device 500, and this recording medium. The processing of each part may be performed by loading the program recorded in the above into a computer system and executing the program. The term "computer system" as used herein includes hardware such as an OS (operating system) and peripheral devices.
"Computer readable recording medium" includes flexible disks, optomagnetic disks, portable media such as ROM (Read Only Memory) and CD-ROM (Compact Disc Read Only Memory), and hard disks built into computer systems. A storage device. Further, the above-mentioned program may be a program for realizing a part of the above-mentioned functions, and may be a program for realizing the above-mentioned functions in combination with a program already recorded in the computer system.

As described above, the embodiment of the present invention has been described in detail with reference to the drawings, but the specific configuration is not limited to this embodiment, and design changes and the like within a range not deviating from the gist of the present invention are also included.

The embodiment of the present invention may be applied to a filter generation device, an estimation device, a face recognition system, a filter generation method, and a recording medium.

1 Face recognition system 100 Face recognition device 110 First communication unit 120 Authentication image acquisition unit 180 First storage unit 181 Authentication database unit 190 First control unit 191 Feature vector calculation unit 192 Classification unit 200, 500 Filter generation device 210 Second Communication unit 280 Second storage unit 290 Second control unit 291, 501 Hostile perturbation acquisition unit 292 Filter generation image acquisition unit 293 Noise addition unit 294 Hostile sample selection unit 295 Acquisition processing unit 296, 502 Filter generation unit 300 Estimator 310 Third communication unit 320 Judgment result processing unit 380 Third storage unit 390 Third control unit 391 First image acquisition unit 392 Filter application unit 393 Score calculation unit 394 Judgment unit

Claims (6)

A hostile perturbation acquisition means for acquiring a plurality of hostile perturbations due to a difference between a face image and a hostile sample based on the face image,
A filter generation means that generates a filter based on a plurality of the hostile perturbations,
A filter generator comprising. The filter generating means calculates the filter by averaging a plurality of the hostile perturbations.
The filter generator according to claim 1. A filter application means for acquiring a second image obtained by applying the filter according to claim 1 or 2 to a first image which is an image used for face recognition.
A score calculation means for calculating a score that quantitatively indicates the degree of difference between the feature vector of the first image and the feature vector of the second image.
Estimator equipped with. A face recognition device that performs face recognition and
The estimation device according to claim 3 and
Face recognition system with. A process of acquiring a plurality of hostile perturbations due to a difference between a face image and a hostile sample based on the face image, and
A process of generating a filter based on a plurality of the hostile perturbations, and
Filter generation method including. On the computer
A process of acquiring a plurality of hostile perturbations due to a difference between a face image and a hostile sample based on the face image, and
A process of generating a filter based on a plurality of the hostile perturbations, and
A recording medium that records a program for executing.

Images