WO2024261856A1 - Information processing device, information processing method, and recording medium - Google Patents

Abstract

An information processing device 1 comprises: a reception unit 11 that receives input of a first image and a second image; a synthesis unit 12 that synthesizes a third image on the basis of the first image and the second image; a difference emphasis unit 13 that emphasizes a difference between the second image and the third image; a calculation unit 14 that calculates an index representing the likeness of the second image to a synthesized image on the basis of the emphasized difference; and a determination unit 15 that determines whether the second image is a synthesized image according to the index.

Description

Information processing device, information processing method, and recording medium

This disclosure relates to the technical fields of information processing devices, information processing methods, and recording media.

Patent Document 1 describes a technology in which a class indicates whether a person's face (object) is real (true) or fake (false), and a synthesized area (i.e., an area added to an original video) is detected as an element of the detection target class to detect whether the video is a fake video (e.g., a false video generated by a synthesis process, etc.).

International Publication No. 2022/054246

The present disclosure aims to provide an information processing device, an information processing method, and a recording medium that are intended to accurately detect whether an input image is a composite image.

One aspect of the information processing device includes a receiving means for receiving an input of a first image and a second image, a synthesis means for synthesizing a third image based on the first image and the second image, a difference emphasis means for emphasizing the difference between the second image and the third image, a calculation means for calculating an index representing the likelihood that the second image is a synthesized image based on the emphasized difference, and a determination means for determining whether the second image is a synthesized image or not according to the index.

One aspect of the information processing method is to receive an input of a first image and a second image, synthesize a third image based on the first image and the second image, emphasize the difference between the second image and the third image, calculate an index representing the likelihood of the second image being a synthesized image based on the emphasized difference, and determine whether the second image is a synthesized image or not based on the index.

In one embodiment of the recording medium, a computer program is recorded to cause a computer to execute an information processing method that accepts input of a first image and a second image, synthesizes a third image based on the first image and the second image, emphasizes the difference between the second image and the third image, calculates an index representing the likelihood that the second image is a synthesized image based on the emphasized difference, and determines whether the second image is a synthesized image according to the index.

The information processing device, information processing method, and recording medium disclosed herein can accurately detect whether an input image is a composite image.

1 is a block diagram showing a configuration of a first information processing device according to the present disclosure. FIG. 2 is a block diagram showing a configuration of a second information processing device according to the present disclosure. 10 is a flowchart showing a processing operation of a second information processing device according to the present disclosure. FIG. 13 is a block diagram showing a configuration of a third information processing device according to the present disclosure. 13 is a flowchart showing a processing operation of a third information processing device according to the present disclosure. FIG. 13 is a block diagram showing a configuration of a fourth information processing device according to the present disclosure. 13 is a flowchart showing a processing operation of a fourth information processing device according to the present disclosure.

Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings.
[1: First embodiment]

A first embodiment of an information processing device, an information processing method, and a recording medium will be described. Hereinafter, a first embodiment of an information processing device, an information processing method, and a recording medium will be described using a first information processing device 1 according to the present disclosure.
[1-1: Configuration of information processing device 1]

FIG. 1 is a block diagram showing a configuration of a first information processing device 1 according to the present disclosure. As shown in FIG. 1, the information processing device 1 includes a receiving unit 11, a synthesis unit 12, a difference emphasizing unit 13, a calculation unit 14, and a determination unit 15. The receiving unit 11 receives input of a first image and a second image. The synthesis unit 12 synthesizes a third image based on the first image and the second image. The difference emphasizing unit 13 emphasizes the difference between the second image and the third image. The calculation unit 14 calculates an index representing the likelihood of the second image being a synthesized image based on the emphasized difference. The determination unit 15 determines whether the second image is a synthesized image according to the index.
[1-2: Technical Effects of Information Processing Device 1]

The first information processing device 1 according to the present disclosure makes a judgment based on the difference between an input image and a composite image, and can therefore accurately detect whether or not the input image is a composite image.
[2: Second embodiment]

A second embodiment of an information processing device, an information processing method, and a recording medium will be described. Hereinafter, a second embodiment of an information processing device, an information processing method, and a recording medium will be described using a second information processing device 2 according to the present disclosure.
[2-1: Fake image]

There is a technology that synthesizes an image of a person based on information from a single photograph of the person's face. For example, deepfake is known as a technology for synthesizing an image of a person. Deepfake is known as a technology for synthesizing a fake image that shows something that does not actually happen. Hereinafter, an image that shows something that does not actually happen may be called a fake image. A synthesized image may also be called a fake image. An image that shows something that actually happened may also be called a real image.
[2-2: Configuration of information processing device 2]

FIG. 2 is a block diagram showing the configuration of the second information processing device 2. As shown in FIG. 2, the information processing device 2 includes a calculation device 21 and a storage device 22. Furthermore, the information processing device 2 may include a communication device 23, an input device 24, and an output device 25. However, the information processing device 2 does not have to include at least one of the communication device 23, the input device 24, and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

The arithmetic device 21 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable and non-transient recording medium using a recording medium reading device (e.g., an input device 24 described later) not shown in the figure that is provided in the information processing device 2. The arithmetic device 21 may acquire (i.e., download or read) a computer program from a device (not shown) located outside the information processing device 2 via the communication device 23 (or other communication device). The arithmetic device 21 executes the read computer program. As a result, a logical functional block for executing the operation to be performed by the information processing device 2 is realized within the calculation device 21. In other words, the calculation device 21 can function as a controller for realizing a logical functional block for executing the operation (in other words, processing) to be performed by the information processing device 2.

2 shows an example of a logical functional block realized in the arithmetic device 21 to execute an information processing operation. As shown in FIG. 2, the arithmetic device 21 realizes a reception unit 211, which is a specific example of the "reception means" described in the appendix described later, a synthesis unit 212, which is a specific example of the "synthesizing means" described in the appendix described later, a difference emphasis unit, which is a specific example of the "difference emphasis means" described in the appendix described later, a calculation unit 214, which is a specific example of the "calculation means" described in the appendix described later, a judgment unit 215, which is a specific example of the "judgment means" described in the appendix described later, and an output unit 216. However, the output unit 216 does not have to be realized in the arithmetic device 21. The difference emphasis unit may have an extraction unit 2131 and an emphasis unit 2132. Details of the operations of the reception unit 211, synthesis unit 212, difference emphasis unit, calculation unit 214, judgment unit 215, and output unit 216 will be described later with reference to FIG. 3.

The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that is temporarily used by the arithmetic device 21 when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the information processing device 2 stores for a long period of time. The storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 22 may include a non-temporary recording medium.

The communication device 23 is capable of communicating with devices external to the information processing device 2 via a communication network (not shown). The communication device 23 may be a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), and USB (Universal Serial Bus).

The input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operating device (e.g., at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be attached externally to the information processing device 2.

The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (so-called a display) capable of displaying an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (so-called a speaker) capable of outputting sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (so-called a printer) capable of printing desired information on paper.
[2-3: Information Processing Operation Performed by Information Processing Device 2]

The information processing operation performed by the information processing device 2 will be described with reference to FIG. 3. FIG. 3 is a flowchart showing the flow of the information processing operation performed by the information processing device 2. Note that this disclosure assumes that the first image is not a fake image but a real image.

As shown in FIG. 3, the reception unit 211 receives input of a first image (step S20). The reception unit 211 may receive input of a face image including a person's face area as the first image. The first image may be a still image. Hereinafter, the "first image" may be referred to as the "source image."

The reception unit 211 receives input of a second image (step S21). The reception unit 211 may receive input of a face image including a person's facial area as the second image. The second image may be a still image. The second image may be a moving image. In the second embodiment, a case where the second image that is a still image is processed will be described. Hereinafter, the "second image" may be referred to as the "image to be determined."

The synthesis unit 212 synthesizes a third image based on the first image and the second image. Hereinafter, the "third image" may be referred to as a "synthetic image". The synthesis unit 212 may generate a synthetic image using, for example, a technique called face swap. Face swap is a technique for exchanging a face area of a source image with a face area of a target image. The synthesis unit 212 may generate a synthetic image by fitting a face area of the judgment target image into a face area of the source image. The synthesis unit 212 may generate a synthetic image having the characteristics of the source image. For example, the synthesis unit 212 may generate a synthetic image that maintains the facial expression of the source image.

The extraction unit 2131 extracts the difference between the image to be determined and the composite image (step S23). In other words, the extraction unit 2131 extracts the portion where the image to be determined and the composite image differ. The extraction unit 2131 may obtain the difference in pixel value for each pixel, and generate a difference image in which each pixel is represented by the difference in pixel value.

The highlighting unit 2132 highlights the difference (step S24). The highlighting unit 2132 may, for example, highlight each pixel value of the difference image by multiplying it by a real number to highlight the difference. That is, the highlighting unit 2132 may increase the pixel value of a pixel in the difference image whose pixel value is not 0 (that is, there is no difference in pixel value between the image to be determined and the composite image). Hereinafter, the difference image in which the difference is highlighted by the highlighting unit 2132 may be referred to as a "difference highlighted image". The highlighting unit 2132 may, for example, generate a difference highlighted image by setting the pixel value of a pixel in the difference image whose pixel value is not 0 to the maximum value that it can take. The highlighting unit 2132 may, for example, generate a difference highlighted image by setting the pixel value of a pixel in the difference image whose pixel value is equal to or greater than a predetermined value to the maximum value that it can take. The highlighting unit 2132 may, for example, generate a difference highlighted image that can take four types of pixel values, for example, large difference, medium difference, small difference, and no difference, by setting a range of pixel values in the difference image. The highlighting unit 2132 may employ any method to highlight the difference.

The calculation unit 214 calculates an index representing the likelihood that the image to be determined is a fake image based on the emphasized difference (step S25). The index calculated by the calculation unit 214 may be a real number. The calculation unit 214 may use a calculation model to calculate an index representing the likelihood that the image to be determined is a fake image. The calculation model is a model that outputs an index representing the likelihood that the image is a fake image when the emphasized difference is input. The calculation model may be a machine-learned model. The learning mechanism that trains the calculation model may train the calculation model using a difference-emphasized image having information indicating the correct answer (the image to be determined is a real image, or the image to be determined is a fake image) as teacher data. The learning mechanism may train the calculation model on a method for calculating the index using the information indicating the correct answer and the index representing the likelihood that the image is a fake image output by the calculation model.

The determination unit 215 determines whether the second image is a composite image or not based on the index (step S26). The determination unit 215 may determine whether the second image is a composite image or not by comparing the index with a predetermined threshold value.

If the index exceeds the predetermined threshold (step S26: Yes), the determination unit 215 determines that the second image is a composite image (step S27). If the index does not exceed the predetermined threshold (step S26: No), the determination unit 215 determines that the second image is a non-composite image (step S28).

The output unit 216 outputs according to the determination result (step S29). The output unit 216 may control the output device 25 to cause the output device 25 to output according to the determination result.
[2-4: Technical Effects of Information Processing Device 2]

When an emphasized difference is used for the comparison judgment between the judgment target image and the composite image, the difference between the judgment target image being a fake image and not being a fake image is easy to understand, and it is easy to judge whether the judgment target image is a fake image or not. The second information processing device 2 of the present disclosure emphasizes the difference between the input judgment target image and the composite image and judges based on the emphasized difference, so that it is possible to accurately detect whether the input judgment target image is a fake image or not. In addition, since the information processing device 2 judges whether the image is a fake image or not by comparing with a threshold value, it is possible to adjust how much an image that seems like a fake image is judged to be a fake image by setting the threshold value.
[3: Third embodiment]

A third embodiment of an information processing device, an information processing method, and a recording medium will be described below. In the following, a third embodiment of an information processing device, an information processing method, and a recording medium will be described using a third information processing device 3 according to the present disclosure. In the third embodiment, a case where the second image is a moving image including a plurality of frames will be described. Hereinafter, the "second image" may be referred to as a "video to be determined". Also, a moving image showing an event that has not actually occurred may be referred to as a fake video. Also, a moving image showing an event that has actually occurred may be referred to as a real video.
[3-1: Fake video]

For example, a genuine video may include a video showing an action performed by person B in front of the camera as captured by the camera. In contrast, a fake video may include a moving image synthesized to make it appear as if person A, a different person from person B, performed the action performed by person B in front of the camera as captured by the camera.

There is a technology called reenactment that creates fake videos in which the facial expression of a person in an original image changes to a desired expression or faces a desired direction. For example, a technology is known that uses at least one facial image of person A and changes the facial expression of person A in the facial image to match the facial expression of person B, creating a moving image that makes it appear as if person A is changing his or her facial expression (hereinafter sometimes referred to as "animating still images").

By using the original image and the original video, a still image can be animated. The original video may be, for example, a video image showing the actions of person B in front of the camera as captured by the camera. The original image may also be a still image showing person A, who is different from person B. When animating a still image, first, landmarks are detected in the original image. Landmarks are also detected from each of the video frames that make up the original video. Next, for each video frame that makes up the original video, the original image is edited so that the landmarks in the original image and the landmarks in the corresponding video frame are matched to generate a composite frame. The generated composite frames are then joined together to generate an animated still image. A landmark may be a characteristic position of a subject that appears in the image.

For example, the facial direction, expression, etc. of person A in the original image can be changed using the facial landmarks of person B in the original video, and a video can be synthesized in which the facial direction, expression, etc. of person A changes. The landmarks that change the facial direction, expression, etc. of a person may be characteristic parts of the face. The characteristic positions on the face may be specific points on parts of the face such as the eyes, nose, mouth, etc.

If the acquired moving image is similar to a composite moving image synthesized using still images, the acquired moving image is likely to be a fake moving image. In this embodiment, this property is used to determine whether or not the moving image is a fake moving image. That is, in the third embodiment, a composite moving image is generated using still images, and the acquired moving image is compared with the composite moving image to determine whether or not the acquired moving image is a fake moving image.
[3-2: Configuration of information processing device 3]

The configuration of the third information processing device 3 will be described with reference to FIG. 4. FIG. 4 is a block diagram showing the configuration of the third information processing device 3.

As shown in FIG. 4, the third information processing device 3 includes a calculation device 21 and a storage device 22, similar to the second information processing device 2. Furthermore, the third information processing device 3 may include a communication device 23, an input device 24, and an output device 25, similar to the second information processing device 2. However, the information processing device 3 may not include at least one of the communication device 23, the input device 24, and the output device 25. The third information processing device 3 differs from the second information processing device 2 in that the synthesis unit 312 includes a detection unit 3121. Other features of the information processing device 3 may be the same as other features of the information processing device 2. For this reason, hereinafter, the parts that are different from the embodiments already described will be described in detail, and other overlapping parts will be appropriately omitted.
[3-3: Information Processing Operation Performed by Information Processing Device 3]

The flow of information processing operations performed by the information processing device 3 will be described with reference to FIG. 5. FIG. 5 is a flowchart showing the flow of information processing operations performed by the information processing device 3.

As shown in FIG. 5, the reception unit 311 receives input of a source image as a first image (step S20). The detection unit 3121 detects landmarks from the source image. The detection unit 3121 may detect characteristic positions in the face area as landmarks from a still image. The detection unit 3121 may detect specific points of parts of the body such as the eyes, nose, and mouth as landmarks from the source image.

The reception unit 311 receives input of the video to be judged as the second image (step S30). The detection unit 3121 detects landmarks from each of the one or more frames included in the video to be judged (step S31). The one or more frames included in the video to be judged may be all frames included in the video to be judged. The one or more frames included in the video to be judged may be any one or more frames included in the moving image. The detection unit 3121 may detect, as landmarks, positions from the image to be judged that are equivalent to the landmarks detected from the source image.

The synthesis unit 312 synthesizes a third image based on the source image, the landmarks of the source image, and the landmarks of one or more frames included in the video to be determined (step S32). In the third embodiment, the third image is a synthetic video including one or more frames. Hereinafter, the "third image" may be referred to as a "synthetic video."

The synthesis unit 312 may first generate a synthesis frame for each input frame constituting the judgment target moving image by editing the landmarks of the source image to match the landmarks of the corresponding input frame. Next, the synthesis unit 312 may connect each synthesis frame together to animate the source images, which are still images, and generate a synthesis moving image.

The extraction unit 3131 extracts the difference between the determination target moving image and the composite moving image (step S33). The extraction unit 3131 extracts the difference between a frame included in the determination target moving image and a frame included in the composite moving image corresponding to the frame. The extraction unit 3131 may extract the difference between the determination target moving image and the composite moving image to generate a difference moving image. When the determination target moving image d _i includes frames 1 to F, the determination target moving image d _i may be expressed as [x _i ¹ , ..., x _i ^F ]. When the composite moving image d _f includes frames 1 to F, the composite moving image d _f may be expressed as [x _f ¹ , ..., x _i ^F ]. In this case, the difference moving image d _diff may be expressed as d _f - _{d i} = [|x _f ¹ - _{x i} ¹ |, ..., |x _f ^F - x _i ^F |]. In step S31, if the detection unit 3121 detects a landmark from any one or more frames included in the video to be judged, the extraction unit 3131 may generate a difference video including difference frames corresponding to the any one or more frames.

The emphasis unit 3132 emphasizes the difference (step S34). The emphasis unit 3132 may generate a difference emphasized moving image including a difference emphasized frame in which a difference between a frame included in the determination target moving image and a frame included in the composite moving image corresponding to the frame is emphasized. The difference emphasized moving image d _diff generated by the emphasis unit 3132 may be expressed as [α|x _f ¹ -x _i ¹ |, ..., α|x _f ^F -x _i ^F |], where α is a real number and a parameter for further emphasizing the difference.

The calculation unit 314 calculates an index representing the likelihood that the video to be judged is a fake video based on one or more frames included in the difference-emphasized video (step S35). The calculation unit 314 may use a calculation model to calculate an index representing the likelihood that the image to be judged is a fake image. In the third embodiment, when one or more frames included in the difference-emphasized video are input, the calculation model outputs an index representing the likelihood that the image is a fake image. The one or more frames included in the difference-emphasized video may be all frames included in the difference-emphasized video. The one or more frames included in the difference-emphasized video may be any one or more frames included in the moving image.

The determination unit 315 determines whether the video to be determined is a fake video or not based on the index (step S36). The determination unit 315 may determine whether the video to be determined is a fake video or not by comparing the index with a predetermined threshold value.

If the index exceeds a predetermined threshold (step S36: Yes), the determination unit 315 determines that the video to be determined is a fake video (step S37). If the index does not exceed the predetermined threshold (step S36: No), the determination unit 315 determines that the video to be determined is not a fake video (step S38). The output unit 316 outputs according to the determination result (step S39).
[3-4: Technical Effects of Information Processing Device 3]

A synthetic video generated using still images and landmarks can capture the characteristics of a fake video generated using a technology such as deep fake. The third information processing device 3 of the present disclosure uses the property that if the characteristics of a synthetic video generated using a source image, which is an input still image, and the input video to be judged are similar, the video to be judged is likely to be a fake video. The third information processing device 3 of the present disclosure can accurately judge whether the video to be judged is a genuine video that is not forged or a fake video that is forged. In addition, the information processing device 3 can accurately detect whether the input video to be judged is a fake video or not based on the difference between each frame. In addition, the information processing device 3 can accurately detect deep fakes generated using landmarks.
[4: Fourth embodiment]

A fourth embodiment of an information processing device, an information processing method, and a recording medium will be described. Hereinafter, a fourth embodiment of an information processing device, an information processing method, and a recording medium will be described using a fourth information processing device 4 according to the present disclosure.
[4-1: Configuration of information processing device 4]

The configuration of the fourth information processing device 4 will be described with reference to FIG. 6. FIG. 6 is a block diagram showing the configuration of the fourth information processing device 4.

6, the fourth information processing device 4 includes a calculation device 21 and a storage device 22, similar to the second information processing device 2 and the third information processing device 3. Furthermore, the fourth information processing device 4 may include a communication device 23, an input device 24, and an output device 25, similar to the second information processing device 2 and the third information processing device 3. However, the information processing device 4 may not include at least one of the communication device 23, the input device 24, and the output device 25. The fourth information processing device 4 differs from the second information processing device 2 and the third information processing device 3 in that a matching unit 417, a spoofing determination unit 418, and an authentication unit 419 are further realized in the calculation device 21. Other features of the information processing device 4 may be the same as other features of at least one of the information processing device 2 and the information processing device 3. Therefore, hereinafter, the parts that are different from the embodiments already described will be described in detail, and other overlapping parts will be omitted as appropriate.

The fourth information processing device 4 is a mechanism capable of performing biometric authentication of a person. The information processing device 4 may be a mechanism capable of performing a matching operation using an image, and determining whether or not a person is impersonating another person using the image, thereby authenticating the person.

The fourth information processing device 4 of the present disclosure may be applied to online identity verification such as electronic know your customer (eKYC). As described above, there is a technology that synthesizes an image of a person based on information of a single face photograph of the person, which poses a threat of impersonation in eKYC. Accurate determination of whether or not a video is fake is an important issue in increasing the reliability of services such as eKYC. The input to eKYC includes a face image of an official document that serves as information for synthesizing the fake video. In other words, as an impersonation method for eKYC, a method of synthesizing and inputting a fake video based on limited information such as a face image of an official document such as a driver's license or a My Number card can be considered.
[4-2: Information Processing Operation Performed by Information Processing Device 4]

With reference to FIG. 7, the flow of information processing operations performed by the information processing device 4 will be described. FIG. 7 is a flowchart showing the flow of information processing operations performed by the information processing device 4. Note that in the fourth embodiment, a case will also be described in which the second image is a moving image including a plurality of frames, and the second image will be referred to as a determination target moving image.

As shown in FIG. 7, the reception unit 311 receives an input of a source image as a first image (step S20). The reception unit 311 may receive an input of a facial photograph on an identification document such as a driver's license or a My Number card as the source image. The reception unit 311 receives an input of a video to be judged as a second image (step S30).

The matching unit 417 matches the facial image of the person (step S40). If the first image is a facial image on an official document such as a driver's license or a My Number card, the matching unit 417 may match the person appearing in the first image with the person appearing in the video to be judged. In this case, if the matching between the person appearing in the first image and the person appearing in the video to be judged fails, the information processing operation may be terminated. Alternatively, the matching unit 417 may match the received first image with a registered facial image that has been registered in advance. Alternatively, the matching unit 417 may match the received video to be judged with a registered facial image that has been registered in advance. In other words, the matching unit 417 may match at least one of the person appearing in the first image and the person appearing in the video to be judged.

In addition, since the source image as the first image and the fake video synthesized based on the source image are similar, even if the video to be judged is a fake video, there is a high possibility that the first image will be successfully matched with the video to be judged.

The spoofing determination unit 418 performs spoofing determination using the video to be determined (step S41). In the fourth embodiment, the video to be determined may be used for spoofing determination along with determining whether it is a fake video or not. For example, the video to be determined may be a video showing an action performed by a person in response to an instruction from the information processing device 4. The information processing device 4 may instruct the face direction, gaze direction, and face position. The information processing device 4 may guide the gaze. The information processing device 4 may instruct a gesture. The spoofing determination unit 418 may perform active liveness determination using the video to be determined.

The detection unit 3121 detects landmarks from each of one or more frames included in the video to be judged (step S31). The synthesis unit 312 generates a composite video based on the source image and the landmarks from each of one or more frames included in the video to be judged (step S32). The synthesis unit 312 generates a composite image based on a facial photograph as the source image.

The extraction unit 3131 extracts the difference between a frame included in the judgment target video and a frame included in the composite video corresponding to that frame (step S33). The emphasis unit 3132 emphasizes the difference (step S34).

The calculation unit 314 calculates an index representing the likelihood that the video to be judged is a fake video based on the video difference (step S35). The determination unit 315 determines whether the video to be judged is a fake video or not based on the index (step S36). The determination unit 315 may determine whether the video to be judged is a fake video or not by comparing the index with a predetermined threshold value.

If the index exceeds the predetermined threshold (step S36: Yes), the determination unit 315 determines that the video to be determined is a fake video (step S37). If the index does not exceed the predetermined threshold (step S36: No), the determination unit 315 determines that the video to be determined is not a fake video (step S38).

When the determination unit 315 determines that the video to be determined is not a fake video, the authentication unit 419 authenticates the person based on the collation result by the collation unit 417 and the determination result by the masquerade determination unit 418 (step S42). The authentication unit 419 may also authenticate the person on the condition that the determination unit 315 determines that the video to be determined is less likely to be a fake image than a predetermined standard and the masquerade determination unit 418 determines that the person has acted in accordance with the instructions. The case where the authentication unit 419 has successfully authenticated the person may be the case where the person's identity has been confirmed. The output unit 416 outputs the authentication result of the person (step S43).
[4-3: Technical Effects of Information Processing Device 4]

The fourth information processing device 4 of the present disclosure can accurately detect whether an input video to be judged is a fake video or not, and can therefore perform identity verification with high accuracy.
[5: Supplementary Note]

The following supplementary notes are further disclosed regarding the above-described embodiment.
[Appendix 1]
A receiving means for receiving an input of a first image and a second image;
a synthesizing means for synthesizing a third image based on the first image and the second image;
a difference enhancing means for enhancing a difference between the second image and the third image;
a calculation means for calculating an index representing a likelihood that the second image is a synthesized image based on the emphasized difference;
and determining whether the second image is a synthesized image or not, based on the index.
[Appendix 2]
The difference emphasis means is
an extraction means for extracting a difference between the second image and the third image;
and highlighting means for highlighting the difference.
[Appendix 3]
the second image is a video including a plurality of frames;
The synthesizing means synthesizes the third image including one or more frames;
The information processing device according to claim 1 or 2, wherein the difference emphasis means emphasizes a difference between a frame included in the second image and a frame included in the third image corresponding to the frame.
[Appendix 4]
the difference emphasizing means generates a difference-emphasized video including a difference frame in which a difference between a frame included in the second image and a frame included in the third image corresponding to the frame is emphasized;
The information processing device according to claim 3, wherein the calculation means calculates an index representing a likelihood that the second image is a synthesized image based on the difference-emphasized moving image.
[Appendix 5]
The synthesis means includes a detection means for detecting a landmark from the second image,
The information processing device according to claim 3, further comprising: a first image processing unit that processes the first image and the landmarks to generate a third image.
[Appendix 6]
The information processing device according to claim 1 or 2, wherein the determining means determines whether the second image is a synthesized image by comparing the index with a predetermined threshold value.
[Appendix 7]
A matching means for matching at least one of an object appearing in the first image and an object appearing in the second image;
and an authentication unit that authenticates the target based on at least one of a determination result by the determination unit and a matching result by the matching unit.
[Appendix 8]
Accepting input of a first image and a second image;
synthesizing a third image based on the first image and the second image;
highlighting a difference between the second image and the third image;
calculating an index representing a likelihood of the second image being a synthesized image based on the emphasized difference;
determining whether the second image is a synthesized image or not according to the indicator.
[Appendix 9]
On the computer,
Accepting input of a first image and a second image;
synthesizing a third image based on the first image and the second image;
highlighting a difference between the second image and the third image;
calculating an index representing a likelihood of the second image being a synthesized image based on the emphasized difference;
determining whether the second image is a synthesized image or not according to the index.

The present disclosure has been described above with reference to the embodiments, but the present disclosure is not limited to the above-mentioned embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

1, 2, 3, 4 Information processing device 11, 211, 311 Reception unit 12, 212, 312 Synthesis unit 13, 213, 313 Difference emphasis unit 2131, 3131 Extraction unit 2132, 3132 Emphasis unit 214, 314 Calculation unit 215, 315 Determination unit 216, 316, 416 Output unit 3121 Detection unit 417 Collation unit 418 Impersonation determination unit 419 Authentication unit

Claims (9)

A receiving means for receiving an input of a first image and a second image;
a synthesizing means for synthesizing a third image based on the first image and the second image;
a difference enhancing means for enhancing a difference between the second image and the third image;
a calculation means for calculating an index representing a likelihood that the second image is a synthesized image based on the emphasized difference;
and determining whether the second image is a synthesized image or not, based on the index. The difference emphasis means is
an extraction means for extracting a difference between the second image and the third image;
The information processing apparatus according to claim 1 , further comprising: emphasis means for emphasizing the difference. the second image is a video including a plurality of frames;
The synthesizing means synthesizes the third image including one or more frames;
The information processing apparatus according to claim 1 , wherein the difference emphasis means emphasizes a difference between a frame included in the second image and a frame included in the third image corresponding to the frame. the difference emphasizing means generates a difference-emphasized video including a difference frame in which a difference between a frame included in the second image and a frame included in the third image corresponding to the frame is emphasized;
The information processing apparatus according to claim 3 , wherein the calculation means calculates an index representing a likelihood that the second image is a synthesized image, based on the difference-emphasized moving image. The synthesis means includes a detection means for detecting a landmark from the second image,
The information processing apparatus according to claim 3 , wherein the third image is synthesized based on the first image and the landmarks. The information processing apparatus according to claim 1 , wherein the determining means determines whether or not the second image is a synthesized image by comparing the index with a predetermined threshold value. A matching means for matching at least one of an object appearing in the first image and an object appearing in the second image;
The information processing apparatus according to claim 1 , further comprising: an authentication unit that authenticates the target based on at least one of a determination result by the determination unit and a comparison result by the comparison unit. Accepting input of a first image and a second image;
synthesizing a third image based on the first image and the second image;
highlighting a difference between the second image and the third image;
calculating an index representing a likelihood of the second image being a synthesized image based on the emphasized difference;
determining whether the second image is a synthesized image or not according to the indicator. On the computer,
Accepting input of a first image and a second image;
synthesizing a third image based on the first image and the second image;
highlighting a difference between the second image and the third image;
calculating an index representing a likelihood of the second image being a synthesized image based on the emphasized difference;
determining whether the second image is a synthesized image or not according to the index.