WO2024195055A1 - Information processing device, information processing method, and recording medium - Google Patents

Abstract

In the present invention, an information processing device 1 comprises: a learning data generation unit 11 that generates second learning data on the basis of first learning data including an eye image in which the eye of a subject appears and information related to an article mounted on the subject appearing in the eye image included in the first learning data; a model generation unit 12 that generates an authentication model using the second learning data; and an authentication unit 13 that authenticates the subject using the authentication model.

Description

Information processing device, information processing method, and recording medium

This disclosure relates to the technical fields of information processing devices, information processing methods, and recording media.

Patent Document 1 describes a technology in which a target iris image, which is the iris image to be processed, is obtained, and one or more similar registered iris images that are similar to the target iris image are searched for from a registered iris image, and if the person corresponding to the target iris image and the person corresponding to the similar registered iris image are different people, the target iris image and the similar registered iris image are determined to be iris images of color contact lenses.
Patent document 2 describes a technology that acquires an image of a subject's eye, compares the image of the eye with a reference image, identifies the coloring pattern of the color contact lenses worn by the subject, and identifies the subject using features of the iris region of the eye other than the coloring region of the coloring pattern.
Patent document 3 describes a technology in which the edge of the iris is detected in an image, texture is obtained from the image, and the edge and texture are combined to generate inner and outer boundaries of the iris, a method is provided for selecting between an ellipse model and a circle model to improve the accuracy of iris boundary detection, and a mask image is obtained using a dome model, and occlusion by the eyelid in the unwrapped image is removed, and the iris is extracted from the image.
Patent Literature 4 describes a technique for inputting a captured image into a first learning model and determining whether iris information can be extracted based on the output from the first learning model. The first learning model is constructed by machine learning using training data that associates a plurality of eye images with labels indicating whether the iris can be extracted.
Patent document 5 describes a technology in which different iris images corresponding to the same person are extracted as cosmetic lens candidates, the iris features of the cosmetic lens candidate are compared with the iris features of other cosmetic lens candidates, a reliability indicating the likelihood of the lens being a cosmetic lens is calculated based on the comparison results, and if the calculated reliability is equal to or greater than a predetermined threshold, the cosmetic lens candidate is determined to be a cosmetic lens.
Non-Patent Document 1 describes a technique for dividing an image into patches, calculating weights based on the degree of occlusion of each patch, and applying weights to areas with less occlusion to estimate facial expressions.

International Publication No. 2022/014001 International Publication No. 2020/065935 Special table 2009-523266 publication International Publication No. 2021/192311 International Publication No. 2021/245932

Y. Li, et al. Occlusion Aware Facial Expression Recognition Using CNN With Attention Mechanism, IEEE TRANSACTIONS ON IMAGE PROCESSING, VOL. 28, NO. 5, MAY 2019

The objective of this disclosure is to provide an information processing device, an information processing method, and a recording medium that aim to improve upon the technology described in prior art documents.

One aspect of the information processing device includes a learning data generation means for generating second learning data based on first learning data including an eye image showing the eyes of a subject and information about the subject's accessories shown in the eye image included in the first learning data, a model generation means for generating an authentication model using the second learning data, and an authentication means for authenticating the subject using the authentication model.

One aspect of the information processing method is to generate second learning data based on first learning data including an eye image showing the subject's eyes and information about the subject's accessories that are shown in the eye image included in the first learning data, generate an authentication model using the second learning data, and authenticate the subject using the authentication model.

In one embodiment of the recording medium, a computer program is recorded to cause a computer to execute an information processing method that generates second learning data based on first learning data including an eye image showing the eyes of a subject and information about the subject's accessories that are shown in the eye image included in the first learning data, generates an authentication model using the second learning data, and authenticates the subject using the authentication model.

FIG. 1 is a block diagram showing the configuration of an information processing apparatus according to the first embodiment. FIG. 2 is a block diagram showing the configuration of an information processing device according to the second embodiment. FIG. 3 is a conceptual diagram showing a flow of information processing operations of the information processing device in the second embodiment. FIG. 4 is a block diagram showing the configuration of an information processing apparatus according to the third embodiment. FIG. 5 is a conceptual diagram showing a flow of information processing operations of the information processing device in the third embodiment. FIG. 6 is a block diagram showing the configuration of an information processing device according to the fourth embodiment. FIG. 7 is a conceptual diagram showing a flow of information processing operations of the information processing device in the fourth embodiment. FIG. 8 is a block diagram showing the configuration of an information processing apparatus according to the fifth embodiment. FIG. 9 is a conceptual diagram showing the flow of information processing operations of the information processing device in the fifth embodiment. FIG. 10 is a block diagram showing the configuration of an information processing device according to the sixth embodiment. FIG. 11 is a conceptual diagram showing the flow of information processing operations of the information processing device in the sixth embodiment. FIG. 12 is a block diagram showing the configuration of an information processing device according to the seventh embodiment. FIG. 13 is a conceptual diagram showing the flow of information processing operations of the information processing device in the seventh embodiment. FIG. 14 is a block diagram showing the configuration of an information processing device according to the eighth embodiment. FIG. 15 is a block diagram showing the configuration of an information processing device according to the ninth embodiment. FIG. 16 is a conceptual diagram showing the flow of information processing operations of the information processing device in the ninth embodiment.

Hereinafter, embodiments of an information processing device, an information processing method, and a recording medium will be described with reference to the drawings.
[1: First embodiment]

A first embodiment of an information processing device, an information processing method, and a recording medium will be described below. In the following, the first embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 1 to which the first embodiment of the information processing device, the information processing method, and the recording medium is applied.
[1-1: Configuration of information processing device 1]

FIG. 1 is a block diagram showing the configuration of an information processing device 1 in the first embodiment. As shown in FIG. 1, the information processing device 1 includes a learning data generation unit 11, a model generation unit 12, and an authentication unit 13.

The training data generating unit 11 generates second training data based on first training data including an eye image showing the eye of a subject and information on an accessory of the subject shown in the eye image included in the first training data. The model generating unit 12 generates an authentication model using the second training data. The authentication unit 13 authenticates the subject using the authentication model.
[1-2: Technical Effects of Information Processing Device 1]

The information processing device 1 in the first embodiment generates second learning data based on information about the subject's attached object shown in the eye image. The information processing device 1 authenticates the subject using an authentication model generated using this second learning data, and therefore can perform highly accurate authentication regardless of whether the subject is wearing an attached object or not.
[2: Second embodiment]

Next, a second embodiment of the information processing device, the information processing method, and the recording medium will be described. In the following, the second embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 2 to which the second embodiment of the information processing device, the information processing method, and the recording medium is applied.
[2-1: Configuration of information processing device 2]

FIG. 2 is a block diagram showing the configuration of the information processing device 2 in the second embodiment. As shown in FIG. 2, the information processing device 2 includes a calculation device 21 and a storage device 22. Furthermore, the information processing device 2 may include a communication device 23, an input device 24, and an output device 25. However, the information processing device 2 does not have to include at least one of the communication device 23, the input device 24, and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

The arithmetic device 21 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable and non-transient recording medium using a recording medium reading device (e.g., an input device 24 described later) not shown in the figure that is provided in the information processing device 2. The arithmetic device 21 may acquire (i.e., download or read) a computer program from a device (not shown) located outside the information processing device 2 via the communication device 23 (or other communication device). The arithmetic device 21 executes the read computer program. As a result, a logical functional block for executing the operation to be performed by the information processing device 2 is realized within the calculation device 21. In other words, the calculation device 21 can function as a controller for realizing a logical functional block for executing the operation (in other words, processing) to be performed by the information processing device 2.

2 shows an example of a logical functional block realized in the computing device 21 to execute an information processing operation. As shown in FIG. 2, the computing device 21 realizes a learning data generating unit 211, which is a specific example of a "learning data generating means" described in the appendix described later, a model generating unit 212, which is a specific example of a "model generating means" described in the appendix described later, an authentication unit 213, which is a specific example of an "authentication means" described in the appendix described later, a learning data acquiring unit 214, which is a specific example of a "learning data acquiring means" described in the appendix described later, an accessory detecting unit 215, which is a specific example of an "accessory detecting means" described in the appendix described later, and an eye image acquiring unit 216. However, at least one of the learning data acquiring unit 214, the accessory detecting unit 215, and the eye image acquiring unit 216 does not have to be realized in the computing device 21. Details of the operation of each of the learning data generation unit 211, the model generation unit 212, the authentication unit 213, the learning data acquisition unit 214, the accessory detection unit 215, and the eye image acquisition unit 216 will be described later with reference to FIG. 3.

The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that is temporarily used by the arithmetic device 21 when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the information processing device 2 stores for a long period of time. The storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 22 may include a non-temporary recording medium.

The communication device 23 is capable of communicating with devices external to the information processing device 2 via a communication network (not shown). The communication device 23 may be a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), and USB (Universal Serial Bus).

The input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operating device (e.g., at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be attached externally to the information processing device 2.

The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (so-called a display) capable of displaying an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (so-called a speaker) capable of outputting sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (so-called a printer) capable of printing desired information on paper.
[2-2: Information Processing Operation Performed by Information Processing Device 2]

An information processing operation performed by the information processing device 2 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of the information processing operation performed by the information processing device 2. Fig. 3(a) is a flowchart showing the flow of the learning operation performed by the information processing device 2, and Fig. 3(b) is a flowchart showing the flow of the authentication operation performed by the information processing device 2.
[2-2-1: Learning Operation]

As shown in FIG. 3(a), the learning data acquisition unit 214 acquires first learning data including an eye image showing the eyes of a target (step S20). In this embodiment, the target includes a person. Also, in this embodiment, the target includes animals such as dogs and snakes. Below, an example will be described in which the target is a person. The first learning data includes eye images of a person not wearing accessories, and eye images of a person wearing accessories. Also, the eye images of a person wearing accessories include eye images of people wearing various types of accessories. In this embodiment, accessories refer to accessories worn around the eyes, such as glasses, contact lenses, masks, etc. Accessories are objects that affect authentication processing, matching processing, etc., using the area of a person's eyes included in the eye image, particularly the area of the person's iris.

The first learning data may be stored in the storage device 22, in which case the learning data acquisition unit 214 may acquire the first learning data from the storage device 22. Alternatively, the learning data acquisition unit 214 may acquire the first learning data from a device external to the information processing device 2 via the communication device 23.

The accessory detection unit 215 detects the accessory of the person appearing in the eye image from the eye image included in the first learning data (step S21). In this embodiment, the type of accessory may be determined in advance. That is, the accessory detection unit 215 may detect whether the person appearing in the eye image is wearing an accessory, and if the person appearing in the eye image is wearing an accessory, what type of accessory the person is wearing.

The type of accessory may be classified according to the effect that the accessory has on the area of the person's eyes included in the eye image. The type of accessory may be classified according to the manner in which the area of the person's eyes included in the eye image is hidden by the accessory. The type of accessory may be classified according to the effect that the area of the person's eyes included in the eye image is hidden by the accessory. The type of accessory may be classified according to the effect that the area of the person's eyes included in the eye image is hidden by the accessory, on the authentication result. For example, eyeglasses as an accessory may be classified into multiple types according to the type of frame, type of lens, etc.

For example, assume that N types of accessories are defined. In this case, the accessory detection unit 215 may detect whether the person in the eye image is (0) not wearing any accessories, (1) wearing a first type of accessory, (2) wearing a second type of accessory, ..., or (N) wearing an Nth type of accessory. The accessory detection unit 215 may calculate the likelihood that the person is not wearing any accessories, and the likelihood that the person is wearing each type of accessory when wearing accessories. The accessory detection unit 215 may detect which of the above (0) to (N) cases is most likely. In addition, the person in the eye image may be wearing multiple accessories (for example, a first type of accessory and a second type of accessory). In this case, the accessory detection unit 215 may detect the above (1) and (2). Below, this embodiment will be described using an example in which N types of accessories are defined.

Furthermore, each of the above (0) to (N) may be referred to as an accessory class WC. An eye image in which a person is not wearing any accessory may be said to belong to the 0th accessory class WC0. An eye image in which a person is wearing the first type of accessory may be said to belong to the first accessory class WC1. ... An eye image in which a person is wearing the Nth type of accessory may be said to belong to the Nth accessory class WCN.

The learning data generating unit 211 generates second learning data from the first learning data (step S22). In this embodiment, the second learning data may be learning data generated for learning iris characteristics. The second learning data may be learning data having desired characteristics related to the wearable item. The second learning data may be learning data having desired characteristics of the iris related to the wearable item. The second learning data may be learning data generated in particular for learning the effect on the iris when a person wears the wearable item.

The learning data generation unit 211 generates second learning data based on the first learning data and information about the accessories of the person appearing in the eye image included in the first learning data. The learning data generation unit 211 generates second learning data from the first learning data according to the detection result by the accessory detection unit 215.

The learning data generation unit 211 may classify the eye images included in the first learning data based on the detection results by the accessory detection unit 215 into one of the following: a 0th accessory class WC0 in which the person is not wearing any accessory, a 1st accessory class WC1 in which the person is wearing a 1st type of accessory, a 2nd accessory class WC2 in which the person is wearing a 2nd type of accessory, ..., an Nth accessory class WCN in which the person is wearing an Nth type of accessory.

The learning data generating unit 211 may generate the second learning data so that the ratio of the number of eye images belonging to each accessory class WC is a desired ratio. The learning data generating unit 211 may generate the second learning data by making adjustments so that there are no accessory classes WC that have too few eye images or no accessory classes WC that have too many eye images. For example, the learning data generating unit 211 may perform undersampling to select the necessary eye images, and oversampling to pad out the necessary eye images. Oversampling may be achieved by copying the relevant eye images.

The learning data generating unit 211 may generate the second learning data so that the number of eye images belonging to each accessory class WC is the same. Alternatively, the learning data generating unit 211 may generate the second learning data based on the detection results of the accessory detection unit 215 so that the ratio of eye images belonging to each accessory class WC is a desired ratio. In other words, the number of eye images belonging to each accessory class WC does not have to be the same. The learning data generating unit 211 may adjust the number of eye images belonging to each accessory class WC according to the number of eye images belonging to each accessory class WC included in the first learning data, starting from a state in which the number of eye images belonging to each accessory class WC is constant.

The second learning data may be learning data that includes eye images, the number of which is adjusted and which belong to each of the attachment classes WC. In other words, the learning data generating unit 211 adjusts the number of eye images included in the first learning data in order to perform the desired iris authentication, and generates the second learning data.

The model generation unit 212 generates an authentication model using the second learning data (step S23). In this embodiment, the authentication model is an iris authentication model. The iris authentication model may be a model that outputs an iris authentication result when an eye image including an iris is input.

It is not necessary to detect attached items within the information processing device 2. In this case, for example, the eye image included in the acquired first learning data may have information regarding attached items of a person appearing in the eye image.
[2-2-2: Authentication Operation]

3B, the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The authentication unit 213 performs iris authentication of the subject using an iris authentication model as an authentication model (step S25).
[2-3: Technical Effects of Information Processing Device 2]

In iris authentication using the iris region of the subject's eye image, identity is determined using the subject's iris region and a pre-stored iris region. Even if the subject's iris region and the pre-registered iris region belong to the same person, there will be differences. For example, if the subject is wearing glasses or similar when an iris photograph is taken, the subject's iris region image will reflect the effects of the glasses or other accessories, which may affect the accuracy of authentication.

In addition, if there is a large variation in the number of eye images belonging to each of the wearable object classes WC contained in the learning data used to construct the authentication model, learning may be performed with less emphasis on wearable object classes WC with fewer eye images. For example, an authentication model may be generated that is constructed by performing heavy learning on the 0th wearable object class WC0, which has the largest number of eye images, but light learning on wearable object classes WC with fewer eye images.

The information processing device 2 in the second embodiment generates second learning data for generating an authentication model from first learning data obtained by detecting and classifying the wear of a person captured in an eye image, and can generate second learning data with a balanced ratio of wear classes suitable for learning according to the wear of a person. The information processing device 2 generates an authentication model using the second learning data, and can therefore perform learning for authentication including not only the iris region but also the wear. The information processing device 2 can construct an authentication model that is robust against occlusion including the wear. The information processing device 2 uses the authentication model to authenticate the subject, and can therefore perform accurate authentication regardless of whether the subject is wearing wear. In other words, the information processing device 2 can perform accurate iris authentication even if the subject is wearing wear.
[3: Third embodiment]

Next, a third embodiment of the information processing device, information processing method, and recording medium will be described. Below, the third embodiment of the information processing device, information processing method, and recording medium will be described using an information processing device 3 to which the third embodiment of the information processing device, information processing method, and recording medium is applied.

4 is a block diagram showing the configuration of an information processing device 3 according to the third embodiment. The information processing device 3 according to the third embodiment differs from the information processing device 2 according to the second embodiment in the operation of a learning data generating unit 311, the operation of a model generating unit 312, and the operation of an authentication unit 313.
[3-1: Information Processing Operation Performed by Information Processing Device 3]

The flow of information processing operations performed by the information processing device 3 in the third embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of information processing operations performed by the information processing device 3 in the third embodiment. Fig. 5(a) is a flowchart showing the flow of a learning operation performed by the information processing device 3, and Fig. 5(b) is a flowchart showing the flow of an authentication operation performed by the information processing device 3.
[3-1-1: Learning Operation]

As shown in FIG. 5(a), the learning data acquisition unit 214 acquires first learning data including an eye image showing a person's eyes (step S20). The accessory detection unit 215 detects the accessory of the person shown in the eye image from the eye image included in the first learning data (step S21).

Based on the detection result of the accessory detection unit 215, the learning data generation unit 311 generates second learning data including classification learning data CD obtained by classifying the first learning data according to whether or not the person in the eye image is wearing an accessory and, if so, what type of accessory the person is wearing (step S30). In other words, the second learning data generated by the learning data generation unit 311 includes 0th classification learning data CD0 consisting of eye images belonging to the 0th accessory class WC0, 1st classification learning data CD1 consisting of eye images belonging to the 1st accessory class WC1, ..., and Nth classification learning data CDN consisting of eye images belonging to the Nth accessory class WCN. The learning data generation unit 311 in the third embodiment may appropriately adjust the number of eye images belonging to each accessory class WC, as described in the second embodiment.

The model generation unit 312 uses each of the categorized learning data CD to generate each of the categorized authentication models CM according to the category (step S31). For example, the model generation unit 312 uses the 0th category learning data CD0 to generate a 0th category authentication model CM0 according to the 0th accessory class WC0. The model generation unit 312 uses the first category learning data CD1 to generate a first category authentication model CM1 according to the first accessory class WC1. ...The model generation unit 312 uses the Nth category learning data CDN to generate an Nth category authentication model CMN according to the Nth accessory class WCN. Note that, in the information processing device in other embodiments, when the model generation unit 312 is realized in a computing device, each of the categorized authentication models CM according to the category is generated.
[3-1-2: Authentication Operation]

5B, the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The authentication unit 313 performs iris authentication of the subject using a classification authentication model CM (step S32). In the third embodiment, the authentication unit 313 may perform iris authentication using at least one of the classification authentication models CM from the 0th classification authentication model CM0 to the Nth classification authentication model CMN.
[3-2: Technical Effects of Information Processing Device 3]

The information processing device 3 in the third embodiment generates second learning data including classification learning data obtained by classifying the first learning data according to whether or not a person is wearing an accessory and, if so, what type of accessory the person is wearing, based on the detection result of the accessory of the person captured in the eye image, so that it is possible to obtain learning data suitable for the accessory class. The information processing device 3 can perform learning using the learning data suitable for the accessory class, so that it can generate each of the classification authentication models CM suitable for the accessory class.
[4: Fourth embodiment]

A fourth embodiment of the information processing device, the information processing method, and the recording medium will be described below. In the following, the fourth embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 4 to which the fourth embodiment of the information processing device, the information processing method, and the recording medium is applied.
[4-1: Configuration of information processing device 4]

The configuration of the information processing device 4 in the fourth embodiment will be described with reference to FIG. 6. FIG. 6 is a block diagram showing the configuration of the information processing device 4 in the fourth embodiment.

As shown in Fig. 6, the information processing device 4 in the fourth embodiment differs from the information processing device 2 in the second embodiment and the information processing device 3 in the third embodiment in that an authentication model selection unit 417 is realized in the calculation device 21. Also, in the information processing device 4, as in the information processing device 3, the model generation unit 312 generates each of the categorized authentication models CM according to the categories. Other features of the information processing device 4 may be the same as other features of at least one of the information processing device 2 and the information processing device 3. Therefore, hereinafter, the parts that differ from the embodiments already described will be described in detail, and the explanation of other overlapping parts will be omitted as appropriate.
[4-2: Information Processing Operation Performed by Information Processing Device 4]

The flow of the information processing operation performed by the information processing device 4 in the fourth embodiment will be described with reference to FIG. 7. The information processing operation performed by the information processing device 4 in the fourth embodiment differs from the information processing operation performed by the information processing device 3 in the third embodiment in terms of authentication operation. FIG. 7 is a flowchart showing the flow of the authentication operation performed by the information processing device 4 in the fourth embodiment.

As shown in Fig. 7, the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The accessory detection unit 415 detects the accessory of the subject from the eye image of the subject (step S40). The authentication model selection unit 417 selects which of the categorized authentication models CM the authentication unit 413 will use based on the detection result of the accessory detection unit 415 (step S41). The authentication model selection unit 417 uses the accessory detection result to select a categorized authentication model CM suitable for the accessory worn by the subject. The authentication unit 413 performs iris authentication of the subject using the selected categorized authentication model CM (step S42).
[4-3: Technical Effects of Information Processing Device 4]

In the fourth embodiment, the information processing device 4 selects a categorized authentication model CM according to the subject's clothing and authenticates the subject using the selected categorized authentication model CM, thereby enabling highly accurate authentication.

When iris authentication is performed using the same authentication model, the authentication result may differ depending on whether the subject is wearing an accessory or not. The information processing device 3 uses the classification authentication model CM according to the accessory class WC, so that the authentication result is stable.
[5: Fifth embodiment]

The fifth embodiment of the information processing device, the information processing method, and the recording medium will be described below. The fifth embodiment of the information processing device, the information processing method, and the recording medium will be described below using an information processing device 5 to which the fifth embodiment of the information processing device, the information processing method, and the recording medium is applied.

8 is a block diagram showing the configuration of an information processing device 5 according to the fifth embodiment. The information processing device 5 according to the fifth embodiment differs from the information processing device 2 according to the second embodiment to the information processing device 4 according to the fourth embodiment in the operation of an authentication unit 513.
[5-1: Information Processing Operation Performed by Information Processing Device 5]

The flow of information processing operations performed by the information processing device 5 in the fifth embodiment will be described with reference to FIG. 9. The information processing operations performed by the information processing device 5 in the fifth embodiment differ in authentication operations from the information processing operations performed by at least one of the information processing device 3 in the third embodiment and the information processing device 4 in the fourth embodiment. Also, in the information processing device 5, as in the information processing device 3, the model generation unit 312 generates each of the categorized authentication models CM according to the categories. FIG. 9 is a flowchart showing the flow of authentication operations performed by the information processing device 5 in the fifth embodiment.

As shown in FIG. 9, the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The authentication unit 513 performs iris authentication of the subject using each of the categorical authentication models CM (step S50). The authentication unit 513 determines whether or not the iris of the subject can be authenticated based on each of the authentication results by each of the categorical authentication models CM (step S51). For example, the authentication unit 513 may perform matching using all of the 0th categorical authentication model CM0 to the Nth categorical authentication model CMN, and determine whether or not authentication can be performed using the highest matching score among the respective matching scores. Alternatively, the authentication unit 513 may perform matching using all of the 0th categorical authentication model CM0 to the Nth categorical authentication model CMN, calculate an average score by averaging each matching score, and determine whether or not authentication can be performed using the average score.
[5-2: Technical Effects of Information Processing Device 5]

The information processing device 5 in the fifth embodiment performs authentication using each of the categorized authentication models CM, and therefore can perform appropriate authentication regardless of whether the subject person is wearing an attachment or not.
[6: Sixth embodiment]

A sixth embodiment of an information processing device, an information processing method, and a recording medium will be described below. In the following, the sixth embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 6 to which the sixth embodiment of the information processing device, the information processing method, and the recording medium is applied.
[6-1: Configuration of information processing device 6]

The configuration of the information processing device 6 in the sixth embodiment will be described with reference to FIG. 10. FIG. 10 is a block diagram showing the configuration of the information processing device 6 in the sixth embodiment.

As shown in FIG. 10, the information processing device 6 in the sixth embodiment differs from the information processing device 2 in the second embodiment to the information processing device 5 in the fifth embodiment in that a classification registration data generating unit 618 and a classification registration data selecting unit 619 are realized in the arithmetic device 21. Also, it differs in that a registration data holding unit 621 is realized in the storage device 22. Also, in the information processing device 6, similarly to the information processing device 3, the model generating unit 312 generates each of the classification authentication models CM according to the classification. Other features of the information processing device 6 may be the same as at least one other feature of the information processing device 2 to the information processing device 5. Therefore, hereinafter, the parts that are different from the already described embodiments will be described in detail, and the description of other overlapping parts will be omitted as appropriate.
[6-2: Information Processing Operation Performed by Information Processing Device 6]

The flow of information processing operations performed by the information processing device 6 in the sixth embodiment will be described with reference to FIG. 11. FIG. 11 is a flowchart showing the flow of information processing operations performed by the information processing device 6 in the sixth embodiment. FIG. 11(a) is a flowchart showing the flow of registration data generation operations performed by the information processing device 6, and FIG. 11(b) is a flowchart showing the flow of authentication operations performed by the information processing device 6.

The enrollment data storage unit 621 in the sixth embodiment stores enrollment data including enrollment images of eyes of a person. The enrollment data may include an eye image of only one eye, or may include an eye image of both eyes.
[6-2-1: Registration data generation operation]

As shown in FIG. 11A, the categorized registration data generating unit 618 acquires registration data (step S60). The accessory detection unit 615 detects the accessory of the person appearing in the eye image from the eye image included in the registration data (step S61). Based on the detection result of the accessory detection unit 615, the categorized registration data generating unit 618 generates categorized registration data CRD that classifies the registration data according to whether the person appearing in the eye image is wearing an accessory and, if so, what type of accessory the person is wearing (step S62). In other words, the categorized registration data generating unit 618 generates 0th categorized registration data CRD0 consisting of eye images belonging to the 0th accessory class WC0, 1st categorized registration data CRD1 consisting of eye images belonging to the 1st accessory class WC1, ..., and Nth categorized registration data CRDN consisting of eye images belonging to the Nth accessory class WCN. The classification registration data generation unit 618 may generate registration data including the 0th classification registration data CRD0, the 1st classification registration data CRD1, ..., and the Nth classification registration data CRDN, and store the data in the registration data storage unit 621.

[6-2-2: Authentication Operation]
As shown in FIG. 11B, the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The attachment detection unit 615 detects the attachment of the subject from the eye image of the subject (step S63). The categorized registration data selection unit 619 selects which of the categorized registration data CRD the authentication unit 613 uses based on the detection result of the attachment detection unit 615 (step S64). The authentication unit 613 performs iris authentication of the subject using the categorized registration data CRD selected by the categorized registration data selection unit 619 (step S65). That is, in the sixth embodiment, the registration data is used differently depending on the attachment of the subject. The authentication unit 613 may also select the categorized authentication model CM to use depending on the detection result of the attachment.
[6-3: Technical Effects of Information Processing Device 6]

The information processing device 6 in the sixth embodiment classifies the registration data based on whether or not the person appearing in the eye image is wearing an accessory and, if so, what type of accessory the person is wearing, and selects the categorized registration data CRD to be used for authentication based on the subject's accessory, thereby enabling more accurate authentication.
[7: Seventh embodiment]

A seventh embodiment of an information processing device, an information processing method, and a recording medium will be described below. In the following, the seventh embodiment of the information processing device, the information processing method, and the recording medium will be described using an information processing device 7 to which the seventh embodiment of the information processing device, the information processing method, and the recording medium is applied.
[7-1: Configuration of information processing device 7]

The configuration of the information processing device 7 in the seventh embodiment will be described with reference to FIG. 12. FIG. 12 is a block diagram showing the configuration of the information processing device 7 in the seventh embodiment.

As shown in FIG. 12, the information processing device 7 in the seventh embodiment is different from the information processing device 2 in the second embodiment to the information processing device 6 in the sixth embodiment in that the authentication unit 713 has a matching unit 7131, a weighting unit 7132, and a judgment unit 7133, and the attachment detection unit 715 has a calculation unit 7151 and a determination unit 7152. Also, in the information processing device 7, as in the information processing device 3, the model generation unit 312 generates each of the categorized authentication models CM according to the categories. Other features of the information processing device 7 may be the same as at least one of the other features of the information processing device 2 to the information processing device 6. Therefore, hereinafter, the parts that are different from the respective embodiments already described will be described in detail, and the explanation of the other overlapping parts will be omitted as appropriate.
[7-2: Information Processing Operation Performed by Information Processing Device 7]

The flow of information processing operations performed by the information processing device 7 in the seventh embodiment will be described with reference to FIG. 13. The information processing operations performed by the information processing device 7 in the seventh embodiment differ in authentication operations from the information processing operations performed by at least one of the information processing device 3 in the third embodiment to the information processing device 6 in the sixth embodiment. FIG. 13 is a flowchart showing the flow of authentication operations performed by the information processing device 7 in the seventh embodiment.

As shown in FIG. 13, the eye image acquisition unit 216 acquires an eye image of the subject (step S24). The calculation unit 7151 calculates the likelihood that the subject is not wearing any accessories, and the likelihood that the subject is wearing each type of accessory (step S70). The determination unit 7152 determines the weight of each accessory class WC according to the likelihood of each accessory class WC (step S71). The determination unit 7152 may use the likelihood value as the weight as is.

The collation unit 7131 performs iris authentication of the subject using each of the categorized authentication models CM (step S72). The weighting unit 7132 weights each authentication result using the weight of the attachment class WC corresponding to the categorized authentication model CM (step S73). For example, the weighting unit 7132 may calculate a weighted average score by weighting the scores output by each of the categorized authentication models CM using weights according to the likelihood. The possibility determination unit 7133 determines whether or not the iris of the subject can be authenticated based on each of the weighted authentication results (step S74).
[7-3: Technical Effects of Information Processing Device 7]

The information processing device 7 in the seventh embodiment determines whether or not to authenticate the subject based on each of the authentication results weighted according to the likelihood of the attachment, and therefore can perform authentication with high accuracy.
[8: Eighth embodiment]

The eighth embodiment of the information processing device, information processing method, and recording medium will be described below. The eighth embodiment of the information processing device, information processing method, and recording medium will be described below using an information processing device 8 to which the eighth embodiment of the information processing device, information processing method, and recording medium is applied.

14 is a block diagram showing the configuration of an information processing device 8 according to the eighth embodiment. The information processing device 8 according to the eighth embodiment differs from the information processing device 2 according to the second embodiment to the information processing device 7 according to the seventh embodiment in the operation of a model generation unit 812 and the operation of an authentication unit 813.
[8-1: Information Processing Operation Performed by Information Processing Device 8]

The authentication model has a function of outputting features of an eye image when an eye image is input. When eye images of the same person are input, the model generation unit 812 generates an authentication model so that it outputs features similar to or more than a predetermined level to features output when an eye image of a person not wearing any accessories is input, regardless of whether the person in the eye image is wearing accessories or not. The model generation unit 812 generates an authentication model so that the features output when an eye image of a person wearing any accessories is input is similar to or more than a predetermined level to the features output when an eye image of a person not wearing accessories is input.

The authentication unit 813 in the eighth embodiment performs iris authentication using an eye image without any accessories as the registration data. Unlike the sixth embodiment described above, the authentication unit 813 in the eighth embodiment can extract features similar to those without any accessories, so the authentication unit 813 can perform iris authentication if only the registration data without any accessories is prepared. That is, in the eighth embodiment, it is sufficient to register only the eye image without any accessories as the registration data.
[8-2: Technical Effects of Information Processing Device 8]

The information processing device 8 in the eighth embodiment can perform authentication if an eye image taken when no attachment is worn is registered.
[9: Ninth embodiment]

A ninth embodiment of an information processing device, an information processing method, and a recording medium will be described below. In the following, the ninth embodiment of the information processing device, the information processing method, and a recording medium will be described using an information processing device 9 to which the ninth embodiment of the information processing device, the information processing method, and a recording medium is applied.
[9-1: Eye Area Recognition]

In the ninth embodiment, eye surrounding authentication is performed in addition to or instead of iris authentication. Eye surrounding authentication may be performed by focusing on the area around the eyes, extracting features from the area around the eyes, and comparing the extraction results with registered data around the eyes. The features around the eyes may include the position and shape of the corners and corners of the eyes, the shape of the eyelids, etc.
[9-2: Information Processing Operation Performed by Information Processing Device 9]

The flow of information processing operations performed by the information processing device 9 in the ninth embodiment will be described with reference to Fig. 16. Fig. 16 is a flowchart showing the flow of information processing operations performed by the information processing device 9 in the ninth embodiment.
[9-2-1: Learning Operation]

As shown in FIG. 16(a), the learning data acquisition unit 214 acquires first learning data including an eye image showing a person's eyes (step S20). In the ninth embodiment, the first learning data includes an eye surrounding image including the area around the eye. Note that the first learning data used in the ninth embodiment may be the same as the first learning data used in the second to eighth embodiments.

The covering object detection unit 915 detects coverings covering the area around the eyes from the eye image included in the first learning data (step S91). The covering object detection unit 915 may detect clothing worn by a person appearing in the eye image from the eye image included in the first learning data. The covering object detection unit 915 may detect coverings other than clothing worn by a person appearing in the eye image from the eye image included in the first learning data. Coverings other than clothing may include, for example, items that cover the area around the eyes, such as makeup. The covering object detection unit 915 may detect factors that affect the features around the eyes from the eye image. The covering object detection unit 915 may detect the state of the area around the eyes, such as occlusion or distortion due to the influence of physical condition, etc. The covering object detection unit 915 may detect which covering object class CC is classified according to the influence on the area around the eyes included in the eye image. In addition, the covering object detection unit 915 may not detect as a covering an object that affects the iris but does not affect the features around the eyes. In other words, the detection operation by the wearing object detection unit 215 and the detection operation by the covering object detection unit 915 may be different.

The learning data generating unit 911 generates third learning data from the first learning data based on the detection result of the covering object detection unit 915 (step S92). In this embodiment, the third learning data may be learning data generated to learn the characteristics of the area around the eyes. The third learning data may be learning data having desired characteristics of the area around the eyes related to the covering. The third learning data may be learning data generated to learn the influence on the area around the eyes when the area around the eyes is covered by a covering. The learning data generating unit 911 may generate the third learning data by adjusting the number of items belonging to each covering object, for example, as described in the second embodiment. Alternatively, the learning data generating unit 911 may generate the third learning data including classification learning data classified for each covering object, for example, as described in the third embodiment.

The model generating unit 912 generates an eye-periphery authentication model using the third learning data (step S93). The model generating unit 912 may generate an eye-periphery authentication model corresponding to any covering, for example, as described in the second embodiment. Alternatively, the model generating unit 912 may generate an eye-periphery authentication model specialized for each covering, for example, as described in the third embodiment.
[9-2-2: Authentication Operation]

As shown in FIG. 16B, the eye surrounding image acquisition unit 916 acquires the eye surrounding image of the subject (step S94). The authentication unit 913 performs iris authentication of the subject using the iris authentication model and the iris area included in the eye surrounding image of the subject (step S95). The authentication unit 913 performs eye surrounding authentication of the subject using the eye surrounding authentication model and the eye surrounding image of the subject (step S96). That is, the authentication unit 913 performs two-factor authentication by performing iris authentication of the subject using the eye surrounding authentication model in addition to iris authentication of the subject using the iris authentication model. The authentication unit 913 may calculate a first score using the iris authentication model, calculate a second score using the eye surrounding authentication model, and authenticate the subject using the first score and the second score.
[9-3: Technical Effects of Information Processing Device 9]

If the area around the eye of the subject has changed since the registration data was registered, this may affect the accuracy of authentication. The information processing device 9 in the ninth embodiment learns the area around the eye in order to perform eye area authentication with high accuracy even when the condition of the area around the eye has changed, such as when there is an area around the eye that is covered. The information processing device 9 can perform eye area authentication with high accuracy even when there is an area around the eye that is covered. In addition, the information processing device 9 can perform two-factor authentication, and therefore can authenticate the subject with higher accuracy.
[10: Supplementary Note]

The following supplementary notes are further disclosed regarding the above-described embodiment.
[Appendix 1]
a learning data generating means for generating second learning data based on first learning data including an eye image showing the eye of a target and information on an attachment of the target shown in the eye image included in the first learning data;
a model generating means for generating an authentication model using the second training data;
and an authentication means for authenticating a target person using the authentication model.
[Appendix 2]
A learning data acquisition means for acquiring the first learning data;
and an attachment detection means for detecting an attachment of a target appearing in an eye image included in the first learning data,
The information processing device according to claim 1, wherein the learning data generating means generates the second learning data from the first learning data in response to a detection result by the attachment detecting means.
[Appendix 3]
The learning data generation means generates the second learning data including classification learning data obtained by classifying the first learning data based on a detection result by the accessory detection means, based on whether the subject shown in the eye image is wearing an accessory and, if so, what type of accessory the subject is wearing.
[Appendix 4]
The information processing device according to claim 3, wherein the model generating means generates each of the categorized authentication models according to the categories by using each of the categorized learning data.
[Appendix 5]
The attachment detection means detects an attachment of the subject from the eye image of the subject,
The information processing device according to claim 4, further comprising an authentication model selection unit that selects which of the categorized authentication models the authentication unit is to use based on a detection result by the attachment detection unit.
[Appendix 6]
The authentication means includes:
authenticating the subject using each of the classification authentication models;
The information processing device according to claim 4, further comprising: determining whether or not the subject can be authenticated based on each of the authentication results obtained by each of the classification authentication models.
[Appendix 7]
a classification means for generating classified registration data based on an accessory of the subject captured in an eye image detected by the accessory detection means from an eye image included in registration data including the eye image showing the subject's eyes, by classifying the registration data according to whether the subject captured in the eye image is wearing an accessory and, if so, what type of accessory the subject is wearing;
The information processing device described in Appendix 2, further comprising a registration data selection means for selecting which of the categorized registration data the authentication means will use based on the attachment of the subject detected by the attachment detection means from the eye image of the subject.
[Appendix 8]
the attachment detection means includes a calculation means for calculating a likelihood that the subject is not wearing an attachment and a likelihood that the subject is wearing each type of attachment when the subject is wearing an attachment, and calculating each of the weights of the classifications in accordance with the likelihoods;
The authentication means includes:
authenticating the subject using each of the classification authentication models;
weighting each authentication result using the category weights corresponding to the category authentication model;
The information processing device according to claim 6, further comprising: determining whether or not to authenticate the subject based on each of the weighted authentication results.
[Appendix 9]
the authentication model has a function of outputting features of the eye image when the eye image is input;
The information processing device described in Appendix 1 or 2, wherein the model generation means generates the authentication model so that, when an eye image of the same subject is input, features similar to or more than a predetermined level are output when an eye image of the subject not wearing any accessories is input, regardless of whether the subject in the eye image is wearing any accessories.
[Appendix 10]
The first learning data includes an eye surrounding image including an area around the eye,
The attachment detection means detects an attachment of a target appearing in the eye-periphery image from the eye-periphery image included in the first learning data,
the learning data generating means generates third learning data from the first learning data based on a detection result by the attachment detecting means;
The information processing device according to claim 2, wherein the model generating means generates an eye-periphery authentication model by using the third learning data.
[Appendix 11]
The authentication using the authentication model is iris authentication,
The information processing apparatus according to claim 10 , wherein the authentication means performs eye-periphery authentication of the subject using the eye-periphery authentication model in addition to iris authentication of the subject using the authentication model.
[Appendix 12]
generating second learning data based on first learning data including an eye image showing the eye of the target and information on an attachment of the target shown in the eye image included in the first learning data;
generating an authentication model using the second training data;
An information processing method for authenticating a subject using the authentication model.
[Appendix 13]
On the computer,
generating second learning data based on first learning data including an eye image showing the eye of the target and information on an attachment of the target shown in the eye image included in the first learning data;
generating an authentication model using the second training data;
A recording medium having a computer program recorded thereon for executing an information processing method for authenticating a subject by using the authentication model.

This disclosure may be modified as appropriate within the scope of the claims and the technical idea that can be read from the entire specification. Information processing devices, information processing methods, and recording media that incorporate such modifications are also included in the technical idea of this disclosure.

1, 2, 3, 4, 5, 6, 7, 8, 9 Information processing device 11, 211, 311, 911 Learning data generation unit 12, 212, 312, 812, 912 Model generation unit 13, 213, 313, 413, 513, 613, 713, 813, 913 Authentication unit 214 Learning data acquisition unit 215, 415, 615, 715 Wearing object detection unit 216 Eye image acquisition unit 417 Authentication model selection unit 618 Classification registration data generation unit 619 Classification registration data selection unit 621 Registration data storage unit 7131 Collation unit 7132 Weighting unit 7133 Possibility determination unit 7151 Calculation unit 7152 Decision unit 915 Covering object detection unit 916 Eye surroundings image acquisition unit

Claims (13)

a learning data generating means for generating second learning data based on first learning data including an eye image showing the eye of a target and information on an attachment of the target shown in the eye image included in the first learning data;
a model generating means for generating an authentication model using the second training data;
and an authentication means for authenticating a target person using the authentication model. A learning data acquisition means for acquiring the first learning data;
and an attachment detection means for detecting an attachment of a target appearing in an eye image included in the first learning data,
The information processing apparatus according to claim 1 , wherein the learning data generating means generates the second learning data from the first learning data in accordance with a detection result by the attachment detecting means. 3. The information processing device according to claim 2, wherein the learning data generation means generates the second learning data including classified learning data obtained by classifying the first learning data based on a detection result by the accessory detection means, according to whether or not the subject shown in the eye image is wearing an accessory and, if so, what type of accessory the subject is wearing. The information processing apparatus according to claim 3 , wherein the model generating means generates each of the categorized authentication models according to the categories by using each of the categorized learning data. The attachment detection means detects an attachment of the subject from the eye image of the subject,
The information processing apparatus according to claim 4 , further comprising an authentication model selection unit that selects which of the categorized authentication models is to be used by the authentication unit based on a detection result by the attachment detection unit. The authentication means includes:
authenticating the subject using each of the classification authentication models;
The information processing apparatus according to claim 4 , further comprising: determining whether or not the subject can be authenticated based on each of the authentication results obtained by each of the classification authentication models. a classification means for generating classified registration data based on an accessory of the subject captured in an eye image detected by the accessory detection means from an eye image included in registration data including the eye image showing the subject's eyes, by classifying the registration data according to whether the subject captured in the eye image is wearing an accessory and, if so, what type of accessory the subject is wearing;
The information processing device according to claim 2 , further comprising a registration data selection means for selecting which of the categorized registration data the authentication means will use based on the subject's clothing detected by the clothing detection means from the subject's eye image. the attachment detection means includes a calculation means for calculating a likelihood that the subject is not wearing an attachment and a likelihood that the subject is wearing each type of attachment when the subject is wearing an attachment, and calculating each of the weights of the classifications in accordance with the likelihoods;
The authentication means includes:
authenticating the subject using each of the classification authentication models;
weighting each authentication result using the category weights corresponding to the category authentication model;
The information processing apparatus according to claim 6 , further comprising: determining whether or not to authenticate the subject person based on each of the weighted authentication results. the authentication model has a function of outputting features of the eye image when the eye image is input;
3. The information processing device according to claim 1, wherein the model generation means generates the authentication model so that, when an eye image of the same subject is input, it outputs features similar to or more than a predetermined level to features output when an eye image of the subject not wearing any accessories is input, regardless of whether the subject in the eye image is wearing any accessories. The first learning data includes an eye surrounding image including an area around the eye,
The attachment detection means detects an attachment of a target appearing in the eye-periphery image from the eye-periphery image included in the first learning data,
the learning data generating means generates third learning data from the first learning data based on a detection result by the attachment detecting means;
The information processing apparatus according to claim 2 , wherein the model generating means generates an eye-periphery authentication model by using the third learning data. The authentication using the authentication model is iris authentication,
The information processing apparatus according to claim 10 , wherein the authentication means performs eye-periphery authentication of the subject using the eye-periphery authentication model in addition to iris authentication of the subject using the authentication model. generating second learning data based on first learning data including an eye image showing the eye of the target and information on an attachment of the target shown in the eye image included in the first learning data;
generating an authentication model using the second training data;
An information processing method for authenticating a subject using the authentication model. On the computer,
generating second learning data based on first learning data including an eye image showing the eye of the target and information on an attachment of the target shown in the eye image included in the first learning data;
generating an authentication model using the second training data;
A recording medium having a computer program recorded thereon for executing an information processing method for authenticating a subject by using the authentication model.