WO2018139847A1 - Personal identification method through facial comparison - Google Patents

Abstract

The present invention relates to a personal identification method through facial comparison, wherein the method is executed in a personal identification system consisting of an imaging unit, a control unit connected to the imaging unit, and a storage unit which is connected to the control unit and in which a personal identification program and an identification image composed of a plurality of frames photographed by the imaging unit are stored, and performs a comparison with data registered in a database. The method comprises a personal identification step consisting of: a facial feature vector extraction step in which a facial feature vector of a frame making up the identification image is derived; and a step in which the Euclidean distance between the facial feature vector extracted in the facial feature vector extraction step and a facial feature vector of the data registered in the database is computed, such that the computed degree of similarity is provided as information for identifying a photographed person.

Description

Personal identification by face comparison

The present invention relates to a method of personal identification through face comparison and a personal identification system in which the method is executed. More particularly, the present invention provides a face that can be identified as a sculpture, identified as the same, and additional information such as race can be provided. A method of personal identification through comparison and a personal identification system in which the method is implemented.

Existing personal identification system is mainly used in access security device or personalized information providing system, etc., and outputs only the result of whether the same person is compared by comparing the face image taken in the field with the already registered image. This method does not apply the method of distinguishing the print out of the human face image from the human face and the color printer or artificially manufactured sculptures, which may cause a fatal security problem and the dichotomous judgment result is wrong. As a result, in spite of being the same person, if it is recognized as a different person, there is a problem that can be judged as the same person even though the person is not the same person, causing inconvenience and the same person.

In the dichotomous judgment system, if the standard similarity judged as the same person is set high, the result is often the same person and the result is not the same person, which causes inconvenience in the entrance system. On the contrary, if the standard similarity is set low, the same person is judged as the same person. Problems will arise.

In general, most access devices and security devices cannot distinguish human faces and printouts, and in case of judgment, it is necessary to set a high standard similarity and take inconvenience. There is a problem of comparing the real person with the naked eye again.

In addition, the existing system for personal identification based on the similarity only automatically collects statistical information about race and gender for the identified individual and is not applicable to a personalized system.

The present invention has been proposed to solve the above problems, and can be distinguished whether the object to be prepared is a real face or printed matter or sculpture printed on the face, provides a similarity data of the face, information on race or gender Another object of the present invention is to provide a personal identification method through face comparison that can be provided together.

To this end, the present invention provides an individual comprising a photographing unit, a control unit connected to the photographing unit, a storage unit storing an identification image and a personal identification program consisting of a plurality of identification images connected to the control unit photographed by the photographing unit. Executed in the identification system to contrast with DB face information registered in the database;

And a personal identification step, wherein the personal identification step includes a facial area derivation step of deriving a face area from an image of an individual photographed, a facial feature point derivation step of deriving facial feature points, and a rotation angle of the derived face area. An operation for calculating the identification face rotation angle, a DB face information contrast step in which a plurality of DB face information images and an identification image are contrasted, a pixel color application step, and an identification face DB which is a face 3D database for the individual to be identified are derived. Consisting of the face DB derivation step; The present invention provides a personal identification method through face comparison in which information for identifying a photographed individual is provided.

In the DB face information contrasting step, a minimum error square value of a plurality of DB face information images is calculated for the identification image, and in the pixel color applying step, the smallest minimum square error value for the identification image among the DB face information images is calculated. The color information of pixels constituting the DB face information image is changed into an identification image and stored in the storage unit.

In the above, further comprising a face identification step before the personal identification step; The face identification step includes eye area detection step of eye area derived from each frame constituting the identification image, eye area derivation step of detecting eye area of eye area, and color information of pixels constituting eye area of each frame. It is characterized in that it comprises a step of contrasting the pupil area color information to be provided, and the sculpture identification information providing step of providing information that determines whether the actual face or sculpture is photographed from the change in the color information of the pixels constituting the pupil area of each frame.

The method may further include a face region deriving step of deriving a face region of a plurality of frames photographed before the eye region is detected, and after the pupil region derivation step, pixel information forming a pupil region in each frame is calculated. Further comprising a computing step; In the color information comparing step, the color information of the pupil area pixels of each frame is contrasted.

In the above, further comprising a face identification step before the personal identification step;

The face identification step may include: an eye area detection step in which an eye region is derived from each frame constituting an image captured by the photographing unit, a face area feature point derivation step in which feature points of a face area of each frame constituting the captured image are derived, and an eye; A pupil area derivation step of detecting a pupil area of an area, a pupil area location derivation step of deriving a pupil area location, and a pupil location from which a distance between a pupil area location and a facial area feature point derived in the pupil area location derivation step is calculated. The distance calculation step and the sculpture identification information providing step of providing information that determines whether the actual face is photographed or the sculpture is photographed from the change in the pupil position distance of each frame.

In the above, further comprising a face identification step before the personal identification step;

The face identification step includes deriving a face region from each frame constituting the image captured by the photographing unit, calculating a direction of the face region, and photographing whether a real face is photographed from a change in the face region direction of each frame. Sculpture identification information providing step is provided, characterized in that the information is determined.

In the above, the system further comprises a distance measuring sensor connected to the control, wherein each frame is stored in the storage with the distance information measured from the distance measuring sensor;

Further comprising a face identification step before the personal identification step; The face identification step includes a face area derivation step in which a face area is derived from each frame constituting the image photographed by the photographing unit, a face area information calculation step in which face area information derived in the face area derivation step is calculated, and each frame It is characterized in that it comprises a sculpture identification information providing step that provides information that determines whether the actual face is photographed or the sculpture is photographed from the face area information and distance information.

In the above, the storage unit stores the composite product neural network module, the learning photographed image and the learning photographed image information, and further comprising the step of providing additional information in addition to the personal identification step; The additional information providing step includes a learning step and a face information calculation step, wherein racial information and gender information of the identification image are provided;

The learning step includes the step of forming a convolutional layer of the training photographing image for each of the race information and the gender information, the step of forming a pooling layer from a map of the convolutional layer, and the information of the pooling of the pooling layer. Hidden layer information operation step of calculating hidden layer node values by operation of hidden layer weight and input information, output layer information calculating step of calculating output layer by operation of hidden layer node value and output layer weight, and error from output layer information and training photographing image information. The updated values of the hidden layer weights and the output layer weights are calculated according to the backpropagation algorithm, and the hidden layer information calculation step and the output layer information calculation step are repeatedly calculated, and the process of comparing the calculated output layer information with the photographed image information for training is repeated. Of the update value and output layer weight Made of a weighting stage which is derived singap is derived;

The hidden layer weight update value and the output layer weight update value derived in the learning step may be the hidden layer weight and the output layer weight for each of the race information and the gender information in the face information calculation step.

In the above operation, the face information calculation step includes a step of forming a convolutional layer of an identification image for each of the race information and the gender information, a step of forming a pooling layer from a map of the convolutional layer, and a pooling information of the pooling layer. A hidden layer information calculating step of calculating the hidden layer node values by the calculation of the hidden layer weight and the input information, and an output layer information calculating step of calculating the output layer by calculating the hidden layer information and the output layer weight; Nodes constituting the output layer in an operation for obtaining race information are equal to the number of races, and two nodes constituting the output layer in an operation for obtaining gender information are provided.

According to the personal identification method through face comparison according to the present invention and the personal identification system on which the method is executed, a sculpture or an output is distinguished from an actual face, so that when applied to an access device or a security device, a sculpture or an output having a malicious purpose may be used. Unauthorized access is blocked, the same person and the same person is identified by similarity step by step, there is an effect that the race, gender and age information can be provided.

Figure 1 illustrates the steps of performing a method of personal identification through face comparison according to the present invention,

2 is a view illustrating the face identification step of FIG.

3 is a schematic flowchart illustrating a face identification step of the present invention;

4 and 5 schematically illustrate the eye region,

6 and 7 are schematic flowcharts illustrating a process of deriving a gaze direction constituting a face identification step,

8 and 9 are graphs illustrating a change in gaze according to a frame.

Hereinafter, a personal identification method through face comparison according to the present invention will be described in detail with reference to the accompanying drawings.

1 is a view illustrating a step of executing a personal identification method through face comparison according to the present invention, FIG. 2 is illustrated to explain the face identification step of FIG. 1, and FIG. 3 is a face identification step of implementing the present invention. 4 and 5 are schematic diagrams illustrating eye regions, and FIGS. 6 and 7 are schematic flowcharts illustrating a process of deriving a gaze direction constituting a face identification step. 8 and 9 are graphs illustrating a change in gaze according to a frame.

According to an aspect of the present invention, there is provided a personal identification method through face comparison, including a photographing unit, a control unit connected to the photographing unit, and a storage unit storing an identification image and a personal identification program including a plurality of frames connected to the control unit photographed by the photographing unit. Runs on a personal identification system.

The personal identification system further includes a distance measuring sensor connected to the control unit to provide the measured distance information to the control unit, and a communication module connected to the control unit to transmit and receive a signal, and the compound multiplication neural network module is stored in the storage unit. The photographing unit, the control unit, the storage unit, and the distance measuring sensor are provided in a main body (not shown).

The personal identification system may be installed and operated at the door. When the power is supplied and the operation command is input, the personal identification program is operated to execute the personal identification method through face comparison according to the present invention.

The storage unit stores a plurality of personal 3D face databases (for example, 300 people), and extends from -30 ° to + 30 ° using an axis (first axis) extending in the vertical direction of the face and passing through the center of the face as a rotation axis. Face image information (pixel color of the face region, coordinates, and feature points of the face region) rotated at intervals of 10 ° are stored. Hereinafter, the face image information is referred to as "DB face information."

The face with a rotation angle of 0 ° is the front face. The first axis is the axis passing through the center of the face that is symmetrical. The process of deriving the feature points of the face region and the contents of the derived feature points are well known in the art, and description thereof will be omitted.

"−" At -30 ° means that the first axis is rotated to the right side of the face on the axis of rotation, and at + 30 °, "+" means that the first axis is rotated to the left side of the face on the axis of rotation. By way of example, it is described as being stored at intervals of 10 °, but may be stored at intervals of 5 ° or 15 °.

-30 ° -20 ° -10 ° 0 ° + 10 ° + 20 ° + 30 ° A Face image information Face image information Face image information Face image information Face image information Face image information Face image information B Face image information Face image information Face image information Face image information Face image information Face image information Face image information C Face image information Face image information Face image information Face image information Face image information Face image information Face image information D Face image information Face image information Face image information Face image information Face image information Face image information Face image information

The table exemplarily shows four persons to explain that the face 3D database of the individual is stored in the storage unit. In the table, A, B, C, and D mean different individuals. The face 3D database may also be stored in a server in communication with a personal identification system in which the present invention is implemented.

As shown in FIG. 1, the personal identification method through face comparison according to the present invention includes a face identification step ST-100, a personal identification step ST-200, and an additional information provision step ST-300. .

In the face identification step (ST-100), information for determining whether an identification image composed of a plurality of frames photographed by the photographing unit and stored in the storage unit is photographed by a real person or a human sculpture or printed matter is photographed.

In the personal identification step (ST-200), a material for identifying a photographed individual is provided by contrasting the individual photographed by the photographing unit and the data registered in the database.

In the additional information providing step ST-300, information for determining the race and gender of the individual photographed is provided. Information derived at each step is transmitted through a communication module and displayed on a monitor such as a security room.

In the face identification step (ST-100), a material for distinguishing whether an image photographed by the photographing unit photographs a real person or a print such as a human sculpture or a person's photo is provided.

The face identification step ST-100 includes an eye region detection step of deriving an eye region from each frame constituting the identification image, a pupil area derivation step of detecting a pupil region of an eye region (ST-120), and a frame Sculpture captures whether a real face is photographed from the eye region color information contrast step (ST-120; blink operation) in which the color information of the pixels constituting the eye region is contrasted and the change in the color information of the pixels constituting the eye region of each frame. Sculpture determination information providing step (ST-140) is provided is provided that is determined to be contrasted with the information.

As illustrated in FIG. 4, the eye region 100 is derived from each frame of the identification image. The eye region 100 may be derived by providing a mask. The mask for deriving the eye region 100 may have a size determined from the size of the face region derived as a rectangle, and the pupil region 110 as shown in FIG. 5 and the peripheral region 120 which is a white region around the pupil as shown in FIG. 5. A pixel portion of the same size as the mask having a size determined and having an arrangement most similar to the pupil region 110 and the peripheral region 120 which is a white region around the pupil in the rectangular region in the derived face region is the eye region 100. The eye region 110 is derived from the color information of the pixel in the derived eye region 100. Among the pixels constituting the pupil region, the pixel having the largest ordinate value and the smallest pixel, and the pixel having the largest abscissa value and the smallest pixel are derived, so that the characteristic points 113 and 115 of the pupil region shown in FIG. Derived. Eye region feature points 121, which are left and right endpoints of the peripheral region of the eye region, may also be derived. The horizontal center coordinates are calculated from the maximum and minimum values of the horizontal coordinates for the pupil, and the vertical center coordinates are calculated from the maximum and minimum values of the vertical coordinates, thereby deriving the pupil center point 111. It is calculated from the information of the pupil area 110 and the pupil center point 111 as described above (ST-130), and the number of eye blinks (ST-131) and the gaze direction (ST-133) are calculated.

In the calculation of the gaze direction, the direction of the face is calculated from the distance change of the facial feature point, and it is also possible to calculate the direction of the face with the gaze direction. For example, the face rotation direction may be calculated from the distance between the left and right end feature points of the left eye area and the distance between the left and right end feature points of the right eye area. When the face is rotated to the right, the distance between the left and right end feature points of the right eye region decreases and the face rotation direction, that is, the eyeline direction, may be calculated at a decreasing rate. The degree of face rotation in the vertical direction may be calculated from the distance change between the feature point of the eye region and the feature point of the lip region, and the subject may be determined whether the photographing object is a sculpture or a real face from the change of the face direction of each frame, which is the face rotation. It is possible to receive data (see Figures 8 and 9).

FIG. 5 schematically illustrates a portion of an eye region of a frame photographed with the eyes closed among the frames constituting the photographed image. Since the state of the eyes is captured, the pupil region or the peripheral region of the eye region is a pixel of the same color as the skin. Will have Meanwhile, the eye region feature point 121 may also be derived from the eye region photographed when the eyes are closed.

As shown in FIG. 5, the color change of the pixels constituting the eye region 100 is derived from the frame photographed with the eyes open and the frame photographed when the eyes are closed in FIG. 6. In the eye region 100, color information of pixels constituting the pupil regions 110 and 110a is derived. Color information of the pixels constituting the pupil region is derived for each frame, an average value of the colors of the pixels constituting the pupil region is calculated, and a change in the color information of the pixels of the pupil region with respect to the time of each frame is displayed through the display. As described above, the number of eye blinks with respect to time is calculated and derived from color information of pixels constituting the eye region in each frame (ST-131).

Specifying a threshold for the change in the average value of the colors of the pixels that make up the pupil area, information that the person is photographed when the average color value of the pupil area pixels exceeds the threshold, and that the print or sculpture is photographed if it does not exceed the threshold. May be provided through the display unit. In addition, when information that photographs or sculptures are photographed is provided, warning means (bells, warning lights, etc.) connected to the control unit may be operated through the control unit. In this case, the face comparison personal identification step (ST-200) does not proceed and ends. Be sure to

Meanwhile, as illustrated in FIG. 5, the horizontal and vertical displacements of the pupil center point 111 with respect to the frame photographed with the eyes open are derived so that the change to the frame may be displayed on the display unit. The horizontal position change and the vertical position change of the pupil center point 111 are derived from the position of the pupil center point 111 in the photographed frame, and the horizontal feature displacement of the pupil center point 111 is the eye feature point 121 and the pupil center point. It is also possible that 111 is calculated and derived.

FIG. 8 is a graph illustrating a change in gaze according to a frame (a change in face direction) when a real person is photographed, and FIG. 9 is a graph illustrating a change in gaze according to a frame (a change in face direction) when a printed matter is taken. As shown in FIG. 8 and FIG. 9, when a real person was photographed in both up, down, left, and right directions, it was confirmed that a change in gaze according to a frame was large. The gaze change may be derived from the rotation angle of the face calculated from the facial feature point.

Specify the threshold value for the line of sight change in the horizontal and vertical directions according to the frame, and judge it as a person when the maximum value of the line of change is greater than the threshold, and when the maximum value of the line of change is less than the threshold, It may also be determined that the result is displayed on the display unit. In addition, when information that photographs or sculptures are photographed is provided, warning means (bells, warning lights, etc.) connected to the control unit may be operated through the control unit. In this case, the face comparison personal identification step (ST-200) does not proceed and ends. Be sure to

Meanwhile, distance information sensed by a distance measuring sensor connected to a controller and measuring a distance from a subject is stored in a storage unit along with each frame. As described in the face region extraction step, the face region of each frame is extracted (ST-110), and the number (area) of pixels constituting the face region is calculated and stored with the distance information. The database stores information on the face area (number of pixels) according to the distance (face area area information according to the distance), and the difference is compared with the face area area according to the distance derived from a frame of the captured identification image. It may be displayed on the display unit. When a threshold value for the absolute value of the face area of the identification image frame according to the distance and the face area area difference according to the distance is specified and is larger than the threshold value, a warning means (bell, beacon, etc.) connected to the control unit is activated through the control unit. It is possible.

In the present invention as described above, before the face comparison personal identification step to determine whether the subject is a person, sculpture or printed matter in advance, by distinguishing the operation time is reduced, as well as steal access to the door and the like by stealing the information registered in the database. There is an effect that can be blocked at source.

The personal identification step ST-200 is provided with a face 3D database (hereinafter referred to as "identification face DB") including a front face image of an individual photographed by the photographing unit. Providing 3D information or a front face image facilitates identification of the photographed individual.

The personal identification step includes a face area derivation step ST-210 in which a face area is derived from an image of a photographed individual (hereinafter referred to as an “identification image”), and a face feature point derivation step in which facial feature points are derived. 220, an identification face rotation angle calculation step (ST-230) for calculating a rotation angle with respect to the first axis of the derived face region, a DB face information comparison step (ST-230), and a pixel color application step (ST) And a face DB derivation step (ST-240) in which an identification face DB, which is a face 3D database for the individual to be identified, is derived.

Deriving a face region from an image of an individual photographed by the photographing unit is well known in the art, and thus description thereof is omitted. For example, an area having a predetermined size (number of pixels) of a region of pixels of a specific color in the image of the photographed individual may be derived as the face region.

A facial specific point is derived from the derived face region (ST-220). The process of deriving a feature point from the face region has been described above by way of example, and the description thereof is also omitted in the prior art.

When the feature point of the face region is derived, the rotation angle of the face, which is the rotation angle with respect to the first axis, is calculated by the calculation of the specific point (ST-230, for information on calculating the face rotation angle, registered in Republic of Korea Registration No. 10-1215751). See patent publication, etc.).

When the face rotation angle of the face region of the identification image is calculated, it is contrasted with face image information of the 3D face database of the plurality of individuals. It is contrasted with the face image of the DB face information of the angle closest to the identification face rotation angle. For example, if the face rotation angle of the identification image is -28 °, it is contrasted with the face image of DB face information having a rotation angle of -30 °. If the face rotation angle of the identification image is a medium value such as −25 °, the face rotation angle of the identification image may be contrasted with the face image of the DB face information having the rotation angle of −30 ° or −20 °.

If the number of pixels constituting the face image of the identification image and the DB face information is different, the step of making the same size of the face image of the identification image and the DB face information is further performed. For example, if the identification image is a 2x2 pixel image and the face image of the DB face information is a 3x3 pixel image, (x1, y1), (x1, y2), (x2, y1) of the identification image ), a pixel having an average value of the color information of the (x1, y1) and (x1, y2) pixels in the (x2, y2) pixels is generated between (x1, y1) and (x1, y2), and (x1, y2) ) The coordinates of the pixel are (x1, y3), and the coordinates of the generated pixel are (x1, y2);

A pixel having an average value of the color information of the (x2, y1) and (x2, y2) pixels is generated between (x2, y1) and (x2, y2), and the coordinate of the (x2, y2) pixel is (x2, y3). ), And the coordinates of the generated pixel are (x2, y2);

A pixel having an average value of the color information of the (x1, y1) and (x2, y1) pixels is generated between (x1, y1) and (x2, y1), and the coordinate of the (x2, y1) pixel is (x3, y1). ), And the coordinate of the generated pixel is (x2, y1);

A pixel having an average value of the color information of the (x1, y2) and (x2, y2) pixels is generated between (x1, y2) and (x2, y2), and the coordinate of the (x2, y2) pixel is (x3, y2). ), And the coordinates of the generated pixel are (x2, y2);

A pixel having an average value of the color information of the (x1, y3) and (x2, y3) pixels is generated between (x1, y3) and (x2, y3), and the coordinate of the (x2, y3) pixel is (x3, y3). ), And the coordinates of the generated pixel are (x2, y3) and stored in the storage unit.

After the rotation angle is calculated, the feature point of the face region is derived again from the identification image, and a feature point such as the feature point of the DB face information is derived.

Then, the least square error value is calculated for the feature image of the identification image and the face image of each DB face information to contrast the DB face information (ST-230). Among the face images of the DB face information, the face image of the DB face information having the smallest least square error is an image to which pixel color is applied (an image of applying color information).

In the pixel color application step ST-240, the color of the pixel constituting the identification image becomes the color of the pixel constituting the color information applied image and is stored in the storage unit. And is displayed on the display unit.

In summary, the least error square value (Equation 1) for the feature points of the face image of the plurality of DB face information having the same face rotation angle as the identification image and the feature points of the identification image among the face images of the plurality of DB face information are calculated. For example, the color of the pixels forming the face image of the DB face information having the smallest minimum error square value of the identification image is changed to the color of the pixel of the identification image and stored in the storage unit. Hereinafter, the DB face information image changed to the color of the pixel of the identification image is called an "identification derived face image".

Where n is the number of feature points, x _i and y _i are the x and y axis coordinates of the i th feature point of the identification face image, x _il and y _il are the x and y axis coordinates of the i th feature point of the DB face information image , dist _rms is the least squares of error.

For example, if the face rotation angle is 30 °, the face image of the DB face information rotated 30 ° having the color of the pixel of the identification image is obtained. Identification image from face image of 30 ° rotated DB face information with color of pixel of identification image by applying to front image information and pixel position information with rotation angle of 0 ° with respect to face image of 30 ° rotated DB face information A front image of the identification face image with a rotation angle of 0 ° having a color of pixels of is obtained.

Through the above process, an identification face DB, which is a face 3D database for an individual to be identified, is derived (ST-240).

Among the face images of the plurality of DB face information having the same rotation angle as the identification image, the face image of the DB face information having the smallest least square error value for the identification image is derived and forms the face image of the derived DB face information. The color information of the pixel is changed to the color information of the pixel constituting the identification image, thereby obtaining the identification face image, and applying the obtained information of the identification face image to the DB face information of the face image to obtain 3D information of the identification face image. And frontal images are obtained.

The personal identification step may include: extracting a face region from each frame photographed to form an identification image of an individual stored in a storage unit; extracting a face feature vector from which a feature vector of the extracted face region is derived; The Euclidean distance between the feature vector of the extracted facial region and the facial feature vector of the data registered in the database may be compared with the facial feature vector.

In the facial feature vector derivation step, a facial region feature vector is derived from the facial region information extracted and frontized in the facial region extraction step, and in the facial feature vector contrast step, the facial feature vector of the captured identification image and the database are registered in the database. The Euclidean distance of the facial feature vector of the data is calculated. Since the method of deriving the facial region feature vector and the Euclidean distance calculation are conventional techniques, description thereof will be omitted.

The Euclidean distance for the facial feature vector of the photographed identification image and the data for the plurality of individuals registered in the database is calculated. The Euclidean distance becomes face similarity. The identification image of the individual photographed together with the calculated Euclidean distance and the image of the individual registered in the database are displayed on a display unit (eg, an LCD panel; not shown) connected to the control unit. Class information is divided into 4 classes based on Euclidean distance, and information about the class is also displayed.

The similarity divided by the experimental value is shown in Table 2 below.

Similarity 0 to 0.25 or less 0.25 (greater than) to 0.5 (less than) 0.5 (greater than) to 0.75 (less than) 0.75 (greater than) to 1 ranking 1 (same person) 2 (similar person) 3 (need to confirm) 4 (non-identical)

As it is provided in the above and the table, it is possible to check and manage the passage of pass in the case of the 1st class, the case of the 2nd and 3rd grade in case of the 1st class when it is installed and operated at the door, and to manage it separately in the case of the 4th grade. Do.

In the facial feature vector, a composite product neural network module is executed, and a face feature vector of the identification image is derived by applying a previously learned personal identification weighting value from the training photographing image.

The personal identification method through face comparison according to the present invention further includes obtaining face information in addition to the face comparison personal identification step. The face information acquiring step includes a learning step and a face information calculation step to provide race information and gender information of the identification image.

The learning step includes a step of forming a convolutional layer of a training photographing image of gender information, a step of forming a pooling layer from a map of a convolutional layer, and information of pooling of a pooling layer as input information, including hidden layer weights and inputs. Hidden layer information operation step of calculating hidden layer node values by operation of information, output layer information calculating step of calculating an output layer by calculation of hidden layer node value and output layer weight, and error calculation from the output layer information and training photographing image information, and backpropagation algorithm. The updated values of the hidden layer weights and the output layer weights are calculated, and the hidden layer information calculating step and the output layer information calculating step are recalculated, and the process of comparing the calculated output layer information with the photographed image information for training is repeated. Weight from which the update value of Derivation phase

In the present invention, 40,000 people (50-100 photographs per person) were prepared by learning training images. Race information and gender information for each of the 40,000 people are stored in the database along with training footage.

 A map is formed of a convolutional layer by using a kernel for each training photographed image. Color information of each pixel constituting the training shot image becomes a component of the training shot image matrix. Before forming the map, it is also possible to convert the training photographed image into a black and white image.

The kernel may be a matrix having various sizes, such as a 2 × 2 matrix, a 3 × 3 matrix, and a 5 × 5 matrix. The inventors formed a map using the 3 × 3 matrix as a kernel, and formed a convolutional layer. Five hidden layers were formed, and in the number of kernels, the first hidden layer had 16 kernels, the second hidden layer 32, the third hidden layer 64, the fourth hidden layer 128, and the fifth hidden layer 512. To form a hidden layer. In the formation of the hidden layer, the accuracy is improved by increasing the number of hidden layers by increasing the number of kernels toward the output layer.

The values of the components of the kernel can be generated using random functions, and the values of each component should not exceed two. A plurality of kernels are provided, and a map for each kernel is provided to form a convolutional layer.

 For example, if the kernel is a 2 × 2 matrix and the components are 1, 0, 0, and 1, the kernel is calculated by mapping the kernel to the first 2 × 2 pixels of the captured training image, and the component values forming the map are calculated. The map component values are sequentially calculated by shifting pixel by pixel along the pixels of the training photographed image to prepare a map that is a matrix. A kernel having a plurality of different component values as described above is provided, and a map for each kernel is provided to form a convolutional layer. If the kernel is a 2x2 matrix and the size of the training photographic image is 4x4 pixels, the map is formed into a 3x3 matrix size.

When the convolution layer is formed, each map constituting the convolution is outputted through an active function. In the present invention, the ReLU function is adopted as the activation function. It is also possible to make the sigmoid function the active function.

A plurality of pools of the pooling layer is formed, and pooling is formed on an output value of each map constituting the convolutional layer. The pooling may be pooled to an average value or to a maximum value. The pooling may be performed for a 2 × 2 matrix size, and the maximum value or the average value of the components having a 2 × 2 matrix size from the components of the map is the component value of the pooling. Therefore, when a map of 4x4 matrix size is pooled to 2x2 matrix size, it becomes a matrix of 2x2 matrix.

Each pooling component becomes an input value and is calculated based on the weight of the hidden layer to calculate the node value constituting the hidden layer.

Equation 2 is an example of calculating each node of the hidden layer, in which m is the number of components forming a pool, Ej is a hidden layer node value, pi is a component of pooling, wij is a weight, and po is 1 as a bias. And φ is an active function. The sigmoid function is used as the activation function. The weight wij may be any value selected from a value between 0 and 2.

The hidden layer may be formed of one or more layers. When the hidden layer is formed of two layers, Equation 2 is a formula in which the node value of the first hidden layer is calculated, and the node value of the second hidden layer is calculated by the node value and the weight of the first hidden layer. The weighting value used in calculating the first node value and the weighting value used in calculating the second node value may be different from each other. As described above, in the case of the plurality of hidden layers, calculations are performed in sequence.

The output layer node value is calculated by the node value and the weight value of the last hidden layer. Since gender is male and female, there are two output layer nodes.

Equation 3 is an example of calculating a node of an output layer, where Tj is an output layer node value, Ei is a node value of the last hidden layer, vij is a weight value, and Eo is 1 as a bias. And φ is an active function. ReLU is used as an active function. The weight vij may be any value selected from a value between 0 and 2.

For example, if the training shot image is male and the node value of the output layer calculated through the above process for the training shot image is larger than 0.5, and the training shot image is female and the training shot image is processed through the above process. When the node value of the output layer is smaller than 0.5, the hidden layer node value calculation weight and the output layer calculation weight are stored in the storage unit. Through the above steps, the output node value is calculated for all of the prepared training images, and the accuracy of the weight is checked. In the classification, if it is larger than 0.5, it can be classified as a male, and if it is smaller than 0.5, it can be classified as a female (and vice versa).

On the other hand, if the training photographing image is male and the output layer node value calculated by the above steps is 0.3, the minimum error is 0.7, and the error value is based on the backpropagation algorithm (the detailed description is omitted since it is a conventional technology). The process of updating the weights of the output layer and the hidden layer, applying the updated weights to calculate the output layer node values again, checking the calculated output layer node values, and updating the weights again if there is an error for the captured images for training. Apply to derive the updated weight.

The output layer node value is calculated through the above process with respect to the identification image. For example, if the output layer node value for the identification image is greater than 0.5, it is classified as a male, and if it is less than 0.5, it is classified and stored.

The race information is calculated through the above process, and the output layer node is set to be the same as the race type. The sigmoid function is used as the active function for the hidden layer node operation, and the softmax function is used as the active function for the output layer node operation. In the case of being classified into three races, the sum of the output layer node value calculation results is 1, the race is classified according to the node value having the maximum value of the three output layer node values, and the larger the maximum value, the higher the accuracy.

According to the personal identification method through the face comparison according to the present invention described above and the personal identification system on which the method is executed, the sculpture or the printout is distinguished from the real face, so when applied to an access device or a security device, the sculpture or the printout having the malicious purpose Unauthorized entry and exit using the back is blocked, the same person and the same person can be identified by the similarity level, and race, gender, and age information is provided to improve business efficiency.

Claims (9)

Is executed in a personal identification system comprising a photographing unit, a control unit connected to the photographing unit, an identification image consisting of a plurality of identification images connected to the control unit, and a storage unit storing a personal identification program, In contrast to the registered DB face information; And a personal identification step, wherein the personal identification step includes a facial area derivation step of deriving a face area from an image of an individual photographed, a facial feature point derivation step of deriving facial feature points, and a rotation angle of the derived face area. An operation for calculating the identification face rotation angle, a DB face information contrast step in which a plurality of DB face information images and an identification image are contrasted, a pixel color application step, and an identification face DB which is a face 3D database for the individual to be identified are derived. Consisting of the face DB derivation step; Personal identification method through face comparison, characterized in that the information for identifying the photographed individual is provided. The method of claim 1, wherein the DB face information contrasting step calculates a minimum error square value of a plurality of DB face information images for the identification image, and in the pixel color applying step, the smallest minimum of the identification image among the DB face information images. And a color information of pixels constituting a DB face information image having an error squared value is converted into an identification image and stored in a storage unit. The method of claim 1, further comprising a face identification step before the personal identification step; The face identification step includes eye area detection step of eye area derived from each frame constituting the identification image, eye area derivation step of detecting eye area of eye area, and color information of pixels constituting eye area of each frame. Comparing the eye region color information to be used, and the object comparison information comprising the step of providing the information to determine whether or not the actual face or sculpture is photographed from the change in the color information of the pixels constituting the eye region of each frame Personal identification method through. The method of claim 3, wherein A pupil area pixel information calculating step further includes a face area deriving step of deriving a face area of a plurality of frames photographed before the eye area is detected, and wherein pixel information constituting a pupil area in each frame is calculated after the eye area derivation step. Further comprises; And the color information of the eye region pixels of each frame is contrasted in the color information comparison step. According to claim 1, Further comprising a face identification step before the personal identification step; The face identification step may include: an eye area detection step in which an eye region is derived from each frame constituting an image captured by the photographing unit, a face area feature point derivation step in which feature points of a face area of each frame constituting the captured image are derived, and an eye; A pupil area derivation step of detecting a pupil area of an area, a pupil area location derivation step of deriving a pupil area location, and a pupil location from which a distance between a pupil area location and a facial area feature point derived in the pupil area location derivation step is calculated. And a distance calculation step and a sculpture discrimination information providing step of providing information determining whether a real face is photographed or a sculpture is photographed based on a change in pupil position distance of each frame. The method of claim 1, further comprising a face identification step before the personal identification step; The face identification step includes deriving a face region from each frame constituting the image captured by the photographing unit, calculating a direction of the face region, and photographing whether a real face is photographed from a change in the face region direction of each frame. Personal identification method according to the face comparison characterized in that it comprises the step of providing sculpture identification information provided information is determined. The system of claim 1, wherein the system further comprises a distance measuring sensor connected to a control unit, wherein each frame is stored in a storage unit with distance information measured from the distance measuring sensor; Further comprising a face identification step before the personal identification step; The face identification step includes a face area derivation step in which a face area is derived from each frame constituting the image photographed by the photographing unit, a face area information calculation step in which face area information derived in the face area derivation step is calculated, and each frame Personal identification method according to the face comparison characterized in that it comprises the step of providing the sculpture identification information is provided from the face region information and distance information to determine whether or not the actual face is photographed. 8. The method of any one of claims 1 to 7, further comprising providing additional information in addition to the personal identification step; The additional information providing step includes a learning step and a face information calculation step, wherein racial information and gender information of the identification image are provided; The learning step includes the step of forming a convolutional layer of the training photographing image for each of the race information and the gender information, the step of forming a pooling layer from a map of the convolutional layer, and the information of the pooling of the pooling layer. Hidden layer information operation step of calculating hidden layer node values by operation of hidden layer weight and input information, output layer information calculating step of calculating output layer by operation of hidden layer node value and output layer weight, and error from output layer information and training photographing image information. The updated values of the hidden layer weights and the output layer weights are calculated according to the backpropagation algorithm, and the hidden layer information calculation step and the output layer information calculation step are repeatedly calculated, and the process of comparing the calculated output layer information with the photographed image information for training is repeated. Of the update value and output layer weight Made of a weighting stage which is derived singap is derived; And a hidden layer weight update value and an output layer weight update value derived in the learning step are hidden layer weights and output layer weights for race information and gender information in the face information calculation step, respectively. The method of claim 8, wherein the calculating of the face information comprises: forming a convolutional layer of the identification image for each of the race information and the gender information, forming a pooling layer from a map of the convolutional layer, and pooling the pooling layer. Is composed of input information, a hidden layer information calculating step of calculating hidden layer node values by calculating hidden layer weights and input information, and an output layer information calculating step of calculating an output layer by calculating hidden layer information and output layer weights; The node forming the output layer in the operation for obtaining race information is equal to the number of races, and the node forming the output layer in the operation for obtaining gender information is 2.

Images