[go: up one dir, main page]

WO2015008567A1 - Procédé, dispositif et programme d'estimation d'impression faciale - Google Patents

Procédé, dispositif et programme d'estimation d'impression faciale Download PDF

Info

Publication number
WO2015008567A1
WO2015008567A1 PCT/JP2014/065823 JP2014065823W WO2015008567A1 WO 2015008567 A1 WO2015008567 A1 WO 2015008567A1 JP 2014065823 W JP2014065823 W JP 2014065823W WO 2015008567 A1 WO2015008567 A1 WO 2015008567A1
Authority
WO
WIPO (PCT)
Prior art keywords
impression
data
degree
estimation
classes
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2014/065823
Other languages
English (en)
Japanese (ja)
Inventor
康行 伊原
将 杉山
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
NEC Solution Innovators Ltd
Tokyo Institute of Technology NUC
Original Assignee
NEC Solution Innovators Ltd
Tokyo Institute of Technology NUC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by NEC Solution Innovators Ltd, Tokyo Institute of Technology NUC filed Critical NEC Solution Innovators Ltd
Priority to JP2015527224A priority Critical patent/JP6029041B2/ja
Publication of WO2015008567A1 publication Critical patent/WO2015008567A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V40/00—Recognition of biometric, human-related or animal-related patterns in image or video data
    • G06V40/10—Human or animal bodies, e.g. vehicle occupants or pedestrians; Body parts, e.g. hands
    • G06V40/16—Human faces, e.g. facial parts, sketches or expressions
    • G06V40/174—Facial expression recognition
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/042—Knowledge-based neural networks; Logical representations of neural networks

Definitions

  • the present invention relates to a method, an apparatus, and a program for estimating an attribute of a person, and more particularly, to a method, an apparatus, and a program for estimating an impression level of a person from a face image of the person.
  • Patent Document 1 discloses an “age estimation apparatus, method, and program” that can obtain a recognition result close to the result perceived by humans.
  • the age estimation device disclosed in Patent Document 1 when an age estimation model is created by regression analysis, the younger generation's estimation accuracy is improved by strengthening the younger group's learning weight.
  • a kernel regularized weighted least squares (KRWLS) is used in a supervised regression problem that predicts the true age of test data from which feature vectors are extracted.
  • the age estimation function is modeled by a linear combination of positive definite kernels.
  • a classifier having a high learning efficiency called a least square probabilistic classifier (LSPC) is also known (see, for example, Non-Patent Document 1 and Non-Patent Document 2).
  • LSPC is an identification method for learning a class posterior probability model under a square loss, and its greatest feature is that a solution can be calculated analytically.
  • LSPC directly estimates the posterior probability for each class in the form of a density ratio, it also has a feature that it is resistant to an imbalance in the number of learning data of each class. In LSPC, the posterior probability is learned using the square loss.
  • ranking learning is a technique of optimization based on supervised learning so that a high score can be given to data according to the degree of relevance and permutation.
  • ranking SVM Ranking Support Vector Machine
  • impressions degree each impression is evaluated (estimated) in a plurality of stages (for example, five stages).
  • stages for example, five stages.
  • This problem is a universal problem peculiar to regression analysis in which it is difficult to ensure the estimation accuracy of the values of the dependent variable (objective variable) for data near both ends, and facial images corresponding to impressions at both ends are sufficient. This is thought to be caused by data collection problems such as difficulty in collecting data. Impressions at both ends (very cute, not pretty at all) are more memorable to human memory than average impressions (average cuteness). What can be estimated is important. Even in the LSPC disclosed in Non-Patent Document 1 and Non-Patent Document 2, there is a problem that the estimation accuracy of impression degree at both ends of a person is poor. In addition, the conventional optimization method using ranking learning has a problem that it is difficult to reflect the degree of difference between two dependent variables (object variables) in the optimization problem.
  • an object of the present invention is to provide a method, an apparatus, and a program for accurately estimating the degree of impression of both ends of a person.
  • One aspect of the present invention is an impression level estimation method for estimating an impression level of a person captured in image data using an impression level estimation device, an extraction step of extracting a face image from image data, and a face image
  • the calculation process of calculating the facial feature vector from the model, and the magnitude relationship between the impressions between the two samples are modeled by a class of least-squares stochastic discriminators, and test data and learning data with the facial image vector as an explanatory variable have already been used.
  • Estimating the impression of a person by scoring based on the certainty of each class obtained by sequentially comparing the obtained plurality of comparison base data with a plurality of ranking least square probabilistic classifiers.
  • the effect of the present invention is that the degree of impression of both ends of a person can be accurately estimated.
  • FIG. 1 is an image diagram of scoring used in the impression degree estimation method according to the embodiment of the present invention.
  • FIG. 2 is a block diagram showing a configuration of an impression degree estimation apparatus according to the first embodiment of the present invention.
  • FIG. 3 is a block diagram showing a configuration of a neural network (continuous amount estimation means) in regression analysis mode used in the impression degree estimation apparatus of FIG.
  • FIG. 4 is a block diagram showing a configuration of a ranking least square probabilistic discriminator (order estimation means) used in the impression degree estimation apparatus of FIG.
  • FIG. 5 is a block diagram showing the configuration of an impression degree estimation apparatus according to the second embodiment of the present invention.
  • FIG. 6 is a block diagram showing a configuration of a ranking least square probabilistic discriminator (order estimation means) used in the impression degree estimation apparatus of FIG.
  • FIG. 7 is a block diagram showing a configuration of an impression degree estimation apparatus according to the third embodiment of the present invention.
  • FIG. 8 is a table showing the experimental results of the recognition rate of each class in each method when “pretty impression” is solved as an identification problem.
  • FIG. 9 is a table showing the experimental results of the absolute error average (MAE) in each method when “pretty impression” is solved as a regression problem.
  • MAE absolute error average
  • LSPC described in Non-Patent Documents 1 and 2 will be described as a first related technique.
  • LSPC is an identification method for learning a class posterior probability model under a square loss, and its greatest feature is that a solution can be calculated analytically.
  • LSPC directly estimates the posterior probability for each class in the form of a density ratio, it also has a feature of being strong against imbalance in the number of learning data of each class. This feature of LSPC is advantageous when solving ranking LSPC according to embodiments of the invention described below.
  • x) of class y in the input vector (face feature amount) x is estimated in the form of the density ratio of the following equation (1).
  • p (x) is the probability density of the sample
  • p (x, y) is the joint probability density.
  • the class with the maximum posterior probability is set as the estimated class represented by the following equation (2).
  • x) is learned using the square loss.
  • x) of class y is modeled by a linear model expressed by the following equation (3).
  • the parameter ⁇ ( ⁇ 1 ,..., ⁇ b ) T is learned so that the square error J 0 expressed by the following equation (4) is minimized.
  • T is a transposed matrix.
  • LSPC finally, a normalization correction is performed so that the sum of the posterior probabilities of all classes becomes 1, thereby obtaining a solution of the posterior probabilities as shown in the following equation (6).
  • the LSPC has a problem that the estimation accuracy of the impression degree at both ends of a person is poor.
  • formulating ranking learning will be described as a second related technique. As described above, ranking learning is a technique of optimization in a supervised learning framework so that a high score can be given to data according to the degree of relevance and permutation.
  • the data space is ⁇
  • n supervised learning data
  • the vector x i is an explanatory variable (feature amount of the face image)
  • the scalar y i is an objective variable (representing the magnitude of the impression degree).
  • the degree of impression of “pretty” the range of y i is set to 0.0 to 4.0 (not pretty: 0.0, very cute: 4.0), and the degree of “pretty” is large. As the number increases, take it.
  • the score function f: ⁇ ⁇ R is modeled by a linear combination of positive definite kernels k (x ′, x) as represented by the following equation (8).
  • a set may be used.
  • ⁇ (x) (k (x 1 , x),... K (x n , x)) T.
  • the main problem of ranking SVM is formulated in the form of the following equation (10). Is a slack variable introduced in consideration of the case where separation is not possible.
  • the impression degree estimation method according to the present embodiment models the “size relationship between two specimens” by multi-class LSPC (introduction of ranking LSPC), and based on the certainty of the class output by LSPC, By performing output scoring, the degree of impression at both ends of the person is accurately estimated.
  • LSPC which is a multi-class probabilistic classifier
  • the impression degree estimation method according to the present embodiment is based on the model of the magnitude relation between the two samples by focusing on “the magnitude relation between the two specimens”, which is relatively easy to obtain a correct answer.
  • This is a technique for improving the estimation accuracy for the face image corresponding to the impression degree at both ends by summing up the estimation results using this model (the magnitude of the impression degree when compared with bearwise).
  • the conventional ranking learning method described in the second related technology and the ranking learning method according to the present embodiment.
  • the conventional ranking learning method when considering a pair-wise approach (for example, an approach considered by comparison between two data), the problem is solved by reducing to a two-class identification problem. Therefore, the conventional ranking learning method only identifies which is “upper or lower”. Also, conventional ranking learning methods often use SVM to solve.
  • the ranking learning method according to the present embodiment when considering a pair-wise approach (for example, an approach considered by comparison between two data), it is solved by reducing to a multi-class identification problem. Therefore, the ranking learning method according to the present embodiment takes into consideration not only the determination of which is “up or down” but also the magnitude of the difference.
  • the ranking learning method according to the present embodiment employs LSPC.
  • the ranking learning is reformulated in the following form.
  • the difference (y i ⁇ y j ) of the dependent variable (object variable) is set according to the degree of magnitude, Classify into C classes.
  • C 2 classes
  • L (y i , y j ) represents the class label to which the difference (y i ⁇ y j ) belongs.
  • This score function may be considered as a generalization of the special case (with linearity) of the conventional score function in the second related technique.
  • the linear kernel of the positive definite kernel of the score function f is used, and the difference of the dependent variable (target variable) is classified into two classes as in the above equation (12). Think about the case.
  • each of the two sets of score functions is expressed by the following equation (15). Note that ranking learning results in an optimization problem expressed by the above equation (14).
  • a pair-wise approach that considers the loss to the pair of learning data is conveniently adopted.
  • a ranking LSPC described below is considered, and the score function is optimized by reducing to a multi-class identification problem.
  • ranking LSPC will be described.
  • the ranking LSPC is based on the pair-wise approach, similar to the ranking SVM described in the second related technology, and is based on the LSPC described in the first related technology by considering the loss to the pair of learning data.
  • the score function f k expressed by the following equation (16) is optimized by reducing to the multi-class identification problem.
  • LSPC is an identification method for learning a posterior probability model of a class under a square loss, and the number of learning times is reduced while maintaining the same pattern recognition accuracy as the conventional method (second related technology). Can be shortened by a factor of 100.
  • the ranking learning based on the pair-wise approach the number of learning data increases dramatically.
  • the original learning data is n
  • the learning data used in the ranking learning is n 2 times. This good learning efficiency of LSPC can alleviate the problem of large-scale sample number in ranking learning.
  • the LSPC since the posterior probability is modeled for each class, the LSPC also has a feature of being strong against unbalance of learning data of each class. When considering the problem of classifying according to the difference of the dependent variable (objective variable), imbalance tends to occur in the number of learning data of each class. However, this feature of LSPC is advantageous to mitigate the negative impact on the accuracy of recognition of the number of data imbalances.
  • the score function for each class can be modeled as described above. In particular, in the LSPC, when considering multi-class ranking learning, which will be described later, it is easy to obtain an “output with a high accuracy rate” depending on the class.
  • the output of ranking learning is normally only the output of the class identification result (predicted class), and does not include information on the reliability of the output.
  • the class identification result is output with the certainty factor of each class, as will be described later, it is convenient when calculating the ranking learning outputs and solving the regression problem.
  • the formulation of ranking LSPC will be described. In other words, the “learning phase” of ranking LSPC will be described. (Object variable)) and the difference (y i -y j ) of the dependent variable (object variable) are classified into C classes according to the degree of magnitude.
  • the vector x i is an explanatory variable representing the feature quantity of the face image
  • y i is an objective variable (dependent variable) representing the numerical value of the degree of impression.
  • the value of y i is a value (known value) that has already been determined as an impression level subjectively (for example, by majority vote).
  • the score function of each class Each is modeled by a linear combination of positive definite kernels ⁇ i, j as represented by the following equation (17).
  • Minimum square error J 0 to learn so as to become a minimum that. It is the same as PC.
  • the “learning phase” of ranking LSPC Next, a technique for solving the regression analysis problem based on the estimation result of ranking LSPC, which is the core part of the impression degree estimation method according to the embodiment of the present invention, will be described.
  • the “recognition (estimation) phase” using ranking LSPC will be described.
  • the feature variable (vector) xte of the test data is a face feature vector calculated by a feature vector calculation unit (described later) from the face image.
  • the objective variable y te of the test data is an unknown variable.
  • the calculation load at the time of model learning can be adjusted by appropriately changing the size of the comparison basis data number b.
  • the impression level y has a lower limit value 0.0 and an upper limit value 4.0, that is, 0.0 ⁇ y ⁇ 4.0.
  • the class classification of the difference of the dependent variable shall be according to the above formula 22.
  • the output of ranking LSPC when a pair of test data (x te , y te ) and comparison base data (b 1 , y 1 ) is input, and the certainty of class 1 and class 2 is 0.7 and 0, respectively. 3 ′ (the probability that y te ⁇ y 1 is 0.7).
  • the impression score of the vector format represented by the following formula (19) Is assigned as shown in the following equation (20) according to the value of the impression degree y (how to obtain the value of y may be arbitrarily determined, but it is desirable to obtain the value so as to be dense with respect to the entire range of y) ).
  • FIG. 1 is an image diagram of this scoring.
  • the horizontal axis represents the impression score
  • the vertical axis represents the certainty factor.
  • Impression score can range from 0.0 to 4.0 Normalization processing is performed on the vector g 1 represented by Next, for the other comparison basis data (b 2 , y 2 ), (b 3 , y 3 ) Normalize to 3 .
  • the output of ranking LSPC when a pair of test data (x te , y te ) and comparison base data (b 1 , y 1 ) is input the certainty of class 1 to class 4 is 0.65, 0 , 2, 0.1, 0.05 ′ (the probability that y te ⁇ y 1 ⁇ 0.5 is highest (0.65)).
  • the impression score of the vector format represented by the following formula (24) Is assigned according to the value of impression degree y as in the following equation (25).
  • scoring is performed for each comparison base data, and an estimated value of the impression degree for the test data (x te , y te ) is obtained.
  • the impression level estimation device 10 shown in the figure is a device that evaluates (estimates) the following five types of impressions of a person in five levels (5 classes) of “0, 1, 2, 3, 4”. 1. 1. Bright and refreshing 2. Cute Business 4. 4. Easy. Healthy And, for example, when expressing the impression of “cute”, the range of the objective variable (dependent variable) y described later is 0.0 to 4.0 (“not cute at all” is set to 0.0, “very cute” ”Is defined in 4.0), and the numerical value increases as the degree of“ pretty ”increases. In the first embodiment, the above five types are given as the impression of the person.
  • the impression level estimation device 10 classifies the impression level (objective variable) y into the following five classes (evaluated in five stages) according to the magnitude. (Class 1) 0.0 ⁇ y ⁇ 0.5 (Class 2) 0.5 ⁇ y ⁇ 1.5 (Class 3) 1.5 ⁇ y ⁇ 2.5 (Class 4) 2.5 ⁇ y ⁇ 3.5 (Class 5) 3.5 ⁇ y ⁇ 4.0
  • the impression level is classified into five classes (evaluated in five stages), but the present invention is not limited to this, and is classified into two or more classes (evaluated in two or more stages). Of course, this may be done.
  • the impression level estimation device 10 includes a head detection unit 12, a face detection unit 14, a neural network 16 in a regression analysis mode, and a ranking least square probabilistic classifier (ranking LSPC) 18.
  • the head detection unit 12 estimates the positions of eyes, nose, and mouth from a color still image file obtained by photographing a person using a head detection program.
  • the face detection unit 14 extracts a face image normalized to 64 ⁇ 64 pick cells based on position information of face parts such as eyes. Therefore, the combination of the head detection unit 12 and the face detection unit 14 serves as a data acquisition unit (face image extraction unit) 32 that extracts a face image from a person image.
  • the neural network 16 comprises an input layer 162, an intermediate layer 164, and an output layer 166, as shown in FIG.
  • the data (intermediate layer data) of the intermediate layer 164 of the neural network 16 is supplied to a ranking least square probabilistic classifier (ranking LSPC) 18 as a face feature vector. Therefore, the combination of the input layer 162 and the intermediate layer 164 of the neural network 16 serves as a feature vector calculation unit 34 that calculates a face feature vector from the extracted face image.
  • the feature vector calculation unit 34 in the neural network 16 is used as means for calculating the face feature vector.
  • the present invention is not limited to this, and other well-known various types. Of course, the face feature vector calculating means may be used.
  • the ranking least square stochastic discriminator 18 shown comprises two classes of ranking least square stochastic discriminators. Therefore, the ranking least-squares stochastic discriminator 18 outputs y * expressed by the above equation (23) as an impression degree estimation result when test data (x te , y te ) is input.
  • FIG. 4 is a block diagram showing the configuration of the two-class ranking least squares probabilistic classifier 18.
  • the two-class ranking least-squares probabilistic classifier 18 includes a magnitude comparison / determination unit 182 using a two-class ranking LSPC, a scoring processing unit 184, and a score sum processing unit 186.
  • test data (x te , y te ) and a plurality of comparison base data (x 1 , y 1 ) are supplied to the magnitude comparison determination unit 182 using two classes of ranking LSPC.
  • the test data (x te , y te ) is intermediate layer data (face feature vector) when the test data in the face image format is input to the neural network 16 in the regression analysis mode.
  • this is used as an explanatory variable for test of test data and expressed as xte .
  • an unknown test objective variable for example, a correct value of the impression degree related to “adorable” is represented by y te .
  • the plurality of comparison basis data (x 1 , y 1 ) is an intermediate layer when the comparison basis data in the face image format (a plurality obtained from the learning data) is input to the neural network 16 in the regression analysis mode.
  • Data face feature vector
  • These are already obtained at the model learning stage of the two classes of ranking LSPC, and are not newly calculated at the stage of estimation processing for test data.
  • these are used as comparison explanatory variables of the comparison base data, and are referred to as x 1 , x 2 ,..., X k ,.
  • correct values of impression degree regarding “pretty”) are respectively represented by y 1 , y 2 ,. .., y k ,...
  • Magnitude comparison determination unit 182 using the ranking LSPC of 2 classes using a ranking LSPC two classes, the comparative base data (x k (a x te with the test explanatory variable) test data and the comparative explanatory variables It is determined whether the impression level is “have” (the same processing is repeated for all comparison base data x k ).
  • the size comparison determination unit 182 using the rank LSPC of two classes has a difference (x te ⁇ x) between the test data and the comparison base data.
  • the magnitude comparison / determination unit 182 using two classes of ranking LSPC outputs the magnitude determination result in the form of confidence of each class. For example, in the case of comparison between x te and x k, magnitude comparison determination unit 182 using the ranking LSPC of 2 classes, for example, Class 1, confidence of class 2 are respectively 0.7,0.3 It outputs as there is (refer FIG. 1).
  • the scoring processing unit 184 performs scoring processing according to the above equations (19) to (21) using the magnitude determination comparison results output for each comparison base data.
  • the score total processing unit 186 adds all the scores calculated for each comparison base data, and outputs the portion where the calculated score is the maximum as the estimation result of the impression degree as shown in the above formula 44.
  • the impression degree estimation result is “estimation result of objective variable (for example, impression degree estimation result regarding“ strange ”) by two classes of ranking LSPC when test data x te is input”.
  • the impression degree estimation apparatus 10 can be realized by a computer that operates by program control.
  • this type of computer includes an input device for inputting data, a data processing device, an output device for outputting processing results in the data processing device, and an auxiliary memory serving as various databases. Device.
  • the data processing device stores the program in a read-only memory (ROM), a random access memory (RAM) used as a work memory for temporarily storing data, and a program stored in the ROM. It consists of a central processing unit (CPU) that processes stored data.
  • the input device operates as a device (not shown) for inputting a person image (image data).
  • the data processing apparatus operates as a data acquisition unit 32, a neural network (continuous quantity estimation unit) 16 in regression analysis mode, and a ranking least square probabilistic classifier (order estimation unit) 18.
  • the auxiliary storage device functions as storage means (not shown) for storing a plurality of comparison base data.
  • each unit of the impression level estimation device 10 may be realized using a combination of hardware and software.
  • each unit is realized as various means by operating hardware such as a control unit (CPU) based on an impression degree estimation program stored in the ROM.
  • the impression degree estimation program may be recorded on a recording medium and distributed.
  • the impression degree estimation program recorded on the recording medium is read into the memory via the wired, wireless, or recording medium itself, and operates the control unit and the like.
  • Examples of the recording medium include an optical disk, a magnetic disk, a semiconductor memory device, and a hard disk.
  • the impression level estimation device 10 having such a configuration can accurately estimate the impression levels of both ends of a person.
  • the illustrated impression level estimation apparatus 10A has the same configuration as the impression level estimation apparatus 10 shown in FIG. 2 and operates except that the configuration of the ranking least squares stochastic discriminator is different. Therefore, the reference sign of 18A is attached to the ranking least squares stochastic discriminator. Constituent elements similar to those shown in FIG. 2 are denoted by the same reference numerals, and description thereof is omitted for the sake of simplicity.
  • the ranking least square stochastic discriminator 18A shown in the figure is composed of four classes of ranking least square stochastic discriminators.
  • FIG. 6 is a block diagram showing a configuration of a 4-class ranking least squares probabilistic classifier 18A.
  • the 4-class ranking least-squares probabilistic classifier 18A includes a magnitude comparison / determination unit 182A using a 4-class ranking LSPC, a scoring processing unit 184A, and a score sum processing unit 186. That is, the 4-class ranking least-squares stochastic discriminator 18A is different from the 2-class ranking least-squares stochastic discriminator 18 in FIG. 4 between the magnitude comparison determination unit and the scoring processing unit using ranking LSPC. Configuration and operation are different. Hereinafter, only the differences will be described for the sake of simplicity.
  • the size comparison determination unit 182A using the ranking LSPC classes 4 using class ranking LSPC determines whether the impression level is “have” (the same processing is repeated for all comparison base data x k ).
  • the size comparison determination unit 182A using the ranking LSPC of 4 classes determines the difference (x te ⁇ x) between the test data and the comparison base data.
  • the magnitude comparison / determination unit 182A using the four classes of ranking LSPC outputs the magnitude determination result in the form of the certainty factor of each class. For example, in the comparative case of x te and x k, magnitude comparison determination unit 182A using the ranking LSPC of 4 classes, for example, confidence of class 1 to class 4, respectively 0.65,0.2,0 ., Output as 0.05.
  • the scoring processing unit 184A performs scoring processing according to the above formulas (24), (25), and (21) using the magnitude determination comparison results output for each comparison base data. Then, the score total processing unit 186 adds all the scores calculated for each comparison base data, and, as shown in the above equation (23), the location where the calculated score is the maximum is obtained as the impression degree estimation result. Output.
  • the impression degree estimation result is “an estimation result of an objective variable by ranking LSPC when test data x te is input (for example, impression degree estimation result regarding“ strange ”)”.
  • the impression degree estimation apparatus 10A according to the second embodiment is configured by an electronic device, the impression degree estimation apparatus 10A can be realized by a computer that operates under program control.
  • this type of computer includes an input device for inputting data, a data processing device, an output device for outputting processing results in the data processing device, and an auxiliary memory serving as various databases.
  • the data processing device stores the program in a read-only memory (ROM), a random access memory (RAM) used as a work memory for temporarily storing data, and a program stored in the ROM. It consists of a central processing unit (CPU) that processes stored data.
  • the input device operates as a device (not shown) for inputting a person image (image data).
  • the data processing apparatus operates as a data acquisition unit 32, a neural network (continuous amount estimation unit) 16 in regression analysis mode, and a ranking least squares probabilistic discriminator (order estimation unit) 18A.
  • the auxiliary storage device functions as storage means (not shown) for storing a plurality of comparison base data.
  • each unit of the impression level estimation device 10A according to the second embodiment may be realized using a combination of hardware and software.
  • each unit is realized as various means by operating hardware such as a control unit (CPU) based on an impression degree estimation program stored in the ROM.
  • the impression degree estimation program may be recorded on a recording medium and distributed.
  • the impression degree estimation program recorded on the recording medium is read into the memory via the wired, wireless, or recording medium itself, and operates the control unit and the like.
  • Examples of the recording medium include an optical disk, a magnetic disk, a semiconductor memory device, and a hard disk.
  • the impression level estimation device 10A having such a configuration can accurately estimate the impression levels of both ends of a person.
  • the feature vector calculation unit 34 in the neural network 16 is used as means for calculating the face feature vector.
  • the present invention is not limited to this, and various other known face feature vectors. Calculation means may be used.
  • the impression level estimation device 10B shown in the figure is the impression level estimation device 10A shown in FIG. 5 except that a least square probabilistic classifier (LSPC) 20 and a weighted integration unit 22 for classifier output are further added. It operates in the same way as the above. Therefore, the same components as those shown in FIG. 5 are denoted by the same reference numerals, and the description thereof is omitted for the sake of simplicity. As described above, by introducing a multi-class ranking LSPC, the accuracy of the impression level at both ends of the person is improved, but the accuracy of the impression level near the middle of the person is not good.
  • LSPC least square probabilistic classifier
  • the neural network 16 in the regression analysis mode is a neural network in the regression mode that is solved as a regression problem that estimates the impression level of a person as a continuous quantity.
  • the neural network 16 in the regression analysis mode is a neural network in the regression mode that is solved as a regression problem that estimates the impression level of a person as a continuous quantity.
  • the score is normalized by the following equation (27).
  • the neural network 16 in the regression analysis mode solves as a regression problem that estimates the impression level of a person as a continuous quantity, and is also called continuous quantity estimation means.
  • the least square probabilistic classifier (LSPC) 20 will be described.
  • the least square probabilistic classifier (LSPC) 20 solves the impression estimation problem as a discrimination problem using the output of the intermediate layer 164 of the neural network 16 as a face feature vector.
  • the range of the impression level of a person is divided into several classes, for example (0.0 ⁇ / 0.5 ⁇ / 1.5 ⁇ / 2.5 ⁇ / 3.5 ⁇ ). Try to identify the class. Since classification is performed according to the degree of impression, this can be interpreted as a kind of ranking learning based on a point-wise approach. Subsequently, scoring of the output of the least square probabilistic classifier (LSPC) 20 will be described.
  • the least square probabilistic classifier (LSPC) 20 outputs the confidence of each class in a probability format.
  • scoring is performed so as to match the format of the first impression degree score (vector) f 1 with respect to the output of the neural network (regression mode) 16 described above.
  • the least square probabilistic classifier Assume that the certainty levels output by 20 are p 1 , p 2 ,..., P 5 in order.
  • the score is normalized by the following equation (29).
  • the weighted integration unit 22 for discriminator output uses the weights w 1 , w 2 , and w 3 and uses the weights w 1 , w 2 , and w 3 for each discriminator 16 at each impression degree y (0.0 ⁇ y ⁇ 4.0, 0.25 interval).
  • the output scores (first to third impression scores) of 20 and 18A are added with weights so that they can be expressed by the following equation (30).
  • the weighted integration unit 22 of the discriminator output is y * represented by the following equation (31) .
  • the such y * a to output as the estimated result of the impression.
  • the weighted integration unit 22 for discriminator output is called integration means.
  • the combination of the neural network 16 in the regression analysis mode, the least square probabilistic classifier (LSPC) 20, the ranking least squares probabilistic classifier 18A, and the weighted integration unit 22 of the classifier output is an integrated classifier (16, 20, 18A, 22).
  • LSPC least square probabilistic classifier
  • the weighted integration unit 22 of the classifier output is an integrated classifier (16, 20, 18A, 22).
  • a weight search method will be described. Using the verification data (data not used for model learning), the optimal weights w 1 , w 2 , and w 3 are searched in a collapsed manner. Specifically, the numerical value width is set to 0 ⁇ w i ⁇ 1 and the search interval is set to 0.05, and the integrated discriminator (16, 20, 18A, 22) is evaluated with the verification data.
  • the impression degree estimation apparatus 10B can be realized by a computer that operates by program control.
  • this type of computer includes an input device for inputting data, a data processing device, an output device for outputting processing results in the data processing device, and an auxiliary memory serving as various databases.
  • the data processing device stores the program in a read-only memory (ROM), a random access memory (RAM) used as a work memory for temporarily storing data, and a program stored in the ROM.
  • ROM read-only memory
  • RAM random access memory
  • the input device operates as a device (not shown) for inputting a person image (image data).
  • the data processing apparatus includes a data acquisition means 32, a regression analysis mode neural network (continuous quantity estimation means) 16, a least squares stochastic discriminator (continuous quantity estimation means) 20, a ranking least squares stochastic discriminator (order estimation means). 18A and a weighted integration unit (integration means) 22 of the discriminator output.
  • the auxiliary storage device functions as storage means (not shown) for storing a plurality of comparison base data.
  • each unit of the impression level estimation device 10B according to the third embodiment may be realized using a combination of hardware and software.
  • each unit is realized as various means by operating hardware such as a control unit (CPU) based on an impression degree estimation program stored in the ROM.
  • the impression degree estimation program may be recorded on a recording medium and distributed.
  • the impression degree estimation program recorded on the recording medium is read into the memory via the wired, wireless, or recording medium itself, and operates the control unit and the like.
  • Examples of the recording medium include an optical disk, a magnetic disk, a semiconductor memory device, and a hard disk.
  • method 1 a technique using the neural network (continuous quantity estimating means) 16 in the regression analysis mode is referred to as “method 1”, and a technique using the least square probabilistic classifier (LSPC) (discrete quantity estimating means) 20 is referred to as “method 1”.
  • Method 2 a method using a two-class ranking least-squares stochastic discriminator (order estimation means) 18 is called “Method 3 ′”, and a four-class ranking least-squares probability classifier (order estimation means).
  • the method using 18A will be referred to as “method 3”.
  • the integration ratio of (Method 1: Method 2: Method 3) in the integrated classifier (16, 20, 18A, 22) is set to 0.35: 0.15: 0.50.
  • the impression degree classification is solved as a 5-class identification problem of (0.0 to /0.5 to /1.5 to /2.5 to /3.5 to 4.0).
  • FIG. 8 shows the recognition rate of each class when solved as the identification problem
  • FIG. 9 shows MAE (absolute error average) when solved as a regression problem. If the recognition rate is 100%, it indicates that the recognition has been made without error. Further, MAE (absolute error average) indicates that the smaller the error, the smaller the error. From FIG. 8 and FIG.
  • Method 3 the accuracy of estimation of the impression degree at both ends of the person is improved from the conventional methods (Method 1 and Method 2). Also, it can be seen that the 4-class ranking LSPC of (Method 3) is slightly better than the 2-class ranking LSPC of (Method 3 ′).
  • the outputs of the plurality of discriminators 16, 20, and 18A are integrated with weights, so that the weak points of each discriminator can be complemented. It is possible to achieve stable discrimination accuracy that cannot be achieved by a single discriminator.
  • the four-class ranking least-squares probabilistic classifier 18A is used as the order estimation means.
  • a two-class ranking least-squares probability classifier 18 is used. May be.
  • the present invention has been described with reference to the embodiments, the present invention is not limited to the above embodiments. Various changes that can be understood by those skilled in the art can be made to the configuration and details of the present invention within the scope of the present invention.
  • the impression degree estimation apparatus of the present invention may integrate the continuous quantity estimation means 16 and the order estimation means 18A (or 18), or the discrete quantity estimation means 20 and the order estimation means 18A (or 18). ) May be integrated.
  • the present invention can be used for sales support of cosmetics and glasses that change the impression of the face.
  • the consumer can check the impression change at the time of makeup at home with a smart device such as a smartphone / tablet. At this time, it is possible to encourage purchase by recommending makeup items used in the simulation. In addition to use at home, it can also be used at drugstore stores and cafes where you can make up. Since the first impression of a person is often interested in social exchanges and activities, the present invention is expected to expand in various usage scenes in the future.

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • General Health & Medical Sciences (AREA)
  • Health & Medical Sciences (AREA)
  • General Physics & Mathematics (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Data Mining & Analysis (AREA)
  • Evolutionary Computation (AREA)
  • Biophysics (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • Computational Linguistics (AREA)
  • Artificial Intelligence (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Human Computer Interaction (AREA)
  • Multimedia (AREA)
  • Image Analysis (AREA)

Abstract

La présente invention concerne, afin d'estimer des impressions de personnes extrêmes avec une précision satisfaisante, un procédé d'estimation d'impression qui estime une impression d'une personne apparaissant dans des données d'image, ledit procédé comprenant : une étape d'extraction consistant à extraire l'image faciale des données d'image ; une étape de calcul consistant à calculer un vecteur de caractéristique faciale à partir de l'image faciale ; et une étape d'estimation consistant à modéliser une relation de niveaux d'impression entre deux échantillons en utilisant les classificateurs probabilistes de méthode des moindres carrés d'une pluralité de classes, en attribuant un score sur la base du degré de confiance de chaque classe obtenu par comparaison séquentielle, avec les classificateurs probabilistes de méthode des moindres carrés de la pluralité de classes, des données de test, le vecteur d'image faciale étant la variable explicative ainsi qu'une pluralité d'instances de données de base de comparaison déjà obtenues à partir de données d'entraînement, permettant ainsi d'estimer l'impression de la personne.
PCT/JP2014/065823 2013-07-18 2014-06-10 Procédé, dispositif et programme d'estimation d'impression faciale Ceased WO2015008567A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
JP2015527224A JP6029041B2 (ja) 2013-07-18 2014-06-10 顔印象度推定方法、装置、及びプログラム

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2013149123 2013-07-18
JP2013-149123 2013-07-18

Publications (1)

Publication Number Publication Date
WO2015008567A1 true WO2015008567A1 (fr) 2015-01-22

Family

ID=52346040

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2014/065823 Ceased WO2015008567A1 (fr) 2013-07-18 2014-06-10 Procédé, dispositif et programme d'estimation d'impression faciale

Country Status (2)

Country Link
JP (1) JP6029041B2 (fr)
WO (1) WO2015008567A1 (fr)

Cited By (10)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015187796A (ja) * 2014-03-27 2015-10-29 富士通株式会社 情報処理装置、評価器の構築方法、及びプログラム
JP2018055377A (ja) * 2016-09-28 2018-04-05 日本電信電話株式会社 マルチタスク処理装置、マルチタスクモデル学習装置、及びプログラム
WO2018207334A1 (fr) * 2017-05-12 2018-11-15 日本電気株式会社 Dispositif, procédé et programme de reconnaissance d'images
WO2019035364A1 (fr) * 2017-08-16 2019-02-21 ソニー株式会社 Programme, procédé de traitement d'informations et dispositif de traitement d'informations
WO2021019748A1 (fr) * 2019-07-31 2021-02-04 富士通株式会社 Procédé de classification, programme de classification, dispositif de classification, procédé d'apprentissage, programme d'apprentissage et dispositif d'apprentissage
WO2021084590A1 (fr) * 2019-10-28 2021-05-06 富士通株式会社 Procédé d'apprentissage, programme d'apprentissage et dispositif d'apprentissage
CN113313007A (zh) * 2021-05-26 2021-08-27 每日互动股份有限公司 一种基于视频的行人静止状态识别方法、电子设备及存储介质
JP2021534439A (ja) * 2018-05-18 2021-12-09 グリーンエデン ユーエス ホールディングス セカンド エルエルシー 自動音声認識システムにおける信頼モデリングのためのマルチクラスアプローチのためのシステム及び方法
JP2022047253A (ja) * 2020-09-11 2022-03-24 日本メナード化粧品株式会社 印象推定装置、印象推定方法および印象推定プログラム
US11564637B1 (en) 2016-04-11 2023-01-31 Pricewaterhousecoopers Llp System and method for physiological health simulation

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP7207857B2 (ja) * 2018-03-15 2023-01-18 サンスター株式会社 フェイシャルエステ提供方法

Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012077476A1 (fr) * 2010-12-08 2012-06-14 Ｎｅｃソフト株式会社 Dispositif d'estimation de valeurs d'attributs, procédé d'estimation de valeurs d'attributs, programme et support d'enregistrement

Patent Citations (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2012077476A1 (fr) * 2010-12-08 2012-06-14 Ｎｅｃソフト株式会社 Dispositif d'estimation de valeurs d'attributs, procédé d'estimation de valeurs d'attributs, programme et support d'enregistrement

Non-Patent Citations (4)

* Cited by examiner, † Cited by third party
Title
JAAK SIMM ET AL.: "Computationally Efficient Multi-task Learning with Least-squares Probabilistic Classifiers", IPSJ TRANSACTIONS ON COMPUTER VISION AND APPLICATIONS, vol. 3, 22 February 2011 (2011-02-22), pages 1 - 8 *
MASASHI SUGIYAMA: "Superfast-Trainable Multi- Class Probabilistic Classifier by Least-Squares Posterior Fitting", IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, vol. E93-D, no. 10, pages 2690 - 2701, XP055125692, Retrieved from the Internet <URL:http://search.ieice.org/bin/summary.php?id=e93-d_10_2690> doi:10.1587/transinf.E93.D.2690 *
RYOMA OAMI ET AL.: "Extraction of Human Behavior and Attribute from Surveillance Videos and Its Applications", THE JOURNAL OF THE INSTITUTE OF ELECTRONICS, INFORMATION AND COMMUNICATION ENGINEERS, vol. 95, no. 5, 1 May 2012 (2012-05-01), pages 452 - 456 *
TSUBASA KOBAYASHI ET AL.: "Early Stopping Heuristics in Pool-Based Incremental Active Learning for Least-Squares Probabilistic Classifier", IEICE TRANSACTIONS ON INFORMATION AND SYSTEMS, vol. E95-D, no. 8, pages 2065 - 2073, Retrieved from the Internet <URL:http://search.ieice.org/bin/summary.php?id=e95-d_8_2065> *

Cited By (22)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2015187796A (ja) * 2014-03-27 2015-10-29 富士通株式会社 情報処理装置、評価器の構築方法、及びプログラム
US11564637B1 (en) 2016-04-11 2023-01-31 Pricewaterhousecoopers Llp System and method for physiological health simulation
US11564638B1 (en) 2016-04-11 2023-01-31 Pricewaterhousecoopers Llp System and method for physiological health simulation
US11564635B1 (en) * 2016-04-11 2023-01-31 Pricewaterhousecoopers Llp System and method for physiological health simulation
US11564636B1 (en) 2016-04-11 2023-01-31 Pricewaterhousecoopers Llp System and method for physiological health simulation
JP2018055377A (ja) * 2016-09-28 2018-04-05 日本電信電話株式会社 マルチタスク処理装置、マルチタスクモデル学習装置、及びプログラム
WO2018207334A1 (fr) * 2017-05-12 2018-11-15 日本電気株式会社 Dispositif, procédé et programme de reconnaissance d'images
JPWO2018207334A1 (ja) * 2017-05-12 2019-11-21 日本電気株式会社 画像認識装置、画像認識方法および画像認識プログラム
JPWO2019035364A1 (ja) * 2017-08-16 2020-09-03 ソニー株式会社 プログラム、情報処理方法、および情報処理装置
US11605002B2 (en) 2017-08-16 2023-03-14 Sony Corporation Program, information processing method, and information processing apparatus
WO2019035364A1 (fr) * 2017-08-16 2019-02-21 ソニー株式会社 Programme, procédé de traitement d'informations et dispositif de traitement d'informations
JP7207309B2 (ja) 2017-08-16 2023-01-18 ソニーグループ株式会社 プログラム、情報処理方法、および情報処理装置
JP2021534439A (ja) * 2018-05-18 2021-12-09 グリーンエデン ユーエス ホールディングス セカンド エルエルシー 自動音声認識システムにおける信頼モデリングのためのマルチクラスアプローチのためのシステム及び方法
JP7223775B2 (ja) 2018-05-18 2023-02-16 ジェネシス クラウド サービシーズ ホールディングス セカンド エルエルシー 自動音声認識システムにおける信頼モデリングのためのマルチクラスアプローチのためのシステム及び方法
AU2019270168B2 (en) * 2018-05-18 2024-01-04 Genesys Cloud Services, Inc. System and method for a multiclass approach for confidence modeling in automatic speech recognition systems
WO2021019748A1 (fr) * 2019-07-31 2021-02-04 富士通株式会社 Procédé de classification, programme de classification, dispositif de classification, procédé d'apprentissage, programme d'apprentissage et dispositif d'apprentissage
JPWO2021084590A1 (fr) * 2019-10-28 2021-05-06
WO2021084590A1 (fr) * 2019-10-28 2021-05-06 富士通株式会社 Procédé d'apprentissage, programme d'apprentissage et dispositif d'apprentissage
JP7322965B2 (ja) 2019-10-28 2023-08-08 富士通株式会社 学習方法、学習プログラム、および学習装置
JP2022047253A (ja) * 2020-09-11 2022-03-24 日本メナード化粧品株式会社 印象推定装置、印象推定方法および印象推定プログラム
JP7525150B2 (ja) 2020-09-11 2024-07-30 日本メナード化粧品株式会社 印象推定装置、印象推定方法および印象推定プログラム
CN113313007A (zh) * 2021-05-26 2021-08-27 每日互动股份有限公司 一种基于视频的行人静止状态识别方法、电子设备及存储介质

Also Published As

Publication number Publication date
JPWO2015008567A1 (ja) 2017-03-02
JP6029041B2 (ja) 2016-11-24

Similar Documents

Publication Publication Date Title
JP6029041B2 (ja) 顔印象度推定方法、装置、及びプログラム
Guo et al. A probabilistic fusion approach to human age prediction
TWI677852B (zh) 一種圖像特徵獲取方法及裝置、電子設備、電腦可讀存儲介質
Zhang et al. Quantifying facial age by posterior of age comparisons
JP6270182B2 (ja) 属性要因分析方法、装置、およびプログラム
CN110503531A (zh) 时序感知的动态社交场景推荐方法
CN105760488B (zh) 基于多层次特征融合的图像表达方法和装置
CN111754345A (zh) 一种基于改进随机森林的比特币地址分类方法
WO2019205729A1 (fr) Procédé utilisé pour identifier un objet, dispositif et support d&#39;informations lisible par ordinateur
Xu Mt-resnet: a multi-task deep network for facial attractiveness prediction
Hewahi et al. Age estimation based on neural networks using face features
CN107563319A (zh) 一种基于图像的亲子间人脸相似性度量计算方法
CN111160119B (zh) 一种用于化妆人脸验证的多任务深度判别度量学习模型构建方法
Shoba et al. Aging facial recognition for feature extraction using adaptive fully recurrent deep neural learning
Dornaika et al. Image-based face beauty analysis via graph-based semi-supervised learning
WO2025194748A1 (fr) Procédé et appareil d&#39;évaluation esthétique d&#39;un visage sur la base d&#39;un méta-apprentissage, dispositif et support
US20250104305A1 (en) Techniques for creating digital collages
Upadhyay et al. Face recognition using EfficientNet
Huang et al. Age-puzzle facenet for cross-age face recognition
Villegas-Cortez et al. Interest points reduction using evolutionary algorithms and CBIR for face recognition
CN113988979B (zh) 一种基于条件相似性语义匹配的多维度商品信息推荐方法
Ma et al. Person-specific age estimation under ranking framework
Li et al. Eyebrow semantic description via clustering based on Axiomatic Fuzzy Set
Dhamija et al. Analysis of age invariant face recognition using quadratic support vector machine-principal component analysis
Zhou et al. Facial Eigen-Feature based gender recognition with an improved genetic algorithm

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 14826431

Country of ref document: EP

Kind code of ref document: A1

ENP Entry into the national phase

Ref document number: 2015527224

Country of ref document: JP

Kind code of ref document: A

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 14826431

Country of ref document: EP

Kind code of ref document: A1