JP6606849B2 - Discriminator generation device, discriminator generation method, estimation device, estimation method, and program - Google Patents

Description

  The present invention relates to a technique for constructing a discriminator from a plurality of images and estimating a specific part (position) in the image.

  Conventionally, the facial organ detection technique defines a facial model and optimizes the input image by applying it to the model, thereby obtaining the facial organ position of the input image based on the facial organ position predefined in the model. (Model-based method) was the mainstream (for example, Non-Patent Documents 1, 2, 3, etc.). In Patent Document 1, high-precision face organ detection is realized by applying an input image to a three-dimensional model.

  However, the model-based method has the following problems. First, the optimization problem is not an alignment error, but an error that is applied to the model is solved. Therefore, even if the error is minimized, the alignment result does not always improve. Second, since it is premised on having a unified model (low dimension) on all faces, it cannot cope with variations such as age, gender, and race.

  On the other hand, in recent years, a method (regression-based method) for solving a correct facial organ position from a suitable initial position as a regression problem has been taken (for example, Non-Patent Documents 4, 5, 6, etc.). In these methods, the alignment error is solved as a regression problem in a high-dimensional space, so that the problem of the model-based method can be overcome and highly accurate alignment can be realized.

Japanese Patent No. 52113778 US Patent Application Publication No. 2014/0185924

Tim Cootes, “An Introduction to Active Shape Models”, http://person.hst.aau.dk/06gr1088d/artikler/Pdf/Cootes%20Introduction.pdf Timothy F. Cootes, Gareth J. Edwards, and Christopher J. Taylor, “ActiveAppearance Models”, IEEETransactions on Pattern Analysis and Machine Intelligence, Vol. 23, No. 6, June 2001, http: //person.hst.aau. dk / 06gr1088d / artikler / Pdf / Cootes% 20Introduction.pdf David Cristinacce and Tim Cootes, “Feature Detection and Trackingwith Constrained Local Models”, http://personalpages.manchester.ac.uk/staff/timothy.f.cootes/papers/BMVC06/cristinacce_bmvc06.pdf Piotr Dollar, Peter Welinder, Pietro Perona, “Cascaded PoseRegression”, Conference on ComputerVision and Pattern Recognition (CVPR) 2010 Xavier P. Burgos-Artizzu, Pietro Perona, Piotr Dollar, “Robust FaceLandmark Estimation Under Occlusion”, International Conference On ComputerVision (ICCV2013) Shaoqing Ren, Xudong Cao, Yichen Wei, Jian Sun, “Face Alignment at3000 FPS via Regressing Local Binary Features”, Conference on Computer Vision and Pattern Recognition (CVPR) 2014

  However, in the regression-based method, since the alignment error is solved as a regression problem in a high-dimensional space, the number of feature combinations for obtaining a regression function becomes enormous, and there is a problem that cannot be solved in real time.

  In this regard, in the existing methods (Patent Document 2, Non-Patent Documents 4, 5, and 6), a plurality of candidate classifiers are created from randomly selected features, and the created classifiers are relatively compared with each other. Thus, the above problem is avoided by determining the best classifier. However, since this method randomly selects features and creates classifiers, the accuracy depends on the number of features to be selected and the number of classifiers to be created. It is difficult to obtain an optimal classifier.

  The present invention has been made in view of the above-described problems, and an object of the present invention is to provide a discriminator generating apparatus and the like that can generate a discriminator having efficient and robust robustness in object detection. That is.

  In order to achieve the above-described object, a discriminator generating device in the present invention is a discriminator generating device that generates a discriminator configured by using a plurality of strong discriminators including a plurality of weak discriminators. Receiving a plurality of learning images including the object and a correct value that is the position of the object of the learning image, extracting a plurality of features from the learning image, and selecting two from the plurality of features, A difference feature is determined from the difference, a histogram is generated from the feature amount of the difference feature using the plurality of learning images, a threshold value for equalizing the areas of the histogram is determined, and the feature amount of the difference feature Generating the weak classifier having a tree structure using the threshold value, and correcting the estimated position of the detection target of the image identified by the weak classifier using the learning image and the correct answer value; And calculates the amount of movement.

  The classifier generates the classifier for each direction by classifying the plurality of learning images for each direction of the face.

  The weak classifier and the strong classifier are generated based on a preset target value.

  The weak discriminator is generated by determining the feature quantity used corresponding to the depth of the tree structure using dynamic programming.

  The feature is a shape-index.

  The estimation apparatus according to the present invention is characterized in that the detection target object is estimated while moving the estimated position of the detection target object in the image using the classifier of the classifier generation apparatus.

In addition, the classifier generation method according to the present invention receives a plurality of learning images including an object to be detected and a correct value that is the position of the object of the learning image, and extracts a plurality of features from the learning image. And selecting two of the plurality of features, determining a difference feature from the difference between them, generating a histogram from the feature amount of the difference feature using the plurality of learning images, and determining the area of the histogram determining the equal threshold, using said feature quantity and the threshold value of the difference feature generates a weak discriminator that Do from the tree structure, using the correct value and the learning image, are identified by the weak classifier A moving amount for correcting the estimated position of the detection target of the detected image is calculated.

The estimation method according to the present invention receives a plurality of learning images including an object to be detected and a correct value that is the position of the object of the learning image, and extracts a plurality of features from the learning image, Two of the plurality of features are selected, a difference feature is determined from the difference between them, a histogram is generated from the feature amount of the difference feature using the plurality of learning images, and the areas of the histogram are equalized determining the threshold value, the using the feature amount of the difference characteristic between the said threshold value to generate a weak discriminator that Do from the tree structure, using the correct value and the learning image, identified by the weak classifier image calculating a movement amount for correcting the estimated position of the detection object by using the weak classifier, estimate the detection target object while moving the estimated position of the detection target object in an image And wherein the Rukoto.

  A program according to the present invention causes a computer to execute the classifier generation method.

  The estimation method is executed by a computer.

  According to the present invention, it is possible to provide a discriminator generating apparatus and the like that generate a discriminator having efficient and robust robustness in object detection.

The figure which shows the hardware structural example of the discriminator production | generation apparatus 1 and the estimation apparatus 2 Conceptual diagram of facial organ estimation based on regression-based method Conceptual diagram of classifier 3 according to the present embodiment Conceptual diagram of classifier 3 (by face direction) according to the present embodiment (A) The figure which shows the tree structure of Fern tree (weak classifier) 5 (b) The figure which shows another expression mode of a Fern tree The figure which shows a mode that a discriminator learns according to a learning target value The figure which shows the feature based on Shape-Index Flow chart explaining the flow of learning processing The figure which shows a mode that threshold value (delta) is determined from the histogram H Conceptual diagram of Fern candidate graph 7Gr Diagram showing how the combination of branch functions is determined The figure which shows a mode that a Fern tree (weak discriminator) is produced | generated from the determined branch function. Figure showing how to calculate the amount of movement Flow chart explaining the flow of estimation processing (A) The figure which shows input data (b) The figure which selects the discriminator 3 according to face direction Figure showing estimation results

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a facial organ is taken as an example as an object, a classifier for estimating the position of the facial organ is learned, and the case of estimating the position of the facial organ using the learned classifier will be described. The invention can also be applied to objects other than facial organs.

  FIG. 1 is a diagram illustrating an example of a hardware configuration of the discriminator generation device 1 and the estimation device 2 according to the present embodiment. As shown in FIG. 1, the discriminator generation device 1 includes a control unit 11, a storage unit 12, a media input / output unit 13, a communication control unit 14, an input unit 15, a display unit 16, a peripheral device I / F unit 27, and the like. Are connected via a bus 18.

  The control unit 11 includes a CPU, ROM, RAM, and the like. The CPU calls and executes a program stored in the storage unit 12, ROM, recording medium, or the like to a work memory area on the RAM, drives and controls each device connected via the bus 18, and is performed by the aggregation device 2. The processing described later is realized. The ROM is a non-volatile memory and permanently holds a computer boot program, a program such as BIOS, data, and the like. The RAM is a volatile memory, and temporarily stores programs, data, and the like loaded from the storage unit 12, ROM, recording medium, and the like, and includes a work area used by the control unit 11 to perform various processes.

  The storage unit 12 is an HDD or the like, and stores a program executed by the control unit 11, data necessary for program execution, an OS, and the like. As for the program, a control program corresponding to the OS and an application program for causing a computer to execute processing to be described later are stored. Each of these program codes is read by the control unit 11 as necessary, transferred to the RAM, read by the CPU, and executed as various means.

  The media input / output unit 13 (drive device) inputs / outputs data, for example, media such as a CD drive (-ROM, -R, -RW, etc.), DVD drive (-ROM, -R, -RW, etc.) Has input / output devices. The communication control unit 14 includes a communication control device, a communication port, and the like, and is a communication interface that mediates communication between a computer and a network, and performs communication control between other computers via the network. The network may be wired or wireless.

  The input unit 15 inputs data and includes, for example, a keyboard, a pointing device such as a mouse, and an input device such as a numeric keypad. An operation instruction, an operation instruction, data input, and the like can be performed on the computer via the input unit 15. The display unit 16 includes a display device such as a liquid crystal panel, and a logic circuit or the like (video adapter or the like) for realizing a video function of the computer in cooperation with the display device. The input unit 15 and the display unit 16 may be integrated like a touch panel display.

  The peripheral device I / F unit 17 is a port, an antenna, or the like for performing data transmission / reception between the computer and the peripheral device. The computer transmits / receives data to / from the peripheral device via the peripheral device I / F unit 17. The connection form with the peripheral device may be either wired (for example, USB) or wireless (for example, Bluetooth (registered trademark)). The bus 18 is a path that mediates transmission / reception of control signals, data signals, and the like between the devices.

(Face organ estimation based on regression-based method)
FIG. 2 is a conceptual diagram showing facial organ estimation based on the regression-based method. In the regression-based method, the facial organ position estimation problem is considered as a regression problem using a displacement amount function (regression function).

2 the input image I and the face appropriate initial position S ⁰ of the organs including the face, as in (a) it is a given. In the regression-based method, the initial position S ⁰ is displaced toward the correct position of the facial organ using the displacement amount function (regression function) R (FIG. 2B), the correct position ^ST is estimated, and the estimation result ( An output image E displaying the regression result is obtained (FIG. 2C). The displacement amount function (regression function) R is usually learned and generated by machine learning. The discriminator generation device and the estimation device in this embodiment generate a plurality of discriminators each including a displacement amount function (regression function) R, generate a discriminator, and estimate the position of an object in an input image.

  The object to be learned and estimated is not limited to this embodiment, and any object can be used. Also, the number can be arbitrarily determined, and an appropriate number (about 1 to 200 points) can be set for each target image.

(Structure of the discriminator 3 according to the present embodiment)
FIG. 3 is a conceptual diagram showing the data structure of the discriminator 3 that is learned and generated in the present embodiment. Discriminator 3, as shown in FIG. 3, the displacement amount function (regression ^{function) R 1, ..., R t} , ..., a plurality of strong classifiers 4-1 functioning as ^{R T, ..., 4-t} , ... 4-T is connected. For simplicity, the strong discriminator may be represented as “R” in the text.

Each strong classifier is configured by connecting a plurality of weak classifiers that similarly function as a displacement amount function (regression function). For example, strong classifier 4-t in FIG. 3, the displacement amount function (regression ^{function) r 1, ..., r k} , ..., a plurality of weak classifiers 5-1 consisting ^{r K, ..., 5-k} , ... 5-K. For simplicity, the weak discriminator may be represented as “r” in the text.

As described above, the identifier 3 of the present embodiment, a two layered structure of the weak classifier 5 with strong classifier 4, to the discriminator 3, the initial position S ⁰ of the face image and the face organ is input while repeating the displacement by the displacement amount function (s weak classifiers 5 included in) the strong classifier 4 consisting of (regression function), and finally operative to estimate a face organ position S ^T.

Further, the classifier 3 according to the present embodiment is generated by learning for each face direction.
FIG. 4 is a diagram illustrating the discriminator 3 generated by learning for each face direction. As shown in the figure, the discriminator 3-1 corresponding to the face direction of “front direction”, the discriminator 3-2 corresponding to the face direction of “diagonal left direction”, and the discrimination corresponding to the face direction of “left direction” Learning classifiers corresponding to various face orientations, such as devices 3-3,. This makes it possible to efficiently estimate the face organ position for images in any face direction.

  Note that it is not always necessary to separately learn the classifier for each orientation, and the classifier can be constructed with various face orientations without being limited to the three patterns of the present embodiment. Here, the orientation of each face is not a unique angle, but a certain range can be learned as a group. At that time, if the object to be learned is an angle included in the entire image, for example, all face images in the direction of 30 ° to the right, 30 ° to the front, and 30 ° to the left can be learned as the direction of the front face.

(Weak classifier 5)
FIG. 5A is a diagram illustrating a tree structure of a Fern tree that is the weak classifier 5 of the present embodiment. Each node of the Fern tree is composed of a “branch function” that branches to one of two branch destination nodes according to the input value of the input image. “F” in the figure represents a branch function. The Fern tree is a kind of decision tree, but is characterized in that the same branch function f can be shared by nodes of the same depth (same layer) as shown in the figure.

FIG. 5B is another representation mode of the Fern tree tree structure. Since the Fern tree uses the same branch function at the same depth (same layer) as described above, the tree structure is a one-dimensional series of branch functions {f ₀ , f ₁ , It can also be expressed by f ₂ ,..., f _S }.

  In the present embodiment, each node of the above branch function is determined by dynamic programming. In the present embodiment, the Fern tree will be described as an example of the tree structure constituting the weak classifier. However, the present invention is not limited to this, and various tree structures that can be generated within the scope of the disclosure of the present invention are used. It is possible to construct a weak classifier.

(Learning according to the learning target value)
In the present embodiment, a learning target value (target to reach the correct position) is set for each strong classifier, and the number of connections of weak classifiers is dynamically determined according to the learning target value. The learning target value is a target value for the estimation accuracy of the final correct position to be achieved when constructing the strong classifier.

  In this way, by determining the learning target value (target to reach the correct answer position) for each strong classifier, the optimum number of weak classifiers corresponding to the target values as much as necessary in each strong classifier can be obtained. Learning generated.

  FIG. 6 is a diagram showing how learning is performed by a certain strong classifier 4-t. A learning target value is set during learning of the strong classifier 4-t. .. Are connected while generating the weak classifiers 5-1, 5-2,... In the strong classifier 4-t while evaluating whether or not the learning target has been achieved. When the learning goal is achieved, the generation of the weak classifier is terminated and the process proceeds to learning of the next strong classifier. In this way, the number of weak classifier connections is automatically determined according to the set learning target value.

  In this embodiment, the learning stages of the strong classifiers 4-1, ..., 4-t, ... 4-T are called "stages", and the weak classifiers 5-1, ..., 5- in the strong classifiers. Each learning stage of k,..., 5-K is called “round”.

(Feature value)
In addition, the feature used for constructing the discriminator in the present embodiment is calculated using the Shape-Index feature described in Non-Patent Document 5. Shape-Index represents the feature of the pixel position calculated using two of a plurality of arbitrary points. By using Shape-Index, there is an advantage that a high invariant feature can be obtained even when the face shape changes greatly. Refer to Non-Patent Document 5 for the details of Shape-Index. In the present embodiment, the calculated luminance difference (Pixel-Difference) between two points of Shape-Index is used as the difference feature amount as the feature amount constituting the node of the weak classifier.

FIG. 7 is a diagram showing Shape-Index. Reference numerals 6A and 6B in FIG. 7 denote Shape-Index pixel positions, which indicate that the pixel values (luminance values) are I _A and I _B , respectively. Brightness difference between two Shape-Index In the present embodiment, as shown in figure ₍₌ I A -I _B) used as the feature amount.

<Learning action>
Next, a process (learning process) in which the classifier generating device 1 according to the present embodiment generates and learns the classifier 3 will be described with reference to the flowchart of FIG.

First, the control unit 11 of the discriminator generation device 1 uses N learning images I _i (i = 1 to N) including faces and correct positions (target positions) of facial organs in each learning image as learning data. An input of S ^* _i (i = 1 to N) is accepted (step S1), and an initial position S ⁰ (initial value of learning) of an arbitrary facial organ to be used for learning is set (step S2).

While setting method of the initial position S ⁰ is arbitrary, in the present embodiment, the initial position of the correct position of the face organ average of (target _{^{position) ((S ※ 1 + ...}} + S ※ N) / N) It is set as S ^0.

  Thereafter, learning of the discriminator 3 is performed. First, the control unit 11 initializes the index t of the current learning stage (learning strong classifier) of the classifier 3 (t = 1, step S3).

And the control part 11 produces | generates the candidate of the branch function which consists of the characteristic and threshold value for producing | generating the weak discriminator in the learning stage t (strong discriminator 4-t ( _Rt ) to learn). Specifically, the control unit 11 first randomly selects P pixel positions (about 40,000 in the present embodiment) for calculating the Shape-Index, and calculates Formula 1 from the selected P pixel positions. M features based on the indicated Shape-Index are calculated for all learning images (step S4).

  M is M = P (P−1) when the luminance difference is calculated for all the combinations of Shape-Index.

  Next, the control unit 11 sets an appropriate threshold δ for each of the M features (step S5). Specifically, the control unit 11 creates a distribution (histogram) of the number of learning images for the feature amount of the feature calculated from all the learning images for each of the M features, and a threshold value serving as a median value that divides the distribution area into two Set δ. Although the threshold value δ can be set based on various indices, in this embodiment, the value of the feature quantity that equalizes the histogram area obtained by dividing the feature distribution into two is specified, and the threshold value δ is set.

FIG. 9 is a diagram conceptually showing the process of step S5. As shown in FIG. 9 (a), the control unit 11 performs, for each learning image, for each difference feature (feature ₁ , feature ₂ ..., Feature _M ) of the pixels at the relative positions in the two Shape-Index. A feature amount is calculated, and histograms H ₁ , H ₂ ... _HM are created for each difference feature. Then, as shown in FIG. 9B, the control unit 11 sets threshold values δ ₁ , δ ₂ ,..., Δ _M that equally divide each histogram into two.

As described above, according to the features (feature ₁ , feature ₂ ,..., Feature _M ) randomly selected in step S4 and the threshold values (δ ₁ , δ ₂ ,..., Δ _M ) set in step S5, M branch function candidates (F ₁ , F ₂ ,..., F _M ) constituting each node of the weak classifier are generated (Formula 2).

In the subsequent processing, the control unit 11 determines a branch function to be assigned to each node of the weak classifier from the branch function candidates (F ₁ , F ₂ ,..., F _M ) in the above formula. As a result, an optimum weak classifier is constructed.
First, the control unit 11 initializes the index k of the current learning round (weak classifier to be learned) (k = 1, step S6).

Subsequently, the control unit 11 creates a “Fern candidate graph Gr” that graphs combinations of all series of branch functions from the branch function candidates (F ₁ , F ₂ ,..., F _M ) (step S1). S7).
FIG. 10 is a conceptual diagram showing the Fern candidate graph 7Gr. The node 7n of the Fern candidate graph 7Gr represents a branch function candidate (F ₁ , F ₂ ,..., F _M ), and the edge 7e represents all combinations connecting the branch function candidates.

  Then, the control unit 11 selects an appropriate path, that is, a combination of branch functions from the Fern candidate graph 7Gr composed of the node 7n and the edge 7e (step S8). Specifically, the control unit 11 minimizes the accumulated cost function obtained by accumulating the cost function at each depth (the sign of the cost function may be reversed, and in this case, the accumulated cost function is maximized). Are selected using dynamic programming. Here, the accumulated cost function will be described. The cumulative cost function is defined as follows:

  In the above equation, S is the depth of the leaf node of the Fern tree (the height of the Fern tree), that is, the number of branch functions constituting the Fern tree. The depth S is a parameter that can be arbitrarily set by the user. Further, s (= 0 to S) is an index indicating the depth of each node of the Fern tree.

In the above equation, U (f _s ) in the first term on the right side is a local cost when the branch function is f _s at each depth s of the Fern tree (hereinafter, U (f _s ) is expressed as “ Also called "data term"). The second term P (f _s , f _s−1 ) is the state transition cost of the branch function, and the constant λ is its weight (hereinafter, P (f _s , f _s−1 ) is expressed as “smoothing term. "). Hereinafter, the data term U _{(f s)} and smoothing term _{P (f s, f s-} 1) will be described in detail.

1. Data term U _(f s)
Data term U (f _s) is an index represents mainly the ability to distinguish the branch function is specifically calculated by the formula shown in Equation 4.

α；調整パラメータ
σ_ｗ１、σ_ｗ２；分割後の各クラスのクラス内分散
σ_ｂ；分割後のクラス間分散

α: Adjustment parameter σ _w1 , σ _w2 ; _Intraclass variance of each class after division σ _b ; Interclass variance after division

ΔS ₁ in the equation is the current position S _i (i∈Ω ₁ ) and correct position S ^* _i (i∈Ω) of each learning image I _i (i∈Ω ₁ ) belonging to one classification divided by a bifurcation function. ₁ ) is a distance average, and ΔS ₂ is a current position S _i (i∈Ω ₂ ) and a correct position S ^{* of} each learning image I _i (i∈Ω ₂ ) belonging to the other classification divided by the bifurcation function. It is a distance average of _i (i∈Ω ₂ ) and is calculated as follows.

ｂ＝１，２
Ω_ｂ；このｂｉｎに含まれる学習画像の集合

b = 1, 2
Ω _b ; set of learning images included in this bin

More branching function the value of the data section U (f _s) is reduced indicates a higher discriminating ability. A branch function to reduce the value of the data section U (f _s) (high branching function of identification performance) is in the first 1, [Delta] S 1 of formula _4, the value of [Delta] S ₂ (that is the value of Equation 5) to increase A branch function. That is, it is a branch function that increases the average distance between the current position and the correct position of each divided learning image. Thereby, the branch function that tries to bring the current position of the learning image closer to the correct position is more easily selected as the optimal branch function.

The data section U and the branch function values to reduce the (f _s) (high branching function of identification performance) is in the second, branch functions to reduce the formula _{4 α (σ ω1 + σ ω2} ) / σ b It is. In other words, the branch function that reduces the intra-class variance of the two distributions after division and increases the inter-class variance (the branch function that increases the separation of the two distributions) is easier to select as the optimal branch function. Become.

2. Smoothing term _{P (f s, f s-} 1)
On the other hand, the smoothing P (f _s , f _s-1 ) is the state transition cost of the branch function, and the branch function (f _s-1 ) at the depth (s-1) immediately before the Fern tree and the current The degree of association between the depth (s) and the bifurcation function (f _s ) is evaluated. Smoothing _{P (f s, f s-} 1) is calculated as Equation 6.

ｂ＝１またはｂ＝２

b = 1 or b = 2

Here, m ₁ is the current depth (s) among the learning images classified into one bin (for example, Ω ₁ ) by the branch function (f _s-1 ) of the previous depth (s-1). It represents the number of branching function (f _s) at similarly classified training image to Omega _1. On the other hand, _{m 2,} of the previous depth (s-1) branch function _{(f s-1)} by the classification learning image to Omega _1, branch function of the current depth _{(s) (f} s) in represents the number of classification learning image to Omega _2.

In the above _equation, if the m 1 = _{m 2,} smoothing term _{P (f s, f s-} 1) the value of the smallest. That is, the branch function that classifies the learning images classified into the bins by the branch function (f _s-1 ) of the previous depth (s-1) evenly into the bins at the current depth (s) without any bias. As f _s , it becomes easier to be selected as an optimal branch function. The purpose of the smoothing term is to select a branch function (feature) having a low relevance as a branch function (feature) assigned to the node at the previous depth (s-1) at the node at the current depth (s). This eliminates the bias in the identification target in each node (each branch function) of the Fern tree (weak classifier), and as a result, reduces the bias in the number of learning images classified into each bin, thereby over-learning and noise. It is to prevent the occurrence.

Above, the data term U _{(f s)} and the smoothing term _{P (f s, f s-} 1) constituting the cumulative cost function was specifically described.
The control unit 11 combines the branch function that minimizes the accumulated cost function.

Are selected (determined) using dynamic programming. In the dynamic programming, the cumulative value of the cost function is calculated at each depth s (= 0 to S). At this time, the nodes of the depth s (candidates of each branch functions), at each node of the previous depth (s-1) (candidate of each branch function), the cost function _{U (f s) + P (} f s , F _s−1 ), each path (partial optimal path) is held as a back pointer. Then, after calculating the accumulated value of the cost function up to the depth S of the lowest layer, the back pointer is traced (trace back) from the node (branch function) where the accumulated value of the cost function becomes the minimum at the depth S of the lowest layer. Select (determine) a combination of branch functions. The difference feature constituting the branch function selected here and the threshold value related to the difference feature are stored as components for each weak classifier depth.

FIG. 11 is a conceptual diagram showing a state in which a combination of branch functions is selected by dynamic programming. The numerical value of FIG. 11 represents the cumulative value of the cost function at each depth (s = 0 to S). In the case of FIG. 11, (F ₁₀₀₃ , F ₃₈₄₃ , F ₈₃₄ ,..., F ₇₄₉₈ , F ₅₁₁₁ ) are selected as a combination of branch functions.

Control unit 11, a combination of the branch function obtained by dynamic programming to construct a weak classifier r ^k (step S9).
Figure 12 is a diagram showing how to construct a weak classifier r ^k from the combination of branching function shown in Equation 7.

Control unit 11 determines a movement amount for correcting the position of the input values of an object in an image to be detected that is identified by weak classifiers r ^k constructed (current estimated position) (step S10).
As a method of determining the movement amount, an arbitrary value (learning value) for the detection target object is input to a learning image whose position (correct value) of the detection target object is known. Next, it is determined whether or not subjected to identification using the weak classifiers r ^k that are built on the learned value of the learning image, are identified in which the leaf node as the identification result. Subsequently, the distance between the learning value and the correct value of the identified learning image is calculated. Such identification processing is repeated with a plurality of different learning images and / or learning values. Then, the average value of the distance between the learning value calculated from all the learning data identified by the same leaf node and the correct value is obtained. The average value obtained here is used as the movement amount of the estimated position of the object of the image identified by the leaf node.
That is, the movement amount ΔS is the Omega _b Fern tree (weak classifiers) ^{r k} of the set of training images in bin classified in each leaf node, respectively, the correct answer of each learning image _I i _{(i∈Ω b)} If the position is S ^* _i (i∈Ω _b ) and the current position is S ^k−1 _i (i∈Ω _b ), it can be formulated as Equation 8.

Ω_ｂ；このｂｉｎに分類された学習画像の集合
ｂ＝１〜２^Ｓ

Ω _b ; set of learning images classified into this bin b = 1 to 2 ^S

FIG. 13 is a diagram showing how the movement amount is ^obtained in the leaf node 51 with the constructed weak classifier rk. As shown in the figure, the movement amounts ΔS ^k ₁ and ΔS ^k ₂ are calculated based on the learning image I ₁ and the learning image I ₂ classified into each bin (Ω ₁ , Ω ₂ ) in the leaf node 51.

Control unit 11, in all of the leaf nodes 51 weak classifiers ^{r k,} the movement amount [Delta] S ^k _1, to calculate the [Delta] S ^k _2, the calculated amount of movement, as an output value of the weak discriminator ^{r k} (regression results) Save it. In estimation processing will be described later, estimates the face organ position where Fern trees saved (weak classifiers) to obtain the output value of r ^k.

At the time of detecting the object position, the control unit 11 calculates the currently estimated position S _i ^k−1 of the facial organ of each learning image I _i (i = 1 to N) based on the calculated movement amount as shown in Equation 9. Update as follows. The movement amount of each learning image is the movement amount calculated for each bin in which each learning image is finally classified in the leaf node as described above.

Further, the control unit 11 uses the constructed classifier to update the learning value in the learning image, thereby evaluating the round error e _round using the following formula (step S11), and round learning (learning of the weak classifier). Determine the end of.

The round error e _round is within the current stage t (strong discriminator 4-t (within ^Rt )) and how long the current position S _i ^{t-1 of} each input learning image I _i is up to the correct position S ^* _i. The approach is represented by a ratio based on the distance between the initial position S ⁰ _i and the correct position S ^* _i of the learning image I _i . At each stage, the control unit 11 compares with a preset learning target value (a ratio of how close the current position is to the correct position) to determine whether to continue learning or to end learning.

If e _round <learning target value (step S12; No), the process proceeds to the next learning round (k ← k + 1, step S27), and the learning of the Fern tree (weak classifier) is continued. On the other _hand, in the case of _{e round english (us)} ≧ objective value (step S12; Yes), terminates the learning Fern tree of the stage (weak classifier), I learned Fern tree (weak ^classifiers) r 1, ..., a ^{r k} Based on this, a strong classifier ^Rt is constructed (step S14).
Thus, the optimum number of connections (number of learning rounds) of the Fern tree (weak classifier) is dynamically determined according to the learning target value set for each stage.

When the learning of the current stage t is completed (step S12; Yes), the control unit 11 further evaluates the stage error e _stage , which is an index indicating the learning convergence state, using the equation 11 (step S15), and the stage learning (End of strong classifier learning) is determined (Note that the following equation is valid when the current stage t is t> = 3).

Here, if e _stage <1 (step S16; No), the control unit 11 determines that learning has not converged, increments the learning stage t (t ← t + 1, step S17), and learns the next strong classifier. Transition to the stage (return to step S4). On the other hand, when e _stage ≧ 1 (step S16; Yes), the control unit 11 determines that the learning has been converged and ends the learning of the strong classifier.

When the learning of the strong classifier is completed, the control unit 11 constructs the classifier 3 from the learned strong classifiers (R ¹ ,..., R ^t ) and stores the classifier 3 in the storage unit 12 (step S18).

  Heretofore, the learning process of the classifier generation device 1 according to the present embodiment has been described in detail.

<Estimated operation>
Next, a facial organ estimation process in which the estimation device 2 according to the present embodiment estimates a facial organ position from a facial image will be described with reference to the flowchart of FIG. It is assumed that the classifier 3 generated by the classifier generation device 2 is stored in the storage unit 12 of the estimation device 2.

The control unit 11 of the estimation apparatus 2 receives input data including the input image I to be estimated, the initial position S ⁰ of the facial organ, and the face direction information D (step S31). Further, the control unit 11 selects and reads the discriminator 3 corresponding to the input face direction information D (step S32).

FIG. 15A shows the input data, and FIG. 15B shows the selected discriminator 3. As shown in FIG. 15A, an input image I, an initial position S ⁰ of a facial organ, and face direction information D (“front”) are given as input data. In this case, as illustrated in FIG. 15B, the control unit 11 selects the discriminator 3-1 corresponding to the face direction information D (“front”). Note that the initial position S ^0, an arbitrary predicted position of an object to be estimated target in the input image I.

Control unit 11 initializes the index estimation stage t by strong classifier ^{R t} discriminator 3 (t = 1, step S33), also, Fern tree (weak classifiers) in estimating the stage ^{r k} by the estimated round k is initialized (k = 1, step S34).

Control unit 11, the position S ^k-1 of the input image I and the previous ^(first initial position S ⁰⁾ to enter into the weak classifier r ^k, with reference to the trained parameter of the weak classifiers (wherein the threshold value) The input image I is classified, and a movement amount ΔS ^k (= r ^k (I, S ^k−1 )) is acquired as its output value (step S35).
Then, the control unit 11 moves the current position from the acquired movement amount ΔS ^k and the previous position S ^k−1 as follows (step S36).

Weak classifier ^{r k} is, if not the last weak classifier in the strong classifier ^{R t} (step S37; No), the control unit 11 increments the estimated round k (k ← k + 1, step S38), the following The estimation process by the weak classifier is continued.
On the other hand, the weak discriminator r ^k is the case of the last weak discriminator in the strong classifier R ^t (step S37; Yes), further, strong classifier R ^t is whether the last strong classifier discriminator 3 Is determined (step S39).

When the strong discriminator R ^t is not the last strong discriminator of the discriminator 3 (step S39; No), the control unit 11 increments the estimation stage t (t ← t + 1, step S40), and returns to step S4. It moves to the estimation process by the strong classifier.

On the other hand, when the strong classifier ^Rt is the last strong classifier of the classifier 3 (step S39; Yes), the control unit 11 ends the estimation process and outputs the estimation result (step S41).

FIG. 16 is a diagram illustrating the estimation result of the facial organs. As shown, the final estimated position S ^T of the face organ is being calculated plotted displayed on the output image E.

  The preferred embodiments of the discriminator generation device 1 and the estimation device 2 according to the present invention have been described above with reference to the accompanying drawings, but the present invention is not limited to such examples. For example, the present invention includes the following modifications.

  For example, the depth of the tree can be determined dynamically. For example, it is possible to determine the optimum depth dynamically by giving a target value in advance and increasing the depth of the tree until the target value is satisfied.

  Further, the weak classifier of the present invention is not necessarily a Fern tree. For example, if a NULL candidate (when no feature is selected) is added to a feature candidate, it is possible to construct a weak classifier having a tree structure that is not a Fern tree by dynamic programming. In addition, a tree structure that is not a Fern tree can be constructed by manually pruning unnecessary branches.

  Further, the control unit 11 of the discriminator generation device 1 sets a plurality of different initial positions in the learning image in step S2 so as to increase the number of learning data for learning the discriminator 3 in a pseudo manner. It may be. For example, the number of learning data can be increased three times by setting three patterns of different initial positions for N learning images.

In general, in the regression-based method, the quality of the estimation result tends to vary depending on how the initial position is given. Therefore, the control unit 11 of the estimation device 2 determines the validity of the initial position when the estimation round advances by a predetermined round (approximately 10% of all estimation rounds) and determines that the initial position is not appropriate. May perform the estimation process again at another initial position. Specifically, when the index of the estimation stage for determining the validity of the initial position is t, the input position S ^t-1 input to the strong discriminator 4-t (R ^t ) and the output position S ^t output If the variance of the displacement amount is calculated and the variance of the displacement amount is smaller than a predetermined threshold value, it is determined that the given initial position is appropriate (highly likely to converge to a good estimated solution). On the other hand, if it is larger than the threshold, it is determined that the given initial position is not valid (the possibility of convergence to a good estimated solution is low). Note that the above-described displacement amount specifically corresponds to the sum of the displacement amounts of the weak classifiers 5 in the strong classifier 4-t (R ^t ) estimated in step S35.

  In the present embodiment, the feature based on Shape-Index is used. However, the present invention is not limited to this, and other features (features) such as Haar-Like, SIFT, and SURF may be used.

  In addition, it is obvious that those skilled in the art can arrive at various changes or modifications within the scope of the technical idea disclosed in the present application, and these naturally belong to the technical scope of the present invention. It is understood.

DESCRIPTION OF SYMBOLS 1; Classifier production | generation apparatus 2; Estimation apparatus 3; Classifier, regression function 4, R; Strong classifier, regression function 5, r; Fern tree (weak classifier), regression function 51; Leaf node 6A, 6B; -Index
H; histogram δ; threshold 7Gr; Fern candidate graph 7n; node 7e of Fern candidate graph 7Gr; edge I _i of Fern candidate graph 7Gr; learning image I; input image E; output image D; face direction information S; Control unit 12; Storage unit 13; Media input / output unit 14; Communication control unit 15; Input unit 16; Display unit 17; Peripheral device I / F unit 18;

Claims (10)

A classifier generating device that generates a classifier configured by using a plurality of strong classifiers composed of a plurality of weak classifiers,
Receiving a plurality of learning images including an object to be detected, and a correct value that is the position of the object of the learning image;
Extracting a plurality of features from the learning image;
Selecting two of the plurality of features and determining a difference feature from the difference between them;
Generating a histogram from the feature quantity of the difference feature using the plurality of learning images, determining a threshold value at which the areas of the histogram are equal;
Using the feature amount of the difference feature and the threshold value, generate the weak classifier having a tree structure,
A discriminator generation device, wherein a movement amount for correcting an estimated position of a detection target of an image identified by the weak discriminator is calculated using the learning image and the correct answer value. The classifier generating apparatus according to claim 1, wherein the classifier classifies the plurality of learning images for each face direction to generate a classifier for each direction. The classifier generation device according to claim 1 or 2, wherein the weak classifier and the strong classifier are generated based on a preset target value. 4. The weak classifier is generated by determining the feature amount to be used corresponding to the depth of the tree structure using dynamic programming. 5. Classifier generator. The discriminator generation device according to any one of claims 1 to 4, wherein the feature is Shape-Index. An estimation apparatus, wherein the detection target object is estimated while moving the estimated position of the detection target object in an image using the classifier of the classifier generation device according to any one of claims 1 to 5. Receiving a plurality of learning images including an object to be detected, and a correct value that is the position of the object of the learning image;
Extracting a plurality of features from the learning image;
Selecting two of the plurality of features and determining a difference feature from the difference between them;
Generating a histogram from the feature quantity of the difference feature using the plurality of learning images, determining a threshold value at which the areas of the histogram are equal;
Using said feature quantity of the difference feature threshold generates a weak discriminator that Do from the tree structure,
A classifier generation method characterized by calculating a movement amount for correcting an estimated position of a detection target of an image identified by the weak classifier using the learning image and the correct answer value. Receiving a plurality of learning images including an object to be detected, and a correct value that is the position of the object of the learning image;
Extracting a plurality of features from the learning image;
Selecting two of the plurality of features and determining a difference feature from the difference between them;
Generating a histogram from the feature quantity of the difference feature using the plurality of learning images, determining a threshold value at which the areas of the histogram are equal;
Using said feature quantity of the difference feature threshold generates a weak discriminator that Do from the tree structure,
Using the learning image and the correct answer value, calculate a movement amount for correcting the estimated position of the detection target of the image identified by the weak classifier,
An estimation method, wherein the detection target object is estimated while moving the estimated position of the detection target object in the image using the weak classifier.   A program for causing a computer to execute the classifier generation method according to claim 7.   A program for causing a computer to execute the estimation method according to claim 8.

Images