CN106570462A - Lip movement feature extraction method and system capable of illumination robustness improvement - Google Patents

Abstract

The present invention provides a pre-processing method for de-illumination, a lip movement feature extraction method and system for improving the robustness of illumination, and the pre-processing method for de-illumination includes a median filtering step, a Gamma correction step, a multi-scale Retinex filtering step, and contrast equalization transformation step. The beneficial effects of the present invention are: the improvement of the LBP feature extraction method of the present invention further improves lip reading recognition to a certain extent, and the feature extraction method is not easily affected by changes in external light and has strong robustness.

Description

Lip movement feature extraction method and system for improving illumination robustness

technical field

The invention relates to the technical field of image processing, in particular to a lip movement feature extraction method and system for improving illumination robustness.

Background technique

With the development of natural human-computer interaction technology, lip reading has become a hot spot in this technology field. However, most of the current research on lip reading technology stays in the ideal lighting environment in the laboratory, and there are very few researches on lip reading recognition under variable lighting.

Contents of the invention

The present invention provides a kind of de-illumination pretreatment method, comprises the following steps:

Median filtering step: used to denoise the image;

Gamma correction step: used to correct the gray distribution of the lip image;

Multi-scale Retinex filtering steps: Gaussian surround function is used, specifically:

S(x,y)=R(x,y)L(x,y) (2)

in σ is the scaling function of the Gaussian surround function, G(x, y) satisfies ∫∫G(x, y)dxdy=1, and λ is a constant that makes ∫∫G(x, y)dxdy=1 normalized;

Contrast equalization step: used to perform contrast equalization processing on the lip image to improve the light distribution of the lip image.

As a further improvement of the present invention, in the median filtering step, a 3*3 median filtering template is used to filter the input lip image to effectively remove impulse noise;

In the Gamma correction step, the formula of Gamma correction is as shown in formula 1:

The selected gamma value of the present invention is 1/2.2;

In the multi-scale Retinex filtering step, the logarithmic domain formula is as shown in formula 4,

K value is 3, ω ₁ = ω ₂ = ω ₃ = 1/3, and the three scale factors σ are 15, 80 and 250 respectively;

In the contrast equalization step, the contrast equalization formula is shown in formula 5-7:

a is the image grayscale compression factor, which can effectively adjust the dynamic grayscale range of the lip image, τ is the threshold used to limit the excessive grayscale value, a=0.2, τ=8, formula 7 is the arctangent transformation , the transformation is a nonlinear transformation, and the image can be effectively normalized to (-τ,τ) by performing this transformation on the lip image.

The present invention also provides a lip motion feature extraction method that improves illumination robustness, the lip motion feature extraction method includes the light-removing preprocessing method, and the lip motion feature extraction method further includes the following steps: The P sampling points are used to compare with the pixel value located in the center of the circle to generate the LBP mode value. When the pixel point is in the center, the bilinear interpolation method is used to obtain the value; after the LBP feature is extracted from the lip, it does not It will be directly recognized as a feature, but the histogram of the LBP coded image will be used as a feature vector.

As a further improvement of the present invention, the lip image is first partitioned, and then the LBP histogram feature is extracted for each partition.

As a further improvement of the present invention, the unified mode of LBP is adopted, and the unified mode of LBP means that the change of 0 and 1 in the binary code of LBP does not exceed 2 times at most.

The present invention also provides a light-removing pretreatment system, comprising:

Median filtering module: used to denoise the image;

Gamma correction module: used to correct the gray distribution of the lip image;

Multi-scale Retinex filter module: Gaussian surround function is used, specifically:

S(x,y)=R(x,y)L(x,y) (2)

in σ is the scaling function of the Gaussian surround function, G(x, y) satisfies ∫∫G(x, y)dxdy=1, and λ is a constant that makes ∫∫G(x, y)dxdy=1 normalized;

Contrast equalization module: used to perform contrast equalization processing on the lip image to improve the light distribution of the lip image.

As a further improvement of the present invention, in the median filter module, a 3*3 median filter template is used to filter the input lip image to effectively remove impulse noise;

In the Gamma correction module, the formula of Gamma correction is as shown in formula 1:

The selected gamma value of the present invention is 1/2.2;

In the multi-scale Retinex filtering module, the logarithmic domain formula is as shown in formula 4,

K value is 3, ω ₁ = ω ₂ = ω ₃ = 1/3, and the three scale factors σ are 15, 80 and 250 respectively;

In the contrast equalization module, the contrast equalization formula is shown in formula 5-7:

a is the image grayscale compression factor, which can effectively adjust the dynamic grayscale range of the lip image, τ is the threshold used to limit the excessive grayscale value, a=0.2, τ=8, formula 7 is the arctangent transformation , the transformation is a nonlinear transformation, and the image can be effectively normalized to (-τ,τ) by performing this transformation on the lip image.

The present invention also provides a lip movement feature extraction system that improves illumination robustness, the lip movement feature extraction system includes the light-removing preprocessing system described in any one of claims 6 to 7, and the lip movement feature extraction system also includes : The P sampling points located on the edge of the circular template are used to compare with the pixel value located at the center of the circle to generate the LBP mode value. When the pixel point is in the center, it adopts the bilinear interpolation method to obtain the value; when the lip is extracted After the LBP feature, it is not directly recognized as a feature, but the histogram of the LBP coded image is used as a feature vector.

As a further improvement of the present invention, the lip image is first partitioned, and then the LBP histogram feature is extracted for each partition.

As a further improvement of the present invention, the unified mode of LBP is adopted, and the unified mode of LBP means that the change of 0 and 1 in the binary code of LBP does not exceed 2 times at most.

The beneficial effects of the present invention are: the improvement of the LBP feature extraction method of the present invention further improves lip reading recognition to a certain extent, and the feature extraction method is not easily affected by changes in external light and has strong robustness.

Description of drawings

Fig. 1 is the flow chart of de-illumination pretreatment method of the present invention;

Figure 2 is a Gamma correction curve;

Figure 3 is a flowchart of the Retinex filtering algorithm;

Fig. 4 is the rendering of the preprocessing method for removing light;

Figure 5 is a schematic diagram of the basic LBP operator;

Figure 6 is a schematic diagram of circular LBP sampling with three different radii;

Figure 7 is a different texture feature map obtained by the unified LBP operator;

Fig. 8 is a schematic diagram of feature extraction of block LBP histogram vector;

Figure 9 is a comparison chart of the recognition rate of different feature extraction methods under natural light;

Figure 10 is a confusion matrix diagram obtained by different feature extraction methods;

Figure 11 is a comparison chart of recognition rates corresponding to different de-illumination preprocessing algorithms;

Fig. 12 is a comparison chart of recognition rates corresponding to different lip movement feature extraction methods.

detailed description

Aiming at the deficiency of the background technology, the present invention proposes a new lip movement feature extraction method. This method is composed of a de-illumination preprocessing chain and an illumination invariant feature extraction operator, which improves the lip reading recognition effect under variable illumination from two aspects.

The de-illumination preprocessing method of the present invention is applied before lip movement feature extraction to filter out the influence of external illumination noise. The entire preprocessing algorithm is composed of four parts, and each step will compensate the light interference to a certain extent, and its program flow chart is shown in Figure 1. The following will describe them respectively according to the sequence of FIG. 1 .

Median filtering: During the formation and transmission of video images collected by the camera, a large amount of impulse noise is often generated due to external noise interference. For analog signals, the impact of impulse noise is not great, but in the transmission of digital signals, impulse noise will greatly affect the image quality. Impulse noise can be mainly divided into salt and pepper noise and random value impulse noise. In order to reduce the influence of noise, various filtering methods can be used to denoise the image. Median filtering has been widely used in image noise reduction processing because it can filter out impulse noise very well and protect some details in the image at the same time. The present invention uses a 3*3 median filtering template to filter the input lip image to effectively remove impulse noise.

Gamma correction: Affected by changes in external lighting, uneven lighting distribution often occurs in the obtained lip images. The most typical situation is that part of the lip is too bright due to light reflection and another part is too dark due to occlusion of light. In order to effectively correct the gray distribution of the lip image in the image preprocessing stage, the present invention further performs gamma correction on the lip image to improve the lip image after the median filter processing. Light distribution. The formula of Gamma correction is shown in Equation 1.

The gamma value selected in the present invention is 1/2.2, and its calibration curve is shown in FIG. 2 . It is not difficult to see from Figure 2 that by performing gamma correction on the image, the gray value of the low gray area (dark area) of the image is stretched, and the gray value of the high gray area (highlight area) is compressed, so that the whole Improved lighting distribution for images of lips.

Multi-scale Retinex filtering: Multi-scale Retinex filtering (MSR) is a de-illumination filtering algorithm widely used in the field of image processing in recent years. Multi-scale Retinex filtering is composed of the most basic single-scale Retinex filter. The Retinex algorithm believes that the image is composed of the incident component L(x, y) and the reflection component R(x, y), as shown in formula 2. Therefore, using the Retinex algorithm to filter an image is essentially to accurately calculate the incident component of an image and eliminate the component in the original image. Since it is a singular problem in the mathematical model to directly calculate its incident component in the input image, it can only be approximated by mathematical methods. The present invention adopts Gaussian surround function to accomplish this task, and concrete method is as follows:

S(x,y)=R(x,y)L(x,y) (2)

in σ is the scaling function of the Gaussian surround function, G(x,y) satisfies ∫∫G(x,y)dxdy=1, and λ is a constant that makes ∫∫G(x,y)dxdy=1 normalized. The advantage of using the logarithmic operation is that the multiplication and division relationship between the incident component and the reflection component can be converted into an addition and subtraction relationship to simplify the operation. On the other hand, the logarithmic operation itself also has a certain light filtering function. The block diagram of the entire filtering algorithm is shown in Figure 3.

In the Gaussian surround function used in the present invention, σ is the most important parameter. When σ is small, the grayscale dynamic range compression ability of the image is strong, which can better highlight the details of the lip image, but at the same time it will also cause a certain degree of image distortion; when σ is large, the lip image has high fidelity , but at the same time correspondingly weakens its ability to compress the dynamic range of the gray scale. In order to make up for this defect, the present invention uses a multi-scale Retinex algorithm to filter images, and its logarithmic domain formula is shown in Equation 4.

Here, in order to ensure that the filter function has the advantages of the high, medium and low scales of the single-scale Retinex filter, the value of K is 3, and the filters of the three scales have the same weight, that is, ω ₁ =ω ₂ = ω ₃ =1/3. After repeated tests, the filter achieves the best filtering effect when the three scale factors σ are 15, 80 and 250 respectively.

Contrast equalization step: After multi-scale Retinex filtering, the illumination of the lip image has been significantly improved. However, since the grayscale of the image after the above filtering is distributed in different grayscale ranges, and the grayscale range is very narrow, the final step of the light-removing preprocessing method proposed in the present invention is to perform contrast equalization on the lip image. In order to achieve the purpose of finally improving the light distribution of the lip image.

The contrast equalization formula of the present invention is shown in Formula 5-7.

Here, a is the image grayscale compression factor, which can effectively adjust the dynamic grayscale range of the lip image. τ is a threshold used to limit excessive grayscale values. This patent takes a=0.2, τ=8. Equation 7 is the arctangent transformation, which is a nonlinear transformation, and the image can be effectively normalized to (-τ,τ) by performing this transformation on the lip image.

Fig. 4 shows the effect diagram after being processed by the de-illumination preprocessing method proposed by the present invention. Each row in the figure is a different lighting situation. The first image in each row is the lip image without any processing, and the remaining 4 images are the effects after different steps of processing. It can be seen from Fig. 4 that the de-illumination preprocessing method proposed by the present invention has a good effect.

Currently, the most representative local feature extraction method is Local Binary Patterns (LBP). Due to its advantages of high resolution and insensitivity to monotonous changes in gray levels, this feature method has been widely used in face recognition, and it has been proved to have good illumination robustness in variable illumination environments. However, the application of this method in the field of lip-reading is rarely mentioned. For this reason, the present invention applies this excellent feature extraction method to the lip-reading subject of research to improve lip-reading recognition under variable light. Effect.

The basic operation unit of the LBP operator is a module composed of 9 pixels, and the center pixel is used as the threshold, and the 8 adjacent pixels around it are processed sequentially. If the value of the adjacent pixel is greater than the center threshold, the result is 1, otherwise it is 0. Then these 8 values are connected in order to form the LBP code value, and finally the binary LBP code value is converted into a decimal number to obtain the LBP mode value of the central pixel. The specific process is shown in Figure 5. Its corresponding mathematical expression is shown in formula 8.

Although the basic LBP operator has achieved a certain recognition effect, it also has the defects of fixed sampling range and limited number of sampling points. In the extended LBP mode, the original square template is replaced by a circular template with radius R. The P sampling points located at the edge of the circular template are used for comparison with the pixel values located at the center of the circle to generate the LBP mode value. When the center of the pixel point is selected, the bilinear interpolation method is used to obtain the value). This effectively enhances the sampling ability of the LBP operator, which can sample different numbers of neighboring pixels in different ranges of a certain pixel in the image, and can extract more effective local features, so the extracted feature recognition ability Significantly outperforms basic LBP features. Figure 6 is the LBP sampling diagram of different radii and sampling points.

Note that the coordinates of the central pixel are (x _c , y _c ), then the coordinates of the sampling points located on the edge of the circular template can be obtained by the following formula:

x _p ＝x _c +Rcos(2πp/P) (8)

y _p =y _c +Rsin(2πp/P) (9)

If the obtained coordinate value is not an integer, you need to use the interpolation method to calculate the corresponding pixel value. Here, the present invention uses a bilinear interpolation method. The mathematical expression of the circular LBP operator is shown in formula 10.

Among them, g _p is the pth pixel on the edge of the circular template, and g _c is the center pixel of the circular template. According to formula 10, the three LBP modes in Fig. 6 can be denoted as LBP _8,1 , LBP _12,2.5 and LBP _16,4 respectively.

After the LBP feature is extracted from the lip, it is not directly recognized as a feature, but the histogram of the LBP coded image is used as a feature vector. In this way, the histogram vector formed by the LBP operator with 8 sampling points has a total of 2 ⁸ =256 dimensions, and the LBP operator with 16 sampling points has 2 ¹⁶ =65536. Obviously, such a feature vector dimension is too high, which will directly reduce the recognition speed of lip reading recognition. To solve this problem, the present invention adopts the unified mode of LBP. The so-called LBP unified mode means that the change of 0 and 1 in the LBP binary code does not exceed 2 times at most. In fact, this means that the changes of 0 and 1 in the LBP unified mode can only be 0 or 2 times, because the LBP codes are distributed circularly. For the LBP encoding of 8 sampling points, there are only two cases where the change of 0 and 1 is 0 times, which are 00000000 and 11111111 respectively, as shown in Figures 7a) and 7b). There are P(P-1) cases where there are 2 changes of 0 and 1. For the LBP unified mode operator, we denote it as LBP _P ^u,2 _R . Since the unified mode of LBP contains most of the information of the lip image, and the non-uniform mode of the lip image contains a lot of noise information, the recognition rate of lip reading obtained by using the unified mode of LBP is higher.

There are two important benefits of adopting the LBP unified mode operator. The first benefit is that it can save memory space. The unified mode LBP operator classifies all the LBP codes with 0 and 1 changes more than 2 times into one category, thus reducing its feature dimension. The dimension of the LBP histogram vector in the non-uniform mode is 2P, while the dimension of the LBP histogram vector in the unified mode is only P(P-1)+3. For example, when the number of sampling points is 16, the dimension of the LBP histogram vector in the unified mode is only 243, which is greatly reduced compared to 65536. The second benefit of adopting the LBP unified mode is that the unified mode itself contains the most important features reflecting local information of the lip image, such as spots, line segments, edges and corners in the image, as shown in Figure 7.

Table 1 Experimental results corresponding to different LBP parameters

In order to determine the optimal LBP operator suitable for the lip-reading subject of the present invention, the present invention uses different LBP parameters to conduct multiple sets of recognition tests. The test results are shown in Table 1. Among them, the recognition rate corresponding to each LBP parameter is the average value of 20 test results. At the same time, in order to evaluate the stability of the recognition result, on the basis of the recognition rate, the present invention also calculates the standard deviation of the recognition rate obtained by different LBP parameters. It is not difficult to see from Table 1 that when P=8 and R=3, the recognition rate is the highest, the standard deviation is the smallest, and the stability of recognition is also the best. Therefore, this patent will use this set of parameters for subsequent experiments.

The above-mentioned LBP feature extraction is carried out in the entire lip image, so the microstructure information describing the lip image is missing in the final LBP histogram vector, which also reduces the performance of the lip reading system to some extent. Recognition rate. The LBP histogram represents the first-order statistical characteristics of the gray value in the image encoded by LBP. However, it is impossible to know the position distribution information of each gray level of the image only by extracting a single LBP histogram, and naturally there is no way to describe the microstructure information in the lip image. Usually, in a lip image, the difference in local information of different regions is relatively large. If the LBP feature extraction operator is applied to the entire lip image to obtain the LBP coded image of the previous lip image, and then on this basis, the histogram statistics of the LBP image are performed to generate the LBP histogram, then the entire lip image In the LBP histogram generated from the lip image, the local difference information in the lip image is usually lost. Therefore, the method adopted in this patent is to partition the lip image first, and then extract the LBP histogram feature for each partition separately, so that the microstructure information of the lip image can be increased while retaining its local information. Based on this idea, this patent first divides a lip image with a size of M×N into A×B partitions of the same size, so that the size of each partition is (M/A)×(N/B), The LBP histogram features are obtained for these partitions whose size is (M/A)×(N/B). By testing different block methods, this patent finally selects the 3×3 block method, and the specific process is shown in FIG. 8 .

The above partition-based LBP feature extraction method can well extract the local features of the lip image, so it has certain robustness to external illumination changes. However, this method cannot extract the global features of the lip image, so the lip features cannot be fully described. In order to further extract the representative features in the lip image, this patent introduces the PCA technology and combines the LBP feature extraction method, so as to extract both the local features of the lip image and its global features. Further improve the lip reading recognition rate. Another advantage brought by the introduction of the PCA method is that it can reduce the dimensionality of lip features. The image sequence corresponding to each number in the lip-reading database collected by this patent has more than 25 frames, so the lip movement feature corresponding to a number is at least 25 times that of a single-frame lip image. Therefore, using the PCA method to reduce the dimensionality of the LBP block histogram vector can not only improve the recognition rate of the lip reading system, filter out the interference caused by some lighting changes, but also improve the recognition rate of the lip reading system.

Based on the self-built lip-reading database, by adopting the SVM recognition method, this patent conducted four different sets of comparative experiments to verify the effectiveness of the illumination preprocessing algorithm and the LBP-based feature extraction algorithm proposed in this patent. Among them, the first group is a comparison test of the effects of different feature extraction methods under natural light. By comparing the lip movement feature extraction algorithm proposed in this patent with the currently commonly used three methods of PCA, DCT and LBP, the recognition effect of the lip reading system proposed in this patent on numbers is reflected. This group of experiments uses subset 1 in the database for experiments, and 5 samples are randomly selected from subset 1 for training each time, and the remaining 20 samples are used for recognition testing. Each feature extraction method carries out the above test process 20 times continuously, and finally takes the average value, so as to obtain the recognition effect of different feature extraction methods for each number. The test results are shown in Figure 9 and Figure 10.

It can be seen from Figure 9 that the recognition of DCT method is the highest under natural light, which is better than other methods. However, the recognition rate of the method proposed in this patent is equivalent to that of PCA, and the recognition effect of the LBP method is the worst. At the same time, for different numbers, the recognition rate of 6 and 7 is the lowest, and the recognition rate of 5 and 8 is the highest. This is because the pronunciation of 5 and 8 is more unique than the pronunciation of other numbers.

Figure 10 is the confusion matrix corresponding to different lip movement feature extraction methods. where each row represents the actual class of a different digit and each column represents the class predicted by the SVM classifier for each digit. It can be seen from Figure 10 that the recognition rate of the two numbers 6 and 7 is not only low, but also the degree of mutual confusion is the most obvious, which is caused by the similarity in the pronunciation of the two numbers. In addition, the degree of confusion between 0 and 1 is second only to 6 and 7, and the recognition rate is not high enough. Generally speaking, the lip-reading recognition system of this patent performs well under natural light conditions, and the average recognition rate reaches about 80%.

The second group of experiments used the improved LBP lip movement feature extraction method proposed in this patent to compare the recognition effects under different lighting preprocessing algorithms. In this experiment, 5 samples in subset 1 are randomly selected for training the SVM model each time. The test samples are the remaining 20 samples in subset 1 and all samples in subset 2, subset 3 and subset 4. The experiment The result is shown in Figure 11. Among them, NONE means that no de-illumination preprocessing algorithm is used, HE means histogram equalization, and HF means homomorphic filtering. It can be seen from Figure 11 that when the lighting conditions change, compared with directly extracting lip movement features, the recognition rate of lip reading will be improved after using the de-illumination preprocessing algorithm. Among them, the effects of histogram equalization and homomorphic filtering are equivalent, and the de-illumination preprocessing algorithm proposed in this patent has the best effect. However, it is worth noting that compared with natural lighting conditions, the lip-reading recognition rate generally drops a lot when the lighting changes. Although this patent adopts compensation measures for the influence of light, it still cannot compare with the effect of lip reading recognition under the condition of constant light.

The third group of experiments used the de-illumination preprocessing algorithm proposed by this patent to compare the recognition rates of different lip movement feature extraction algorithms. The test procedure is the same as that of the second group of experiments, and the experimental results are shown in Figure 12. It can be seen from Figure 12 that although feature extraction methods such as PCA and DCT have a high recognition rate in subset 1, for other subsets, the recognition rate is much lower than that of LBP and the method proposed in this patent. It can be seen that LBP is indeed a light-robust feature extraction method, and the improvement of the LBP feature extraction method in this patent has also improved lip reading recognition to a certain extent. In contrast, traditional feature extraction methods are vulnerable to external lighting changes and have poor robustness.

The above content is a further detailed description of the present invention in conjunction with specific preferred embodiments, and it cannot be assumed that the specific implementation of the present invention is limited to these descriptions. For those of ordinary skill in the technical field of the present invention, without departing from the concept of the present invention, some simple deduction or replacement can be made, which should be regarded as belonging to the protection scope of the present invention.

Claims (10)

1. one kind removes light irradiation preprocess method, it is characterised in that comprise the steps：

Median filtering step：For carrying out denoising to image；

Gamma aligning steps：For being corrected to the intensity profile of lip image；

Multiple dimensioned Retinex filter steps：Function is surround using Gauss, specially：

S (x, y)=R (x, y) L (x, y) (2)

$\begin{array}{c}\log R(x,y)=log\[S(x,y)/L(x,y)\]\\ =\log S\left(x,y\right)-\log \[S\left(x,y\right)\⊗G\left(x,y\right)\]\end{array}---\left(3\right)$

Whereinσ is scaling function of the Gauss around function, and G (x, y) meets ∫ ∫ G (x, y) Dxdy=1, λ are to make the normalized constants of ∫ ∫ G (x, y) dxdy=1；

Contrast equalization procedures：Improve lip image irradiation to reach for contrast equalization processing is carried out to lip image Distribution situation.

2. it is according to claim 1 to remove light irradiation preprocess method, it is characterised in that in the median filtering step, to adopt The lip image being input into is filtered effectively to remove impulsive noise with the medium filtering template of a 3*3；

In the Gamma aligning steps, the formula of Gamma corrections is as shown in Equation 1：

$I_{(x,y)}^{\′}={\left(\frac{I_{(x,y)}}{255}\right)}^{\mathrm{\γ}}\×255,\mathrm{\γ}\∈(0,1)---\left(1\right)$

γ-value selected by the present invention is 1/2.2；

In the multiple dimensioned Retinex filter steps, log-domain formula is as shown in Equation 4,

$\log R\left(x,y\right)=\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{\mathrm{\ω}}_k\left\{\log S\left(x,y\right)-\log \[S\left(x,y\right)\⊗G_k\left(x,y\right)\]\right\}---\left(4\right)$

K values are 3, ω₁=ω₂=ω₃=1/3,3 scale factors σ are respectively 15,80 and 250；

In the contrast equalization procedures, contrast equalizes formula as shown in formula 5-7：

$I(x,y)=\frac{I(x,y)}{{\left(mean\right(|I\left(x^{\′},y^{\′}\right){|}^a\left)\right)}^{1/a}}---\left(5\right)$

$I(x,y)=\frac{I(x,y)}{(mean{\left(min{\left(\mathrm{\τ},\left|I\left(x^{\′},y^{\′}\right)\right|\right)}^a\right)}^{1/a}}---\left(6\right)$

$I(x,y)=\mathrm{\τ}\frac{2}{\mathrm{\π}}\mathrm{arc}tan\left(I\right(x,y)/\mathrm{\τ})---\left(7\right)$

A is gradation of image compressibility factor, can effectively adjust the dynamical gray scope of lip image, and τ is excessive for limiting Gray value threshold values, a=0.2, τ=8, formula 7 be contact transformation anyway, and this is transformed to nonlinear transformation, by lip image Doing the conversion can effectively by image normalization to (- τ, τ).

3. a kind of lip for improving illumination robustness moves feature extracting method, it is characterised in that the dynamic feature extracting method of the lip includes Light irradiation preprocess method is removed described in any one of claim 1 to 2, the dynamic feature extracting method of the lip also comprises the steps：It is located at The P sampled point at circular shuttering edge is used for and the pixel value positioned at the center of circle makes comparisons to generate LBP mode values, pixel dot center When, adopt bilinear interpolation to carry out value to which；After LBP features are extracted to lip, can't be directly as spy Levy and be identified, but using the rectangular histogram of LBP code patterns as characteristic vector.

4. lip according to claim 3 moves feature extracting method, it is characterised in that carry out subregion to lip image first, Then LBP histogram features are extracted respectively to each subregion again.

5. lip according to claim 3 moves feature extracting method, it is characterised in that using the More General Form of LBP, LBP systems One pattern refers to that in LBP binary codings 0 and 1 change is no more than 2 times.

6. one kind removes illumination pretreatment system, it is characterised in that include：

Medium filtering module：For carrying out denoising to image；

Gamma correction modules：For being corrected to the intensity profile of lip image；

Multiple dimensioned Retinex filtration modules：Function is surround using Gauss, specially：

S (x, y)=R (x, y) L (x, y) (2)

$\begin{array}{c}\log R(x,y)=log\[S(x,y)/L(x,y)\]\\ =\log S\left(x,y\right)-\log \[S\left(x,y\right)\⊗G\left(x,y\right)\]\end{array}---\left(3\right)$

Whereinσ is scaling function of the Gauss around function, and G (x, y) meets ∫ ∫ G (x, y) Dxdy=1, λ are to make the normalized constants of ∫ ∫ G (x, y) dxdy=1；

Contrast equalizes module：Improve lip image irradiation to reach for contrast equalization processing is carried out to lip image Distribution situation.

7. it is according to claim 6 to remove illumination pretreatment system, it is characterised in that in the medium filtering module, to adopt The lip image being input into is filtered effectively to remove impulsive noise with the medium filtering template of a 3*3；

In the Gamma correction modules, the formula of Gamma corrections is as shown in Equation 1：

$I_{(x,y)}^{\′}={\left(\frac{I_{(x,y)}}{255}\right)}^{\mathrm{\γ}}\×255,\mathrm{\γ}\∈(0,1)---\left(1\right)$

γ-value selected by the present invention is 1/2.2；

In the multiple dimensioned Retinex filtration modules, log-domain formula is as shown in Equation 4,

$\log R\left(x,y\right)=\underset{k=1}{\overset{K}{\mathrm{\Σ}}}{\mathrm{\ω}}_k\left\{\log S\left(x,y\right)-\log \[S\left(x,y\right)\⊗G_k\left(x,y\right)\]\right\}---\left(4\right)$

K values are 3, ω₁=ω₂=ω₃=1/3,3 scale factors σ are respectively 15,80 and 250；

In contrast equalization module, contrast equalizes formula as shown in formula 5-7：

$I(x,y)=\frac{I(x,y)}{{\left(mean\right(|I\left(x^{\′},y^{\′}\right){|}^a\left)\right)}^{1/a}}---\left(5\right)$

$I(x,y)=\frac{I(x,y)}{(mean{\left(min{\left(\mathrm{\τ},\left|I\left(x^{\′},y^{\′}\right)\right|\right)}^a\right)}^{1/a}}---\left(6\right)$

$I(x,y)=\mathrm{\τ}\frac{2}{\mathrm{\π}}\mathrm{arc}tan\left(I\right(x,y)/\mathrm{\τ})---\left(7\right)$

A is gradation of image compressibility factor, can effectively adjust the dynamical gray scope of lip image, and τ is excessive for limiting Gray value threshold values, a=0.2, τ=8, formula 7 be contact transformation anyway, and this is transformed to nonlinear transformation, by lip image Doing the conversion can effectively by image normalization to (- τ, τ).

8. a kind of lip for improving illumination robustness moves Feature Extraction System, it is characterised in that the dynamic Feature Extraction System of the lip includes Illumination pretreatment system is removed described in any one of claim 6 to 7, the dynamic Feature Extraction System of the lip also includes：Positioned at circular shuttering The P sampled point at edge is used for and the pixel value positioned at the center of circle is made comparisons to generate LBP mode values, during pixel dot center, to which Value is carried out using bilinear interpolation；After LBP features are extracted to lip, directly can't be known as feature Not, but using the rectangular histogram of LBP code patterns as characteristic vector.

9. lip according to claim 8 moves Feature Extraction System, it is characterised in that carry out subregion to lip image first, Then LBP histogram features are extracted respectively to each subregion again.

10. lip according to claim 8 moves Feature Extraction System, it is characterised in that using the More General Form of LBP, LBP systems One pattern refers to that in LBP binary codings 0 and 1 change is no more than 2 times.