JP2001022928A - Image processing method - Google Patents

Abstract

PROBLEM TO BE SOLVED: To exactly extract only feature information that an image has by applying two-dimensional(2D) vector wavelet transformation to a provided vector, resolving the vector into multiple resolutions and performing arithmetic processing among vectors at respective resolution levels. SOLUTION: Gradation value data of pixels are reconstituted as vector data, these data are resolved into vector data at multiple resolution levels by applying multiple resolution analysis, based on 2D vector wavelet transformation thereto, and the information of features is extracted exactly from all the information contained in the image, by applying various kinds of operation to the vector data at respective resolution levels. The gradation data composing of the image are regarded as potentials. The image is approximated by an analytical function and vectorization is executed by operating an incline or rotation in respect to this function. At such times, when the gradation value that each of pixels in the image has is regarded as a component H of a vector potential vertical to the pixel as shown in the figure, the rate of change in the gradation value of the pixel is calculated not by slope calculation but by rotation calculation.

Description

DETAILED DESCRIPTION OF THE INVENTION

[0001]

BACKGROUND OF THE INVENTION 1. Field of the Invention The present invention relates to an image processing method. More specifically, the invention of this application relates to an image processing method effective for accurately extracting a feature included in an image.

[0002]

2. Description of the Related Art With the remarkable development of computer technology, industrial applications of image processing have been actively performed. Attempts to automatically recognize image information obtained by an image input device such as a CCD by a computer, as typified by the terms computer vision and robot vision, are actively being made, and some of those currently in practical use are being used. Not a few. These technologies are thought to contribute to labor saving and time saving by human labor, and their application fields are expected to expand in the future.

[0003] The present inventors perform an arithmetic operation by a computer on an image acquired by a CCD camera or the like,
We have been studying methods to evaluate and judge the surface condition of objects. 2. Description of the Related Art A technique for evaluating the surface state of an object by image processing has been introduced for use in quality control such as detection of defective products at a site where products are manufactured. Specifically, various appearance inspections such as evaluation of the coating state of the product, detection of scratches and chips, and contaminants have been performed, and these can be performed automatically and in a non-contact manner. Further technology development is expected for the technology for evaluating the surface state of an object.

[0004] Analytical processing methods for images used in the technology for evaluating the surface state of an object include binarization or multi-valued images, edge extraction by various differential operators, region segmentation, texture analysis, and the like. The target feature is extracted by these processing methods. Much research and development has been done on these various processing techniques.

In general, information included in an image includes, in addition to information on a target feature, a texture whose gray level changes continuously or intermittently due to illumination or disturbance light, or an electrical cause of an image acquisition device. It also contains information such as noise caused by the noise and dust and dirt on the product surface. The problem is that unnecessary information other than these features has a negative effect on image analysis, and there is a need for an image analysis method that reduces the effects of these unnecessary information and more accurately extracts image features. Have been.

[0006] The invention of this application has been made in view of the above circumstances, and is an image processing method for accurately extracting only information on the characteristics of an image in performing an appearance inspection of a product. It is intended to provide.

[0007]

In order to solve the above-mentioned problems, the invention of this application considers the gray value stored in a pixel matrix forming an image as a potential and uses the gray value of an adjacent pixel. Reconstructed as vector data by the gradient operation or rotation operation, and apply a two-dimensional vector wavelet transform to the obtained vector,
This vectorized data is decomposed into multiple resolutions,
An image processing method by a computer is provided, wherein an image is reconstructed by performing arithmetic processing between vectors at each resolution level.

The image processing method by the computer according to the invention of the present application reconstructs a coordinate system by a point where four adjacent pixels are in contact with each other with respect to pixels constituting an image, and coordinates (M,
N) is a vector whose horizontal and vertical components are (j _x , j _y ), respectively,

[0009]

(Equation 4)

(H _{i, j} , h _{i + 1, j} , h _i , _{j + 1} , h _{i + 1, j + 1}
Is calculated by using the grayscale values of four adjacent pixels, thereby performing vectorization of the image (claim 2). The image processing method by the computer according to the invention of the present application decomposes vector data obtained by vectorizing an image into vector data of multiple resolution levels by two-dimensional vector wavelet transform, and extracts vector data of each resolution level. Unnecessary information is obtained by calculating the inner product of the resolution-level vector data containing more information that does not require the target feature information and the resolution level vector data that contains both the unnecessary information and the feature information to be extracted. Reconstructing an image in which is reduced (claim 3). Specifically, the image processing method by a computer according to claim 1, wherein

[0011]

(Equation 5)

(A _i <b _i (J _xai , J _yai ) is the vector data of the resolution level in which the information of the feature to be extracted is included more than unnecessary information, and (J _xbi , J _ybi ) is unnecessary. Vector data of a resolution level including both information and information of a feature to be extracted, w _i is (J _xai ,
J _yai) and (J _xbi, calculates a matrix J _m by the inner product operation represented by the weight) to the inner product of J _ybi), reconstructing an image from the matrix J _m (Claim 4). Alternatively, the computer-implemented image processing method according to the invention of the present application is the computer-implemented image processing method according to claim 1, wherein the vector data obtained by vectorizing the image is subjected to two-dimensional vector wavelet transform to provide multiple resolution levels. The vector data at each resolution level is extracted.The vector data at the resolution level that contains more information that does not require the feature information to be extracted and the resolution that contains both the unnecessary information and the feature information that you want to extract An image in which unnecessary information is reduced is reconstructed by performing a convolution integral operation with the level vector data (claim 5).

[0013]

(Equation 6)

(A _i <b _i (J _xai , J _yai ) is the resolution level vector data in which the information of the feature to be extracted is included more than unnecessary information, and (J _xbi , J _ybi ) is unnecessary. Vector data of a resolution level including both information and information of a feature to be extracted, w _i is (J _xai ,
J _yai) and (J _xbi, calculates a matrix J _m by the integration convolution represented by weight) with respect to the inner product of J _ybi), reconstructing an image from the matrix J _m (Claim 6). With respect to the matrix J _m calculated by the inner product operation or the convolution integral operation, the image is reconstructed using the values of the respective elements constituting the matrix J _m as gray values (claim 7).

Further, the invention of the present application provides a computer-readable recording medium in which a program for executing the image processing method having the above-mentioned characteristics is recorded (claim 8).

[0016]

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS As described above, the image processing method of the present invention reconstructs gray level data of pixels as vector data, and then multiplexes the reconstructed vector data by two-dimensional vector wavelet transform. By performing resolution analysis, it is decomposed into vector data of multiple resolution levels, and by performing various operations on vector data of each resolution level, feature information is accurately extracted from all information contained in the image. It is possible to do.

The image processing method according to the invention of the present application considers the gray value data constituting an image as a potential, approximates the image with an analytical function, and performs a gradient operation, a rotation operation, and the like on the function. Perform vectorization. In particular,
When vectorization is performed using a discrete value system, vectorization is performed by calculating the difference between adjacent pixels. Hereinafter, an example of a method of reconstructing grayscale value data of a pixel as vector data will be described.

In general, image data is composed of pixels in which gray values are arranged in a matrix, and the gray values of the pixels are regarded as scalar values. In the example of the method of reconstructing the gray value data of the pixel as vector data, the gray value of the pixel is interpreted as a component of the vector potential in one direction. As shown in FIG. 1, assuming that the gray value of each pixel in the image is a vector potential component H perpendicular to the pixel, the change rate of the gray value of the pixel is calculated not by a gradient operation but by a rotation operation. . By the rotation operation, the rate of change of the gray value is expressed as vector components J _x and J _{y in the} x direction and the y direction in the pixel plane, and the following equation is established between the pixel and H.

[0019]

(Equation 7)

For each pixel, equation (I) is approximated by the following equation as a discrete value using the central difference:

[0021]

(Equation 8)

From the formula (II), considering four adjacent pixels as one unit and reconstructing coordinates consisting of contact points of the four pixels, Δx and Δy are unit lengths between the pixels, and the coordinates (M , N) in the vector (j _{x (M, N)} , j _{y (M, N)} )
Are the gray values h _{i, j} , h _{i + 1, j} of the four pixels in the periphery,
Using h _{i, j + 1} and h _{i + 1, j + 1} , it is expressed by the following equation.

[0023]

(Equation 9)

The magnitude of the vector (j _{x (M, N)} , j _{y (M, N)} ) obtained by the equation (III) is large at a portion where the difference in gray value between pixels is large, and is small at a portion where the difference is small. Then it becomes smaller. The direction of the vector (jx _{(M, N)} , zy _{(M, N)} ) obtained by the equation (III) is equal to the direction in which the boundary generated by the difference in shading is continuous. Next, a method of performing a discrete-value wavelet transform on vector data distributed two-dimensionally in the image processing method of the present invention will be described.

In general, the coordinate system of an image is a rectangular coordinate system consisting of a horizontal direction and a vertical direction. Therefore, the horizontal base x and the vertical base y are orthogonal to each other. Therefore, the coordinates (M,
Bases x and y that make up the vector Z in N);
Wherein each component j _x of the vector of each pixel obtained by the vector of (III), consisting of j _y n rows and n columns of the matrix J _x, J _y
And the inner product of the wavelet transform matrix W are

[0026]

(Equation 10)

## EQU1 ## Therefore, a wavelet transform of a vector is performed by performing a two-dimensional wavelet transform on a vector for each basis. A method for performing a multi-resolution analysis on a vector existing at n × n two-dimensional coordinates will be described. First, based on J _x and J _y , S _x and S _y which are wavelet spectrum matrices of n rows and n columns are obtained from the following equations.
Respectively. Here, n is an integer of a power of 2, and the matrices J _x and J _y are decomposed into log ₂ n + 1 levels of resolution.

[0028]

[Equation 11]

Wavelet spectrum matrix S_x, S_y
For each resolution level,
N, leaving only corresponding elements and leaving other elements as 0
Row n column wavelet spectrum matrix S_x1, S
_y1By inverse wavelet transform by the following equation
Multi-resolution analysis is performed, and a matrix J of each resolution level is obtained.
_x1, J _y1Is constructed. Where l indicates the resolution level
And 1 ≦ l ≦ log_Twon + 1. Resolution level l
Indicates that the larger the value is, the higher the spatial frequency is.

[0030]

(Equation 12)

Elements s ₁ (M, N) constituting a wavelet spectrum matrix corresponding to each resolution level 1 are:
Elements s that make up the original wavelet spectrum matrix
It is determined as follows using the value of (M, N).

[0032]

(Equation 13)

A method for performing an arithmetic process on the multi-resolution analysis image to reduce the influence of information other than necessary characteristic information included in the original image will be described below. A texture whose gray level changes continuously or intermittently due to lighting or the like is information that spreads over a relatively wide area of an image, and thus information is collected at a low level by multi-resolution analysis. Therefore, a low-level resolution matrix J considered to contain texture information
_x1 and _Jy1 are all 0,

[0034]

[Equation 14]

As a result, the matrices J _x1 and J _y1 of the respective resolution levels are obtained.
By calculating the sum of the textures, the information on the texture can be reduced. Noise due to the electrical causes of the image acquisition device, dust and dirt on the product surface are information having a high granular spatial frequency, so that information is collected at a low level by multi-resolution analysis. Therefore, the elements of the low-level resolution matrices J _x1 and J _y1 , which are considered to contain information on noise due to electrical causes and dust and dirt on the product surface, are all set to 0, and each resolution level is calculated by equation (VIII). By _calculating the sum of the matrices J _x1 and J _y1 , it is possible to reduce information on noise due to electrical causes and dust and dirt on the product surface.

When a feature having a high spatial frequency, such as a scratch on a product, is to be extracted, the resolution level including the information on the feature of interest is determined by unnecessary information such as noise due to electrical causes, dust and dirt on the product surface. May match or overlap with the resolution level that contains. In such a case, the above method cannot be used. The computer image processing method according to the invention of the present application presupposes that feature information is distributed with a wider spatial extent than unnecessary information, and the following method addresses this problem.

Since the feature information includes the information in a wider range of resolution levels than the unnecessary information,

[0038]

(Equation 15)

The matrix Jm, which is a component whose information on the feature to be extracted is dominant, is calculated by the inner product operation shown by. Here, a _i <b _i , (J _xai , J _yai ) is vector data of a resolution level containing more feature information than unnecessary information, and (J _xbi , J _{y bi} ) is feature information and It shows vector data of a resolution level that includes both unnecessary information. W _i is a weight for the inner product of the vectors. Resolution level a used for the calculation of equation (IX)
_i and b _i and weights w _i are appropriately selected empirically according to the features to be extracted.

Similarly,

[0041]

(Equation 16)

By the convolution operation shown in the above, a matrix J _m in which information of the feature to be extracted is a dominant component is calculated. Here, a _i <b _i , and (J _xai , J _yai ) is resolution-level vector data that includes more feature information than unnecessary information, and (J _xbi , J _ybi ) is feature information and unnecessary 2 shows vector data of a resolution level including both pieces of information.

By reconstructing the image using the values of the elements constituting the matrix J _m calculated by the inner product operation or the convolution integral operation as the grayscale values, it is possible to obtain an image in which the features in the image are extracted. Becomes In addition, the matrix J _{m is} subjected to threshold processing, dilation / erosion processing, multiplication with the elements of the original image, or convolution of the matrix J _m and the pixel matrix of the original image to perform image processing. It is possible to more accurately extract the features inside.

The image processing method by the computer according to the invention of this application is recorded as a program on a computer-readable recording medium. The invention of this application has the above-mentioned features, and will be described in more detail with reference to examples below.

[0045]

EXAMPLE A flaw is extracted from the sample image shown in FIG. 2 using the above method. This sample image is not an image actually acquired by a CCD or the like, but an image artificially created to specifically explain the above description. The sample image includes a stain (2) and a texture due to illumination, in addition to the scratch (1). Lighting texture
The setting is such that the gray value at the upper left of the image is the lowest, and becomes higher toward the lower right. Sample image is 33 × 33
The image data which is a pixel and is vectorized by the equation (III) has 32 × 32 elements. FIG. 3 shows the vectorized image.

The vector constituting the vectorized image has a vector in the flaw portion where the direction of the vector is equal to the direction in which the flaw extends.
The vectors have the same size and direction. Furthermore, in the dirt part, the distribution is made so as to draw a circle in the dirt part.
Paying attention to the above points, multi-resolution analysis by wavelet transform is performed on the vector image, information on texture and dirt is removed from the image, and flaws are extracted.

FIG. 4 shows vector data of each resolution level obtained as a result of the multi-resolution analysis. An image with a low resolution level contains many components in which the direction of a vector fluctuates at a low spatial frequency. In particular, it is considered that the resolution level 1 probably includes the surface information of the texture from the direction of the vector.

The resolution levels that contain a lot of dirt information are 5 and 6, but these levels also contain crack information. However, level 4 contains little dirt information. Therefore, the matrix was calculated by obtaining the sum of the inner product of the level 4 and level 5 vectors and the inner product of the level 5 and level 6 vectors. FIG. 5 shows the result of imaging the values of the elements of this matrix. From FIG. 5, it can be seen that only the information of the features included in all three levels remains for each element, and an image in which the information of dirt and texture is reduced is created.

The above method is applied to an image of a concrete wall having a flaw, which is actually photographed by a CCD, and flaws are extracted. FIG. 6 shows an image of a concrete wall surface having a flaw that was actually photographed by a CCD. FIG. 7 shows the result of vectorizing the image shown in FIG. Further, the image of the flaw obtained by performing multi-resolution analysis by vector wavelet transform, calculating the inner product for each element of the vectors from resolution level 5 to resolution level 8, and then adding the matrices consisting of the inner products. (12
8 × 128 pixels) is shown in FIG. As shown in FIG. 8, information on dirt and texture is also removed from an actual image.

[0050]

According to the invention of this application, it is possible to accurately extract information of necessary features from an image in which unnecessary information is mixed, and it is possible to perform inspection by various image processing such as visual inspection of a product. It is expected to be applied.

[Brief description of the drawings]

FIG. 1 is a schematic diagram showing vectorization of an image according to a computer image processing method of the invention of the present application.

FIG. 2 is a sample image used in an embodiment of an image processing method by a computer of the present invention.

FIG. 3 is an image obtained as a result of vectorizing the sample image shown in FIG. 2;

FIG. 4 is a vector image at each resolution level obtained as a result of applying multi-resolution analysis by two-dimensional vector wavelet transform to the vectorized image shown in FIG. 3;

FIG. 5 is an image obtained by applying a computer image processing method of the invention of the present application to the image shown in FIG. 2 and extracting a flaw obtained.

FIG. 6 is an image of a damaged concrete wall surface obtained by a CCD camera.

FIG. 7 is an image obtained as a result of vectorizing the sample image shown in FIG. 6;

8 is an image obtained by applying a computer image processing method of the invention of the present application to the image shown in FIG.

[Explanation of symbols]

 1 wound 2 dirt

────────────────────────────────────────────────── ───

[Procedure amendment]

[Submission date] July 9, 1999 (1999.7.9)

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0013

[Correction method] Change

[Correction contents]

[0013]

(Equation 6)

[Procedure amendment]

[Submission date] November 10, 1999 (1999.11.
10)

[Procedure amendment 1]

[Document name to be amended] Statement

[Correction target item name] 0021

[Correction method] Change

[Correction contents]

[0021]

(Equation 8)

[Procedure amendment 2]

[Document name to be amended] Statement

[Correction target item name] 0022

[Correction method] Change

[Correction contents]

From the formula (II), considering four adjacent pixels as one unit and reconstructing coordinates consisting of contact points of the four pixels, Δx and Δy are unit lengths between the pixels, and the coordinates (M each component _(j x _{(M, n)} of the vector you have normalized is made to _{n), j y (M,} n)) is the gray value h _i of the four pixels in the _{neighborhood, j,} h i _{+ Using 1, j} , h _{i, j + 1} , h _{i + 1, j + 1} ,
It is expressed by the following equation.

 ──────────────────────────────────────────────────続 き Continuing on the front page (72) Inventor Hiroshi Aoki 1-1390 Kamimaruko Sannocho, Nakahara-ku, Kawasaki-shi, Kanagawa 406 F-term (reference) 5L096 BA03 DA01 EA37 EA39 FA26 GA07 GA09 HA13 JA05 JA11 JA18

Claims (8)

[Claims] 1. A gray scale value stored in a pixel matrix forming an image is regarded as a potential, and is reconstructed as vector data by a gradient calculation or a rotation calculation using the gray scale values of adjacent pixels. Image processing characterized by reconstructing an image by applying a two-dimensional vector wavelet transform to the vectorized data, decomposing the vectorized data into multiple resolutions, and performing arithmetic processing between vectors at each resolution level. Method. 2. The image processing method according to claim 1, wherein:
The coordinate system is reconstructed by the points where four adjacent pixels are in contact with the pixels constituting the image, and a vector in which the horizontal component and the vertical component at the coordinates (M, N) are (jx, zy) is expressed by the following equation. ] (H _{i, j} , h _{i + 1} , j, h _i , _{j + 1} , h _{i + 1, j + 1} is the gray value of four adjacent pixels) An image processing method characterized by the following. 3. The image processing method according to claim 1, wherein the vector data obtained by vectorizing the image is decomposed into vector data of multiple resolution levels by a two-dimensional vector wavelet transform. The inner product operation is performed between the vector data of the resolution level containing more unnecessary information of the feature information for which the vector data is to be extracted and the vector data of the resolution level containing both the unnecessary information and the feature information to be extracted. And reconstructing an image from which unnecessary information has been reduced. 4. The image processing method according to claim 1, wherein: _{(A i <b i (J} xai, J yai) , the resolution level vector data information of the extracted is subject features may be included more than unnecessary information, and (J _xbi, J _ybi) are unnecessary information extraction The vector data of the resolution level including both the information of the target feature, w _i is (J _xai , J _yai ) and (J _xbi ,
An image processing method comprising: calculating a matrix J _m by an inner product operation represented by (weight of the inner product of J _ybi )); and reconstructing an image from the matrix J _m . 5. The image processing method according to claim 1, wherein the vector data obtained by vectorizing the image is decomposed into vector data of multiple resolution levels by two-dimensional vector wavelet transform. Convolution integral operation of vector data of the resolution level containing more unnecessary information of the feature information from which vector data is to be extracted and vector data of the resolution level containing both unnecessary information and the feature information to be extracted And reconstructing an image from which unnecessary information has been reduced. 6. The image processing method according to claim 1, wherein: _{(A i <b i (J} xai, J yai) , the resolution level vector data information of the extracted is subject features may be included more than unnecessary information, and (J _xbi, J _ybi) are unnecessary information extraction The vector data of the resolution level including both the information of the target feature, w _i is (J _xai , J _yai ) and (J _xbi ,
An image processing method comprising: calculating a matrix J _m by a convolution integral operation represented by a weight of the inner product of J _ybi ), and reconstructing an image from the matrix J _m . 7. The image processing method according to claim 4 or 6, wherein the image is reconstructed using the values of the respective elements constituting the matrix J _m calculated by the inner product operation or the convolution integration operation as gray values. An image processing method comprising: 8. A computer-readable recording medium on which a program for executing the image processing method according to claim 1 is recorded.