TWM525498U - Image processing system for calculating image sub-pixel - Google Patents

Description

Image processing system for calculating image sub-pixels

The present invention provides an image processing system for calculating image sub-pixels, and more particularly, a system for storing sub-pixels by sub-pixel displacement information and image matching.

Obtaining small movements Under the current trend of technology, it is indispensable information, from high-precision machining, integrated circuit exposure alignment, thermal expansion coefficient correction, etc., all need to accurately and calculate a small amount of displacement.

There are many methods for calculating the displacement of sub-pixels using images, which are mainly divided into two different ways, one is to use image feature point calculation, such as: SIFT (Scale Invariant Feature Transform), Corner, and so on. By extracting image information, it is determined whether the feature points of each image belong to the same image feature. The result of the detection is to divide the points on the image into different subsets. These subsets often belong to isolated points, continuous curves or continuous regions, and then perform similar operations by two different feature points. The calculation of the feature points is performed to obtain the displacement between the two images. The feature point or feature vector defined by this method includes the displacement information of the sub-pixel, that is to say, the feature described is not located on the integer pixel, so the calculated matching result also includes the displacement of the sub-pixel. Information, this method can obtain accurate image displacement information by calculating feature information, but it must use more complicated algorithms and a large number of calculations. Most of them are calculated on PC Base, and it is not easy to be presented in a hardware architecture.

The other method is image template matching, such as: SAD, SSD, or NCC, but the method performs image similarity calculation by pixel subtraction or image folding of the image, and the calculation unit has a calculation unit. The pixel is a unit, and the image conversion of the last pixel needs to be matched to obtain the displacement information of the sub-pixel. If no additional processing is added, the displacement information of the sub-pixel cannot be obtained.

Use the image template matching method to take the sub-pixel information, you must first take the original image The information of the sub-pixel image is first established by the difference in the pixel, and then the similarity calculation is performed with the image to be matched to obtain the sub-pixel information. Alternatively, the image of the sub-pixel displacement information is stored in advance, and the displacement information of the sub-pixel is determined by the similarity by image matching.

Among them, using the interpolation method for image processing can be regarded as segmentation of pixels to simulate smaller pixel information, but the obtained sub-pixel information has a limited effective resolution, and when the number of divisions is high, the image is more likely to be generated. The error on the match.

Another method for obtaining sub-pixel displacement information by using image template matching is to record the information by recording the image displacement smaller than the pixel size, and then obtaining the sub-pixel displacement by the image template matching comparison method, and recording the sub-pixel within the sub-pixel. Image information is used to obtain image displacement changes. Because image template matching is used, the entire image must be stored. If the distance to be observed is large, a large amount of image data must be stored, and the image matching is performed when the image is matched. More calculation time.

Please refer to Fig. 1, using the database storage method to calculate the sub-pixel displacement information. It is necessary to establish the sub-pixel image data to a 20cm diameter rotating platform. If the resolution to be obtained is 0.5um, and the used The pixel size of the image capture device is 7.5 um, and the image storage content of the image database is 64 pixels x 64 pixels = 4096 (pixel 2) = 4K bytes. A full circle of images will be recorded: 1256638 x 4K is about 4.794G of storage space. If the calculation is performed by the traditional image matching algorithm, it will generate a great image storage requirement.

Therefore, using a smaller storage space and reducing the complexity of the operation is the direction that needs to be improved in the calculation of the image sub-pixel.

In view of the above-mentioned deficiencies, the present invention provides an image processing system for calculating image sub-pixels, which is composed of a database system and a comparison system, wherein the database system includes: an image capturing device, a reference object surface, and an image storage device. The image processing unit and the matrix storage unit are configured; the comparison system comprises: an image calculation unit and a matrix calculation unit, wherein the database system is configured to provide data coordinates and matching information corresponding to the instant image; wherein the image capture device respectively The light source and the image sensing device are configured to generate an image; the reference object is used to provide light The information generated by the source after reflection or scattering is entered into the sensing element for imaging; the image storage device is used to store the image information obtained by the image capturing device; the image processing unit is used to convert the image into a matrix for image compression; the matrix storage unit The matrix data is compressed by the image processing unit; the comparison system is configured to calculate the sub-pixel displacement information by using the comparison result; wherein the image calculation unit is configured to use the image to calculate the image similarity to provide integer pixel displacement information; The matching result of calculating the similarity between matrices is used to provide sub-pixel displacement information.

An object of the present invention is to provide a system for effectively reducing image matching of data storage space for obtaining displacement results of sub-pixels.

A‧‧‧Database System

A1‧‧‧Image capture device

A1.1‧‧‧Light source

A1.2‧‧‧Image sensing device

A2‧‧‧ reference surface

A3‧‧‧Image storage device

A4‧‧‧Image Processing Unit

A5‧‧‧ Matrix Storage Unit

B‧‧‧ comparison system

B1‧‧‧Image Computing Unit

B2‧‧‧Matrix calculation unit

Figure 1: Schematic diagram of the storage of data in the image format.

Figure 2: A block diagram of a preferred embodiment of the present invention.

Figure 3: Schematic diagram of the image processing of this creation.

Figure 4: Schematic diagram of the matching of this creative image template.

Figure 5: Schematic diagram of the sub-pixel displacement matching of this creation.

Figure 6: Schematic diagram of the present embodiment.

Please refer to FIG. 2, which is a block diagram of a preferred embodiment of the present invention. As can be clearly seen from the figure, the image processing system for sub-pixel image calculation of the present invention comprises a database system A and a comparison system B. The database system A includes an image capturing device A1, a reference object surface A2, an image storage device A3, an image processing unit A4, and a matrix storage unit A5. The comparison system B includes an image calculation unit B1 and a matrix calculation unit B2.

The image capturing device A1 is respectively a light source A1.1 and an image sensing device A1.2, and is irradiated on the reference object surface A2 by using the light source A1.1, and the reflected light source or the scattered light source information is introduced into the image sensing device A1.2. .

The integer pixel displacement image generated by the image capturing device A1 illuminating the reference object surface A2 and the corresponding coordinate information are stored in the image storage device A3. The sub-pixel information smaller than the integer displacement is converted by the image processing unit A4 and stored in the matrix storage unit A5. .

Referring to FIG. 3, the image processing unit A4 converts the 64 pixel x 64 pixel image into a conversion mode; adds the grayscale values of the base rows in each column in units of columns, and the grayscale values of the even rows are After subtracting the last value, a 64-pixel x 64-pixel image is converted into a 1 x 64 array by the above description, wherein each column value is represented by 2 Bytes, and the image is represented by 2 x 64 Bytes and stored in In the matrix storage device A5.

The image capturing device A1 illuminates the real-time image generated by the reference object surface A2, and performs similarity calculation through the image computing unit B1 in the comparison system B and the image storage device A3 in the database system A to obtain the highest similarity matching. Image location. Then, the corresponding matrix data is taken out by the matrix storage device A5 in the database system A, and the real-time image is converted into matrix data by the image processing unit A4, and the matrix data in the matrix storage device A5 is matched with the matrix calculation unit B2, and the calculation is performed. The matrix with the highest similarity and the displacement of the sub-pixel.

Referring to FIG. 4, the image calculation unit B1 in the comparison system B performs image template matching, and the image in the image storage device A3 in the database system A is extracted from the real image to perform template matching.

Please refer to Figure 5, if the image displacement is less than one pixel, as shown in the figure. Referring to FIG. 6, the matrix calculation unit B2 in the comparison system B performs a matrix operation on the matrix of the real-time image generated by the instant image transmission image calculation unit B1 and the matrix storage device A5 in the database system A, and the minimum value is obtained. The basis of the highest similarity, and then the sub-pixel displacement information.

Please refer to Fig. 6. If the image is recorded with a resolution of 0.5um at a pixel size of 7.5um, all the images are matrix-converted by the image processing unit A4, and the matrix information is stored in the database system A after conversion. In the matrix storage device A5, the partial pixel of the integer pixel stores the image of 64 pixels x 64 pixels stored in the image storage device A3 in the database system A, taking the circumference of 20 cm in diameter as an example: the number of images stored in integer pixels It is 200 π mm / 7.5 um = 83776 images. The number of matrices stored in Phase Shift is 200 π mm/0.5um=1256638, and the required storage is approximately: 83776 * 4K bytes + 1256638 * (64 * 2) bytes = 470.422MB < 512MB.

This result is much smaller than the database storage space that is reduced by about 10 times by simply using image storage, and the number of calculations is greatly reduced.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the scope of the present invention, that is, the shape, structure, characteristics and spirit described in the scope of the patent application are equally changed. Modifications shall be included in the scope of the patent application of this creation.

A‧‧‧Database System

A1‧‧‧Image capture device

A1.1‧‧‧Light source

A1.2‧‧‧Image sensing device

A2‧‧‧ reference surface

A3‧‧‧Image storage device

A4‧‧‧Image Processing Unit

A5‧‧‧ Matrix Storage Unit

B‧‧‧ comparison system

B1‧‧‧Image Computing Unit

B2‧‧‧Matrix calculation unit

Claims (3)

An image processing system for calculating image sub-pixels, comprising: a database system comprising an image capturing device, a reference object surface, an image storage device, an image processing unit and a matrix storage device, wherein the image capturing device The device is electrically connected to the image storage device and the image processing unit, and the image processing unit is electrically connected to the matrix storage device; and a comparison system includes an image calculation unit and a matrix calculation unit; wherein the image capture The device can project a projection light to illuminate the reference object surface, and the reflected light of the reference object surface is received by the image capturing device and transferred to the image storage device or the image processing unit for storage. The image processing system can be used for calculating an image sub-pixel as described in claim 1, wherein the image processing unit adds a single row of images and subtracts the even rows to obtain a matrix compression array. An image processing system for calculating image sub-pixels, as described in claim 1, wherein the image capturing device includes a light source and an image sensing device, the light source can generate the projection light, and the image sensing device The reflected light can then be received.