WO2019178717A1 - Binocular matching method, visual imaging device and device with storage function - Google Patents

Abstract

Provided is a binocular matching method, a method device of visual imaging and a device with storage function, wherein the binocular matching method comprises the following steps: obtaining a binocular image of the target area; obtaining a specific pixel point in the binocular image; obtaining a gray value or a tone value of a pixel point having a predetermined interval from the specific pixel point based on the specific pixel point; determining the similarity between respective specific pixel points on the same matching pole line based on the gray value or the tonal value, and defining at least one pair of specific pixel points having the highest similarity on the same matching pole line as a robust support point; the binocular image is matched by the robust support point. Through the above method, the moving range of image matching is reduced, the calculation amount of the algorithm is reduced, and the purpose of image fast matching is achieved.

Description

Binocular matching method, visual imaging device, and device having storage function

[Technical Field]

The present invention relates to the field of visual imaging, and in particular to a method of binocular matching, a method of visual imaging, and a device having a storage function.

 【Background technique】

With the development of computer vision, stereo vision technology is widely used in the fields of manufacturing, inspection, document analysis, medical diagnosis and military. Stereoscopic vision mainly uses the disparity value of the same point in space to calculate the three-dimensional coordinates of the spatial point on the two camera planes, and the disparity is obtained by stereo matching. Unlike ordinary image template matching, stereo matching is through Two or more pairs of images with viewpoint differences, geometric distortion, grayscale distortion, and noise interference.

The stereo matching method generally includes the following three problems:

1. The selection of primitives, that is, selecting appropriate image features (such as points, lines, phases, etc.) as matching primitives;

2. Matching criteria, which express some inherent features of the physical world as a number of rules that must be followed to match, so that the matching results can truly reflect the true face of the scene;

3. Algorithm structure, by using appropriate mathematical methods to design a stable algorithm that can correctly match the selected primitives.

According to different matching primitives, stereo vision matching algorithms are mainly divided into three categories, namely region matching, phase matching and feature matching.

Currently widely used is a binocular matching algorithm based on region matching, which mainly uses image similar regions of two reference images to achieve image matching. The performance of such algorithms depends on the choice of metrics and the strategy of the search.

The information of the image is detected by changing the size and form of the matching window. However, this calculation method has a large amount of calculation, and reducing the matching window in order to reduce the calculation amount may cause the computer to obtain insufficient gray scale change information.

 [Summary of the Invention]

The technical problem to be solved by the present invention is to provide a binocular matching method, a visual imaging method device and a device having a storage function, and perform binocular matching by acquiring a robust support point, thereby reducing the amount of calculation.

To solve the above problems, the present invention provides a method for binocular matching, comprising the following steps:

Obtaining a binocular image of the target area; acquiring a specific pixel point in the binocular image; wherein, the specific pixel point is the pixel point with the smallest hue value of the color statistical histogram in the binocular image, the pixel point with the largest difference in the neighborhood hue or the binocular a pixel point having the largest difference in edge histogram edge histogram in the image; acquiring a gray value or a tone value of a pixel point having a predetermined interval from a specific pixel point based on the specific pixel point; according to the gray value or the tonal value Determining the similarity between each specific pixel on the same matching pole line, and defining at least one pair of specific pixel points with the highest similarity on the same matching pole line as a robust support point; Binocular images are matched.

In order to solve the above problems, the present invention further provides a visual imaging device including an image collector, a processor and a memory coupled to each other, the image collector for acquiring a binocular image of a target area, and a memory for storing a binocular image. a specific pixel point, a robust support point, and program data run by the processor; the processor causes the visual imaging device to perform the following steps when acquiring the program data: acquiring a binocular image of the target area; acquiring a specific pixel point in the binocular image; Wherein, the specific pixel point is the pixel point with the smallest hue value of the color statistical histogram in the binocular image, the pixel point with the largest difference in the neighborhood hue or the pixel with the largest difference in the edge histogram in the binocular image; Obtaining a gray value or a tone value of a pixel point having a predetermined interval from a specific pixel; determining a similarity between respective specific pixel points on the same matching pole line according to the gray value or the tone value, and At least one pair of specific pixel points with the highest similarity on the same matching pole line is defined as a robust support point; through the robust support point pair binocular map Match.

In order to solve the above problems, the present invention also provides an apparatus having a storage function, a device having a storage function storing program data, and program data being executable to implement any of the above-described binocular matching methods.

The invention has the beneficial effects that: different from the prior art, the present invention obtains a robust support point by defining a specific pixel point in the binocular image, and then performs image matching by using the robust support point as a boundary point, thereby reducing image matching. The range of motion reduces the amount of computation of the algorithm and achieves the goal of fast image matching.

 [Description of the Drawings]

1 is a schematic flow chart of an embodiment of a method for binocular matching according to the present invention;

2 is a binocular view of selecting a specific pixel point in the binocular matching method of the present invention;

3 is a binocular view of obtaining a robust support point in the binocular matching method of the present invention;

4 is another binocular view of obtaining a robust support point in the binocular matching method of the present invention;

5 is a schematic diagram of an embodiment of a robust support line of the binocular matching method of FIG. 1;

Figure 6 is a schematic structural view of a visual imaging device of the present invention;

FIG. 7 is a schematic structural diagram of an embodiment of a computer readable storage device of the present invention.

【detailed description】

Referring to FIG. 1 , FIG. 1 is a schematic flowchart of a method for binocular matching according to an embodiment of the present invention. The method for binocular matching in this embodiment includes the following steps:

101: Acquire a binocular image of the target area.

The so-called binocular image refers to an image of an object taken by an image acquisition device (for example, a camera) fixed to different positions on one feature point on the object. Further, the feature point can be obtained by using the binocular image: firstly, the coordinates of the feature point on the two camera image planes are respectively obtained, and then the exact relative positions of the two cameras are obtained, and the geometric method can be used to obtain the feature point. The feature point is the coordinate in the coordinate system of a camera, that is, the position of the feature point is determined.

102: Acquire a specific pixel point in the binocular image; wherein, the specific pixel point is a pixel point with the smallest hue value of the color statistical histogram in the binocular image, the pixel point with the largest difference in the neighborhood hue or the edge histogram difference in the binocular image. The pixel with the largest value.

In a specific embodiment, the specific pixel is the pixel having the smallest hue value of the color statistical histogram in the binocular image. The color histogram describes the proportion of different colors in the entire image, so the minimum tonal value indicates that the pixel has the lowest probability of repeated occurrence.

Based on similar considerations, in another specific embodiment, the specific pixel point may also be the pixel point with the largest difference in the neighborhood hue in the binocular image, preferably the top 5% pixel point with the largest difference in the neighborhood hue. Specifically, for any pixel point (i, j), a set (i+p, j+q) formed by surrounding pixels is referred to as a neighborhood of pixel points (i, j). Where p and q are integers, and the specific values of p and q are adjusted according to the size of the currently defined neighborhood. The top 5% of the pixels with the largest difference in hue in the neighborhood are the pixels with the first 5% difference in hue between the other pixels in this neighborhood.

Based on similar considerations, in another preferred embodiment, the specific pixel point may also be the pixel point with the largest difference in the edge histogram in the binocular image, preferably the top 5% pixel with the largest difference in the edge histogram. point. Specifically, the edge of the image is important as visual perception information and one of the most basic features of the image. The so-called image edge refers to the set of pixels in the image where the surrounding pixel values have transitional changes or roof changes, that is, the most significant part of the image local variation. Firstly, the color edge information in the original binocular image is extracted by detection, and then three kinds of edge histograms which can fully reflect the edge contour content can be directly constructed, which are edge color histogram, edge distance histogram and edge direction histogram. In this embodiment, in order to simplify the calculation, only one of the histograms may be selected to extract the pixel points for subsequent calculation, such as extracting the first 5% of the pixel points having the largest difference in the edge color histogram.

Since such pixels have large differences and low probability of repeated occurrences, it is easy to display the detailed information of the target area, and the position information of the target area can be better confirmed.

In a specific embodiment, as shown in FIG. 2, FIG. 2 is a binocular view in which a specific pixel point is selected in the binocular matching method of the present invention, and the binocular image mainly includes three pixels of different colors (203, 204, 205), the target area is 201 area in the left image, and the target area is 202 area in the right image. After all the point pixels of the left and right images are acquired, since the pixel point 205 has the least proportion in the color statistics, Pixel point 205 is selected as a particular pixel point.

103: Obtain a gray value or a tone value of a pixel point having a predetermined interval from a specific pixel point based on a specific pixel point.

In a specific embodiment, as shown in FIG. 3, FIG. 3 is a binocular view of obtaining a robust support point in the binocular matching method of the present invention. The binocular image plane coordinate system is established, wherein the X axis is a horizontal direction, and the right direction is a positive direction; the Y axis is a vertical direction, and the upward direction is a positive direction (the following steps are subject to this, and will not be described again). The binocular image is subjected to gradation processing, and the gradation value of each pixel point position or the gradient direction of the tone value is calculated. The gray value of the pixel or the gradient of the tonal value mainly refers to the direction in which the gray value or the tonal value of the pixel point increases the fastest. The most common method is to first convolve the original image with the [-1,0,1] gradient operator to obtain the gradient component gradscalx in the x direction, and then use the [1,0,-1]T gradient operator pair. The original image is convoluted to obtain the gradient component gradscaly in the y direction, and then the gradient size and direction of the pixel are calculated.

In a specific embodiment, the gradient value of the gray value or the tonal value of the specific pixel is calculated according to the above manner, and the pixel with a predetermined interval from the specific pixel is selected according to the characteristic of the gradient direction, and the corresponding pixel is obtained. Gray value or tonal value. Specifically, when the gradient value of the gray value or the tonal value of the specific pixel point and the horizontal direction in the preset binocular image plane coordinate system are smaller than a preset angle, for example, 45 degrees, the acquisition is performed with a specific pixel point. A gray value or a tone value of a pixel point having a predetermined interval on both sides in the horizontal direction, preferably a gray value or a tone value of a 1-5 pixel dot buffer of the specific pixel point on both sides in the horizontal direction. When the gradient value of the gray value or the tone value of the specific pixel point and the horizontal direction in the preset binocular image plane coordinate system are greater than or equal to a preset angle, for example, 45 degrees, the vertical pixel direction is acquired. A gray value or a tone value of a pixel having a predetermined interval on both sides, preferably a gray value or a tone value of a 1-5 pixel dot buffer of the specific pixel on both sides in the vertical direction.

Still taking FIG. 3 as an example, in FIG. 3, there are specific pixel points A, B, C, D, and E, and the gradient direction of the specific pixel points A, C, and D in the figure and the preset binocular image plane are calculated. The horizontal angle in the coordinate system is less than 45 degrees. Therefore, the gray value or the tonal value of the 1-5 pixel buffers of the specific pixel points A, C, and D on both sides in the horizontal direction are obtained. The angle between the gradient direction of the specific pixel points B and E in the figure and the horizontal direction in the preset binocular image plane coordinate system is greater than 45 degrees, so that 1-5 of the specific pixel points B and E are obtained on both sides in the vertical direction. The gray value or tonal value of the pixel buffer.

104: determining, according to the gray value or the tonal value, a similarity between respective specific pixel points on the same matching pole line, and defining at least one pair of specific pixel points having the highest similarity on the same matching pole line as Robust support points.

The matching pole line is a kind of constraint. Specifically, it is the mapping of the same point on two images. Knowing the mapping point p1 on one picture, the mapping point p2 in the other picture must be relative to p1. On the polar line, this reduces the number of points to be matched.

In a specific embodiment, referring to FIG. 3, a specific pixel point A in the left figure is used as a matching pole line 301. In this embodiment, the matching limit 301 is also found by the specific pixel points C and D in the right figure. a specific pixel point corresponding to a specific pixel point A in the right picture, a gray value or a tone value of a pixel point at a predetermined interval according to a specific pixel point A, and a gray level of a pixel point at a predetermined interval of a specific pixel point C and D The value or the tonal value, the similarity between the specific pixel point A and the specific pixel point C and the specific pixel point D is calculated, and the specific pixel point with the highest similarity is taken as the specific pixel point corresponding to the specific pixel point A in the right image. . For example, since the specific pixel point A on the matching pole line 301 is in the horizontal direction, the negative direction is the gray value corresponding to white, and the horizontal direction positive direction is the gray corresponding gray value, and on the same matching pole line, only the specific pixel The negative direction of the horizontal direction is the white corresponding gray value, and the horizontal direction positive direction is the gray corresponding gray value, that is, the specific pixel A and the specific pixel point C have the highest similarity, so the specific pixel is Point A and a specific pixel point C are defined as robust support points, and the position information of this particular pixel point is recorded.

In another specific embodiment, referring to FIG. 4, FIG. 4 is another binocular view of obtaining a robust support point in the binocular matching method of the present invention. In the figure, the black square is a specific pixel point, and the specific pixel point J is used as the matching pole line 401. In this embodiment, the matching pole line 401 sequentially passes through the specific pixel points I, J, K, L, M, and N, at the matching pole line 401. In the above, it is assumed that only the gray value of the left and right preset ranges is selected, and the pixel points of the specific pixel points J and M on both sides in the horizontal direction are white corresponding gray values, and the specific pixel points I, K, L and N is gray corresponding to the pixel points on both sides of the horizontal direction, that is, the similarity of the specific pixel points J and M is the highest. Therefore, specific pixel points J and M are defined as robust support points, and position information of this particular pixel point is recorded. That is to say, when a plurality of pairs of similar specific pixel points appear in the same matching polar line, a pair of specific pixel points with the highest similarity are selected as the robust support points. In addition, if there is no corresponding specific pixel on the current matching pole line, the selection of the robust support point is performed through other matching pole lines. If there is no corresponding specific pixel on all matching pole lines, then the range of gray value or tone value selection is reduced and the robust support point is selected (or the specific pixel point is directly removed). It should be noted that the specific pixel points corresponding to the above selection do not require the two to be completely consistent, and it is only necessary to select a specific pixel point that is most similar to the error range.

105: Matching binocular images by robust support points.

The specific process of image matching of binocular images by robust support points is basically the same as the existing method of image matching by feature points, except that in the present invention, the robust support points are the boundary points of image matching. . As shown in FIG. 5, FIG. 5 is a schematic diagram of an embodiment of the robust support line of the binocular matching method of FIG. 1. The specific pixel points are filtered by the above manner to determine a specific pixel point F, G, H and a corresponding F′. , G′, H′ is the robust support point of the binocular image, and the left graph 501 and the right graph 502 are divided into 1 by the vertical support points F, G, H, F′, G′, H′. Eight areas, these boundary lines are called robust support lines. Corresponding to the corresponding robust support points, the regions are corresponding, respectively, regions 1 and 5, 2 and 6, 3 and 7, 4 and 8, matching only in the corresponding regions, effectively reducing image matching by the above method The range of movement reduces the amount of calculation.

In another specific embodiment, if the matching result cannot be obtained by the above robust support points, the selection range of the specific pixel point may be expanded, for example, selecting the top 10% of the pixels with the largest difference in neighborhood tones, or selecting the edge. The first 10% of the pixels with the largest difference in histogram, repeat the above steps to obtain a new robust support point, and match the binocular image with the previously obtained robust support point and the new robust support point until a match is obtained. result.

Different from the prior art, the present invention obtains a robust support point by defining a specific pixel point in the binocular image, and then performs image matching by using the robust support point as a boundary point, thereby reducing the moving range of image matching and reducing the algorithm. The amount of calculations achieves the purpose of fast matching of images.

Please refer to FIG. 6. FIG. 6 is a schematic structural view of a visual imaging device of the present invention. The present invention also provides a visual imaging device including an image collector 601, a processor 602, and a memory 603 coupled to each other. The image collector 601 is configured to acquire a binocular image of the target area, and the memory 603 is configured to store a binocular image, a specific pixel point, a robust support point, and program data run by the processor, and the processor 602 implements the program data. Any of the above methods of binocular matching. For detailed steps on the method of binocular matching, please refer to the foregoing description, and details are not described herein again.

Different from the prior art, the embodiment provides a visual imaging device. The visual imaging device obtains a robust support point by defining a specific pixel point in the binocular image during operation, and then uses the robust support point as a demarcation point. Image matching is performed, which reduces the moving range of image matching, reduces the calculation amount of the algorithm, and achieves the purpose of image fast matching.

Further, the present invention further provides a device having a storage function. As shown in FIG. 7, FIG. 7 is a schematic structural diagram of an embodiment of a computer readable storage device according to the present invention. The device 701 having a storage function stores program data. 702. The program data 702 can be executed to implement any of the binocular matching methods described above. In a specific embodiment, the storage device 701 may be a storage chip in the terminal, a hard disk, or a portable hard disk or other readable and writable storage tool such as a flash memory, an optical disk, or the like, or a server or the like.

Different from the prior art, the embodiment provides a device with a storage function, in which program data is stored, and when the program data is executed, a robust support point can be obtained by defining a specific pixel point in the binocular image. Then, the robust support point is used as the boundary point for image matching, which reduces the moving range of image matching, reduces the calculation amount of the algorithm, and achieves the purpose of fast image matching.

The above-described embodiments are merely illustrative of the embodiments of the present invention, and the description thereof is more specific and detailed, but is not to be construed as limiting the scope of the invention. It should be noted that a number of variations and modifications may be made by those skilled in the art without departing from the spirit and scope of the invention. Therefore, the scope of the invention should be determined by the appended claims.

Claims (20)

A method for binocular matching, comprising the steps of: Obtain a binocular image of the target area; Obtaining a specific pixel point in the binocular image; wherein the specific pixel point is a pixel point with a smallest hue value of a color statistical histogram in the binocular image, a pixel point having the largest difference in neighborhood color tone, or the binocular The pixel with the largest difference in edge histogram in the image; Obtaining a gray value or a tone value of a pixel point having a predetermined interval from the specific pixel point based on the specific pixel point; Determining the similarity between respective specific pixel points on the same matching pole line according to the gray value or the tonal value, and defining at least one pair of specific pixel points having the highest similarity on the same matching pole line as robust Support point The binocular image is matched by the robust support points. The binocular matching method according to claim 1, wherein the specificity of the gray value or the tonal value of the pixel point having a predetermined interval from the specific pixel point is obtained based on the specific pixel point The steps include: Obtaining a gradient value of the gray value or the tonal value of the specific pixel point based on the specific pixel point, selecting a pixel point having a predetermined interval from the specific pixel point according to the characteristic of the gradient direction, and acquiring a corresponding pixel Gray value or tonal value. The binocular matching method according to claim 2, wherein the selecting a pixel point having a predetermined interval from the specific pixel point according to the characteristic of the gradient direction, and acquiring a corresponding gray value or tone value The steps specifically include: When the angle of the gray value or the tone value of the specific pixel point is smaller than the preset angle in the horizontal direction in the preset binocular image plane coordinate system, acquiring the specific pixel point on both sides in the horizontal direction a gray value or a tone value of a pixel having a predetermined interval; When the gradient direction of the gray value or the tonal value of the specific pixel point and the horizontal direction in the preset binocular image plane coordinate system are greater than or equal to the preset angle, obtaining the vertical direction with the specific pixel point The side has a gray value or a tone value of a pixel of a predetermined interval. The method of binocular matching according to claim 3, wherein said predetermined interval is 1-5 pixels. The method of binocular matching according to claim 2, wherein the step of acquiring a gradient value of the gray value or the tonal value of the specific pixel point comprises: establishing a binocular image plane coordinate system, The mesh image is subjected to grayscale processing, and the gradient direction of the grayscale value of the specific pixel point position is calculated accordingly. The method of binocular matching according to claim 1, wherein the pixel point having the largest difference in the color tone of the neighborhood comprises the first 5% of the pixel points having the largest difference in the color tone of the neighborhood. The binocular matching method according to claim 1, wherein the pixel point having the largest difference in the edge histogram includes the first 5% of the pixel having the largest difference in the edge histogram. The method of binocular matching according to claim 1, wherein the step of matching the binocular image by the robust support point comprises: connecting the robust support points to be robust Support lines that match the image by the robust support lines. The binocular matching method according to claim 1, wherein after the step of matching the binocular image by the robust support point, the method further comprises: If the matching result cannot be obtained, the selection range of the specific pixel point is expanded to obtain a new robust support point; The binocular image is matched in combination with the robust support point and the new robust support point to obtain a matching result. A visual imaging device, comprising: an image collector, a processor and a memory coupled to each other, The image collector is configured to acquire a binocular image of a target area; The memory is configured to store binocular images, specific pixel points, robust support points, and program data run by the processor; The processor causes the visual imaging device to perform the following steps when executing program data: Obtaining a binocular image of the target area; acquiring a specific pixel point in the binocular image; wherein the specific pixel point is a pixel point with the smallest hue value of the color statistical histogram in the binocular image, and the neighborhood color difference is the largest a pixel point or a pixel point having the largest difference in edge histogram in the binocular image; acquiring, according to the specific pixel point, a gray value or a tone value of a pixel point having a predetermined interval from the specific pixel point; Determining the similarity between respective specific pixel points on the same matching pole line according to the gray value or the tonal value, and defining at least one pair of specific pixel points having the highest similarity on the same matching pole line as robust a support point; the binocular image is matched by the robust support point. The visual imaging apparatus according to claim 10, wherein said processor acquires a gradation value of a pixel point having a predetermined interval from said specific pixel point or based on said specific pixel point The specific steps of the tone value include: The processor acquires a gradient value of a gray value or a tone value of the specific pixel point based on the specific pixel point, and selects a pixel point having a predetermined interval from the specific pixel point according to the characteristic of the gradient direction, And get the corresponding gray value or tonal value. The visual imaging apparatus according to claim 11, wherein said processor selects a pixel point having a predetermined interval from said specific pixel point according to a characteristic according to said gradient direction, and acquires a corresponding gray value Or the step of the tone value specifically includes: Obtaining, when the gradient direction of the gray value or the tonal value of the specific pixel point and the horizontal direction in the preset binocular image plane coordinate system are smaller than a preset angle, acquiring a gray value or a tone value of a pixel point having a predetermined interval on both sides in the horizontal direction; Obtaining a vertical angle with the specific pixel point when the angle between the gradient value of the gray value or the tonal value of the specific pixel point and the horizontal direction in the preset binocular image plane coordinate system is greater than or equal to the preset angle The side has a gray value or a tone value of a pixel of a predetermined interval. The visual imaging apparatus according to claim 12, wherein said predetermined interval is 1-5 pixels. The visual imaging apparatus according to claim 11, wherein the step of the processor performing the obtaining a gradient direction of a gray value or a tonal value of the specific pixel point comprises: the processor establishing a double The eye image plane coordinate system performs grayscale processing on the binocular image, and calculates the gradient direction of the gray value of the specific pixel point position accordingly. The visual imaging apparatus according to claim 10, wherein the pixel point having the largest difference in the neighborhood hue includes the first 5% of the pixel points having the largest difference in the neighborhood hue. The visual imaging apparatus according to claim 10, wherein the pixel point having the largest difference in edge histogram includes the first 5% of the pixel having the largest difference in edge histogram. The visual imaging apparatus according to claim 10, wherein the step of the processor performing the matching of the binocular image by the robust support point comprises: the processor The robust support points are connected into a robust support line, and the images are matched by the robust support lines. The visual imaging apparatus according to claim 10, wherein the processor further comprises: after performing the step of matching the binocular image by the robust support point: If the processor is unable to obtain a matching result, expanding a selection range of a specific pixel point to obtain a new robust support point; combining the robust support point and the new robust support point to the binocular image Match and get the matching result. A device having a storage function, characterized in that the device having a storage function stores program data, and the program data can be executed by the following steps: Obtain a binocular image of the target area; Obtaining a specific pixel point in the binocular image; wherein the specific pixel point is a pixel point with a smallest hue value of a color statistical histogram in the binocular image, a pixel point having the largest difference in neighborhood color tone, or the binocular The pixel with the largest difference in edge histogram in the image; Obtaining a gray value or a tone value of a pixel point having a predetermined interval from the specific pixel point based on the specific pixel point; Determining the similarity between respective specific pixel points on the same matching pole line according to the gray value or the tonal value, and defining at least one pair of specific pixel points having the highest similarity on the same matching pole line as robust Support point The binocular image is matched by the robust support points. The device having a storage function according to claim 19, wherein said obtaining, based on said specific pixel point, a specific value of a gray value or a tone value of a pixel point having a predetermined interval from said specific pixel point The steps include: Obtaining a gradient value of the gray value or the tonal value of the specific pixel point based on the specific pixel point, selecting a pixel point having a predetermined interval from the specific pixel point according to the characteristic of the gradient direction, and acquiring a corresponding pixel Gray value or tonal value.