CN107991665A - It is a kind of based on fixed-focus camera to target three-dimensional coordinate method for continuous measuring - Google Patents

Abstract

The invention discloses a method for continuously measuring three-dimensional coordinates of a target based on a fixed-focus camera, which belongs to the technical field of monocular vision. It includes: pre-arranging the dot array target at the target position; reading the real-time video stream through the fixed-focus camera, adjusting the camera, aiming the camera at the dot target, so that all the dots in the target enter the camera field of view; Calculate the dots in the field of view, detect the marked dots, and calculate the center coordinates; calculate the camera extrinsic parameter matrix based on the obtained circle center coordinates. Calculate the three-dimensional coordinates according to the internal and external parameter matrix of the camera. By applying the imaging principle, the present invention introduces a target as a coordinate constraint in monocular vision, and obtains accurate required data by setting markers that are easy to identify and using a computer to accurately extract the marker image, and realizes it simply and effectively. high-precision distance measurement.

Description

A method for continuous measurement of three-dimensional coordinates of a target based on a fixed-focus camera

technical field

The invention belongs to the technical field of monocular vision, in particular to a method for continuously measuring three-dimensional coordinates of a target based on a fixed-focus camera.

Background technique

In the process of monocular 3D coordinate measurement, a certain constraint in the image sequence is usually used to transform the 3D coordinates in the target coordinate system into a coordinate sequence in a known coordinate system by means of coordinate system conversion.

In the existing non-contact ranging technology, the most widely used theories and methods are mainly multi-eye ranging and laser ranging. Multi-eye ranging is to use multiple camera devices with the same structure and performance to shoot the same object, determine the constraint relationship through feature extraction of the target in multiple groups of images, and then calculate the object in the camera coordinate system by geometric methods coordinates to determine its relative position. Multi-eye ranging is limited by the relative positions of multiple cameras. The parameters in the calculation are small and the calculation amount for high-definition images is relatively large. It is difficult to run in real time, and the feature extraction is difficult, and the error is not easy to eliminate. It can only be used for blurring. Measurement. Laser ranging is mainly carried out by using a laser rangefinder, which emits a very thin beam of laser light to the target when it is working, and the photoelectric element receives the laser beam reflected by the target, and the timer measures the time from emission to reception of the laser beam, and calculates The distance from the observer to the target. Although laser ranging has high precision, it has strong directionality, very low divergence, and relatively high cost. How to simultaneously solve the problems of large resource consumption and insufficient precision in multi-eye ranging, as well as the problems of too strong targeting and too low divergence in laser ranging, has always been a key research issue for those skilled in the art, but at present No breakthrough research progress has been published.

Contents of the invention

In order to solve the problems of large resource consumption and insufficient precision in multi-eye ranging, and the problems of too strong targeting and low divergence in laser ranging, the present invention provides a fixed-focus camera-based benchmarking A method for continuous measurement of three-dimensional coordinates of a target.

To achieve the above object, the present invention adopts the following technical solutions:

A method for continuously measuring three-dimensional coordinates of a target based on a fixed-focus camera, comprising the following steps:

Step 1, prearranging the dot array target at the target position;

Step 2, read in the real-time video stream through the fixed-focus camera. The source of the video stream is required to be a camera equipped with a fixed-focus lens, that is, the focal length of the camera is a fixed value, which does not change with the adjustment of other parameters, and the camera does not specifically require image acquisition. is a single-channel image;

Step 3, aim the fixed-focus camera at the dot array target, and adjust the fixed-focus camera appropriately so that all the dots in the dot array target enter the field of view of the fixed-focus camera;

Step 4, calculate the dots in the field of view of the fixed-focus camera, detect the marked dots, and calculate the coordinates of the center of the circle;

Step five, calculate the fixed-focus camera extrinsic parameter matrix according to the obtained circle center coordinates;

Step six, calculate the three-dimensional coordinates according to the fixed-focus camera extrinsic parameter matrix.

Further, the shape of the dot array target in step 1 is a regular shape for easy identification, such as rectangle, square, etc.;

For the dot array target in step 1, targets of different sizes and materials should be selected according to the application scenario. The size requirement is that all the dots in the target should be image-acquired and output on the dot array target in step 3 In the video field of view, occupying more than 60% of the size of the video field of view at the same time;

For the dot array target in step 1, targets of different sizes and materials should be selected according to the application scenario, and the material requirements are to ensure a high contrast between the target and the surrounding environment.

Furthermore, the dot array target in step 1 requires material requirements including lighting operations in application scenarios with weak light to ensure high contrast between the target and the surrounding environment, or the dot array target in step 1 The target directly uses the LED screen to project images to ensure high contrast between the target and the surrounding environment.

Further, after the circle point array target in step 1 takes its upper left corner circle point as the origin and the plane where it is located as the plane to establish the world coordinate system, the coordinates of all circle points are obtained by including the measurement method, or the circle in step 1 The dot array target is directly arranged with regularly distributed dot matrix dots, which simplifies the production of the target.

Further, the step four is to detect the dots, which refers to performing a series of image processing operations on the image containing the target, and calculating the coordinates of each dot in the image in its coordinate system through geometric knowledge. The specific steps are as follows:

Step 7, calculate the internal parameter matrix K through the relevant parameters of the fixed-focus camera lens;

Step 8, performing grayscale processing and binarization processing on the image acquired by the fixed-focus camera;

Step 9, perform ellipse detection on the image, and detect dots in the image;

Step ten, if all the dots cannot be detected in step nine, adjust the relevant parameters of the fixed-focus camera lens including the camera aperture parameter and re-detect until all the dots are detected;

Step 11: Calculate the circle center coordinate set corresponding to each circle point according to the detected circle point set.

Further, the internal reference matrix K in step 7 is directly calculated according to the relevant parameters of the fixed-focus camera lens, including the focal length of the fixed-focus camera, the size of the unit pixel of the sensor, and the resolution of the output image. Different fixed-focus cameras and camera lenses should have different internal reference matrices, but for the same fixed-focus camera and camera lens, the internal reference matrix does not change with other parameters.

Further, the grayscale processing in step 8 is to obtain a grayscale image after collecting the color image by the fixed-focus camera for grayscale or directly collect the grayscale image by the fixed-focus camera.

Further, the specific steps of the ellipse detection in step nine are as follows:

Step 12, firstly, edge detection is performed on the image to obtain a binarized edge contour map, and a set of point coordinates on the edge map is stored;

Step 13, for each point on the image, calculate the distance from the point in the point coordinate set obtained in step 12, and output the maximum and minimum values of the distance, which is the center of the ellipse (p, q), and the maximum value of the distance is is the length of the major axis of the ellipse a;

In step fourteen, the value of each point in the point coordinate set and the three elliptic parameters p, q, and a obtained in step thirteen are substituted into the ellipse equation, and the equation is:

In step fifteen, the parameters b and θ are counted in the two-dimensional parameter space, and a set of parameters whose peak value exceeds a certain threshold is obtained, which is an ellipse.

Further, the calculation method of the fixed-focus camera extrinsic parameter matrix in step five is as follows:

Step sixteen, according to the corresponding relationship between the coordinates in the image coordinate system and the coordinates in the world coordinate system, the initial value of the external parameter matrix [RT] is obtained by solving the linear equation system;

Step seventeen, iteratively obtain the optimal transformation matrix [R T] by nonlinear least squares method.

Further, the calculation method of the three-dimensional coordinates in step six is as follows:

According to the external parameter matrix obtained in step five, the relative coordinates of any known point in the world coordinate system in the camera coordinate system are obtained through matrix transformation, so as to obtain the target three-dimensional coordinate point set.

Beneficial effect:

By applying the imaging principle, the present invention introduces a target as a coordinate constraint in monocular vision, and obtains accurate required data by setting markers that are easy to identify and using a computer to accurately extract the marker image, and realizes it simply and effectively. high-precision distance measurement.

Description of drawings

Fig. 1 is a flowchart of an embodiment of the present invention;

Fig. 2 is a flow chart of an extrinsic parameter matrix calculation module according to an embodiment of the present invention.

Detailed ways

The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

This embodiment proposes a method for continuous measurement of three-dimensional coordinates of a target based on a fixed-focus camera, which includes the following steps:

Step 1, pre-arranging the dot array target at the target position.

Specifically, the size of the target should be selected according to the actual application scenarios such as the focal length and measurement distance of the selected fixed-focus camera. It must be ensured that all the dots in the target are within the video field of view within the measurement range. It is guaranteed to occupy more than 60% of the size of the video field of view, that is, the target cannot be too large or too small in the video field of view. If the ambient light in the application scene is weak, the target can be illuminated by supplementary light measures to cause a strong contrast between the target and the environment, or the LED display can be used directly to display the content of the target on it, through Adjust the brightness to create strong contrast. The target can be an array of black dots on a white background or an array of white dots on a black background. The following uses an array of black dots on a white background as an example and will be described in conjunction with FIG. 1 .

S101, after the selection of the target is completed, initialize the coordinate system according to the coordinates of the dots in the target. The direction is the x-axis to establish a world coordinate system, the unit length is mm, and all the dot coordinates are input into the coordinate system to obtain the dot coordinate set in the world coordinate system.

Step 2, read in the real-time video stream through the fixed-focus camera.

Specifically, after the fixed-focus camera is connected to the computer, the image content obtained by the fixed-focus camera can be accessed through the computer, and the video transmission rate can reach more than 24fps.

Step 3: Aim the fixed-focus camera at the dot array target, and adjust the fixed-focus camera appropriately so that the target and the dots therein all enter the field of view of the fixed-focus camera.

S102, specifically, move the fixed-focus camera or move the target so that the target and the dots therein are within the video field of view, and at the same time ensure that they occupy more than 60% of the video field of view, that is, the target is within the video field of view. The video field of view can neither be too large nor too small. The adjustment of the fixed-focus camera includes the adjustment of parameters such as the aperture size of the camera. The present invention does not have high requirements for the clarity of the image. It only requires that the dots are all in the image and not too blurred, so that all The coordinates of the center of the circle are the datum.

S103, Step 4, calculate the circular point in the field of view of the fixed-focus camera, detect the marked circular point, and calculate the coordinates of the center of the circle.

S104. Specifically, for the fixed-focus camera used, first calculate the intrinsic parameter matrix K of the fixed-focus camera.

in, They are called the normalized focal length on the u-axis and v-axis respectively; f is the focal length of the fixed-focus camera, and dX and dY represent the size of the unit pixel on the u-axis and v-axis of the sensor, respectively. u ₀ and v ₀ represent the optical center, that is, the intersection point of the optical axis of the fixed-focus camera and the image plane, usually located at the center of the image, so their values are usually half of the resolution.

For the above-mentioned required values, they should be obtained directly from the manual of the fixed-focus camera or camera lens, and can be substituted into the calculation.

After the internal reference matrix K is calculated, the image acquired by the fixed-focus camera is processed. As shown in Figure 2, the specific operations are as follows:

S201. Perform grayscale processing and binarization processing on an image acquired by a fixed-focus camera. It is additionally explained here that if the source image is directly collected as a single-channel grayscale image, the step of grayscale processing can be omitted; if it is a three-channel image, grayscale operation must be performed.

S202. Binarize the grayscale image according to the following formula:

The threshold used can be directly passed in as a parameter by using 127.

S203, adjusting the parameters of the fixed-focus camera, performing ellipse detection on the image, and detecting dots in the image, the specific steps are as follows:

Let the elliptic equation be

First, edge detection is performed on the original image to obtain a binarized edge contour map, and the point coordinates on the edge map are stored.

For each point on the image, calculate the distance from the points in the point coordinate set obtained in the previous step, and output the maximum and minimum values of the distance, which is the center of the ellipse (p, q), and the maximum distance is the length a of the major axis of the ellipse.

Substitute the value of each point in the point coordinate set and the three ellipse parameters p, q, and a obtained in the previous step into the above ellipse equation.

The parameters b and θ are counted in the two-dimensional parameter space, and a set of parameters whose peak value exceeds a certain threshold is obtained, which is the ellipse set.

S204. Calculate a circle center coordinate set corresponding to each circle point according to the detected circle point set.

S105, step five, calculate the camera extrinsic parameter matrix according to the obtained circle center coordinates, the calculation method is as follows:

According to the corresponding relationship between the coordinates in the image coordinate system and the coordinates in the world coordinate system, the initial value of [RT] is obtained by solving the linear equations.

The optimal transformation matrix [RT] is obtained iteratively by nonlinear least squares method.

S106, Step 6, calculating three-dimensional coordinates according to the fixed-focus camera extrinsic parameter matrix.

According to the external parameter matrix obtained in step five, the relative coordinates of any known point in the world coordinate system in the camera coordinate system can be obtained. For any point in world coordinates There is a point in the camera coordinate system Satisfy

Among them, K is the desired internal reference matrix, [RT] is the desired external parameter matrix, and the calculation can be completed according to the above formula. By default, the coordinates can be entered directly Then the output 3D coordinate value represents the 3D coordinate of the circle point in the upper left corner of the target relative to the camera coordinate system.

For the limitation of the protection scope of the present invention, those skilled in the art should understand that on the basis of the technical solution of the present invention, various modifications or deformations that those skilled in the art can make without creative labor are still within the protection scope of the present invention within.

Claims (10)

1. A method for continuously measuring three-dimensional coordinates of a target based on a fixed-focus camera is characterized by comprising the following steps:

firstly, prearranging a dot array target at a target position;

reading in a real-time video stream through a fixed-focus camera;

step three, aligning the fixed focus camera to the dot array target, and properly adjusting the fixed focus camera to enable all dots in the dot array target to enter the view field of the fixed focus camera;

calculating dots in the field of view of the fixed-focus camera, detecting the target dots, and calculating the coordinates of the circle center;

calculating an external parameter matrix of the fixed focus camera according to the obtained circle center coordinates;

and step six, calculating three-dimensional coordinates according to the fixed-focus camera external parameter matrix.

2. The method for continuously measuring the three-dimensional coordinates of the target based on the fixed-focus camera according to claim 1, wherein the target shape of the dot array of the first step is a regular shape for easy identification;

all dots in the dot array target in the first step are in the video view field of image acquisition and output of the dot array target in the third step, and the dot array target occupies more than 60% of the video view field;

the dot array target of step one has a high contrast with the surrounding environment.

3. The method for continuously measuring the three-dimensional coordinates of the target based on the fixed-focus camera as claimed in claim 2, wherein the dot array target of the first step is subjected to a lighting operation under the application scene with weak light to generate high contrast with the surrounding environment, or the dot array target of the first step is directly projected by using an LED screen to generate high contrast with the surrounding environment.

4. The method for continuously measuring the three-dimensional coordinates of the target based on the fixed-focus camera as claimed in claim 2, wherein the dot array target of the first step is obtained by taking the leftmost dot as the origin and the plane where the dot array target is located as the plane to establish the world coordinate system, and then the coordinates of all the dots are obtained by a measurement method, or the dot array target of the first step is directly arranged by using regularly distributed dot array type dots.

5. The method for continuously measuring the three-dimensional coordinates of the target based on the fixed-focus camera according to claim 1, wherein the fourth step of detecting the dots comprises the following specific steps:

step seven, calculating an internal parameter matrix K according to the relevant parameters of the lens of the fixed-focus camera;

eighthly, carrying out gray processing and binarization processing on the image acquired by the fixed-focus camera;

step nine, carrying out ellipse detection on the image, and detecting dots in the image;

step ten, if all the round points cannot be detected in the step nine, adjusting relevant parameters of the fixed-focus camera lens, including camera aperture parameters, to detect again until all the round points are detected;

and step eleven, calculating a circle center coordinate set corresponding to each dot according to the detected dot set.

6. The method for continuously measuring the three-dimensional coordinates of the target based on the fixed-focus camera as claimed in claim 5, wherein the reference matrix K of the seventh step is directly calculated according to the relevant parameters of the fixed-focus camera lens, including the focal length of the fixed-focus camera, the size of the unit pixel of the sensor and the resolution of the output image.

7. The method for continuously measuring the three-dimensional coordinates of the target based on the fixed-focus camera as claimed in claim 5, wherein the graying processing of the step eight is to acquire the gray image after the color image is grayed by the fixed-focus camera or directly acquire the gray image by the fixed-focus camera.

8. The method for continuously measuring the three-dimensional coordinates of the target based on the fixed-focus camera according to claim 5, wherein the ellipse detection of the ninth step comprises the following specific steps:

step twelve, firstly, carrying out edge detection on the image to obtain a binary edge profile map, and storing a point coordinate set on the edge map;

step thirteen, calculating the distance between each point on the image and the point coordinate concentration point obtained in the step twelve, and outputting the maximum value and the minimum value of the distance, namely the center (p, q) of the ellipse, wherein the maximum value of the distance is the length a of the long axis of the ellipse;

step fourteen, substituting the numerical value of each point in the point coordinate set and the three ellipse parameters p, q and a obtained in the step thirteen into an ellipse equation, wherein the equation is as follows:

<mrow> <mfrac> <msup> <mrow> <mo>&amp;lsqb;</mo> <mrow> <mo>(</mo> <mi>x</mi> <mo>-</mo> <mi>P</mi> <mo>)</mo> </mrow> <mi>c</mi> <mi>o</mi> <mi>s</mi> <mi>&amp;theta;</mi> <mo>+</mo> <mrow> <mo>(</mo> <mi>y</mi> <mo>-</mo> <mi>Q</mi> <mo>)</mo> </mrow> <mi>s</mi> <mi>i</mi> <mi>n</mi> <mi>&amp;theta;</mi> <mo>&amp;rsqb;</mo> </mrow> <mn>2</mn> </msup> <msup> <mi>a</mi> <mn>2</mn> </msup> </mfrac> <mo>+</mo> <mfrac> <msup> <mrow> <mo>&amp;lsqb;</mo> <mo>-</mo> <mrow> <mo>(</mo> <mi>x</mi> <mo>-</mo> <mi>P</mi> <mo>)</mo> </mrow> <mi>s</mi> <mi>i</mi> <mi>n</mi> <mi>&amp;theta;</mi> <mo>+</mo> <mrow> <mo>(</mo> <mi>y</mi> <mo>-</mo> <mi>Q</mi> <mo>)</mo> </mrow> <mi>c</mi> <mi>o</mi> <mi>s</mi> <mi>&amp;theta;</mi> <mo>&amp;rsqb;</mo> </mrow> <mn>2</mn> </msup> <msup> <mi>b</mi> <mn>2</mn> </msup> </mfrac> <mo>=</mo> <mn>1</mn> <mo>;</mo> </mrow>

and step fifteen, counting the parameters b and theta in a two-dimensional parameter space to obtain a group of parameters with peak values exceeding a certain threshold value, namely the ellipse.

9. The method for continuously measuring the three-dimensional coordinates of the target based on the fixed-focus camera according to claim 1, wherein the method for calculating the external reference matrix of the fixed-focus camera in the fifth step is as follows:

sixthly, according to the corresponding relation between the coordinates in the image coordinate system and the coordinates in the world coordinate system, solving a linear equation set to obtain an initial value of an external parameter matrix [ R T ];

seventhly, iteration is carried out by utilizing a nonlinear least square method to obtain an optimal transformation matrix [ R T ].

10. The method for continuously measuring the three-dimensional coordinates of the target based on the fixed-focus camera according to claim 1, wherein the three-dimensional coordinate calculation method in the sixth step is as follows:

and D, solving the relative coordinates of any known point in the world coordinate system in the camera coordinate system through matrix transformation according to the external reference matrix obtained in the step five, so as to obtain a target three-dimensional coordinate point set.