JP7593131B2 - Projection device, detection device, and random dot pattern design method - Google Patents

Description

The present invention relates to a projection device, a detection device, and a method for designing a random dot pattern.

A technology is known in which a detection light is projected onto an object, an image is taken, and distance information to each position on the object is obtained from the obtained image data to create distance distribution data, also known as shape information or a three-dimensional map (see Patent Document 1).
The method disclosed in Patent Document 1 uses a structured light method in which a light-projecting unit projects a predetermined pattern of detection light and detects distance from changes in the position and shape of the pattern on the target object.

In the structured light method, for example, a random dot pattern is projected, and distance distribution data is calculated using the positional deviation of each dot. Here, it may seem difficult to identify the position of these random dots because they are randomly arranged, but the position of the random dots can be identified with high accuracy by calculating the autocorrelation value.

However, even if the dot pattern is random, it is not possible to pinpoint the position with high accuracy regardless of the dot pattern. For example, if a specific randomly arranged dot pattern is repeatedly arranged, there will be multiple positions where the same autocorrelation value is obtained, making it impossible to pinpoint the position accurately.

In addition, if the number of dots in a random dot pattern is large, the computational load may become too large, making high-speed processing difficult.

JP 2010-504522 A

The objective of the present invention is to provide a method for designing a projection device, a detection device, and a random dot pattern that can identify an accurate position and reduce the computational load.

The present invention solves the above problems by the following means. Note that, for ease of understanding, the following description will use symbols corresponding to the embodiments of the present invention, but the present invention is not limited to these.

The first invention is a projection device (10) that projects a random dot pattern, characterized in that the random dot pattern is a pattern in which one dot is randomly arranged in each of a predetermined size of arrangement area centered on the intersection position of a square lattice arranged at equal intervals n, and the dots are arranged so that the shortest distance between each dot is equal to or greater than a predetermined interference distance d.

The second invention is the projection device (10) described in the first invention, characterized in that the placement area is a square area of 2 m × 2 m centered on the intersection position, where m is a predetermined value indicating distance, and the relationship between the interval n and the predetermined value m satisfies m < n/2.

The third invention is a projection device (10) according to the first or second invention, characterized in that the random dot pattern is arranged so that adjacent dots can be connected continuously by straight lines with each dot being the vertex of a triangle, the straight lines that form the sides of the triangle do not intersect midway, and the triangles can be closely arranged.

The fourth invention is a detection device (1) including a projection device (10) according to any one of the first to third inventions, and an image capturing unit (20) that captures the random dot pattern projected by the projection device (10).

The fifth invention is the detection device (1) according to the fourth invention, characterized in that it includes a calculation unit that uses an index list of polygon mesh data in which the intersection positions of the square lattice are configured as the vertex positions of triangles as an index list for each dot of the random dot pattern.

The sixth invention is a method for designing a random dot pattern to be projected onto an object using a projection device (10), comprising: an arrangement step of randomly arranging one dot in each of an arrangement area of a predetermined size centered on an intersection position of a square lattice arranged at equal intervals n; a confirmation step of confirming whether the shortest distance between the dots arranged in the arrangement step is equal to or greater than a predetermined interference distance d; and a rearrangement step of moving the position of dots whose shortest distance between the dots is not equal to or greater than the predetermined interference distance d based on the result of the confirmation step.

The seventh invention is a method for designing a random dot pattern according to the sixth invention, characterized in that the arrangement area is a square area of 2 m × 2 m centered on the intersection position, where m is a predetermined value indicating distance, and the relationship between the interval n and the predetermined value m satisfies m < n/2.

The present invention provides a projection device, a detection device, and a method for designing a random dot pattern that can pinpoint a precise position.

1 is a diagram illustrating an outline of a configuration of an embodiment of a three-dimensional measurement apparatus 1 according to the present invention and distance calculation. FIG. 13 is a diagram illustrating an example of a dot pattern. 11A and 11B are diagrams illustrating a method for detecting an area that matches a dot pattern in a specific rectangular area. FIG. 13 is a diagram illustrating a method for generating a dot pattern. FIG. 13 is a diagram showing a triangular polygon mesh set in a dot pattern before being randomly arranged. FIG. 6 is a diagram showing a triangular polygon mesh set in a case where the dot pattern in FIG. 5 is rearranged to create a random dot pattern. FIG. 13 is a diagram that allows visual confirmation of a triangular polygon mesh when a randomly arranged dot pattern of the present invention is projected onto a hand.

The best mode for carrying out the present invention will be described below with reference to the drawings.

(Embodiment)
FIG. 1 is a diagram for explaining the configuration of an embodiment of a three-dimensional measurement apparatus 1 according to the present invention and an overview of distance calculation.
Note that the figures shown below, including FIG. 1, are schematic views, and the size and shape of each part are exaggerated or omitted as appropriate for ease of understanding.
In the following description, specific values, shapes, materials, etc. are given, but these can be changed as appropriate.
The three-dimensional measuring device 1 of this embodiment includes a light-projecting unit (projection device) 10, a camera 20, and a calculation unit (not shown). The camera 20 includes an image sensor unit 21 and a photographing lens 22. The three-dimensional measuring device 1 of this embodiment employs a Structured Light method in which a laser light source of a single wavelength in any one of the bands of ultraviolet light, visible light, near-infrared light, and far-infrared light is used, a pattern light is projected that projects light pulses in a circular or rectangular shape, the pattern light projected on an object is optically observed, and a distance is calculated from the observed parallax of the pattern light using trigonometry.

The light projection unit 10 includes a light projection lens 10a, a single-wavelength laser light source, and a pattern generating unit for projecting a dot pattern, as described above, and projects a randomly arranged dot pattern (detection light). The pattern generating unit may be, for example, a diffractive optical element (DOE).
The image sensor unit 21 is a camera equipped with an image sensor that receives the dot pattern projected by the light projecting unit 10 onto the detection target.

In the state shown in FIG. 1, the following relationship exists due to the similarity of triangles.
Here, the distance between the light projecting lens 10a and the photographing lens 22 is the base length d, the distance from the light projecting lens 10a to the subject is Z, the focal length of the light receiving lens is f, and the position of the subject projected on the image sensor unit is x.
Z = d f / x
Z _min = d・f/x _max
Furthermore, if the number of horizontal pixels in the image sensor is n _x and the horizontal field of view (FoV) of the camera is θ _x , then the following relationship holds.
_xmax = _nx /2
tan(θ _x /2)=x _max /f
As described above, the distance calculation unit 34 can perform distance calculations by optical trigonometry.

A pattern projector or the like can be used as the light projecting unit that projects the random dot pattern. By arranging the pattern projector and the imaging camera at horizontally offset positions as shown in Figure 1, the random dot pattern projected onto the measurement target is always observed at a horizontally offset position depending on the distance due to the triangular similarity relationship.

FIG. 2 is a diagram showing an example of a dot pattern.
On the dot pattern shown in Fig. 2, there are only one area that matches a dot pattern in a specific rectangular area, not multiple areas. The method of creating the dot pattern will be described later. The autocorrelation characteristic of the random dot pattern of this embodiment is sharp, and it overlaps with itself only in the case of a zero shift, and has the property that the correlation value drops sharply if it is shifted by even one pixel. The cross-correlation between the partial pattern of the dot pattern and the original pattern also has a sharp correlation, and overlaps only in the case of a zero shift, but otherwise the cross-correlation value shows a low value.
The dot pattern projected onto the measurement target is always observed at a horizontally shifted position depending on the distance due to the triangular similarity relationship.

FIG. 3 is a diagram for explaining a method for detecting an area that matches a dot pattern in a specific rectangular area.
To detect an area that matches a dot pattern in a specific rectangular area, first quantization is performed by setting the dots in the dot pattern to +1 and the non-dots to -1. Then, the pixel values in the specific rectangular area are multiplied and accumulated using the following formula.
Here, the correlation value representing the similarity is denoted as _φj .
The pixels of the template image in a specific rectangular area to be extracted are quantized according to the presence or absence of a dot pattern, and the quantized value is set as a vector value _Qi .
The pixels of the image of a specific rectangular area to be compared with and the correlation to be determined are quantized according to the presence or absence of a dot pattern, and the quantized value is taken as a vector value P _i+j , where n is the number of pixels in the rectangular area.

By calculating the cross-correlation between a specific rectangular area pattern and the captured horizontal dot pattern, the pixel phase position for distance measurement using the optical triangulation method shown in Figure 1 can be obtained from the peak position of the correlation value.

Next, the dot pattern projected by light projector 10 of three-dimensional measuring device 1 of this embodiment will be described in more detail.
The dot pattern projected by the light-projecting unit 10 in this embodiment is generated by generating random screen coordinates using random numbers, and if the dot does not overlap within a certain distance with an already placed dot, the dot is placed as is, and if the dot overlaps with an already placed dot, the next random number is obtained and the dot placement is repeated.

FIG. 4 is a diagram for explaining a method for generating a dot pattern.
In the example of FIG. 4, only nine dots D1 to D9 are shown, but in reality many more dots are arranged.
First, as shown in the figure, dots D1 to D9 are arranged in a square with a pitch of n.
Next, each dot is successively rearranged (moved), but here, an example will be described in which dot D5 is moved. Dot D5 is located at the intersection of a square lattice that is equally spaced at intervals n.

The position of dot D5 before movement is used as the center, and the position after movement is determined by a random number. The area that can be moved is called the placement area. The hatched placement area PA in Figure 4 is the placement area of dot D5. Placement area PA has a predetermined size and is centered on dot D5 (the intersection position of the square lattice). Specifically, placement area PA is a 2 m x 2 m square area centered on dot D5 (the intersection position), with m being a predetermined value indicating distance, and the relationship between interval n and the predetermined value m satisfies m < n/2.

Under the restriction of movement within the placement area PA, the destination (relocation destination) is determined by random numbers, and the destination is provisionally determined (placement process). For example, the dot D5 in FIG. 4 is relocated to the position of the dot D5B. A determination (confirmation) is made as to whether the shortest distance between the relocated dot D5B and other dots is equal to or greater than a predetermined interference distance d (confirmation process). If the distance between the other dot (in the example of FIG. 4, the dot D4 is the closest) and the relocated dot D5B is smaller than the interference distance d, the relocated position of the dot D5B is determined to be inappropriate, and this is canceled, and under the restriction of movement within the placement area PA, a new operation of determining the destination (relocation destination) by random numbers (relocation process) is performed. Also, if the distance between the other dot (in the example of FIG. 4, the dot D4 is the closest) and the relocated dot D5B is equal to or greater than the interference distance d, the relocated dot D5B is determined to be appropriate, its position is confirmed, and the relocation operation of the next dot is performed in the same manner as above. By repeating the above operations, when the shortest distance between all dots finally becomes equal to or greater than the interference distance d, the randomly arranged dot pattern of this embodiment is completed.
When a diffractive optical element is used to project the dot pattern, the dot positions formed by the zero-order light cannot be moved, and therefore should be fixed.

FIG. 5 is a diagram showing a triangular polygon mesh set in a dot pattern before being randomly arranged.
FIG. 6 is a diagram showing a triangular polygon mesh set in the case where the dot pattern in FIG. 5 is rearranged to create a random dot pattern.
With the above-mentioned light spot position arrangement method, even after all the arrangements are completed, there is no change in the vertex numbers and the vertex relationships of the triangular polygons that make up the mesh, as shown in Fig. 6. It is only necessary to modify the x and y coordinates of the vertices. Therefore, in this embodiment, a triangular polygon mesh is constructed as shown in Fig. 5 with dots arranged on a square lattice, and after the dots are rearranged, the coordinates of the vertices are modified, so that a dot arrangement in which the straight lines that form the sides of the triangles do not intersect in the middle of the sides and the triangles can be arranged closely can be obtained simply and reliably, and a triangular polygon mesh without gaps or overlaps can be constructed into a randomly arranged dot pattern.
Therefore, in the calculation unit 30 of this embodiment, an index list of polygon mesh data configured with the intersection positions of a square lattice as the vertex positions of triangles is used as an index list for each dot of the random dot pattern created by rearrangement.

FIG. 7 is a diagram that allows visual confirmation of a triangular polygon mesh when the randomly arranged dot pattern of the present invention is projected onto a hand.
In the dot pattern projected by the light-projecting unit 10 of this embodiment, a mesh pattern such as that shown in FIG. 7 is not projected. However, in the processing after photographing the dot pattern, since each dot is related to another by a mesh, in terms of data processing, it is possible to treat it as a mesh model such as that shown in FIG. 7.

As shown in Figure 7, the depth value (distance) of each light point of a random dot pattern projected onto an object can be determined using trigonometry by cross-correlation processing between a partial pattern, which is a small rectangular area containing the surrounding light points, and the original pattern.
In addition, the depth value within each triangle is converted into a three-dimensional world coordinate system for each dot based on the screen coordinates (x, y) and depth value z, and the depth values corresponding to all pixels within the triangle can be calculated at high speed using the polygon fill (triangle filling) function of three-dimensional graphics as mesh data. Since it is triangular polygon mesh data, the calculation load is very light because a three-dimensional graphics pipeline can be used.
Depth data is obtained by adding a depth value z to the screen coordinates (x, y) of the mesh data to make each vertex data (x, y, z), and filling the mesh data with a polygon fill.

As described above, according to this embodiment, the computation load associated with the cross-correlation calculation can be reduced by the ratio of the number of light points of the random dot pattern to the total number of pixels that make up the screen.
In addition, 3D mesh data, point cloud data, and depth values can be obtained simultaneously.
Furthermore, the filling process of the 3D mesh data can be performed in a 3D graphics pipeline, and can be executed at high speed using, for example, a GPU built into a smartphone.
The present invention is not limited to the above-described embodiment.

Reference Signs List 1 Three-dimensional measuring device 10 Light projection unit 10a Light projection lens 20 Camera 21 Image sensor unit 22 Photographing lens 30 Calculation unit

Claims (5)

A projection device for projecting a random dot pattern, comprising:
The random dot pattern is a pattern in which one dot is randomly rearranged in each of arrangement areas of a predetermined size centered on dots before rearrangement that are arranged in a square at equal intervals n,
The dots are arranged so that the shortest distance between each other is equal to or greater than a predetermined interference distance d.
the arrangement area is a 2 m × 2 m square area centered on the dot position before the rearrangement, where m is a predetermined value indicating a distance,
The relationship between the interval n and the predetermined value m is as follows:
m<n/2
To satisfy the following:
A projection device comprising: 2. The projection device according to claim 1 ,
The random dot pattern is arranged so that adjacent dots can be connected by straight lines continuously with each dot being the vertex of a triangle, and the straight lines forming the sides of the triangles do not cross midway, and the triangles can be arranged in close contact with each other.
A projection device comprising: A projection device according to claim 1 or 2 ;
an imaging unit that images the random dot pattern projected by the projection device;
A detection device comprising: 4. The detection device according to claim 3 ,
a calculation unit that uses an index list of polygon mesh data in which the positions of the dots before rearrangement, which are arranged in a square at equal intervals of n, are configured as vertex positions of a triangle, as an index list of each dot of the random dot pattern;
A detection device comprising: A method for designing a random dot pattern to be projected onto an object using a projection device, comprising the steps of:
an arrangement step of randomly arranging one dot in each of arrangement areas of a predetermined size centered on the dots before rearrangement that are arranged in a square at equal intervals of n;
a confirmation step of confirming whether or not the shortest distance between the dots arranged in the arrangement step is equal to or greater than a predetermined interference distance d;
a rearrangement step of moving the positions of dots whose shortest distance between dots is not equal to or greater than a predetermined interference distance d based on the result of the confirmation step;
A method for designing a random dot pattern comprising:
the placement area is a 2 m × 2 m square area centered on the dot position before rearrangement, where m is a predetermined value indicating a distance,
The relationship between the interval n and the predetermined value m is as follows:
m<n/2
To satisfy the following:
A method for designing a random dot pattern comprising:

Images