CN107330930A - Depth of 3 D picture information extracting method - Google Patents

Abstract

The present application discloses a method for extracting the depth of a three-dimensional object, which solves the problem that the algorithm is too complicated in the process of extracting the depth information of the current 3D image. The method for extracting the depth of a three-dimensional object obtains the depth of the pixel by calculating the similarity of adjacent pixels in each element image in the two-dimensional plane image. This extraction method is a fast and accurate method for depth extraction of 3D objects, considering integrated imaging (II) and synthetic aperture integral imaging, by assuming that a 3D object is a surface composed of many aspects, a depth extraction method is developed based on the Patch‑Match algorithm Mathematical frame. The present application also discloses a device using the method for extracting the depth of a three-dimensional object.

Description

3D Image Depth Information Extraction Method

technical field

The invention relates to an information extraction technology in the field of optical information engineering, in particular to a technology for extracting depth information of a three-dimensional image.

Background technique

As a next-generation display technology, three-dimensional (3D) imaging technology has developed rapidly in recent years. Integrated imaging (II) (English: Integrated imaging (II)) involves its high resolution and full parallax. It is compatible with traditional image processing techniques such as super-resolution and image matching. In order to realize 3D imaging and display. Integrated imaging (II) requires different perspectives from 3D objects (called elemental images), which are usually picked up by small lens arrays in general integrated imaging (II) systems. Due to the use of standard 2D images, multiscale 3D imaging systems can be constructed using a single inexpensive camera with an array of small lenses or an array of inexpensive imagers. Existing technologies have achieved many research results, including 3D display and automatic object recognition.

Depth extraction is known as one of the most important problems in integrated imaging (II). Many researchers have paid attention to depth extraction for integrated imaging (II). However, the methods that have been proposed in the prior art suffer from low-resolution elemental images or complex algorithms.

Contents of the invention

According to one aspect of the present application, a method for extracting depth information of a three-dimensional image is proposed, which solves the problem that the algorithm is too complicated in the current process of extracting depth information of a three-dimensional image. This extraction method is a fast and accurate method for extracting the depth of three-dimensional objects. It considers integrated imaging (II) and synthetic aperture integral imaging (English: synthetic aperture integral imaging). By assuming that three-dimensional objects are surfaces composed of many aspects, based on Patch -Match algorithm develops a mathematical framework for deep extraction.

The method for extracting the depth of a three-dimensional object obtains the depth of the pixel by calculating the similarity of adjacent pixels in each element image in the two-dimensional plane image.

Preferably, when calculating the similarity of pixels in each element image in a two-dimensional plane image, it is assumed that adjacent pixels are on the same plane, and multiple small planes are used to model the surface.

Preferably, the calculation of the similarity of adjacent pixels in each element image in the two-dimensional plane image adopts a multi-cycle algorithm of adjacent pixel propagation and random optimization.

Preferably, it includes the steps of initializing the horizontal pixels to a random plane and iteratively calculating the similarity of adjacent pixels.

Further preferably, the initialization of the horizontal pixels to a random plane includes the steps of:

Initialize horizontal pixels to a random plane;

The initial depth of each pixel is set to a random value, and the normal vector of the surface through each pixel is set to a random unit vector.

Further preferably, said initializing the horizontal pixels as random planes includes the following process:

The plane passing through the depth coordinates of the horizontal pixels is represented by formula (5),

z＝f ₁ ▽p _x +f ₂ ▽p _y +f ₃ formula (5)

Wherein, z is the depth coordinate of the horizontal pixel, and p _x and p _y are random planes, and f ₁ , f ₂ and f ₃ are as in formula (6-1), formula (6-2) and formula (6- 3) as shown,

f ₁ =-n ₁ /n ₃ formula (6-1)

f ₂ =-n ₂ /n ₃ formula (6-2)

f ₃ =(n ₁ ·x ₀ +n ₂ ·y ₀ +n ₃ ·z ₀ )/n ₃ Formula (6-3)

In formula (6-1), formula (6-2) and formula (6-3), n ₁ , n ₂ and n ₃ are scalars, which are numerical vectors shown in formula (7) Indicates all possible planes where the minimum aggregation cost is located, x ₀ and y ₀ are the coordinate values of the initialized horizontal pixels respectively, z ₀ is the initial depth value of the initialized horizontal pixels,

In formula (7), m is provided by formula (8),

In formula (8), w is used to implement adaptive weighting, and w is provided by formula (9); E represents the similarity calculation factor, and E is provided by formula (10); ▽ represents the gradient value, and W _p represents a square window,

In formula (9), ||I _p -I _q || represents the distance between two adjacent pixels p and q, p is a horizontal pixel, q is an adjacent pixel in the same plane as p,

E＝α||I _i -I _j ||+(1-α)||▽I _i -▽I _j || Formula (10)

In formula (10), I is the intensity of the pixel in the element image, the subscripts i and j are the index of the element image, I _i , I _j represent the intensity of the corresponding pixel in the i-th and j-th element image respectively, I _i and I _j will be projected to the same space point, the coordinates of I _i and I _j are calculated by formula (11), ||I _i -I _j || is the Manhattan distance of the colors of I _i and I _j in RGB space , ▽I _i and ▽I _j are the gray value gradient of the pixel, ||▽I _i -▽I _j || represents the absolute difference of the gray gradient calculated at I _i and I _j , α is the weight factor without unit , used to balance the influence of color and gradient items;

In Equation (11), u _i are the local coordinates of the pixel corresponding to the point with coordinates y and z in each element image.

As a specific embodiment, the method is executed on a computer using an integrated imaging (II) system.

Further preferably, the iterative calculation of the similarity of adjacent pixels comprises the steps of:

a. Initialize a horizontal pixel in a random plane and calculate its depth coordinate and vector value, calculate its aggregation cost, and use this aggregation cost as a reference aggregation cost;

b. Calculate the aggregation cost of any adjacent pixel in the same plane as the horizontal pixel in step a;

c. comparing the reference aggregation cost in step a with the aggregation cost of adjacent pixels in step b;

d. Use the pixel corresponding to the relatively small aggregation cost in step c as a new reference value;

e. Set the pixel corresponding to the new reference value in step d to be adjacent to the upper left of the corresponding pixel of the comparison reference value;

f. Setting conditions: the depth value corresponding to the new reference value in step d is within the maximum allowable range;

g. If the condition of step f is established, execute step a to step f in a loop;

l. The condition of step f is not established, and the last loop step e is used as the leftmost pixel of the image;

m. On the basis of step l, the lower right of the image is performed to descend even iterations;

n. Calculate the number of calculations for each pixel according to the number of iterations in step m.

Further preferably, the iterative calculation of the similarity of adjacent pixels includes the steps of spatial propagation and plane refinement;

In the step of spatial propagation, adjacent pixel points are set to be on the same plane, and the cost m of different situations is first evaluated by formula (8),

In formula (8), p represents the current pixel, f _p is the vector of its corresponding plane, and q is the adjacent pixel of p, which is calculated by f _p and f _q respectively under p(x ₀ , y ₀ ) to evaluate The cost of these two cases; the check condition is shown in formula (12),

m(x ₀ , y ₀ , f _p ')<m(x ₀ , y ₀ , f _p ); formula (12)

and in formula (12) are respectively obtained by formula (8);

If the expression shown in formula (12) is established, then f _q is accepted as a new vector of p, i.e. f _p =f _q ;

In odd iterations, q is the left and upper bounds;

In even iterations, q is the right and bottom bounds;

In the step of plane refinement, f _p is converted into normal vector n _p , two parameters ▽z and ▽n are defined to limit the maximum allowable variation of z ₀ and n respectively, and z ₀ ' is calculated as z ₀ '= z ₀ +▽z, where ▽z is located at [-▽z ^max , ▽z ^max ], and n'=u(n+▽n), u() represents the calculation unit vector, ▽n is located at [-▽n ^max , ▽ ^nmax ];

Finally, get a new f _p ' through p and n', if m(x ₀ , y ₀ , f _p ')<m(x ₀ , y ₀ , f _p ), then f _p =f _p ';

In the step of plane refinement, start from setting ▽z ^max =maxdisp/2, where maxdisp is the maximum allowable parallax, ▽n ^max =1, after each refinement, the parameter will be updated to ▽z ^max =▽z ^max /2, ▽n ^max ＝▽n ^max /2; until ▽z ^max <resolution/2, the minimized resolution is obtained; for odd iterations, start from the left side of the image and proceed to even iterations down to the right;

After iteration, the similarity of adjacent pixels is obtained, and then the depth of the three-dimensional object is obtained.

It is further preferred that z ₀ is initialized to a fixed value for all pixels, and a refinement step is added before the iteration.

According to one aspect of the present application, a device for obtaining 3D image information from a 2D image is provided, which solves the problem that algorithms are too complicated in the current 3D image depth information extraction process. The device for obtaining three-dimensional image information from two-dimensional pictures is a fast and accurate three-dimensional object depth extraction device, which considers comprehensive imaging (II) and synthetic aperture integral imaging (English: synthetic aperture integral imaging), by assuming that the 3D object is A surface composed of many facets, a mathematical framework for depth extraction was developed based on the Patch-Match algorithm.

The device for obtaining three-dimensional image information from two-dimensional pictures includes a picture acquisition unit, an image storage unit and an image processing unit;

The picture acquisition unit is electrically connected to the image storage unit, and the image storage unit is electrically connected to the image processing unit;

The image processing unit adopts at least one of the above-mentioned 3D object depth extraction methods to obtain the depth information of the object in the 2D picture and create a 3D image.

The beneficial effects of the technical solution of the present invention include but are not limited to:

(1) This application proposes a new computational method for depth extraction of 3D objects, which computes the similarity of pixels in each element image while projecting them to possible depths.

(2) The method for extracting the depth information of the 3D image provided by this application takes into account the continuity of the surface to improve the resolution.

(3) The method for extracting the depth information of the three-dimensional image provided by the application is used in the comprehensive imaging (II) system, and the patch matching method is used to greatly reduce the amount of calculation and speed up the calculation, which can make the equipment applied to the algorithm of the present invention more common change.

Description of drawings

Fig. 1 is a schematic diagram of the principle of the 3D image depth information extraction method of the present invention; wherein Fig. 1 (a) is a schematic diagram of the picking part of the comprehensive imaging (II); Fig. 1 (b) is a schematic diagram of the projection part of the comprehensive imaging (II).

Figure 2 is a diagram of light propagation in an integrated imaging (II) system of body surface and free space voxels.

Fig. 3 is a 3D object extracted from image information, wherein Fig. 3(a) is an object imaged by comprehensive imaging (II); Fig. 3(b) is an object imaged by synthetic aperture integral imaging.

Fig. 4 is a comparison between the imaging result of the application method and the imaging result of the prior art method through mathematical software technology; wherein Fig. 4 (a) is the result obtained by using the microneedle depth information extraction method; Fig. 4 (b) is Adopt the result that the catadioptric omnidirectional extraction method obtains; Fig. 4 (c) is the comprehensive imaging (II) result that adopts the method of the present invention to obtain; Fig. 4 (d) is the synthetic aperture integral imaging that adopts the method of the present invention to obtain result.

Figure 5 is a comparison of the results after reducing the influence of white noise; among them, Figure 5(a) is the result of comprehensive imaging (II) before reducing the influence of white noise; Figure 5(b) is the result of comprehensive imaging (II) after reducing the influence of white noise ; Figure 5(c) is the synthetic aperture integral imaging result before reducing the influence of white noise; Figure 5(d)) is the synthetic aperture integral imaging result after reducing the influence of white noise.

Fig. 6 is the rear projection diagram adopting the 3D image depth information extraction method of the present invention; Wherein Fig. 6 (a) is Fig. 3 (a) adopts the rear projection diagram of the 3D image depth information extraction method of the present invention; Fig. 6 (b) is a diagram 3(b) The back projection diagram of the 3D image depth information extraction method of the present invention.

detailed description

The technical solution of the present invention will be described in detail below in conjunction with the embodiments.

The present invention proposes a new calculation method for depth extraction of 3D objects in an integrated imaging (II) system, calculating the similarity of pixels in each element image while projecting them to possible depths. Surface continuity is also taken into account for improved resolution. And use the patch matching method to speed up the computation.

Figure 1(a) and Figure 1(b) are schematic diagrams of the principle of integrated imaging (II). 1(a) is a schematic diagram of the pick-up part of the comprehensive imaging (II), and FIG. 1(b) is a schematic diagram of the projection part of the comprehensive imaging (II).

Referring to Figure 1(a), the letter A denotes a three-dimensional (3D) object. Z (capital letter) represents the distance between the object and the lenslet array, and g represents the distance between the lens array and the image plane. The intensity and direction of the light rays from the 3D object are recorded at different positions by the lenslets. Different element images are also shown in Figure 1(a), see the three images from top to bottom on the right.

In the projection part as shown in Fig. 1(b), each element image is projected to the target space through the corresponding pinhole. In Figure 1(b), z (lowercase letter) represents the projection distance, and g represents the distance between the image and the pinhole. And these projected images are magnified by a factor of z/g in the reconstruction plane. Finally, these upscaled images are overlaid and accumulated on the corresponding pixels of the output plane.

In the systems shown in Fig. 1(a) and Fig. 1(b), the projection distance z is set by the experimenter, so when the projection distance z does not match its spatial depth Z, the projected images are blurred pixels. Therefore, if the blurriness of each pixel in different projection distances is known, the corresponding projection distances can be calculated.

Figure 2 is a diagram of light propagation in an integrated imaging (II) system of body surface and free space voxels. Fig. 2 shows the 2D structure of the method of the present invention, showing the yz plane in 3D space, and the imaging object is shown as a plane on the left. The lenslet array is on the y-axis. And the z axis is the depth direction. The coordinates of each lenslet are labeled S _i . The imaged object plane is labeled u. The distance from u to the lenslet array is g. As shown in Fig. 2, when the elemental image is projected onto the z ₀ plane, the high similarity of (y ₀ , z ₀ ) to its corresponding pixel in the resulting image will be clear. As a comparison, when projected onto the z ₁ plane, (y ₁ , z ₁ ) is blurred because the pixels at (y ₁ , z ₁ ) and (y ₀ , z ₀ ) come from different parts of the object u, as shown in 2 shown in different colors. The local coordinate point corresponding to each pixel of the projection can be obtained by formula (1):

where u _i are the local coordinates of the pixel corresponding to point (y, z) in each element image. g is the distance between the image plane and the lenslet array. s _i are the coordinates of the lenslet, that is, the index of the lenslet. Using this equation, the similarity of pixels projected to the same point can be estimated by equation (1).

E＝α||I _i -I _j ||+(1-α)||▽I _i -▽I _j || Formula (2)

In this equation, E is the evaluation factor of similarity. The smaller E is, the more similar the pixels are. I is the intensity of the pixel in the element image. The subscripts i, j are the indices of the element image. I _i , I _j represent corresponding pixels in the i-th and j-th element images respectively. I _i and I _j will project to the same spatial point, and their coordinates are calculated by equation (1). ||I _i -I _j || Calculate the L1 distance (ie Manhattan distance) of the colors of I _i and I _j in the RGB space. ▽I is the gray value gradient of the pixel; ||▽Ii-▽Ij|| represents the absolute difference of the gray gradient calculated at I _i and I _j . α is a weight factor without units and is a user-defined parameter to balance the influence of color and gradient terms. From this equation the similarity between pixels projected to the same spatial location can be calculated.

Therefore, in 3D mode, the depth of the imaged object surface with a lateral point (x, y) can be extracted by finding Z such that E(x, y, z) is in the range Z = [Z min, Z max] minimize. The mathematical expression of this assumption is shown in formula (3):

The answer can be found by examining all possible z. However, the possible z's discovered in this way are discrete and limited by resolution. This approach ignores the continuity of the surface and is computationally intensive, requiring more surface information to obtain sub-resolution effects. The present invention takes into account the continuity of the surface of the object. Adjacent pixels are assumed to be on the same plane, so the present invention models the surface with many facets.

The surface passing through (x ₀ , y ₀ , z ₀ ) can be expressed as formula (4):

n ₁ x+n ₂ y+n ₃ z＝n ₁ x ₀ +n ₂ y ₀ +n ₃ z ₀ Formula (4)

n(n ₁ , n ₂ , n ₃ ) is a normal vector. In the present invention, the horizontal pixel is called p(p _x , p _y ), and z is the required depth coordinate, so formula (4) can be transformed to obtain formulas (5) and (6):

z＝f ₁ ▽p _x +f ₂ ▽p _y +f ₃ formula (5)

f ₁ =-n ₁ /n ₃ , f ₂ =-n ₂ /n ₃ , f ₃ =(n ₁ x ₀ +n ₂ y ₀ +n ₃ z ₀ )/n ₃ Formula (6)

Therefore, the problem of finding z becomes finding f, and the vector f is one of the minimum aggregated matching costs in all possible planes, which can be expressed as Equation (7):

where F represents the set of all vectors of infinite size. The aggregation cost m of matching p(p _x , p _y ) according to the vector f is calculated by formula (8) and formula (9):

In formula (9), w _p represents a square window centered on p(p _x , p _y ); w is used to realize adaptive weight stereo matching, which can overcome the problem of edge proliferation; γ is a user-defined parameter; I _p Indicates the pixel intensity of image p; I _q indicates the pixel intensity of image q. The E of nearby pixels is also computed with the same vector f, revealing that they are in the same plane. The set of all vectors F is an infinite label space, and the usual practice of simply checking all possible labels cannot be adopted.

The 3D image depth information extraction method proposed by the present invention is based on Patch-Match, and its basic idea is that most adjacent pixels should be on the same plane. Based on this assumption, the present invention develops a multi-loop algorithm involving neighbor pixel propagation and stochastic optimization.

Based on the above analysis, the 3D image depth information extraction method proposed by the present invention comprises the following steps:

Step 1: Initialize

Initialize the horizontal pixel p(x ₀ , y ₀ ) as a random plane;

The plane can be determined by point and normal vector. Each pixel's z ₀ is initialized with a random value, and the normal vector of the surface through that pixel is set to a random unit vector n(n ₁ , n ₂ , n ₃ ). The vector f can be derived from the normal n and the point p(x ₀ , y ₀ , z ₀ ).

The depth coordinate of the horizontal pixel is z(x ₀ , y ₀ , z ₀ ), the random plane is expressed as p(p _x , p _y ), and the plane passing through z can be expressed as:

z=f ₁ ▽p _x +f ₂ ▽p _y +f ₃

Wherein refer to formula (6), f ₁ =-n ₁ /n ₃ f ₂ =-n ₂ /n ₃ , f ₃ =(n ₁ ·x ₀ +n ₂ ·y ₀ +n ₃ ·z ₀ )/n ₃ , n is a scalar and a numeric vector Denotes all possible planes where the minimum aggregate cost lies,

The horizontal pixel initialized in this step is p(x ₀ , y ₀ ), its corresponding depth value is z0, and its aggregation cost m is:

in, ||I _p -I _q || indicates the distance between two adjacent pixels p and q, p is a horizontal pixel, q is an adjacent pixel in the same plane as p, w is used to realize adaptive weighting, E means the same The calculation factor of , ▽ represents the gradient value, W _p represents a square window centered on p(px, p _y ).

The same calculation factor E can be expressed as:

E＝α||I _i -I _j ||+(1-α)||▽I _i -▽I _j

where I is the intensity of the pixel in the element image. The subscripts i, j are the indices of the element image. I _i , I _j represent the intensity of the corresponding pixel in the i-th and j-th element images respectively. I _i and I _j will project to the same spatial point, and their coordinates are calculated by equation (1). That is, the local coordinate points corresponding to each pixel of the projection are:

where u _i are the local coordinates of the pixel corresponding to point (y, z) in each element image. g is the distance between the image plane and the lenslet array (see Figure 1). s _i are the coordinates of the lenslet, that is, the index of the lenslet. I _i and I _j can be obtained using this formula.

||I _i -I _j || Calculate the L1 distance (ie Manhattan distance) of the colors of I _i and I _j in the RGB space. ▽I is the gray value gradient of the pixel; ||▽I _i -▽I _j || represents the absolute difference of the gray gradient calculated at I _i and I _j . α is a weight factor without units and is a user-defined parameter to balance the influence of color and gradient terms. The similarity between pixels projected to the same spatial location can be calculated by this equation.

Thanks to many predicted values, after this random initialization, the pixels of at least one region with a plane in the region are close to the correct value. After a propagation step where the plane is passed to other pixels, a good prediction is enough for the algorithm to work.

Step 2: Iterate

In iterations, two stages are run per pixel: first spatial propagation, followed by planar refinement.

(2-1) Spatial propagation

Adjacent points are assumed to be on the same plane as is usually thought of. This is the key point of communication. p represents the current pixel, and f _p is the vector of its corresponding plane. q is the neighbor pixel of p. Equation (8) is calculated with f _p and f _q under p(x ₀ , y ₀ ), respectively, to evaluate the cost of these two cases. Check the condition m(x ₀ , y ₀ , f _p ')<m(x ₀ , y ₀ , f _p ).

m(x ₀ , y ₀ , f _p ') and m(x ₀ , y ₀ , f _p ) in formula (12) are respectively obtained from formula (8);

If this expression holds, then f _q is accepted as a new vector of p, ie f _p =f _q . In odd iterations, q is the left and upper bounds, and in even iterations, q is the right and lower bounds.

(2-2) Plane finishing

The goal of plane refinement is to improve the parameters of the plane at pixel p. In order to further reduce Z in formula (6) to extract the depth of the imaging object surface with lateral points.

Convert f _p to normal vector n _p . Two parameters ▽z and ▽n are defined to limit the maximum allowable variation of z ₀ and n, respectively. z ₀ ' is calculated as z ₀ '=z ₀ +▽z, where ▽z is located at [-▽z ^max ,▽z ^max ]. And n'=u(n+▽n), u() represents a calculation unit vector, and ▽n is located at [-▽n ^max , ▽n ^max ]. Finally, pass p and n' to get a new f _p '. If m(x ₀ , y ₀ , f _p ')<m(x ₀ , y ₀ , f _p ), then f _p =f _p '.

The method starts by setting ▽z ^max =maxdisp/2, where maxdisp is the maximum allowed disparity and ▽n ^max =1. After each refinement, the parameters will be updated as ▽z ^max = ▽z ^max /2, ▽n ^max = ▽n ^max /2, thus reducing the search range. We return to the special propagation again until ▽z ^max <resolution/2, where the resolution is as in the literature [DaneshPanah M, JavidiB. "Profilometry and optical slicing by passive three-dimensional imaging[J]". Optics letters, 2009, 34( 7):1105-1107] as shown in the minimization. For odd iterations, start from the left of the image and work down to the right for even iterations. The final result is obtained after iterations.

To test the utility of this approach, two type II experiments are presented. And also do the normal way of checking all possible z for comparison. First, a model of the tractor was used as a 3D object in a computer-integrated imaging system. The object is shown in Figure 3(a). The result has 20x28 element images with 100x100 pixels each. The focal length of the small lens is 1.5mm. The depth of the tractor is α is set to 0.5, and γ is set to 5. By calculation, the minimum depth resolution is 0.005mm.

In synthetic aperture integral imaging, the elemental image is shown in Fig. 3(b). The 3D objects included a building block, a doll, and a toy elephant, located at 53-57 cm, 89-93 cm, and 131-136 cm, respectively. The system contains a 6×6 perspective. And the images are captured on a regular grid with 5mm spacing. The focal length of the camera is 16mm. α is set to 0.5, and γ is set to 5. By calculation, the minimum depth resolution is 2mm.

The algorithm is calculated in Matlab. The comparison of the results received by the method proposed in this paper with the results of commonly used methods is shown in the figure. In Figure 4(b) and (d), the horizontal stripes at the bottom are folds of the background.

As can be seen from these results, both methods perform well in the object part. However, the results obtained by common methods are shown in Fig. 4(a) and 4(b), and the edge fattening is quite obvious. In the method proposed in this paper, as shown in Fig. 4(c) and 4(d), the blank part of the result is filled with white noise. Insensitive to depth changes in these regions, as differences are difficult to discern. So the depth of this part is still its initialized value. There is a bit of inaccuracy inside the object that needs to be removed with a few more iterations.

To reduce the influence of white noise, z ₀ is initialized to all pixels with a fixed value, and a refinement step is added before iterations. A better result is obtained as shown in Fig. 5, the white noise is well removed. In Fig. 5(a), the depth of the tractor is precisely extracted especially at the edge of the object. But there are still some white noises at the edge of the object in Fig. 5(b), which may be limited by the experimental conditions.

Although the continuity of the surface is considered, it is not intuitive from this result that the depth is constantly changing. This may be due to the coarser resolution of this geometric model.

To further verify the proposed method, each pixel is projected to the depth calculated by the proposed method, as shown in Fig. 6. The depth used in this step is Gaussian smoothed and the background filtered out. Since the algorithm has not been optimized, the calculation time makes it difficult to evaluate the computational efficiency of this method. Thus, m can be evaluated by the computation time of the kernel factor, and not all z spaces need to be computed; computing resources are paid for sub-resolution computations. In the simulated light field, with maxdisp set to 10 mm and a resolution of 0.005 mm, 12 iterations were calculated in the algorithm. In each iteration, m is computed 4 times in each pixel (f _p , f _q , f _p' , where q are two adjacent pixels). 12 x 4 = 48 calculations per pixel in this method is nearly 40 times m compared to 10/0.005 = 2000 times the usual method of calculating each possible position. From this point of view, the proposed algorithm can effectively reduce the amount of computation.

The above are only several specific implementations of the present invention, and do not limit the present invention in any form. Although the present invention is disclosed as above with preferred embodiments, it is not intended to limit the present invention. Anyone familiar with this field , without departing from the scope of the technical solution of the present application, any changes or modifications made using the technical content disclosed above are equivalent to equivalent implementation cases, and all belong to the scope of the technical solution.

Claims (10)

1. A three-dimensional object depth extraction method is characterized in that the depth of a pixel is obtained by calculating the similarity of adjacent pixels in each element image in a two-dimensional plane image.

2. The method of extracting a depth of a three-dimensional object according to claim 1, wherein in calculating the similarity of pixels in each elemental image in the two-dimensional plane image, it is assumed that adjacent pixels are on the same plane, and the surface is modeled with a plurality of small planes.

3. The method for extracting the depth of the three-dimensional object according to claim 1, wherein the calculating the similarity of the adjacent pixels in each elemental image in the two-dimensional plane image adopts a multi-cycle algorithm of adjacent pixel propagation and random optimization.

4. The method of claim 1, comprising the steps of initializing a horizontal pixel to a random plane and iteratively calculating the similarity of neighboring pixels.

5. The method of claim 4, wherein the initializing of the horizontal pixels to a random plane comprises:

initializing a horizontal pixel to a random plane;

the initial depth of each pixel is set to a random value, and the normal vector passing through the surface of each pixel is set to a random unit vector.

6. The method of claim 4, wherein the initializing the horizontal pixels to a random plane comprises:

a plane passing through the depth coordinate of the lateral pixel, represented by equation (5),

z＝f₁▽p_x+f₂▽p_y+f₃formula (5)

Wherein z is the depth coordinate of the lateral pixel, and p_xAnd p_yIs a random plane, f₁、f₂And f₃Respectively represented by formula (6-1), formula (6-2) and formula (6-3),

f₁＝-n₁/n₃formula (6-1)

f₂＝-n₂/n₃Formula (6-2)

f₃＝(n₁·x₀+n₂·y₀+n₃·z₀)/n₃Formula (6-3)

In the formulae (6-1), (6-2) and (6-3), n₁、n₂And n₃Is a scalar quantity, is a numerical vector represented by the formula (7)All possible planes, x, representing the minimum aggregation cost₀And y₀Respectively initialized coordinate values of said transverse pixels, z₀For the initialized initial depth values of the lateral pixels,

formula (7) wherein m is provided by formula (8),

in the formula (8), W is used to realize adaptive weighting, W is provided by the formula (9), E represents a similarity calculation factor, E is provided by the formula (10), ▽ represents a gradient value, W_pA square window centered on p is shown,

in the formula (9), | | I_p-I_qAnd | | l represents the distance between two adjacent pixels p and q, p being a transverse pixel, q being an adjacent pixel in the same plane as p,

E＝α||I_i-I_j||+(1-α)||▽I_i-▽I_j| | formula (10)

In the formula (10), I is the intensity of a pixel in an element image, subscripts I, j are indices of the element image, I_i，I_jRespectively representing the intensity, I, of the corresponding pixel in the ith and jth elemental images_iAnd I_jWill project to the same spatial point, I_iAnd I_jIs calculated by the formula (11) | | I_i-I_j| is in RGB spaceI of (A)_iAnd I_jManhattan distance of color of (1), ▽ I_iAnd ▽ I_jIs the gray value gradient of the pixel, | | ▽ I_i-▽I_jI is expressed in_iAnd I_jThe absolute difference of the calculated gray gradients, α is a weighting factor without units for balancing the influence of color and gradient terms;

in the formula (11), u_iIs the local coordinate of the pixel in each elemental image corresponding to the point with coordinates y and z.

7. The method of claim 4, wherein the method of three-dimensional object depth extraction is performed on a computer using an Integrated Imaging (II) system, and the iterative computation of the similarity of neighboring pixels comprises the steps of:

a. initializing a horizontal pixel in a random plane, calculating a depth coordinate and a vector value of the horizontal pixel, calculating the aggregation cost of the horizontal pixel, and taking the aggregation cost as a reference aggregation cost;

b. calculating the aggregation cost of any adjacent pixel in the same plane with the horizontal pixel in the step a;

c. comparing the reference aggregation cost in the step a with the aggregation cost of the adjacent pixels in the step b;

d. taking the pixel corresponding to the smaller aggregation cost in the step c as a new reference value;

e. setting the pixel corresponding to the new reference value in the step d as the upper left adjacent to the pixel corresponding to the comparison reference value;

f. setting conditions: d, the corresponding depth value of the new reference value in the step d is in the maximum allowable range;

g. if the condition of the step f is satisfied, circularly executing the step a to the step f;

step f, if the condition is not satisfied, taking the last cycle of step e as the leftmost pixel of the image;

m, on the basis of the step l, performing descending even iteration on the right lower part of the image;

n, calculating the calculation times of each pixel according to the iteration times of the step m.

8. The method of claim 4, wherein the iterative computation of the similarity of neighboring pixels comprises steps of spatial propagation and planar refinement;

in the step of spatial propagation, adjacent pixels are set to be on the same plane, the cost m of different situations is evaluated by the formula (8),

in the formula (8), p represents the current pixel, f_pIs the vector of its corresponding plane, q is the neighboring pixel of p, under p, f_pAnd f_qCalculating to evaluate the cost of both cases; the examination conditions are as shown in the formula (12),

m(x₀，y₀，f_p')<m(x₀，y₀，f_p) (ii) a Formula (12)

M (x) in formula (12)₀，y₀，f_p') and m (x)₀，y₀，f_p) Respectively obtained by formula (8);

if the expression shown in equation (12) holds, f_qNew vector accepted as p, i.e. f_p＝f_q；

In odd iterations, q is the left and upper bounds;

in even iterations, q is the right and lower bounds;

in the step of plane finishing, f is_pConversion to normal vector n_pTwo parameters ▽ z and ▽ n are defined to limit z, respectively₀And the maximum allowable variation of n, z₀' calculation as z₀'＝z₀+ ▽ z, wherein ▽ z is located at [ - ▽ z^max，▽z^max]And n' ═ u (n + ▽ n), where u denotes the calculation unit vector, and ▽ n is located at [ - ▽ n^max，▽n^max]；

Finally, a new f is obtained by p and n_p', if m (x)₀，y₀，f_p')<m(x₀，y₀，f_p) Then f is_p＝f_p'；

In the step of plane finishing, from setting ▽ z^maxMaxdisp/2 starts, where maxdisp is the maximum allowed disparity, ▽ n^maxAfter each refinement, the parameter will be updated to ▽ z as 1^max＝▽z^max/2、▽n^max＝▽n^max/2 up to ▽ z^max<resolution/2, obtaining the minimized resolution; for odd iterations, starting from the left side of the image, performing even iterations towards the right and downwards;

and after iteration, obtaining the similarity of adjacent pixels so as to further obtain the depth of the three-dimensional object.

9. The method of claim 8, wherein z is₀All pixels are initialized with fixed values and a refinement step is added before the iteration.

10. The device for obtaining three-dimensional image information from a two-dimensional picture is characterized by comprising a picture acquisition unit, an image storage unit and an image processing unit;

the image acquisition unit is electrically connected with the image storage unit, and the image storage unit is electrically connected with the image processing unit;

the image processing unit adopts the three-dimensional object depth extraction method of any one of claims 1 to 9 to obtain the depth information of the object in the two-dimensional picture and establish a three-dimensional image.