CN106815865A - Image depth estimation method, depth drawing generating method and device - Google Patents
Image depth estimation method, depth drawing generating method and device Download PDFInfo
- Publication number
- CN106815865A CN106815865A CN201510859849.9A CN201510859849A CN106815865A CN 106815865 A CN106815865 A CN 106815865A CN 201510859849 A CN201510859849 A CN 201510859849A CN 106815865 A CN106815865 A CN 106815865A
- Authority
- CN
- China
- Prior art keywords
- image
- map
- depth estimation
- depth
- local variance
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10028—Range image; Depth image; 3D point clouds
Landscapes
- Image Processing (AREA)
Abstract
A kind of image depth estimation method, depth drawing generating method and device.Described image depth estimation method includes：Obtain the N image { I shot in same visual angle, identical focal length but different image distances1,I2,…,IN}；Respectively according to described image { I1,I2,…,INThe corresponding local variance figure of generation；Search for the maximum image of the corresponding variance of each pixel from each local variance figure, and the maximum corresponding object distance of image of the variance that will be searched is used as the estimation of Depth value of the pixel.The complexity and hardware cost of picture depth estimation procedure can be reduced using described image depth estimation method.
Description
Technical Field
The invention relates to the technical field of image processing, in particular to an image depth estimation method, a depth map generation method and a depth map generation device.
Background
Currently, in the implementation of image processing algorithms such as refocusing, augmented reality (augmented reality), and object detection and recognition (depth map), a depth map (depth map) is required. Depth estimation is performed on the image to obtain a corresponding depth map, which plays an important role in processing the image.
In practical applications, the image depth estimation process is as follows: firstly, carrying out Image matching (Image Alignment) on an original Image to obtain a corresponding parallax estimation Image; then, performing original depth estimation according to the parallax estimation image to obtain an original depth estimation image (raw depth map estimation); and finally, post-processing (post-processing) is carried out on the original depth estimation map to obtain a final depth map.
Currently, in the process of depth estimation of an original image, a calculation is mainly performed based on a stereo matching (stereo) algorithm. On one hand, the stereo matching process has high computational complexity and is not suitable for real-time application, and on the other hand, the stereo matching process needs to use two cameras at different viewing angles to respectively obtain corresponding original images, so that the hardware cost is high, and the image depth estimation process has high computational complexity and hardware cost.
Disclosure of Invention
The problem to be solved by the invention is how to reduce the complexity and hardware cost of the image depth estimation process.
In order to solve the above problem, an embodiment of the present invention provides an image depth estimation method, where the method includes:
acquiring N images { I) shot at the same view angle and the same focal length but different image distances1，I2，...，INN is more than or equal to 3 and is a positive integer;
respectively according to the image { I1，I2，...，INGenerating a corresponding local variance map;
and searching the image with the maximum variance corresponding to each pixel point from each local variance map, and taking the object distance corresponding to the searched image with the maximum variance as the depth estimation value of the pixel point.
Optionally, said separately from said images { I1，I2，...，INGenerating a corresponding local variance map, comprising:
for the acquired N images { I1，I2，...，INCarrying out image alignment processing;
according to the aligned images respectively{I1，I2，...，INSolving the local variance to obtain the local variance of the image { I }1，I2，...，INAnd f, respectively corresponding local variance maps.
The embodiment of the invention also provides a method for generating the depth map, which comprises the following steps:
calculating the depth estimation value of each pixel point, comprising the following steps: acquiring N images { I) shot at the same view angle and the same focal length but different image distances1，I2，...，INN is more than or equal to 3 and is a positive integer; respectively according to the image { I1，I2，...，INGenerating a corresponding local variance map; searching an image with the maximum variance corresponding to each pixel point from each local variance map, and taking the object distance corresponding to the searched image with the maximum variance as the depth estimation value of the pixel point;
obtaining a corresponding original depth estimation image according to the depth estimation value;
removing a smooth region from the original depth estimation map to obtain a corresponding sparse depth estimation map;
and filling the sparse depth estimation map to obtain a complete depth map.
Optionally, said separately from said images { I1，I2，...，INGenerating a corresponding local variance map, comprising:
for the acquired N images { I1，I2，...，INCarrying out image alignment processing;
according to the aligned images, the images { I are respectively aligned1，I2，...，INSolving the local variance to obtain the local variance of the image { I }1，I2，...，INAnd f, respectively corresponding local variance maps.
Optionally, the removing the smooth region from the original depth estimation map to obtain a corresponding sparse depth estimation map includes:
respectively averaging the variances of all the pixel points to obtain average local variance graphs respectively corresponding to all the local variance graphs;
selecting a region corresponding to a pixel point of which the depth estimation value is smaller than a preset value in the average local variance image, and taking the selected region as the smooth region;
and removing the smooth region from the original depth estimation map, and taking the removed original depth estimation map as the sparse depth estimation map.
Optionally, the filling the sparse depth estimation map to obtain a complete depth map includes:
averaging each of the aligned images to obtain an average image of the aligned images;
and filling the sparse depth estimation image by using the average image to obtain a complete depth image.
Optionally, the filling the sparse depth estimation map includes:
and filling the sparse depth estimation map by adopting 2-by-2 overlapping windows.
An embodiment of the present invention further provides an image depth estimation device, where the device includes:
an image acquisition unit adapted to acquire N images { I } captured at the same angle of view, the same focal length, but different image distances1，I2，...，INN is more than or equal to 3 and is a positive integer;
a local variance map generation unit adapted to generate a local variance map from the images { I }1，I2，...，INGenerating a corresponding local variance map;
and the depth calculation unit is suitable for searching the image with the maximum variance corresponding to each pixel point from each local variance map, and taking the object distance corresponding to the searched image with the maximum variance as the depth estimation value of the pixel point.
Optionally, the local variance map generating unit includes:
an alignment subunit adapted to align the acquired N images { I }1，I2，...，INCarrying out image alignment processing;
a computing subunit adapted to compute said images { I } from each aligned image, respectively1，I2，...，INSolving the local variance to obtain the local variance of the image { I }1，I2，...，INAnd f, respectively corresponding local variance maps.
An embodiment of the present invention further provides a depth map generating device, where the device includes:
the image depth estimation unit is suitable for calculating the depth estimation value of each pixel point and comprises the following steps: an image acquisition subunit adapted to acquire N images { I } captured at the same viewing angle, the same focal length, but different image distances1，I2，...，INN is more than or equal to 3 and is a positive integer; a local variance map generation subunit adapted to generate a local variance map from the images { I }1，I2，...，INGenerating a corresponding local variance map; the depth calculation operator unit is suitable for searching the image with the maximum variance corresponding to each pixel point from each local variance map, and taking the object distance corresponding to the searched image with the maximum variance as the depth estimation value of the pixel point;
the first image generation unit is suitable for obtaining a corresponding original depth estimation image according to the depth estimation value;
the second image generation unit is suitable for removing a smooth region from the original depth estimation image to obtain a corresponding sparse depth estimation image;
and the third image generation unit is suitable for filling the sparse depth estimation map to obtain a complete depth map.
Optionally, the local variance map generating subunit includes:
an alignment module adapted to align the acquired N images { I }1，I2，...，INCarrying out image alignment processing;
a computing module adapted to align the images { I } according to the aligned images, respectively1，I2，...，INSolving the local variance to obtain the local variance of the image { I }1，I2，...，INAnd f, respectively corresponding local variance maps.
Optionally, the second image generation unit includes:
the calculation subunit is suitable for respectively averaging the variances of all the pixel points to obtain an average local variance map corresponding to each local variance map subsection;
a selecting subunit, adapted to select a region corresponding to a pixel point in the original depth estimation image whose depth estimation value is smaller than a preset value, and take the selected region as the smooth region;
a processing subunit adapted to remove the smoothed region from the original depth estimation map and to use the removed original depth estimation map as the sparse depth estimation map.
Optionally, the third image generation unit includes:
the average subunit is suitable for averaging all the aligned images to obtain an average image of the aligned images;
and the filling subunit is suitable for filling the sparse depth estimation map by using the average image to obtain a complete depth map.
Optionally, the filling subunit is adapted to perform a filling process on the sparse depth estimation map with 2 × 2 overlapping windows.
Compared with the prior art, the technical scheme of the invention at least has the following advantages:
n images shot at the same visual angle and the same focal length but different image distances are obtained, corresponding local variance maps are generated according to the obtained images, the image with the maximum variance corresponding to each pixel point is searched from each local variance map, the object distance corresponding to the searched image with the maximum variance is used as the depth estimation value of the pixel point, and depth estimation is not needed to be carried out through a stereo matching method, so that the complexity in the whole image depth estimation process can be reduced. In addition, the acquired images are images with the same visual angle, the same focal length and different image distances, so that the images can be acquired by only using one camera, and the hardware cost of the whole image depth estimation process can be reduced.
Drawings
FIG. 1 is a flow chart of a method for estimating image depth according to an embodiment of the present invention;
FIG. 2 is a diagram of an input image according to an embodiment of the present invention;
FIG. 3 is a schematic diagram of the input image of FIG. 2 after image alignment processing;
FIG. 4 is a flow chart of a depth map generation method in an embodiment of the present invention;
FIG. 5 is a schematic diagram of a process of generating a sparse depth map according to an embodiment of the present invention;
FIG. 6 is a full depth map corresponding to the input image of FIG. 2;
FIG. 7 is a schematic diagram of a generation process of a full depth map according to an embodiment of the present invention;
FIG. 8 is a schematic structural diagram of an image depth estimation apparatus according to an embodiment of the present invention;
fig. 9 is a schematic structural diagram of a depth map generating apparatus according to an embodiment of the present invention.
Detailed Description
Currently, when performing depth estimation of an image, two cameras located at different viewing angles are used to obtain corresponding original images, and then the two obtained original images are subjected to stereo matching to obtain corresponding disparity estimation images, and finally depth estimation is performed according to the disparity estimation images.
In the above image depth estimation process, on one hand, the complexity of the stereo matching process is high, and therefore the stereo matching process is not suitable for real-time application, and therefore the complexity of the whole image depth estimation process is high. On the other hand, the original image must be obtained by two cameras with different view angles, thereby resulting in higher hardware cost of the whole image depth estimation process.
In view of the above problems, an embodiment of the present invention provides an image depth estimation method, where N images captured at the same viewing angle and the same focal length but different image distances are first acquired, and then corresponding local variance maps are generated according to the acquired images, so as to search for an image with the largest variance corresponding to each pixel point from each local variance map, and use an object distance corresponding to the searched image with the largest variance as a depth estimation value of the pixel point, without performing depth estimation by using a stereo matching method, so that complexity in the whole image depth estimation process can be reduced. In addition, the acquired images are images with the same visual angle, the same focal length and different image distances, so that the images can be acquired by only using one camera, and the hardware cost of the whole image depth estimation process can be reduced.
In order to make the aforementioned objects, features and advantages of the present invention comprehensible, embodiments accompanied with figures are described in detail below.
As shown in fig. 1, an embodiment of the present invention provides an image depth estimation method, which may include the following steps:
step 11, acquiring N images { I) shot at the same view angle, the same focal length and different image distances1，I2，...，INN is more than or equal to 3, and N is a positive integer.
In a specific implementation, due to the image { I }1，I2，...，INAll the images are images with the same visual angle, so that the images can be obtained by only using one camera. Specifically, the focal point of the camera is adjusted, so that the camera focuses at different positions, and the shot objects can be located in the same focal distance. Due to the image { I1，I2，...，INAll the images are images with the same view angle, so that the number of pixel points included in each image and the pixel value of each pixel point are the same.
For example, FIG. 2 shows 8 images { I } acquired by the above method1，I2，...，I8}. From an image I1To image I8The focusing position is from near to far. Wherein, the image I1Closest in focus position, image I8Is infinity.
Step 12, respectively according to the images { I1，I2，...，INGenerate the corresponding local variance map.
In a specific implementation, the image { I is obtained by continuously adjusting the focusing position1，I2，...，INThe variation of the focusing position and the camera shake usually destroy the corresponding relationship between the obtained pixel points in each image. For example, in FIG. 1, image I1-I8The scene positions corresponding to the pixel points at the same position in the image are different.
Therefore, in order to reduce the influence of the change of the focusing position and the shake of the camera on the corresponding relation between pixel points among the images, the images { I are respectively obtained1，I2，...，INWhen generating a corresponding local variance map, firstly acquiring N images { I }1，I2，...，INCarrying out image alignment processing, and then respectively aligning the images { I) according to the aligned images1，I2，...，INSolving the local variance to obtain the local variance of the image { I }1，I2，...，INAnd f, respectively corresponding local variance maps.
In particular implementations, multiple methods may be employed for obtaining N images { I }1，I2，...，INCarry on the picture and align and deal with. For example, fig. 3 shows that 8 images acquired in fig. 1 are subjected to image alignment processing by using an affine transformation (affine transformation) -based image alignment algorithm, and an aligned image { I } is obtained correspondingly1’，I2’，...，I8'}. Wherein, the image I1Is' an image I1Image subjected to image alignment processing, image I2Is' an image I2Image after image alignment processing, image I8Is' an image I8And (5) carrying out image alignment processing on the image. After image alignment processing, in image I1～I8The scene positions corresponding to the pixel points at the same position in the images are basically the same, namely, the corresponding relation of the pixel points among the aligned images is basically consistent.
In a specific implementation, to obtain the image { I }1，I2，...，INOne-to-one correspondence of local variance maps to the image { I }1，I2，...，INAfter alignment processing, the image { I ] is calculated according to each aligned image1，I2，...，INI.e. computing the local variance of the image I1，I2，...，INAnd the local variance of each pixel point in the pixel.
In an embodiment of the present invention, the local variance g (x, y) of the pixel point (x, y) can be calculated by using the following formula:
wherein M is a local region centered on (x, y)The size of (d); (x ', y') is a regionThe position variable in (1) represents each pixel point in the position variable; μ (x, y) is the regionAnd μ (x, y) satisfies the following formula:
from formulas (1) and (2), the image { I } can be obtained1，I2，...，INAnd f, respectively corresponding local variance maps.
And step 13, searching an image with the maximum variance corresponding to each pixel point from each local variance map, and taking the object distance corresponding to the searched image with the maximum variance as the depth estimation value of the pixel point.
In a specific implementation, pixel points (x, y) at the same position have different variance values in each local variance map, and since the image distance of any target point is the distance between the image plane and the main plane when focusing (in focus) is performed with the target point as a focus, the variance of the pixel point corresponding to the target point in the local area where the pixel point is located is the largest, after each local variance map is obtained, an image with the largest variance corresponding to each pixel point is searched from each local variance map, and the object distance corresponding to the image with the largest variance is the depth estimation value of the pixel point.
As can be seen from the above, in the image depth estimation method in the embodiment of the present invention, the depth estimation value of each pixel point is obtained by focusing and ranging, that is, N images shot at the same viewing angle and the same focal length but different image distances are obtained, and then corresponding local variance maps are generated according to the obtained images, so as to search for an image with the largest variance corresponding to each pixel point from each local variance map, and use the object distance corresponding to the searched image with the largest variance as the depth estimation value of the pixel point, so that not only the complexity in the whole image depth estimation process can be reduced, but also the hardware cost in the whole image depth estimation process can be reduced.
As shown in fig. 4, an embodiment of the present invention further provides a depth map generating method, where the method may include the following steps:
and step 41, calculating the depth estimation value of each pixel point.
In a specific implementation, the depth estimation value of each pixel point can be obtained by using the image depth estimation method in the embodiment of the present invention. The embodiment shown in fig. 1 may be implemented specifically with reference to the description above, and will not be described herein again.
And step 42, obtaining a corresponding original depth estimation image according to the depth estimation value.
In specific implementation, after the depth estimation value of each pixel point is obtained, the corresponding original depth estimation image can be obtained
And 43, removing the smooth region from the original depth estimation image to obtain a corresponding sparse depth estimation image.
In a specific implementation, due to the image { I }1，I2，...，INThere is a smooth area in the natural object scene corresponding to, i.e. there is a lack of textureAnd the depth estimation for the smoothed region is not reliable. Therefore, in order to improve the accuracy of the depth map, the original depth estimation map may be first estimatedRemoving the smooth region to obtain corresponding sparse depth estimation imageThen the sparse depth estimation map is subjected toPerforming subsequent processing to obtain complete depth map
In particular implementations, a variety of methods may be employed to estimate the map from the original depthThe smooth region is removed. For example, an average local variance map may be obtained according to the local variance map, and then the smooth region may be located by using a threshold operation to obtain a sparse depth estimation map
Specifically, according to formula (3), the local variance g of the pixel point (x, y) is obtained from each local variance mapn(x, y) and averaging to obtain the average local variance of the pixel point (x, y)Wherein n is an image InCorresponding local variance map, N ∈ [0, N]：
After the average local variance of each pixel point is obtained, an average local variance map can be obtained.
Then, selecting a region corresponding to a pixel point with a depth estimation value smaller than a preset value H from the average local variance map, and obtaining a sparse depth estimation map according to the following formula
Wherein,is the original depth estimate for pixel point (x, y),is the sparse depth estimate for pixel point (x, y).
As shown in FIG. 5, FIG. 5a) and FIG. 5b) are respectively for the input image { I ] in FIG. 11，I2，...，I8And when depth estimation is carried out, obtaining an original depth estimation image and an average local variance image. In fig. 5b), the region corresponding to the black color portion is the selected smooth region, and the region corresponding to the black color portion is removed from fig. 5a), so as to obtain the sparse depth estimation map shown in fig. 5 c).
And 44, filling the sparse depth estimation map to obtain a complete depth map.
In specific implementation, the sparse depth estimation map is subjected to depth estimation by assuming that adjacent pixel points with similar brightness have similar depth estimation valuesFilling processing is carried out, and the original depth estimation map can be reliably processedThe estimated value of the region, namely the non-smooth region, is diffused to the whole image to obtain the complete depth map
In particular implementations, various algorithms may be employed to map the sparse depth estimateAnd (6) performing filling treatment. In an embodiment of the present invention, the aligned images may be averaged to obtain an average image corresponding to each aligned image, and then the sparse depth estimation map is obtained by using the average imageFilling processing is carried out to obtain a complete depth map
In a specific implementation, after obtaining the average image, a corresponding value of the matching Laplacian matrix L may be obtained according to the following formula:
l is an N matrix, N is the total number of pixel points of the average image, wherein the (p, q) th element corresponds to the relationship between the pixel points p and q, both p and q are one-dimensional coordinates corresponding to two-dimensional coordinates (x, y) of the image, and p or q is a local area with (x, y) as the centerAny pixel point in (1).p，qIs a Kronecker symbol (Kronecker delta), mux，yAnd gx，yAre respectively regionsM is the mean and variance value of∈ is a regularization parameter (regularization parameter) that is set to avoid the occurrence of "divide by zero", and is generally set to 1. Ip、IqPixel values of pixel points p and q, respectively.
It should be noted that, in the implementation, each region corresponds to a window. As defined above, if p and q are not in the same window, the L (p, q) value is 0. Therefore, L is a large sparse matrix, and operation optimization can be performed by utilizing the characteristic. In addition, the sparsity is inversely proportional to the corresponding computational complexity, and in order to further reduce the complexity, the value of L may be obtained by performing calculation using 2 × 2 overlapping windows.
After obtaining the value of L, the sparse depth estimation map may be minimized by quadratic programming to minimize the cost function E in equation (5)And (3) filling treatment:
and U is a diagonal matrix, when a pixel point corresponding to a diagonal element in the diagonal matrix is a pixel point of a reliable region, the diagonal element is 1, and otherwise, the diagonal element is 0. The scalar lambda is an adjusting parameter of the smoothness of the output image and can be set by a person in the art according to actual conditions, and the smaller lambda is, the complete depth map isThe smoother the surface.
According to equation (5), when the value of E is minimized, we get the optimal solutionBy further derivation, the quadratic programming problem can be simplified to an Ax-b problem, which can be realized by efficient L, U decomposition, i.e. the quadratic programming problem can be simplified to an Ax-b problem
Mapping the sparse depth estimate mapSubstituting the sparse depth estimation value of the middle pixel point into the formula (6), the complete depth map of the pixel point can be obtained respectivelySo that the complete depth map can be obtained
FIG. 6 is a diagram of a method for generating a depth map for the input image { I ] of FIG. 1 according to an embodiment of the present invention1，I2，...，I8The complete depth map generated. As can be seen from fig. 6, the complete depth map both guarantees that neighboring areas of similar luminance have similar depth values and leaves a depth discontinuity at the edge position of the input image.
As shown in fig. 7, in order to more clearly implement the method for generating the depth map in the embodiment of the present invention, the embodiment of the present invention further provides a schematic diagram of a process for generating the depth map. As shown in fig. 7, the process of generating the depth map is illustrated by using the processing results of the images in different stages as a main line and combining different processing operations. The generation process of the depth map is described below with reference to fig. 7:
the input images 71 with the same view angle, the same focal length and different image distances are subjected to image alignment processing to obtain corresponding aligned images 72. The local variance is calculated for each of the aligned images 72 to obtain corresponding local variance maps 73. Then, the local variance map 73 is focused and measured to obtain an original depth estimation map 74.
To further obtain the full depth map 75, after obtaining the original depth estimate map 74, the local variance map 73 is first averaged to obtain an average local variance map 76. Threshold operation is then performed on the original depth estimation map 74 and the mean local variance map 76 to obtain a sparse depth estimation map 77. The aligned images are averaged to obtain a corresponding average image 78. Finally, the complete depth map 75 is obtained by using the average image 78 and the sparse depth estimation map 77.
In order that those skilled in the art will better understand and realize the present invention, the following detailed description is provided for the device corresponding to the above method.
As shown in fig. 8, an embodiment of the present invention provides an image depth estimation device 80. The apparatus 80 may include: an image acquisition unit 81, a local variance map generation unit 82, and a depth calculation unit 83. Wherein:
the image acquisition unit 81 is adapted to acquire N images { I } captured at the same viewing angle, the same focal length, but different image distances1，I2，...，INN is more than or equal to 3, and N is a positive integer. The local variance map generation unit 82 is adapted to generate the local variance map from the images { I }, respectively1，I2，...，INGenerate the corresponding local variance map. The depth calculating unit 83 is adapted to search the local variance maps for the image with the largest variance corresponding to each pixel point, and use the object distance corresponding to the searched image with the largest variance as the depth estimation value of the pixel point.
In a specific implementation, the local variance map generating unit 82 may include: an alignment subunit 821 and a computation subunit 822. Wherein the alignment subunit 821 is adapted to align the acquired N images { I }1，I2，...，INCarry on the picture and align and deal with. The meterThe operator unit 822 is adapted to align the image { I) separately from each aligned image1，I2，...，INSolving the local variance to obtain the local variance of the image { I }1，I2，...，INAnd f, respectively corresponding local variance maps.
As shown in fig. 9, an embodiment of the present invention further provides a depth map generating apparatus 90. The apparatus 90 may include: an image depth estimation unit 91, a first image generation unit 92, a second image generation unit 93, and a third image generation unit 94. Wherein:
the image depth estimation unit 91 is adapted to calculate depth estimates for respective pixel points. The first image generation unit 92 is adapted to obtain a corresponding original depth estimation map from the depth estimation valuesThe second image generation unit 93 is adapted to estimate a depth map from the original depth mapRemoving the smooth region to obtain corresponding sparse depth estimation mapThe third image generation unit 94 is adapted to apply the sparse depth estimation mapFilling processing is carried out to obtain a complete depth map
In a specific implementation, the image depth estimation unit 91 may include: an image acquisition sub-unit 911, a local variance map generation sub-unit 912, and a depth calculation sub-unit 913. Wherein, the image acquiring subunit 911 is adapted to acquire N images { I } captured at the same angle of view, the same focal length, but different image distances1，I2，...，IN}，N is not less than 3 and is a positive integer. The local variance map generation subunit 912 is adapted to generate the local variance maps according to the images { I }1，I2，...，INGenerate the corresponding local variance map. The depth operator unit 913 is adapted to search the local variance maps for an image with the largest variance corresponding to each pixel point, and use an object distance corresponding to the searched image with the largest variance as the depth estimation value of the pixel point.
Wherein the local variance map generating sub-unit 912 may include: an alignment module (not shown) and a calculation module (not shown). Wherein the alignment module is adapted to align the acquired N images { I }1，I2，...，INCarry on the picture and align and deal with. The computing module is adapted to align the image { I) according to the aligned images, respectively1，I2，...，INSolving the local variance to obtain the local variance of the image { I }1，I2，...，INAnd f, respectively corresponding local variance maps.
In a specific implementation, the second image generating unit 93 may include: a compute subunit 931, a select subunit 932, and a process subunit 933. The calculating subunit 931 is adapted to average the variances of the pixels, respectively, to obtain an average local variance map corresponding to each of the local variance map partitions. The selecting subunit 932 is adapted to select the original depth estimation mapAnd taking the selected area as the smooth area, wherein the depth estimation value is smaller than the area corresponding to the pixel point with the preset value. The processing subunit 933 is adapted to derive the smooth region from the original depth estimation mapRemoving the original depth estimation map, and obtaining the removed original depth estimation mapAs the sparse depth estimation map
In a specific implementation, the third image generating unit 94 may include: an average sub-cell 941 and a padding sub-cell 942. The averaging subunit 941 is adapted to average each aligned image to obtain an average image of the aligned images. The filler sub-unit 942 is adapted to utilize the average image for the sparse depth estimation mapFilling processing is carried out to obtain a complete depth map
In a specific implementation, the fill subunits 942 are on the sparse depth estimation mapWhen the filling processing is carried out, 2-by-2 overlapping windows can be adopted to carry out the filling processing on the sparse depth estimation mapAnd (6) performing filling treatment.
Those skilled in the art will appreciate that all or part of the steps in the methods of the above embodiments may be implemented by associated hardware instructed by a program, which may be stored in a computer-readable storage medium, and the storage medium may include: ROM, RAM, magnetic or optical disks, and the like.
Although the present invention is disclosed above, the present invention is not limited thereto. Various changes and modifications may be effected therein by one skilled in the art without departing from the spirit and scope of the invention as defined in the appended claims.
Claims (14)
1. An image depth estimation method, comprising:
acquiring N images { I) shot at the same view angle and the same focal length but different image distances1,I2,…,INN is more than or equal to 3 and is a positive integer;
respectively according to the image { I1,I2,…,INGenerating a corresponding local variance map;
and searching the image with the maximum variance corresponding to each pixel point from each local variance map, and taking the object distance corresponding to the searched image with the maximum variance as the depth estimation value of the pixel point.
2. The image depth estimation method of claim 1, wherein the images { I } from which the images are respectively based1,I2,…,INGenerating a corresponding local variance map, comprising:
for the acquired N images { I1,I2,…,INCarrying out image alignment processing;
according to the aligned images, the images { I are respectively aligned1,I2,…,INSolving the local variance to obtain the local variance of the image { I }1,I2,…,INAnd f, respectively corresponding local variance maps.
3. A method for generating a depth map is characterized by comprising the following steps:
calculating the depth estimation value of each pixel point, comprising the following steps: acquiring N images { I) shot at the same view angle and the same focal length but different image distances1,I2,…,INN is more than or equal to 3 and is a positive integer; respectively according to the image { I1,I2,…,INGenerating a corresponding local variance map; searching an image with the maximum variance corresponding to each pixel point from each local variance map, and taking the object distance corresponding to the searched image with the maximum variance as the depth estimation value of the pixel point;
obtaining a corresponding original depth estimation image according to the depth estimation value;
removing a smooth region from the original depth estimation map to obtain a corresponding sparse depth estimation map;
and filling the sparse depth estimation map to obtain a complete depth map.
4. Method for generating a depth map as claimed in claim 3, characterized in that said respective basis is a function of said image { I }1,I2,…,INGenerating a corresponding local variance map, comprising:
for the acquired N images { I1,I2,…,INCarrying out image alignment processing;
according to the aligned images, the images { I are respectively aligned1,I2,…,INSolving the local variance to obtain the local variance of the image { I }1,I2,…,INAnd f, respectively corresponding local variance maps.
5. The method for generating a depth map of claim 4, wherein removing the smooth region from the original depth estimation map to obtain a corresponding sparse depth estimation map comprises:
respectively averaging the variances of all the pixel points to obtain average local variance graphs respectively corresponding to all the local variance graphs;
selecting a region corresponding to a pixel point of which the depth estimation value is smaller than a preset value in the average local variance image, and taking the selected region as the smooth region;
and removing the smooth region from the original depth estimation map, and taking the removed original depth estimation map as the sparse depth estimation map.
6. The method for generating a depth map of claim 4, wherein the filling the sparse depth estimation map to obtain a complete depth map comprises:
averaging each of the aligned images to obtain an average image of the aligned images;
and filling the sparse depth estimation image by using the average image to obtain a complete depth image.
7. The method for generating a depth map of claim 6, wherein the filling the sparse depth estimation map comprises: and filling the sparse depth estimation map by adopting 2-by-2 overlapping windows.
8. An image depth estimation device, characterized by comprising:
an image acquisition unit adapted to acquire N images { I } captured at the same angle of view, the same focal length, but different image distances1,I2,…,INN is more than or equal to 3 and is a positive integer;
a local variance map generation unit adapted to generate a local variance map from the images { I }1,I2,…,INGenerating a corresponding local variance map;
and the depth calculation unit is suitable for searching the image with the maximum variance corresponding to each pixel point from each local variance map, and taking the object distance corresponding to the searched image with the maximum variance as the depth estimation value of the pixel point.
9. The image depth estimation apparatus according to claim 8, wherein the local variance map generation unit includes:
an alignment subunit adapted to align the acquired N images { I }1,I2,…,INCarrying out image alignment processing;
a computing subunit adapted to compute said images { I } from each aligned image, respectively1,I2,…,INSolving the local variance to obtain the local variance of the image { I }1,I2,…,INAnd f, respectively corresponding local variance maps.
10. A depth map generation apparatus, comprising:
the image depth estimation unit is suitable for calculating the depth estimation value of each pixel point and comprises the following steps: an image acquisition subunit adapted to acquire N images { I } captured at the same viewing angle, the same focal length, but different image distances1,I2,…,INN is more than or equal to 3 and is a positive integer; a local variance map generation subunit adapted to generate a local variance map from the images { I }1,I2,…,INGenerating a corresponding local variance map; a depth calculation operator unit adapted to search each local variance map for an image with the largest variance corresponding to each pixel point and to search forThe object distance corresponding to the image with the largest variance is used as the depth estimation value of the pixel point;
the first image generation unit is suitable for obtaining a corresponding original depth estimation image according to the depth estimation value;
the second image generation unit is suitable for removing a smooth region from the original depth estimation image to obtain a corresponding sparse depth estimation image;
and the third image generation unit is suitable for filling the sparse depth estimation map to obtain a complete depth map.
11. The depth map generating apparatus of claim 10, wherein the local variance map generating subunit includes:
an alignment module adapted to align the acquired N images { I }1,I2,…,INCarrying out image alignment processing;
a computing module adapted to align the images { I } according to the aligned images, respectively1,I2,…,INSolving the local variance to obtain the local variance of the image { I }1,I2,…,INAnd f, respectively corresponding local variance maps.
12. The depth map generating apparatus according to claim 11, wherein the second image generating unit includes:
the calculation subunit is suitable for respectively averaging the variances of all the pixel points to obtain an average local variance map corresponding to each local variance map subsection;
a selecting subunit, adapted to select a region corresponding to a pixel point in the original depth estimation image whose depth estimation value is smaller than a preset value, and take the selected region as the smooth region;
a processing subunit adapted to remove the smoothed region from the original depth estimation map and to use the removed original depth estimation map as the sparse depth estimation map.
13. The depth map generating apparatus according to claim 11, wherein the third image generating unit includes:
the average subunit is suitable for averaging all the aligned images to obtain an average image of the aligned images;
and the filling subunit is suitable for filling the sparse depth estimation map by using the average image to obtain a complete depth map.
14. The depth map generating apparatus of claim 13, wherein the padding subunit is adapted to pad the sparse depth estimation map with 2 x 2 overlapping windows.
Priority Applications (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510859849.9A CN106815865A (en) | 2015-11-30 | 2015-11-30 | Image depth estimation method, depth drawing generating method and device |
Applications Claiming Priority (1)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201510859849.9A CN106815865A (en) | 2015-11-30 | 2015-11-30 | Image depth estimation method, depth drawing generating method and device |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| CN106815865A true CN106815865A (en) | 2017-06-09 |
Family
ID=59155834
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| CN201510859849.9A Pending CN106815865A (en) | 2015-11-30 | 2015-11-30 | Image depth estimation method, depth drawing generating method and device |
Country Status (1)
| Country | Link |
|---|---|
| CN (1) | CN106815865A (en) |
Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101312539A (en) * | 2008-07-03 | 2008-11-26 | 浙江大学 | Hierarchical Image Depth Extraction Method for 3D Television |
| CN102103248A (en) * | 2009-12-21 | 2011-06-22 | 索尼公司 | Autofocus with confidence measure |
| US20110181770A1 (en) * | 2010-01-27 | 2011-07-28 | Zoran Corporation | Depth from defocus calibration |
| CN102509294A (en) * | 2011-11-08 | 2012-06-20 | 清华大学深圳研究生院 | Single-image-based global depth estimation method |
| CN103049906A (en) * | 2012-12-07 | 2013-04-17 | 清华大学深圳研究生院 | Image depth extraction method |
| CN103559701A (en) * | 2013-09-26 | 2014-02-05 | 哈尔滨商业大学 | Two-dimensional single-view image depth estimation method based on DCT coefficient entropy |
-
2015
- 2015-11-30 CN CN201510859849.9A patent/CN106815865A/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN101312539A (en) * | 2008-07-03 | 2008-11-26 | 浙江大学 | Hierarchical Image Depth Extraction Method for 3D Television |
| CN102103248A (en) * | 2009-12-21 | 2011-06-22 | 索尼公司 | Autofocus with confidence measure |
| US20110181770A1 (en) * | 2010-01-27 | 2011-07-28 | Zoran Corporation | Depth from defocus calibration |
| CN102509294A (en) * | 2011-11-08 | 2012-06-20 | 清华大学深圳研究生院 | Single-image-based global depth estimation method |
| CN103049906A (en) * | 2012-12-07 | 2013-04-17 | 清华大学深圳研究生院 | Image depth extraction method |
| CN103559701A (en) * | 2013-09-26 | 2014-02-05 | 哈尔滨商业大学 | Two-dimensional single-view image depth estimation method based on DCT coefficient entropy |
Non-Patent Citations (2)
| Title |
|---|
| KROTKOV E P: "Focusing", 《INTERNATIONAL JOURNAL OF COMPUTER VISION》 * |
| 江静，张雪松: "基于计算机视觉的深度估计方法", 《光电技术应用》 * |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US12423849B2 (en) | Systems and methods for hybrid depth regularization | |
| CN103945118B (en) | Image weakening method, device and electronic equipment | |
| CN112884682B (en) | A stereoscopic image color correction method and system based on matching and fusion | |
| US10129455B2 (en) | Auto-focus method and apparatus and electronic device | |
| CN105374039B (en) | Monocular image depth information method of estimation based on contour acuity | |
| CN107316326B (en) | Edge-based disparity map calculation method and device applied to binocular stereo vision | |
| CN103473743B (en) | A kind of method obtaining image depth information | |
| CN110853151A (en) | Three-dimensional point set recovery method based on video | |
| US20090092282A1 (en) | System and Method for Tracking Objects with a Synthetic Aperture | |
| KR102415505B1 (en) | Method and apparatus for matching stereo images | |
| WO2014068472A1 (en) | Depth map generation from a monoscopic image based on combined depth cues | |
| US20160093028A1 (en) | Image processing method, image processing apparatus and electronic device | |
| JP7378219B2 (en) | Imaging device, image processing device, control method, and program | |
| KR101852085B1 (en) | Depth map acquisition device and depth map acquisition method | |
| GB2585197A (en) | Method and system for obtaining depth data | |
| KR100943635B1 (en) | Method and apparatus for generating disparity map using digital camera image | |
| EP3506206B1 (en) | Method and apparatus for depth-map estimation | |
| JP6624785B2 (en) | Image processing method, image processing device, imaging device, program, and storage medium | |
| CN113763449A (en) | Deep recovery method, device, electronic device and storage medium | |
| JP2016148588A (en) | Depth estimation model generation device and depth estimation device | |
| KR101921608B1 (en) | Apparatus and method for generating depth information | |
| TWI639135B (en) | Restoration method for blurred image | |
| CN107403448B (en) | Cost function generation method and cost function generation device | |
| KR20110133677A (en) | 3D image processing device and method | |
| RU2580466C1 (en) | Device for recovery of depth map of scene |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| PB01 | Publication | ||
| PB01 | Publication | ||
| SE01 | Entry into force of request for substantive examination | ||
| SE01 | Entry into force of request for substantive examination | ||
| RJ01 | Rejection of invention patent application after publication |
Application publication date: 20170609 |
|
| RJ01 | Rejection of invention patent application after publication |