CN105957043B - The blurred picture blind restoration method activated automatically based on gradient - Google Patents

Abstract

The invention discloses a kind of blurred picture blind restoration method activated automatically based on gradient, the technical issues of the practicability is poor for solving existing blurred picture blind restoration method.Technical solution is the ambiguous estimation core on image gradient, during iteration updates image gradient estimation and image fuzzy core, is estimated using the most important gradient components of the automatic activating part of incremental form as nonzero element and to non-zero gradient value.And then fuzzy core only is updated with the gradient image containing sparse nonzero element.The accurate fuzzy core obtained using estimation.The present invention can reliably be restored for multi-Fuzzy core and the blurred picture containing different content, and obtain the clear image of high quality.In model to the sparse constraint of gradient activity factor and algorithm in stop condition design enable the algorithm to estimation obtain sparse and accurate image gradient, ensure that algorithm to the robust of noise and the insensitive of parameter and preferable timeliness.

Description

Fuzzy image blind restoration method based on gradient automatic activation

Technical Field

The invention relates to a blind restoration method of a blurred image, in particular to a blind restoration method of a blurred image based on gradient automatic activation.

Background

In image capture, blur is a common form of image degradation. Blur in an image is typically caused by camera lens mis-focus or relative motion between the camera and the photographic target during exposure. Restoring the blurred image has important significance for improving the visual effect of the image and extracting image information. In the image deblurring problem, a form of blurring is generally represented using a blurring kernel. Aiming at the problem of blind restoration of a fuzzy image with unknown fuzzy kernel, the problem of difficult and problematic kernel is to estimate the accurate fuzzy kernel.

The document "Cho, singhyun, and mounting lee," Fast motion deblocking. "ACMTransactions on Graphics (TOG). vol.28.no.5.acm, 2009" proposes a blurred image blind restoration method based on the object edge. The method comprises the steps of iteratively estimating a clear image and an image blur kernel, wherein in the iterative process, the blur kernel obtained by the previous step of estimation is firstly fixed in each iteration, the clear image is estimated by solving an optimization problem, and then the blur kernel is estimated by fixing the clear image obtained by estimation. When the fuzzy kernel is updated, the method utilizes the bilateral filter to carry out edge enhancement processing on the clear image obtained by estimation, so that only the strong edge which accords with the definition of the bilateral filter is reserved in the processed image, and then the filtered image is utilized to estimate the fuzzy kernel. The method relies on the fact that the image contains strong and isolated edges, and that the edges can respond strongly under the action of a bilateral filter.

Disclosure of Invention

In order to overcome the defect of poor practicability of the conventional blurred image blind restoration method, the invention provides a blurred image blind restoration method based on gradient automatic activation. The method estimates a blur kernel on an image gradient, and in the process of iteratively updating the image gradient estimation and the image blur kernel, part of the most important gradient components are automatically activated in an incremental mode to be non-zero elements and non-zero gradient values are estimated. And then updating the blur kernel with only the gradient image containing sparse non-zero elements. And utilizing the estimated accurate fuzzy core. The invention can reliably restore various fuzzy cores and fuzzy images containing different contents, and obtain high-quality clear images. Sparse constraint on gradient activation factors in the model and stop condition design in the algorithm enable the algorithm to estimate the obtained sparse and accurate image gradient, and the robustness of the algorithm to noise, the insensitivity of parameters and good timeliness are guaranteed.

The technical scheme adopted by the invention for solving the technical problems is as follows: a fuzzy image blind restoration method based on gradient automatic activation is characterized by comprising the following steps:

firstly, giving a blurred image y, and firstly solving the gradient of the blurred image yUsing the gradient of the image in the horizontal and vertical directions to defineIs composed of

WhereinAndrepresenting the gradient of the blurred image y in the vertical and horizontal directions, respectively, i.e. orderAndand denotes a convolution operation.

Step two, settingEstimation of sharp image gradients by solving equation (2)Blur kernel k and gradient activation factor τ:

wherein tau is a gradient activation factor, and elements in the gradient activation factor only take values of 0 or 1 and are used for indicatingWhether the element in the corresponding position is activated, i.e. when τ_jWhen 1 is trueIs non-zero, i.e. when τ_jWhen equal to 0Must be 0, j represents τ and⊙ represents a point-by-element multiplication operation, Λ ═ τ: | | | τ | | non-volatile memory₀≤κ,τ∈{0,1}ⁿIs the value range of tau, n representsThe number of elements (A) and (Y) are non-negative penalty coefficients, zero norm | | · | | luminance₀Denotes the number of non-zero elements in τ, and κ is a positive integer of 1 or more. Constraint term | | k | | non-woven phosphor in formula (2)₁1, k ≧ 0 indicates that the elements in k should be non-negative and the sum is 1. For the solution of equation (2), first, forCarry out initialization toAnd then circularly executing the following steps 1 and 2 until the set iteration number T.

1. FixingAnd solving an image blurring kernel k. The specific method comprises the following steps:

then, the negative number in the solution of k is made to be zero, and k is normalized, that is, the order is madeWherein k is_iRepresenting the ith element of k. The formula (2) has a closed form solution, the fast Fourier change is utilized to accelerate the solving speed, and the solving formula is as follows:

wherein, F (-) and F^-1(. to) respectively represent a two-dimensional fast fourier transform and an inverse transform,is the conjugate of F (-), and I is the identity matrix. The blur kernel in the case of a fixed image is obtained by equation (4).

2. Fix k, solve sharp image gradientAnd a corresponding gradient activation vector tau. The method comprises introducing intermediate variablesAnd solution formula (5)

Equation (5) is solved iteratively by the following steps.

1) Initialization of variables, orderτ⁰0, k is 0, where 0 denotes a zero vector and k denotes an iteration count. Defining an objective function valueAnd calculate f⁰. The k value is estimated. The current fuzzy kernel k is obtained after clockwise rotation by 180 DEGIs calculated to obtainTaking the absolute value of g to obtainTo findMaximum value of (1) is g_maxStatistics ofMiddle greater than delta x g_maxThe number of elements of (d) is set to κ, where δ is a real number with a value range of [0, 1).

2) Firstly, k is equal to k +1, and the current fuzzy kernel k is obtained by clockwise rotating by 180 degreesIs calculated to obtaing^kAndis the same size as τ by g^kValue determination of the Medium element τ^kIs 0 or 1, i.e. whether the gradient at the respective position is activated. Detailed description of the inventionTo g is^kTaking the absolute value to obtainAccording to the size of element valueTaking the first k elements inAnd recorded in the set C^kIn (1). Due to the fact thatAnd g^k、And τ being the same size, C^kEach element j ∈ C in (C)^kMay correspond to g^k、And the corresponding position in τ. In each iteration, it should be guaranteed that C is put arbitrarily^kElement j in (E) C^kCorrespond toI.e. the newly activated gradient should be inactive in the previous iteration. In recording C^kWhile firstly let τ be^k＝τ^k-1Then updatedFor theDefinition of S^kFor a recording τ^kSet of all 1 element positions in (S)^k)^CDenotes S^kThe complete set is the position set of all pixel points in the image.

3) At C^oK and S, o ═ 1^kOn the basis ofSolving equation (6) to obtain

WhereinAndto representOf which the respective element corresponds to S^kAnd C^oThe position specified by the element(s).The medium element still keeps the original spatial position unchanged, so thatTo aim at S^kTwo-dimensional convolution of the medium elements. UpdatingBy solving the formula (6), letThe activated image gradient region isOrder toWith inactive part of zero, i.e. To representPosition set S of quilt^kThe portion of the indication is that portion of the indication,to representSet of middle and absent positions S^kThe indicated section. Equation (6) is solved iteratively by:

i) initializationWhereinIs composed ofQuilt S^kThe value of the indicated section, t, is the iteration count. Defining an objective function valueAnd calculate g⁰。

ii) let t be t + 1. ComputingWhereinThe gradient descent control method is obtained by clockwise rotating the fuzzy kernel k by 180 degrees, and rho is a real number larger than zero and represents gradient descent step length.

iii) according to C^oK, where the elements indicate positions, k sub-maps are extracted from dRespectively calculating the two-norm value of each subgraph toFor k v_oSorting k in descending order to obtain a sequence w₁≥...≥w_k. Based on the sequence w₁,...,w_kThe following condition (7) is satisfied by finding a maximum sequence number value in 1.

ComputingAnd the sequence α is obtained by the calculation of the formula (8)₁,...,α_k。

Using sequence α₁,...,α_kD is processed to make d correspond to the position set C^oThe elements in o ═ 1.. k are transformed according to the principles in equation (9).

Then updated

vi) calculating the objective function value g^tIf the condition | g is satisfied^t-g^t-1|/|g⁰|<ε_inOr t is more than or equal to t_maxThen stop updating and returnTo updateAfter thatIf the condition is not met, jumping to step ii) and continuing to execute steps ii) -vi).

4) UpdatingAnd calculates an objective function value f^kIf the condition | f is satisfied^t-f^t-¹|/|f⁰|<Epsilon or k is not less than k_maxThen stop the iteration and return toTo be updatedIf the condition is not met, jumping to step 2) and continuing to iterate steps 2) -4).

After iterating step 1 and step 2 for T times, obtaining an estimated fuzzy kernel k.

And thirdly, carrying out non-blind deconvolution on the blurred image y by using the estimated blurred kernel k to obtain a clear image x. Non-blind deconvolution can be achieved by solving a problem (10).

Wherein,andequation (10) is a fully-variational model, by introducing an intermediate variable z_vAnd z_hLet us orderAnd isThe problem is equivalently transformed into equation (11) and solved using the alternating direction multiplier method.

Wherein β is a penalty factor.

The invention has the beneficial effects that: the method estimates a blur kernel on an image gradient, and in the process of iteratively updating the image gradient estimation and the image blur kernel, part of the most important gradient components are automatically activated in an incremental mode to be non-zero elements and non-zero gradient values are estimated. And then updating the blur kernel with only the gradient image containing sparse non-zero elements. And utilizing the estimated accurate fuzzy core. The invention can reliably restore various fuzzy cores and fuzzy images containing different contents, and obtain high-quality clear images. Sparse constraint on gradient activation factors in the model and stop condition design in the algorithm enable the algorithm to estimate the obtained sparse and accurate image gradient, and the robustness of the algorithm to noise, the insensitivity of parameters and good timeliness are guaranteed.

The present invention will be described in detail with reference to the following embodiments.

Detailed Description

The blind restoration method of the blurred image based on gradient automatic activation comprises the following specific steps:

step one, given a blurred image y, the representation of the blurred image y in the gradient domain is solvedUsing the gradient of the image in the horizontal and vertical directions to defineIs composed of

WhereinAndrepresenting the gradient of the blurred image y in the vertical and horizontal directions, respectively, i.e. orderAndand denotes a convolution operation.

Step two, settingEstimation of sharp image gradients by equation (2)Blur kernel k and gradient activation factor τ:

wherein tau is a gradient activation factor, and elements in the gradient activation factor only take values of 0 or 1 and are used for indicatingWhether the element in the corresponding position is activated, i.e. when τ_jWhen 1 is trueMay be non-zero, i.e. when τ_jWhen equal to 0Must be 0, j represents τ and⊙ represents a point-by-element multiplication operation, Λ ═ τ: | | | τ | | non-volatile memory₀≤κ,τ∈{0,1}ⁿIs the value range of tau, n representsThe number of elements (A) and (Y) are non-negative penalty coefficients, zero norm | | · | | luminance₀Denotes the number of non-zero elements in τ, and κ is a positive integer of 1 or more. λ is set to 0.001 and γ is set to 0.5. Constraint term | | k | | non-woven phosphor in formula (2)₁1, k ≧ 0 indicates that the elements in k should be non-negative and the sum is 1. For the solution of the formula (2), first, forCarry out initialization toAnd then circularly executing the following steps 1 and 2 until the set iteration number T. T is set to 15.

1. FixingAnd solving an image blurring kernel k. The specific method comprises the following steps:

then, the negative number in k obtained by solving is made to be zero, and k is normalized, namely, the order is madeWherein k is_iRepresenting the ith element of k. The formula (2) has a closed form solution, and the fast Fourier change can be used for accelerating the solving speed, and the solving formula is as follows:

wherein, F (-) and F^-1(. to) respectively represent a two-dimensional fast fourier transform and an inverse transform,is the conjugate of F (-), and I is the identity matrix. The blur kernel in the case of a fixed image can be obtained by equation (4).

2. Fix k, solve sharp image gradientAnd a corresponding gradient activation vector tau. The method comprises introducing intermediate variablesAnd solution formula (5)

Equation (5) is a non-convex problem that can be solved iteratively after convex relaxation.

1) Initialization of variables, orderτ⁰0, k is 0, where 0 denotes a zero vector and k denotes an iteration count. Defining an objective function valueAnd calculate f⁰. The k value is estimated. The current fuzzy kernel k is obtained after clockwise rotation by 180 DEGIs calculated to obtainTaking the absolute value of g to obtainTo findMaximum value of (1) is g_maxStatistics ofMiddle greater than delta x g_maxThe number of elements (c) is represented by κ. Let δ be 0.6.

2) Firstly, k is equal to k +1, and the current fuzzy kernel k is obtained by clockwise rotating by 180 degreesIs calculated to obtaing^kAndis the same as τ in size and can pass through g^kValue determination of the Medium element τ^kIs 0 or 1, i.e. whether the gradient at the respective position is activated. The specific method is to g^kTaking the absolute value to obtainAccording to the size of element valueTaking the first k elements inMiddle spatial position and recorded in set C^kIn (1). Due to the fact thatAnd g^k、And τ being the same size, C^kEach element j ∈ C in (C)^kMay correspond to g^k、And the position of the corresponding element in τ. In each iteration, it should be guaranteed that C is put arbitrarily^kElement j in (E) C^kCorrespond toI.e. the newly activated gradient should be inactive in the previous iteration. In recording C^kWhile firstly let τ be^k＝τ^k-1Then updatedFor theDefinition of S^kFor a recording τ^kSet of all 1 element positions in (S)^k)^CDenotes S^kThe complement set of (1) isThe position set of all elements in.

3) At C^oK and S, o ═ 1^kOn the basis of (2), the solution of the formula (6) can obtain

WhereinAndto representOf which the respective element corresponds to S^kAnd C^oThe position specified by the element(s).The medium element still keeps the original spatial position unchanged, so thatTo aim at S^kTwo-dimensional convolution of the medium elements. UpdatingBy solving equation (6), letThe activated image gradient region isOrder toWith inactive part of zero, i.e. To representPosition set S of quilt^kThe portion of the indication is that portion of the indication,to representZhongyaoSet of quilt positions S^kThe indicated section. Equation (6) can be solved iteratively by:

i) initializationWhereinIs composed ofQuilt S^kThe value of the indicated section, t, is the number of iterations. Defining an objective function valueAnd calculate g⁰。

ii) let t be t + 1. ComputingWhereinThe gradient descent control method is obtained by clockwise rotating the fuzzy kernel k by 180 degrees, and rho is a real number larger than zero and represents gradient descent step length. ρ is 0.5.

iii) according to C^oK, where the elements indicate positions, k sub-maps are extracted from dSeparately calculating the two-norm values of each sub-graphFor k v_oSorting k in descending order to obtain a sequence w₁≥...≥w_k. Based on the sequence w₁,...,w_kThe following condition (7) is satisfied by finding a maximum sequence number value in 1.

ComputingAnd the sequence α is obtained by the calculation of the formula (8)₁,...,α_k。

Using sequence α₁,...,α_kD is processed to make d correspond to the position combination C^oThe elements in o ═ 1.. k are transformed according to the principles in equation (9).

Then updated

vi) calculating the objective function value g^tIf the condition | g is satisfied^t-g^t-1|/|g⁰|<ε_inOr t is more than or equal to t_maxThen stop updating and returnTo be updatedIf the condition is not met, jumping to ii) and continuing to execute steps ii) -vi). Setting epsilon_in＝10^-6And t_max＝15。

4) UpdatingAnd calculates an objective function value f^kIf the condition | f is satisfied^t-f^t-1|/|f⁰|<Epsilon or k is not less than k_maxThen stop the iteration and return toTo be updatedIf the condition is not met, jumping to step 2) and continuing to iterate steps 2) -4). Setting ε to 10^-5And k_max＝15。

And after the step 1 and the step 2 are iterated for T times, stopping the iteration to obtain an estimated fuzzy kernel k.

And thirdly, carrying out non-blind deconvolution on the blurred image y by using the estimated blurred kernel k to obtain a clear image x. The non-blind deconvolution can be realized by equation (10).

Wherein,andthe formula (10) is a full variation model, and an intermediate variable z can be introduced_vAnd z_hLet us orderAnd isThe problem is equivalently transformed into equation (11) and solved using the alternating direction multiplier method.

Wherein | · | purple sweet₁Is 1 rangeAnd the value of β real number range (0.005 and 0.1) is set as a penalty factor β, and parameters can be adjusted according to the actual condition of the image.

Claims (1)

1. A blurred image blind restoration method based on gradient automatic activation is characterized by comprising the following steps:

firstly, giving a blurred image y, firstly solving the gradient ▽ y of the blurred image y, and defining ▽ y as ▽ y by utilizing the gradient of the image in the horizontal and vertical directions

▽y＝[(▽_vy)^T,(▽_hy)^T]^T(1)

▽ therein_vy and ▽_hy represents the gradient of the blurred image y in the vertical and horizontal directions, respectively, i.e. ▽_hy＝[-1,1]Y and ▽_vy＝[-1,1]^TY, and denotes a convolution operation;

step two, given ▽ y, the sharp image gradient ▽ x, the blur kernel k and the gradient activation factor τ are estimated by solving equation (2):

wherein τ is a gradient activation factor, and the elements therein only take values of 0 or 1, and are used to indicate ▽ x whether the elements at the corresponding positions are activated, that is, when τ is_j▽ x when equal to 1_jIs non-zero, i.e. when τ_j▽ x when equal to 0_jMust be 0, j represents the same position in τ and ▽ x, ⊙ represents a point-by-element multiplication operation, # ═ τ: | | τ | |, luminance₀≤κ,τ∈{0,1}ⁿThe method is a value taking domain of tau, n represents the number of elements ▽ x, lambda and gamma are nonnegative penalty coefficients, zero norm | · | | torry₀Represents the number of nonzero elements in the tau, and kappa is a positive integer greater than or equal to 1; constraint term | | k | | non-woven phosphor in formula (2)₁For the solution of the formula (2), firstly, ▽ x is initialized, ▽ x is ▽ y, and then the following steps 1 and 2 are executed in a loop until the set iteration number T;

step 1, fixing ▽ x, and solving an image fuzzy kernel k, wherein the specific method comprises the following steps:

then, the negative number in the solution of k is made to be zero, and k is normalized, that is, the order is madeWherein k is_iThe ith element representing k; the formula (2) has a closed form solution, the fast Fourier change is utilized to accelerate the solving speed, and the solving formula is as follows:

wherein, F () and F^-1(. cndot.) represents two-dimensional fast Fourier transform and inverse transform, respectively, conj (F (▽ x)) is the conjugate of F (. cndot.), and I is a unit matrix;

step 2, fixing k, and solving the gradient ▽ x of the clear image and the corresponding gradient activation vector tau, wherein the specific method comprises the steps of firstly introducing an intermediate variable ξ ═ ▽ y-k ═ ▽ x ⊙ tau, and solving a formula (5)

The formula (5) is solved iteratively by the following steps;

1) variable initialization, let ▽ x⁰＝0，ξ⁰＝▽y，τ⁰0, k is 0, where 0 represents a zero vector and k represents an iteration count; defining an objective function valueAnd calculate f⁰(ii) a Estimating a kappa value; the current fuzzy kernel k is obtained after clockwise rotation by 180 DEGIs calculated to obtainTaking the absolute value of g to obtainTo findMaximum value of (1) is g_maxStatistics ofMiddle greater than delta x g_maxThe number of elements of (a) is set as k, wherein δ is a real number with a numeric range of [0, 1);

2) firstly, k is equal to k +1, and the current fuzzy kernel k is obtained by clockwise rotating by 180 degreesIs calculated to obtaing^kSame size as ▽ x and τ, by g^kValue determination of the Medium element τ^kIs 0 or 1, i.e. whether the gradient at the respective position is activated; the specific method is to g^kTaking the absolute value to obtainAccording to the size of element valueTaking the first k elements inAnd recorded in the set C^kPerforming the following steps; due to the fact thatAnd g^k▽ x and tau are the same size,

C^keach element j' e C in (1)^kCorresponds to g^k▽ x and tau, and in each iteration, any placement of C should be guaranteed^kElement j' e in (C)^kCorrespond toI.e. the newly activated gradient should be inactive in the previous iteration; in recording C^kWhile firstly let τ be^k＝τ^k-1Then updatedFor thej′∈C^k(ii) a Definition of S^kFor a recording τ^kSet of all 1 element positions in (S)^k)^CDenotes S^kThe complete set is a position set of all pixel points in the image;

3) at C^oO 1, k and S^kOn the basis of (1), solving the formula (6) to obtain

WhereinAndsub-images representing ▽ x, the respective elements corresponding to S^kAnd C^oThe position specified by the element;the medium element still keeps the original spatial position unchanged, so thatTo aim at S^kTwo-dimensional convolution of middle element, update ▽ x^kBy solving equation (6), ▽ x^kThe activated image gradient region isLet ▽ x^kWith inactive part of zero, i.e. Representation ▽ x^kPosition set S of quilt^kThe portion of the indication is that portion of the indication,representation ▽ x^kSet of middle and absent positions S^kA portion of the indication; equation (6) is solved iteratively by:

i) initializationt is 0, whereinIs ▽ x^k-1Quilt S^kThe value of the indicated portion, t is the iteration count; defining an objective function valueAnd calculate g⁰；

ii) let t ═ t + 1; computingWhereinThe gradient descent control method is obtained by clockwise rotating a fuzzy kernel k by 180 degrees, wherein rho is a real number larger than zero and represents gradient descent step length;

iii) extracting k sub-maps from d according to the positions indicated by the elements in Co, o 1Respectively calculating the two-norm value of each subgraph toFor k v_oSorting the k descending order into an orderColumn w₁≥...≥w_k(ii) a Based on the sequence w₁,...,w_kFinding a maximum sequence number value in 1,.. k satisfies the following condition (7);

computingAnd the sequence α is obtained by the calculation of the formula (8)₁,...,α_k；

Using sequence α₁,...,α_kProcessing d, converting elements in d corresponding to a position set Co, o being 1, k according to a principle in an expression (9);

then updated

vi) calculating the objective function value g^tIf the condition | g is satisfied^t-g^t-1|/|g⁰|＜ε_inOr t is more than or equal to t_maxThen stop updating and returnTo be updatedIf the condition is not met, jumping to the step ii) and continuously executing the steps ii) -vi);

4) update ξ^k＝▽y-k*▽x^kAnd calculating the objective function value f^kIf the condition | f is satisfied^t-f^t-1|/|f⁰If | < epsilon or k is more than or equal to k_maxThen the iteration is stopped and returns to ▽ x^k▽ x after updating, if the condition is not satisfied, jumping to the step 2) and continuously iterating the step 2) -4);

obtaining an estimated fuzzy kernel k after iterating for T times in the step 1 and the step 2;

thirdly, carrying out non-blind deconvolution on the blurred image y by using the blurred kernel k obtained by estimation to obtain a clear image x; the non-blind deconvolution is realized by solving the formula (10);

wherein, ▽_hx＝[-1,1]X and ▽_vx＝[-1,1]^TX; equation (10) is a fully-variational model, by introducing an intermediate variable z_vAnd z_hLet z_v＝▽_vx and z_h＝▽_hx, equivalently converting the problem into an expression (11), and solving by using an alternative direction multiplier method;

wherein β is a penalty factor.