CN118735806A - Conditional diffusion low-light image enhancement method and system based on dual-branch attention feature encoding - Google Patents

Abstract

The present invention discloses a conditional diffusion low-light image enhancement method and system based on dual-branch attention feature coding. The method constructs a low-light image feature coding model and a conditional diffusion model based on dual-branch attention. The low-light image feature coding model includes a gradient recovery module and a brightness recovery module. A channel attention mechanism and a spatial attention mechanism are used to transfer the features of the low-light image to the gradient recovery module and the brightness recovery module, and the feature map and the gradient map of the low-light image are restored. The gradient map features and the brightness map features are used as prior information to guide the training of the diffusion model, so that the enhanced normal light image has better quality and clarity.

Description

Conditional diffusion low-light image enhancement method and system based on dual-branch attention feature encoding

Technical Field

The present invention relates to the field of computer vision and artificial intelligence technology, and specifically to a conditional diffusion low-light image enhancement method and system based on dual-branch attention feature encoding.

Background Art

With the continuous development of multimedia technology, images have become the most common information medium in people's daily lives, and a lot of information is stored and transmitted in the form of images. Therefore, the quality of images is closely related to the amount and accuracy of information obtained. However, during the image capture process, due to the influence of environmental factors, such as cloudy days or night scenes, the image quality is dark, the contrast and signal-to-noise ratio are low, and other problems. Low-light image computational enhancement technology helps to improve the quality and clarity of low-light images, and can also provide more reliable and high-quality data for related applications in the fields of digital image processing, computer vision, artificial intelligence, etc.

Existing low-light image enhancement methods are mainly divided into two categories: the first category is the traditional low-light image enhancement method, which is further divided into enhancement methods based on histogram equalization and enhancement methods based on Retinex decomposition models; the second category is low-light image enhancement methods based on deep learning. The histogram equalization-based method enhances the smaller pixel values by adjusting the pixel distribution of the input low-light image, thereby improving the contrast and clarity of the image. The image restored using this method may have problems such as excessive brightness, loss of local details, and color distortion, resulting in unsatisfactory visual effects. The enhancement method based on the Retinex model decomposes the perceived image into two components: reflection and illumination, so that changes in illumination will not affect the color of the object, but the enhanced image will still have color distortion and over-enhancement.

Therefore, how to deal with the noise in the image has always been a difficult point in the low-light image enhancement method, because while enhancing the image brightness, the noise in the image will also be enhanced, resulting in overexposure, color distortion, etc. in the enhanced result.

In view of the limitations of existing technical means in processing low-light image noise, the present invention first adds noise in order from small to large intensity to convert the input low-light image into Gaussian noise, and then gradually samples from the Gaussian noise to finally obtain a highly realistic normal-light image. The present invention can effectively improve the brightness, contrast and clarity of low-light images, and effectively suppress image noise. The color and brightness of the enhanced image are closer to the normal-light image, and more image detail information can be retained.

Summary of the invention

In view of the problem of low-light image noise processing in the existing technical means, the present invention provides a conditional diffusion low-light image enhancement method based on dual-branch attention feature encoding, which adds noise in order from small to large intensity to convert the input low-light image into Gaussian noise, and then gradually samples from the Gaussian noise to finally obtain a high-fidelity normal-light image. It can effectively improve the brightness, contrast and clarity of low-light images, and effectively suppress image noise. The color and brightness of the enhanced image are closer to the normal-light image, and more image detail information can be retained.

The present invention is achieved through the following technical solutions:

A conditional diffusion low-light image enhancement method based on dual-branch attention feature encoding, comprising the following steps:

Step 1: construct a low-light image feature encoding model based on dual-branch attention, including a gradient recovery module and a brightness recovery module connected in parallel. The channel attention mechanism and the spatial attention mechanism are used to transfer the features of the low-light image to the gradient recovery module and the brightness recovery module. The gradient recovery module outputs the gradient map features of the low-light image, and the brightness recovery module inputs the brightness map features of the low-light image.

Step 2: construct a conditional diffusion model, splice the gradient map features, the brightness map features and the original low-light image and input them into the conditional diffusion model, add random noise to the spliced image to turn it into pure noise, and obtain a normal light image from the pure noise;

Step 3: Using structural similarity loss and color consistency loss as constraints, and optimizing model parameters by minimizing the loss function, the low-light image feature encoding model and the conditional diffusion model are trained until the network model converges, thereby obtaining a conditional diffusion low-light image enhancement model;

Step 4: Use the conditional diffusion low-light image enhancement model to enhance the low-light image.

Preferably, the gradient recovery module includes multiple neural network layers, the neural network layer includes a two-dimensional convolution layer, a normalization layer, a ReLU activation function layer and a residual connection layer. The input of the gradient recovery module is a multi-channel numerical matrix of the low-light image, and the output is a gradient feature map.

Preferably, the brightness restoration module includes multiple neural network layers, the neural network layer includes a two-dimensional convolution layer, a normalization layer, a ReLU activation function layer and a fully connected layer, and the brightness restoration module calculates the local and global brightness distribution characteristics of the image from the low-light image numerical matrix to obtain the brightness map characteristics.

Preferably, the conditional diffusion model includes a diffusion noise addition module and a diffusion denoising module;

The diffusion noise adding module is used to add noise to the input feature map by gradually adding random Gaussian noise until the obtained feature map becomes a pure noise image obeying Gaussian distribution;

The diffusion denoising module is used to denoise the pure noise image and construct a normal light image.

Preferably, the conditional diffusion model further includes a contrast correction module for using the color of the noise image as a feature vector to suppress the color cast phenomenon occurring in the diffusion denoising process.

Preferably, the method of adding random noise is as follows:

Where _x0 is the input image, _xt is a pure noise image, _αt is a hyperparameter, and _zt is Gaussian noise.

The desiccation method is as follows:

Among them, _xt is the pure noise image obtained after t-step noise addition, xt _-1 is the image before the t-th step noise addition, _αt is the hyperparameter, and _εθ is the noise predicted by the neural network.

Preferably, the loss function in step 3 is expressed as follows:

L _total = L _g + _Li + L _n

Among them, _Lg is the gradient loss, _Li is the brightness loss, and _Ln is the denoising loss.

Preferably, the expression of the gradient loss is as follows:

L _g ＝||G(I)-I _grad || ₁

Among them, G(I) is the gradient map output by the gradient recovery module, and I _grad is the gradient map of the normal light image;

The expression of the brightness loss is as follows:

_Li ＝||L(I) _-Ilight || ₁

Wherein, L(I) is the brightness image output by the brightness restoration module, and I _light is the brightness image of the normal light image;

The expression of the denoising loss is as follows:

L _n =||ε-ε _θ || ²

Among them, ε is the noise artificially added in the diffusion noise addition module, and ε _θ is the noise predicted in the diffusion denoising module.

A system for conditional diffusion low-light image enhancement method based on dual-branch attention feature encoding, comprising the following steps:

A feature encoding module is used to construct a feature encoding model for low-light images based on dual-branch attention, including a gradient recovery module and a brightness recovery module connected in parallel. The channel attention mechanism and the spatial attention mechanism are used to transfer the features of the low-light image to the gradient recovery module and the brightness recovery module. The gradient recovery module outputs the gradient map features of the low-light image, and the brightness recovery module inputs the brightness map features of the low-light image.

The noise module is used to build a conditional diffusion model, splice the gradient map features, the brightness map features and the original low-light image and input them into the conditional diffusion model, add random noise to the spliced image to turn it into pure noise, and obtain a normal light image from the pure noise;

The enhancement module is used to use structural similarity loss and color consistency loss as constraints, and optimize the model parameters by minimizing the loss function. The low-light image feature encoding model and the conditional diffusion model are trained until the network model converges to obtain the conditional diffusion low-light image enhancement model, and the conditional diffusion low-light image enhancement model is used to enhance the low-light image.

Compared with the prior art, the present invention has the following beneficial technical effects:

The present invention provides a conditional diffusion low-light image enhancement method based on dual-branch attention feature coding. In view of the limitations of the existing technical means in processing the low-light image noise problem, the present invention introduces feature coding technology and a conditional diffusion model. Since the feature coding technology can provide prior information for the training of the neural network, the normal light image enhanced by the present invention has better quality and clarity. At the same time, since the present invention gradually samples from Gaussian noise and finally obtains a normal light image, it has richer and more realistic detail information. In addition, since the image generated by the diffusion model often has the problem of insufficient contrast, this problem is solved by a contrast correction module, thereby further improving the quality of the generated image.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG1 is a flow chart of a conditional diffusion weak-light image enhancement method according to the present invention;

FIG2 is an overall framework diagram of a conditional diffusion network based on dual-branch attention feature encoding of the present invention;

FIG3 is a diagram of a dual-branch attention structure of the present invention;

FIG. 4 is a structural diagram of a contrast correction module of the present invention.

DETAILED DESCRIPTION

The present invention will be further described in detail below in conjunction with the accompanying drawings, which are intended to explain the present invention rather than to limit it.

Referring to FIG1 , a conditional diffusion low-light image enhancement method based on dual-branch attention feature encoding includes the following steps:

Step 1: construct a low-light image feature encoding model based on dual-branch attention to obtain the gradient map and brightness map of the low-light image;

The low-light image feature coding model includes a gradient recovery module and a brightness recovery module;

The gradient recovery module is used to recover the gradient map from the low-light image. The gradient map provides information about the change of local grayscale values in the image and can describe the change between edges, textures and regions in the image.

The brightness restoration module is used to restore a brightness map from a low-light image. The brightness map contains the brightness information of each pixel in the image, can describe the light and dark distribution in the image, and reflects the overall brightness characteristics of the image.

The low-light image feature encoding model adopts a dual-branch attention mechanism. By combining the channel attention mechanism and the spatial attention mechanism, it can effectively transfer the features in the low-light image to the gradient recovery module and the brightness recovery module. Specifically, the dual-branch attention mechanism combines the spatial attention weight with the channel attention weight, which means that when the model calculates the attention weight of each channel, it not only considers the relationship between the channels, but also the relationship between the spatial positions. Therefore, it can effectively handle detail blurring and structural loss in low-light images. This scheme enables the input of the gradient recovery module to contain more texture information, thereby improving the quality of the gradient map.

The low-light image feature encoding model is constructed using a deep neural network, and the gradient recovery module and the brightness recovery module are connected in parallel. The gradient recovery module recovers the gradient feature map from the low-light image, which is used to describe the first-order and second-order grayscale changes between the edges, textures and regions in the image. The gradient recovery module has no less than 4 neural network layers, in which the two-dimensional convolution layer, the normalization layer, and the ReLU activation function layer are combined in sequence, and may include one or more groups of the above combinations, each of which may include one residual connection layer, or multiple groups using a residual connection layer connected by the head and the tail. The input of the gradient recovery module is a multi-channel numerical matrix of the low-light image, and the output is a tensor map calculated by the above-mentioned deep neural network layer, that is, a gradient feature map. The number of rows and columns of the input and output numerical matrices of this module and the number of channels remain consistent.

The brightness restoration module calculates the local and global brightness distribution characteristics of the image from the numerical matrix of the low-light image. The brightness restoration module contains at least 5 neural network layers, among which the two-dimensional convolution layer, the normalization layer, and the ReLU activation function layer are combined in sequence, and can contain one or more groups of the above combinations. Each group can contain one residual connection layer, or multiple groups use a residual connection layer connected by the head and the tail, and finally a Sigmoid activation function layer quantizes the calculation results and outputs them. The number of rows and columns of the numerical matrix of the brightness restoration module remains the same, but the number of output channels becomes a single channel.

The attention module uses both the channel attention mechanism and the spatial attention mechanism, and uses the gradient recovery module and the brightness recovery module to perform local and global feature calculations on the input low-light image. For the two modules, the input features are multiplied by three trainable parameter matrices of the same resolution to obtain the key (k), query (q) and value (v). The dot product of the key (k) and the query (q) is calculated to obtain the similarity between the two, and then normalized using the softmax function to obtain the weight ɑ calculated by the attention mechanism. The attention weight ɑ is weighted averaged with the value (v) to obtain the attention calculation result of the module. Finally, the feature maps of the channel attention and spatial attention calculation results are added element by element to obtain the output features of the feature encoding module. The calculation process is shown in the following formula:

Attention(Q,K,V)=Softmax(QK ^T )V

Step 2: construct a conditional diffusion model, including a diffusion noise addition module, a diffusion denoising module, and a contrast correction module;

The gradient map and brightness map obtained in step 1 are spliced together with the original low-light image and input into the conditional diffusion model for enhancement and denoising. The model first gradually adds random noise to the input image until it becomes pure noise, then learns the diffusion process to construct the required normal light image from pure noise. By restoring the normal light image from pure noise, the noise problem that is difficult to handle by traditional low-light image enhancement methods can be effectively solved, further improving the image quality. However, the diffusion model usually results in insufficient contrast in the generated image. Therefore, the contrast correction module is used to further improve the image quality while effectively removing noise. The specific method is as follows:

The conditional diffusion model is constructed using a deep neural network, and is composed of a diffusion noise addition module and a diffusion denoising module connected in parallel. In the diffusion noise addition module, random Gaussian noise is gradually added to the input feature map for noise addition, and the number of noise addition steps is T, T is not less than 2 steps, until the obtained feature map becomes a pure noise image that obeys the Gaussian distribution. The output feature map of the gradient recovery module and the brightness recovery module are spliced with the pure noise image along the channel dimension to obtain the input of the diffusion denoising module. The diffusion denoising module adopts a U-shaped network structure, which includes no less than 3 neural network layers, which must include no less than 1 downsampling and 1 upsampling process to obtain feature calculation results of different scales. Each two-dimensional convolutional layer of the U-shaped network is combined with normalization, ReLU activation function, and linear layer in sequence. The diffusion noise addition module can obtain a pure noise image _xt through the number of steps t and the input image x0 _. The noise addition process is shown as follows:

Among them, _x0 is the input image, _xt is the pure noise image, _αt is the hyperparameter, and _zt is Gaussian noise.

The diffusion denoising module can obtain the image x _t-1 before the t-th step of denoising through the pure noise image x _t and the noise ε _θ predicted by the neural network, and finally obtain the normal light image x ₀ after t iterations. The denoising process is shown in the following formula:

Where _xt is the pure noise image obtained after t-step noise addition, xt _-1 is the image before the t-th step noise addition, _αt is the hyperparameter, and _εθ is the noise predicted by the neural network.

Secondly, a contrast correction module is introduced into the conditional diffusion model, which reduces the color cast in the diffusion denoising process while retaining the generated edges and textures, thus solving the problem of insufficient contrast in images generated by the diffusion model.

The contrast correction module includes at least 4 neural network layers, wherein the two-dimensional convolution layer and the ReLU activation function layer of the module trunk are sequentially combined, and may include one or more groups of the above combinations. The two-dimensional convolution layer, the ReLU activation function layer, and the global average pooling layer of the module branch are sequentially combined and include only one group of the above combinations. The contrast correction module adds the color information of the noise image _xt obtained by t-step denoising to the network as a feature vector through the above neural network layer, so that the color of the image _xt-1 after one-step denoising will not deviate too much from _xt , thereby suppressing the color cast phenomenon that occurs in the diffusion denoising process.

Step 3: Use the image dataset to train the low-light image feature encoding model and the conditional diffusion model, use the structural similarity loss and color consistency loss as the constraints for tuning, and optimize the model parameters by minimizing the loss function until the network model converges. Use the test set data to evaluate the performance of the model, save the parameters of the neural network after training, and obtain the conditional diffusion low-light image enhancement model.

The model parameters are optimized by minimizing the loss function terms, which are composed of gradient loss, brightness loss and denoising loss. The gradient loss is the L1 norm of the gradient feature map output by the gradient recovery module and the gradient feature map of the normal light image; the brightness loss is the L1 norm of the brightness feature map output by the brightness recovery module and the brightness feature map of the normal light image; the denoising loss is the L2 norm of the noise predicted in the diffusion denoising module and the noise added in the diffusion denoising module.

The expression of the loss function is as follows:

L _total = L _g + _Li + L _n (1)

Wherein, _Lg is the gradient loss, _Li is the brightness loss, and _Ln is the denoising loss, as shown in formula (2), formula (3), and formula (4), respectively.

L _g ＝||G(I)-I _grad || ₁ (2)

Among them, G(I) is the gradient map output by the gradient recovery module, and I _grad is the gradient map of the normal light image.

L _i =||L(I)-I _light || ₁ (3)

Among them, L(I) is the brightness image output by the brightness restoration module, and I _light is the brightness image of the normal light image.

L _n =||ε-ε _θ || ² (4)

Among them, ε is the noise artificially added in the diffusion noise addition module, and ε _θ is the noise predicted in the diffusion denoising module.

After the model training is completed, the low-light images in the test set are input to obtain the enhanced images. The enhanced images and the normal-light images in the test set are compared with each other using two image quality evaluation indicators, peak signal-to-noise ratio (PSNR) and structural similarity (SSIM), to measure the model performance.

Step 4: Use the conditional diffusion weak-light image enhancement model to enhance the weak-light image. The weak-light image is input into the conditional diffusion weak-light image enhancement model to generate an enhanced image, thereby enhancing the weak-light image.

The present invention first obtains a gradient map and a brightness map through a low-light image feature encoding model based on a dual-branch attention mechanism, then obtains a normal-light image from Gaussian noise through a conditional diffusion model, and further improves the image quality through a contrast correction module.

Example 1

Referring to FIG. 2-4, a conditional diffusion low-light image enhancement method based on dual-branch attention feature encoding, which uses deep learning technology to enhance low-light images into normal-light images, includes the following steps:

Step 1: Build a low-light image feature encoding model based on dual-branch attention, including a gradient recovery module and a brightness recovery module.

S1.1, Gradient recovery module: This module consists of multiple sets of two-dimensional convolutional layers, ReLU activation functions, normalization and residual blocks. Assume that the input of this module is a 3×256×256 low-light image, and its output is a 3×256×256 gradient feature map.

S1.2, Brightness restoration module: This module consists of multiple sets of two-dimensional convolutional layers, ReLU activation function, Sigmoid activation function, normalization and residual blocks. Assume that the input of this module is a 3×256×256 low-light image, and its output is a 1×256×256 grayscale image.

The process of building a low-light image feature encoding model based on dual-branch attention is as follows:

The low-light image is used as the original input to the gradient recovery module and the brightness recovery module respectively. The first layer of the gradient recovery module contains a two-dimensional convolution, ReLU activation function, dual-branch attention and residual block, the convolution kernel size is 3×3, the output channel is 6, and the convolution step is 2. The second layer contains a two-dimensional convolution, ReLU activation function and residual block, the convolution kernel size is 3×3, the output channel is 12, and the convolution step is 2. The third layer contains a two-dimensional convolution, ReLU activation function and residual block, the convolution kernel size is 1×1, the output channel is 6, and the convolution step is 1. The fourth layer contains a two-dimensional convolution, ReLU activation function and residual block, the convolution kernel size is 1×1, the output channel is 3, and the convolution step is 1.

The first layer of the brightness restoration module contains a 2D convolution, ReLU activation function and normalization. The convolution kernel size is 3×3, the output channel is 3, and the convolution step is 1. The second layer contains two 2D convolutions, ReLU activation function, dual-branch attention and normalization. The first 2D convolution has a convolution kernel size of 3×3, an output channel of 6, and a convolution step of 2. The second 2D convolution has a convolution kernel size of 3×3, an output channel of 6, and a convolution step of 1. The third layer has a similar structure to the second layer, containing two 2D convolutions, ReLU activation function, and normalization. The first 2D convolution has a convolution kernel size of 3×3, an output channel of 12, and a convolution step of 2. The second 2D convolution has a convolution kernel size of 3×3, an output channel of 12, and a convolution step of 1. The fourth layer contains two 2D convolutions, ReLU activation function and normalization. The first 2D convolution has a kernel size of 1×1, an output channel of 6, and a convolution step of 1. The second 2D convolution has a kernel size of 3×3, an output channel of 6, and a convolution step of 1. The fifth layer contains two 2D convolutions, a Sigmoid activation function, and normalization. The first 2D convolution has a kernel size of 1×1, an output channel of 3, and a convolution step of 1. The second 2D convolution has a kernel size of 3×3, an output channel of 1, and a convolution step of 1.

Step 2: Build a conditional diffusion model, including a diffusion noise addition module, a diffusion denoising module, and a contrast correction module.

S2.1, Diffusion Noise Adding Module: This module first adds random noise to the normal light image of size 3×256×256 in 1000 steps until it becomes a pure noise image. Then the 3×256×256 gradient feature map and 1×256×256 grayscale image output by step 1 are concatenated with the pure noise image along the channel, and finally a noise image of size 7×256×256 is output.

S2.2, Diffusion denoising module: This module consists of multiple sets of two-dimensional convolutional layers, ReLU activation functions, normalization, linear layers, and contrast correction modules. It takes the 7×256×256 noise map output by the denoising module as input, learns the diffusion denoising process, and suppresses the color cast that occurs during the denoising process through the contrast correction module, constructs the distribution of the normal light image from the pure noise, and finally outputs a 3×256×256 normal light image.

The process of building the conditional diffusion model is as follows:

Random noise is added to the normal light image of size 3×256×256 in 1000 steps until it becomes a pure noise image. Then the gradient feature map of size 3×256×256 and the grayscale image of size 1×256×256 output by step 1 are concatenated with the pure noise image along the channel to form a noise image of size 7×256×256. The 7×256×256 noise image is input into the diffusion denoising module, and finally a normal light image of size 3×256×256 is obtained. The diffusion denoising module first passes the 7×256×256 noise image through a two-dimensional convolution layer and outputs a 32×256×256 feature map. This feature map is upsampled five times and downsampled five times respectively. The first downsampling layer contains a two-dimensional convolution layer, ReLU activation function and normalization. The convolution kernel size is 3×3, the output channel is 32, and the convolution step size is 2. The next four downsampling layers are similar in structure to the first downsampling layer, but the 2D convolution output channels are 64, 128, 256, and 1024, respectively. After four downsamplings, the feature map size becomes 1024×16×16. The first upsampling layer contains a 2D convolution layer, ReLU activation function, normalization, and upsampling. The convolution kernel size is 3×3, the output channel is 256, and the convolution step is 1. The next four upsampling layers are similar in structure to the first upsampling layer, but the 2D convolution output channels are 128, 64, 32, and 3, respectively. After five upsamplings, the network generates a predicted noise map of size 3×256×256. The result of subtracting the predicted noise from the noise map is input into the contrast correction module, and finally the normal light image of size 3×256×256 is output.

Step 3: Use no less than 500 sets of paired image data of low-light and normal-light to train the low-light image feature encoding model and the conditional diffusion model, and optimize the model parameters by minimizing the loss function until the network model converges. Save the parameters of the trained neural network to obtain the conditional diffusion low-light image enhancement model, and finally use the test set data to evaluate the performance of the model.

Step 4: Input the low-light image into the conditional diffusion low-light image enhancement model, and the conditional diffusion low-light image enhancement model outputs the enhanced low-light image.

Example 2

A conditional diffusion low-light image enhancement system based on dual-branch attention feature encoding, comprising the following steps:

A feature encoding module is used to construct a feature encoding model for low-light images based on dual-branch attention, including a gradient recovery module and a brightness recovery module connected in parallel. The channel attention mechanism and the spatial attention mechanism are used to transfer the features of the low-light image to the gradient recovery module and the brightness recovery module. The gradient recovery module outputs the gradient map features of the low-light image, and the brightness recovery module inputs the brightness map features of the low-light image.

The noise module is used to build a conditional diffusion model, splice the gradient map features, the brightness map features and the original low-light image and input them into the conditional diffusion model, add random noise to the spliced image to turn it into pure noise, and obtain a normal light image from the pure noise;

The enhancement module is used to use structural similarity loss and color consistency loss as constraints, and optimize the model parameters by minimizing the loss function. The low-light image feature encoding model and the conditional diffusion model are trained until the network model converges to obtain the conditional diffusion low-light image enhancement model, and the conditional diffusion low-light image enhancement model is used to enhance the low-light image.

The present invention discloses a conditional diffusion low-light image enhancement method based on feature coding. The method first extracts a gradient map and a brightness map from a low-light image, then gradually samples from Gaussian noise, and solves the problem of insufficient contrast of the result generated by the diffusion model through a contrast correction module, and finally obtains a normal light image with high authenticity. The method is characterized in that the gradient map and the brightness map are obtained by a low-light image feature coding model based on dual-branch attention, and are used as prior information to guide the training of the diffusion model.

In addition, a conditional diffusion model is designed to construct the required normal light image from pure noise, which can better deal with the noise problem in low light images. The enhanced image can well simulate the distribution of natural images and has richer and more realistic detail information. The present invention also adds a contrast correction module, which can suppress the color cast that occurs during the diffusion denoising process and further improve the quality of the generated image.

The above contents are only for explaining the technical idea of the present invention and cannot be used to limit the protection scope of the present invention. Any changes made on the basis of the technical solution in accordance with the technical idea proposed by the present invention shall fall within the protection scope of the claims of the present invention.

Claims (9)

1. The conditional diffusion weak light image enhancement method based on the double-branch attention feature coding is characterized by comprising the following steps of:

Step 1, constructing a weak light image feature coding model based on double-branch attention, wherein the model comprises a gradient recovery module and a brightness recovery module which are connected in parallel, the features of the weak light image are transferred to the gradient recovery module and the brightness recovery module by adopting a channel attention mechanism and a space attention mechanism, the gradient recovery module outputs the gradient map features of the weak light image, and the brightness recovery module inputs the brightness map features of the weak light image;

Step 2, constructing a conditional diffusion model, splicing the gradient map features, the brightness map features and the original weak light images, inputting the spliced images into the conditional diffusion model, adding random noise into the spliced images to change the spliced images into pure noise, and acquiring normal light images from the pure noise;

Step 3, adopting structural similarity loss and color consistency loss as constraint conditions, optimizing model parameters by minimizing a loss function, and training a weak light image feature coding model and a conditional diffusion model until a network model converges to obtain a conditional diffusion weak light image enhancement model;

And 4, enhancing the weak light image by adopting a conditional diffusion weak light image enhancement model.

2. The method for enhancing a conditional diffusion weak light image based on dual-branch attention feature coding according to claim 1, wherein the gradient recovery module comprises a plurality of neural network layers, the neural network layers comprise a two-dimensional convolution layer, a normalization layer, a ReLU activation function layer and a residual error connection layer, the input of the gradient recovery module is a multi-channel numerical matrix of the weak light image, and the output of the gradient recovery module is a gradient feature map.

3. The method for enhancing a conditional diffuse low-light image based on dual-branch attention feature coding according to claim 1, wherein the brightness restoration module comprises a plurality of neural network layers, the neural network layers comprise a two-dimensional convolution layer, a normalization layer, a ReLU activation function layer and a full connection layer, and the brightness restoration module calculates local and global brightness distribution features of the image from a low-light image numerical matrix to obtain brightness map features.

4. The conditional diffusion weak light image enhancement method based on double-branch attention feature coding according to claim 1, wherein the conditional diffusion model comprises a diffusion noise adding module and a diffusion noise removing module;

the diffusion noise adding module is used for adding noise to the input feature map in a mode of gradually adding random Gaussian noise until the obtained feature map becomes a pure noise image obeying Gaussian distribution;

the diffusion denoising module is used for denoising the pure noise image and constructing a normal light image.

5. The method for enhancing a conditional diffusion weak light image based on dual branch attention feature coding as claimed in claim 4, wherein the conditional diffusion model further comprises a contrast correction module for suppressing color cast phenomenon generated in the diffusion denoising process by using the color of the noise figure as a feature vector.

6. The method for enhancing a conditional diffuse low-light image based on dual branch attention feature coding of claim 4, wherein the method for adding random noise is as follows:

Wherein x ₀ is an input image, x _t is a pure noise image, α _t is a super parameter, and z _t is gaussian noise;

the drying method comprises the following steps:

Wherein x _t is a pure noise graph obtained by t-step noise addition, x _t-1 is an image before t-step noise addition, alpha _t is a super parameter, and epsilon _θ is noise predicted by a neural network.

7. The method for enhancing a conditional diffuse low-light image based on dual branch attention feature coding of claim 4, wherein the expression of the loss function in step 3 is as follows:

L_total＝L_g+L_i+L_n

Where L _g is gradient loss, L _i is brightness loss, and L _n is denoising loss.

8. The conditional diffuse low light image enhancement method based on dual branch attention feature encoding of claim 4, wherein the gradient loss is expressed as follows:

L_g＝||G(I)-I_grad||₁

Wherein G (I) is a gradient map output by the gradient recovery module, and I _grad is a gradient map of a normal light image;

The expression of the brightness loss is as follows:

L_i＝||L(I)-I_light||₁

wherein L (I) is a luminance graph output by the luminance recovery module, and I _light is a luminance graph of a normal light image;

the expression of the denoising penalty is as follows:

L_n＝||ε-ε_θ||²

wherein epsilon is artificially added noise in the diffusion noise adding module, and epsilon _θ is predicted noise in the diffusion noise removing module.

9. A system for performing the conditional diffuse low light image enhancement method based on dual branch attention feature encoding of any one of claims 1-8, comprising the steps of:

the feature coding module is used for constructing a weak light image feature coding model based on double-branch attention and comprises a gradient recovery module and a brightness recovery module which are connected in parallel, the features of the weak light image are transferred to the gradient recovery module and the brightness recovery module by adopting a channel attention mechanism and a space attention mechanism, the gradient recovery module outputs the gradient map features of the weak light image, and the brightness recovery module inputs the brightness map features of the weak light image;

the noise module is used for constructing a conditional diffusion model, splicing the gradient map features, the brightness map features and the original weak light images, inputting the gradient map features, the brightness map features and the original weak light images into the conditional diffusion model, adding random noise into the spliced images to change the random noise into pure noise, and acquiring normal light images from the pure noise;

The enhancement module is used for optimizing model parameters by adopting structural similarity loss and color consistency loss as constraint conditions and minimizing a loss function, training the weak light image feature coding model and the conditional diffusion model until the network model converges to obtain a conditional diffusion weak light image enhancement model, and enhancing the weak light image by adopting the conditional diffusion weak light image enhancement model.