[go: up one dir, main page]

CN111667445B - Image compressed sensing reconstruction method based on Attention multi-feature fusion - Google Patents

Image compressed sensing reconstruction method based on Attention multi-feature fusion Download PDF

Info

Publication number
CN111667445B
CN111667445B CN202010479734.8A CN202010479734A CN111667445B CN 111667445 B CN111667445 B CN 111667445B CN 202010479734 A CN202010479734 A CN 202010479734A CN 111667445 B CN111667445 B CN 111667445B
Authority
CN
China
Prior art keywords
convolution
image
channel
size
information
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202010479734.8A
Other languages
Chinese (zh)
Other versions
CN111667445A (en
Inventor
曾春艳
王正辉
武明虎
孔帅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Hubei University of Technology
Original Assignee
Hubei University of Technology
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Hubei University of Technology filed Critical Hubei University of Technology
Priority to CN202010479734.8A priority Critical patent/CN111667445B/en
Publication of CN111667445A publication Critical patent/CN111667445A/en
Application granted granted Critical
Publication of CN111667445B publication Critical patent/CN111667445B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/50—Image enhancement or restoration using two or more images, e.g. averaging or subtraction
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T3/00—Geometric image transformations in the plane of the image
    • G06T3/40—Scaling of whole images or parts thereof, e.g. expanding or contracting
    • G06T3/4046—Scaling of whole images or parts thereof, e.g. expanding or contracting using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T5/00—Image enhancement or restoration
    • G06T5/90—Dynamic range modification of images or parts thereof
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20212—Image combination
    • G06T2207/20221—Image fusion; Image merging

Landscapes

  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Theoretical Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Molecular Biology (AREA)
  • Computing Systems (AREA)
  • General Engineering & Computer Science (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Mathematical Physics (AREA)
  • Software Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Image Analysis (AREA)

Abstract

本发明提出了一种基于Attention的多特征融合的图像压缩感知重建方法，首先利用全卷积神经网络进行采样，然后使用反卷积网络进行初始重建。接下来利用不同扩张速率的扩张卷积得到不同感受野的扩张卷积核捕获图像中不同尺度的信息，得到多个尺度特征信息；在不同扩张速率的卷积通道，我们在每个通道加入了残差块，构成残差网络，通过残差网里多个跳连接来绕过大量的低频信息从而使网络专注于学习高频信息。在得到多个尺度特征信息后，使用Attention机制对输出的不同特征进行加权，更好的利用特征中更多有用信息，然后将多个特征信息进行融合完成图像的深度重建。本发明能有效提高采样效率，并使图像重建的质量得到大幅提升。

Figure 202010479734

The invention proposes a multi-feature fusion image compressive sensing reconstruction method based on Attention, which firstly uses a full convolutional neural network for sampling, and then uses a deconvolutional network for initial reconstruction. Next, dilated convolutions with different dilation rates are used to obtain dilated convolution kernels of different receptive fields to capture information of different scales in the image, and obtain multiple scale feature information; in the convolution channels of different dilation rates, we added a The residual block constitutes a residual network, which bypasses a large amount of low-frequency information through multiple jump connections in the residual network so that the network can focus on learning high-frequency information. After obtaining multiple scale feature information, the attention mechanism is used to weight the different features of the output to better utilize more useful information in the features, and then the multiple feature information is fused to complete the depth reconstruction of the image. The invention can effectively improve the sampling efficiency and greatly improve the quality of image reconstruction.

Figure 202010479734

Description

Image compressed sensing reconstruction method based on Attention multi-feature fusion
Technical Field
The invention relates to the field of image processing, in particular to an Attention-based multi-feature fusion image compressed sensing reconstruction method.
Background
Compressed Sensing (CS) can accurately recover the original signal from measurements much less than nyquist sampling, and thus is widely applied to measurements of data. Most of the traditional compressed sensing measurement methods adopt random measurement matrixes, and a large amount of memory is occupied when high-dimensional data is measured; the traditional reconstruction method has high calculation complexity and is difficult to realize real-time. In recent years, a deep learning-based method is successfully applied to many fields, and the application of a deep learning technology to the field of compressed sensing can greatly reduce the computational complexity and improve the reconstruction performance. A stack de-noising automatic encoder (SDA) is used as an unsupervised feature learner to obtain the statistical correlation among different elements of a signal and improve the signal reconstruction performance; the DeepInverse network learns a representation of the signal using a deep convolutional neural network, and performs the inverse transformation from the measurement vector to the original signal similar to a greedy algorithm or a convex relaxation algorithm; ReconNet is also a non-iterative reconstruction network that uses a convolutional neural network to learn the mapping from CS measurements to the original image; a DR2-Net network is proposed on the basis of Reconnet, the network adopts a depth residual error network to reconstruct an original image from a measured value, the network mainly comprises a linear mapping network and a residual error learning network, the fully-connected linear mapping network reconstructs a primary image with higher quality, and the residual error network composed of a plurality of residual error learning blocks further improves the quality of the reconstructed image. Although the deep learning based method enables the CS to achieve a large improvement in image reconstruction performance, most of the work is to reconstruct an image with a single-channel convolutional neural network, and the weight of each feature spectrum is the same. In the field of neuroscience, the visual receptive field is considered to be automatically adjustable by external stimulation, so that the size of a convolution kernel of a convolution neural network needs to be adaptively adjustable, and different characteristic spectrums need to be endowed with different weights according to the importance degree.
Therefore, based on general observation of the human visual system, the human visual nervous system is simulated, expansion convolution kernels of different receptive fields are obtained by utilizing expansion convolution of different rates, information of different scales in an image is obtained, different output features are weighted through an Attention mechanism, and then a plurality of feature information are fused to complete depth reconstruction of the image. A residual block is used in each channel to form a residual network, so that the depth of the network is improved, and the reconstruction performance is further improved.
Disclosure of Invention
The technical problem of the invention is mainly solved by the following technical scheme:
an image compressed sensing reconstruction method based on Attention multi-feature fusion is characterized by comprising the following steps:
step 1, collecting image data, measuring the image data based on a neural network and outputting measured data;
step 2, outputting an initial reconstruction image after performing initial reconstruction on the measurement signal by adopting a deconvolution neural network;
and 3, processing the reconstructed image in the step 2 by using a three-channel network, and outputting multi-scale characteristic information.
And 4, performing weighting processing on the output different characteristics by an Attention mechanism to obtain final reconstructed image data.
In the foregoing image compressed sensing reconstruction method based on Attention multi-feature fusion, the step 1 specifically includes:
and 2.1, selecting an image with the size of n multiplied by n, and converting the image into a gray-scale image.
And 2.2, measuring by using a full convolution neural network. Simultaneously measuring the input data using m BxBx 1 convolution kernels, wherein
Figure BDA0002516905260000021
Figure BDA0002516905260000022
To measure the rate, B is the size of the convolution kernel of the sample layer, and B is 32, it should be noted that in the convolution operation of this step, there is no offset, and there is no Pad zero padding, and the convolution step size is also set to B.
Step 2.3, obtaining the product after the measurement of the convolution layer
Figure BDA0002516905260000023
The measured data of (2).
In the above image compressed sensing reconstruction method based on Attention multi-feature fusion, step 2.2 specifically includes:
the measurement by using the convolutional neural network is a mode simulating the traditional measurement: y isi＝ΦB×xiThe measurement matrix phi can beBEach row of (a) is considered a convolution kernel. Where the matrix phi is measuredBIn existence of
Figure BDA0002516905260000024
Line (A)
Figure BDA0002516905260000025
For measurement rate) to obtain m measurement points. In the experiment we apply convolution kernelsIs set to be B multiplied by B, the step length is also set to be B, which is equivalent to non-overlapping sampling; each convolution kernel thus outputs a measurement value. When measuring the rate
Figure BDA0002516905260000026
If B is set to 32, then m is the number of convolution kernels in the measurement layer should be 102.
In the foregoing image compressed sensing reconstruction method based on Attention multi-feature fusion, the step 2 specifically includes:
and 3.1, performing initial reconstruction on the sampling signal by using a deconvolution neural network. At the deconvolution level, the size of the convolution kernel is set to 32 × 32, and the step size is also 32.
And 3.2, obtaining an n multiplied by n initial reconstruction image after the convolution operation of the step 3.1.
In step 3.1, the image compressed sensing reconstruction method based on Attention multi-feature fusion specifically includes:
for the deconvolution process, a convolution kernel after the convolution process is reversed is adopted, and the convolution kernel is equivalent to the inverse process of the convolution. After convolution measurement, the resolution of the image may become low and part of the information may be missing. The deconvolution is intended to restore the original image as much as possible, but complete restoration is impossible, and it is important to obtain a size matching the original image. The output tensor (image) size due to deconvolution is: 0 ═ N-1 × S + K-2P, where N is the input image size, O is the output image size, S is the step size, K is the size of the convolution kernel, and P is edge filling. After the measurement has been carried out, the measurement is carried out,
Figure BDA0002516905260000031
there is no Pad operation. In order to obtain the same size as the original, S is 32, K is 32, and P is 0 since there is no Pad operation. Thus, after deconvolution, an initial reconstructed image with the same size as the original image is obtained.
In the image compressed sensing reconstruction method based on Attention multi-feature fusion, in step 3, expansion convolutions with different expansion rates are used for forming three parallel channels, a residual block is added into each channel to form a residual network, and a plurality of feature information with different scales are obtained; the three parallel channels specifically include three channels, namely a left channel, a middle channel and a right channel, each channel has four residual blocks, each residual block has three layers, but the expanding convolution rate of each channel is different, specifically:
the leftmost lane has a dilation rate of 1, which is a common 3 x 3 convolution kernel. The first layer of each residual block is 64 convolution kernels of size 3 × 3; the second layer is 32 convolution kernels with the size of 3 x 3; the third layer is 1 convolution kernel with the size of 3 multiplied by 3; the step size of all convolutional layers of the part is 1 and the activation function is ReLU. In order to keep the dimension of the output data the same as the dimension of the input data, Pad operation is performed and there is no Pool operation, and then Pad is 1.
The middle channel is again four residual blocks, each having three convolutional layers. The expansion rate of each convolution kernel is 2, and 5 × 5 convolution kernels are obtained by performing expansion on 3 × 3 convolution kernels. The first layer is 64 convolution kernels; the second layer is 32 convolution kernels; the third layer is a convolution kernel of 1; the step size of all convolutional layers of the part is 1 and the activation function is ReLU. In order to keep the dimension of the output data the same as the dimension of the input data, Pad operation is performed and there is no Pool operation, and then Pad is 2.
The number of convolution layers of the right channel and other channels is the same as that of convolution kernels, but the rate of expanding convolution is 3, namely 7 multiplied by 7 convolution kernels are obtained. Therefore, to keep the output data dimensions the same as the input data dimensions, Pad is now complemented by 3. After each convolution, the ReLU is used as the activation function.
In the above image compressed sensing reconstruction method based on Attention multi-feature fusion, step 4 specifically includes:
after multi-scale feature information is extracted from multiple channels, feature information of each channel is directly fused, and the information is treated equally among the channels, which hinders the representation capability of CNN. In fact, the information of each channel has different proportions to play a role in image reconstruction. However, the per-channel scaling is not considered a setting, and is learned through network training. Extracting features from multiple convolution channels and outputting X ═ X1,…,xc,…,xCAmong them, there are C feature maps with size n × n. Obtaining channel statistics by a global pooling operation
Figure BDA0002516905260000041
The c is shown as
Figure BDA0002516905260000042
We then also introduce a gating control mechanism to learn the nonlinear interaction between channels. Here we choose to use a simple sigmoid function as the door mechanism: s ═ f (W)AR(WBz)) where f (-) and R are sigmoid and ReLU activation functions, WAAnd WBAre the convolved weights. Finally, we obtain the final channel statistics s for rescaling the input X, finally
Figure BDA0002516905260000043
S is the scale factor of the channel, i.e. the characteristic information between each channel is adaptively rescaled by the Attention mechanism.
Therefore, the invention has the following advantages: 1. the convolution measurement is used for replacing the traditional fast measurement mode, and the blocking effect generated after the image reconstruction is eliminated. 2. Extracting multi-scale information in the image through a multi-channel network of expansion convolution with different expansion rates; a residual block is added into the channel to form a residual network, and a large amount of low-frequency information is bypassed through a plurality of hop connections in the residual network, so that the network is focused on learning high-frequency information; and the image reconstruction quality is improved. 3. Adding an Attention mechanism to enable the characteristic information among the channels to be adaptively rescaled to enhance the discrimination capability of the network; then, the multiscale information is fused through Concat, so that the information in the extracted image is more comprehensive.
Drawings
FIG. 1 is a schematic flow diagram of the process of the present invention.
Detailed Description
The technical scheme of the invention is further specifically described by the following embodiments and the accompanying drawings.
Example (b):
the method mainly comprises four steps, wherein the first step is a sampling stage of an image, and input data is measured by utilizing a full convolution neural network; the second step is an initial reconstruction stage, and a deconvolution network is adopted to perform initial reconstruction on the measured data; acquiring multi-scale characteristic information, and acquiring information of different scales in an expansion convolution kernel capture image of different receptive fields by using expansion convolutions with different expansion rates to acquire a plurality of scale characteristics; and the fourth step is to weight the output different characteristics through an Attention mechanism, and then fuse the characteristic information of a plurality of scales to complete the final image reconstruction.
Step 1: and (5) an image measuring stage.
1-1) selecting an image with the size of n multiplied by n, and converting the image into a gray scale image.
1-2) measurements were performed using a full convolution neural network. The measurement by using the convolutional neural network is a mode simulating the traditional measurement: y isi＝ΦB×xiThe measurement matrix phi can beBEach row of (a) is considered a convolution kernel. Where the matrix phi is measuredBIn existence of
Figure BDA0002516905260000051
Line (A)
Figure BDA0002516905260000052
For measurement rate) to obtain m measurement points. In the experiment, the size of a convolution kernel is set to be B multiplied by B, the step length is also set to be B, and the method is equivalent to non-overlapping sampling; each convolution kernel thus outputs a measurement value. When measuring the rate
Figure BDA0002516905260000053
If B is set to 32, then m is the number of convolution kernels in the measurement layer should be 102. Note that in the convolution operation of this step, there is no offset and no Pad zero padding, and the convolution step is also set to B.
1-3) obtaining the product after the measurement of the convolution layer
Figure BDA0002516905260000054
The measured data of (2).
Step 2: an initial reconstruction phase.
2-1) carrying out initial reconstruction on the sampling signal by using a deconvolution neural network. For the deconvolution process, a convolution kernel after the convolution process is reversed is adopted, and the convolution kernel is equivalent to the inverse process of the convolution. After convolution measurement, the resolution of the image may become low and part of the information may be missing. The deconvolution is intended to restore the original image as much as possible, but complete restoration is impossible, and it is important to obtain a size matching the original image. During measurement, the size of a convolution kernel is set to be B, and the step length is also B; also when the size and step size of the deconvolution kernel are both set to B, this results in an initial reconstruction that is the same size as the original image. A
2-2) obtaining an n multiplied by n initial reconstruction image after the convolution operation of 2-1).
And step 3: obtaining multi-scale characteristic information, forming three parallel channels by using expansion convolutions with different expansion rates, adding a residual block into each channel to form a residual network to obtain a plurality of different-scale characteristic information. Each channel in the three parallel channel networks has four residual blocks, each residual block has three layers, but the rate of the expanding convolution is different for each channel.
The leftmost lane has a dilation rate of 1, which is a common 3 x 3 convolution kernel. The first layer of each residual block is 64 convolution kernels of size 3 × 3; the second layer is 32 convolution kernels with the size of 3 x 3; the third layer is 1 convolution kernel with the size of 3 multiplied by 3; the step size of all convolutional layers of the part is 1 and the activation function is ReLU. In order to keep the dimension of the output data the same as the dimension of the input data, Pad operation is performed and there is no Pool operation, and then Pad is 1.
The middle channel is again four residual blocks, each having three convolutional layers. The expansion rate of each convolution kernel is 2, and 5 × 5 convolution kernels are obtained by performing expansion on 3 × 3 convolution kernels. The first layer is 64 convolution kernels; the second layer is 32 convolution kernels; the third layer is a convolution kernel of 1; the step size of all convolutional layers of the part is 1 and the activation function is ReLU. In order to keep the dimension of the output data the same as the dimension of the input data, Pad operation is performed and there is no Pool operation, and then Pad is 2.
The number of convolution layers of the right channel and other channels is the same as that of convolution kernels, but the rate of expanding convolution is 3, namely 7 multiplied by 7 convolution kernels are obtained. Therefore, to keep the output data dimensions the same as the input data dimensions, Pad is now complemented by 3. After each convolution, the ReLU is used as the activation function.
After the three parallel channel networks with different rates and convolutions are expanded, a plurality of feature information with different scales are obtained, a residual block is added into each channel to form a residual network, and a large amount of low-frequency information is bypassed through a plurality of hop connections in the residual network, so that the network is focused on learning high-frequency information.
And 4, step 4: after obtaining a plurality of feature information, weighting and Concat different output features through an Attention mechanism to fuse the multi-scale information. After multi-scale feature information is extracted from multiple channels, feature information of each channel is directly fused, and the information is treated equally among the channels, which hinders the representation capability of CNN. In fact, the information of each channel has different proportions to play a role in image reconstruction. The per-channel scaling is not considered a setting, but rather learned through network training. Through network training, the proportion of the channel information which is beneficial to image reconstruction is amplified, the proportion of the channel information which is not beneficial to image reconstruction is reduced, and the weight between the channel information and the channel information is automatically learned. It can be seen from the figure that the color change of each channel after the Attention mechanism, that is, the feature information between each channel is adaptively rescaled through the Attention mechanism, further enhancing the discrimination capability of the network. More useful information in the features is better utilized, then a plurality of pieces of feature information with different scales are fused, and then 1 convolution kernel with the size of 3 multiplied by 3 is used to obtain a final reconstructed image.
The specific embodiments described herein are merely illustrative of the spirit of the invention. Various modifications or additions may be made to the described embodiments or alternatives may be employed by those skilled in the art without departing from the spirit or ambit of the invention as defined in the appended claims.

Claims (5)

1. An image compressed sensing reconstruction method based on Attention multi-feature fusion is characterized by comprising the following steps:
step 1, collecting image data, measuring the image data based on a neural network and outputting measured data;
step 2, outputting an initial reconstruction image after performing initial reconstruction on the measurement signal by adopting a deconvolution neural network;
step 3, processing the initial reconstruction image in the step 2 by using a three-channel network and outputting multi-scale characteristic information;
step 4, weighting different output characteristics by an Attention mechanism to obtain final reconstructed image data;
the step 2 specifically comprises:
step 3.1, performing initial reconstruction on the measurement signal by using a deconvolution neural network;
step 3.2, obtaining an n multiplied by n initial reconstruction image after the convolution operation of the step 3.1;
in step 3.1, the method specifically comprises the following steps: for the deconvolution process, a convolution kernel after the convolution process is transferred is adopted, which is equivalent to the inverse process of the convolution; after convolution measurement, the resolution of the image becomes low, and partial information is lost; the deconvolution is to restore the original image, and it is important to obtain the same size as the original image; the output tensor size due to deconvolution is: o ═ N-1 × S + K-2P, where N is the input image size, O is the output image size, S is the step size, K is the size of the convolution kernel, and P is edge filling; after having been subjected to the full convolution measurement,
Figure FDA0003303713950000011
no Pad operation; in order to obtain the same size as the original, S is 32, K is 32, and P is 0 since there is no Pad operation; thus, after deconvolution, an initial reconstructed image with the same size as the original image is obtained.
2. The method according to claim 1, wherein the step 1 specifically comprises:
2.1, selecting an image with the size of n multiplied by n, and converting the image into a gray scale image;
step 2.2, measuring by using a full convolution neural network; simultaneously measuring the input data using m BxBx 1 convolution kernels, wherein
Figure FDA0003303713950000012
For measuring the rate, no offset and Pad zero padding are set, and the convolution step length is set as B;
step 2.3, obtaining the product after the measurement of the convolution layer
Figure FDA0003303713950000013
The measured data of (2).
3. The method for compressed sensing reconstruction of image based on Attention multi-feature fusion as claimed in claim 2, wherein the step 2.2 specifically comprises: the measurement by using the convolutional neural network is a mode simulating the traditional measurement: y isi＝ΦB×xiWill measure the matrix phiBEach row of (a) is treated as a convolution kernel; measurement matrix phiBIn existence of
Figure FDA0003303713950000014
The rows of the image data are, in turn,
Figure FDA0003303713950000015
obtaining m measuring points for measuring rate; setting the size of the convolution kernel to be B multiplied by B, and setting the step length to be B, which is equivalent to non-overlapping sampling; each convolution kernel thus outputs a measurement value.
4. The image compressed sensing reconstruction method based on Attention multi-feature fusion according to claim 3, characterized in that in step 3, expansion convolutions with different expansion rates are used to form three parallel channels and a residual block is added to each channel to form a residual network, and a plurality of feature information with different scales are obtained; the three parallel channels specifically include three channels, namely a left channel, a middle channel and a right channel, each channel has four residual blocks, each residual block has three layers, but the expanding convolution rate of each channel is different, specifically:
the expansion rate of the leftmost lane is 1, namely the ordinary convolution kernel of 3 multiplied by 3; the first layer of each residual block is 64 convolution kernels of size 3 × 3; the second layer is 32 convolution kernels with the size of 3 x 3; the third layer is 1 convolution kernel with the size of 3 multiplied by 3; the step length of all the convolution layers in the part is 1 and the activation function is ReLU; performing Pad operation and no Pool operation in order to keep the dimension of output data identical to the dimension of input data, wherein Pad is 1;
the middle channel is also four residual blocks, and each residual block is provided with three convolution layers; the expansion rate of each convolution kernel is 2, and 5 × 5 convolution kernels are obtained by expanding on 3 × 3 convolution kernels; the first layer is 64 convolution kernels; the second layer is 32 convolution kernels; the third layer is 1 convolution kernel; the step length of all the convolution layers in the part is 1 and the activation function is ReLU; performing Pad operation and no Pool operation in order to keep the dimension of output data identical to the dimension of input data, wherein Pad is 2;
the convolution layer number of the right channel and other channels is the same as the number of convolution kernels, but the expanding convolution rate is 3, and then a 7 multiplied by 7 convolution kernel is obtained; therefore, in order to keep the dimension of the output data the same as that of the input data, the Pad zero padding number is 3 at this time; after each convolution, the ReLU is used as the activation function.
5. The method according to claim 4, wherein the step 4 specifically includes: after multi-scale feature information is extracted from multiple channels, feature information of each channel is directly fused, and the information is treated equally among the channels, so that the representation capability of CNN is hindered; actually, the information of each channel has different proportions playing a role in image reconstruction; however, the per-channel scaling is not set artificially, but by network trainingLearning to obtain; extracting features from multiple convolution channels and outputting X ═ X1,…,xc,…,xCC characteristic graphs with the size of n 'multiplied by n' are provided; obtaining channel statistics by a global pooling operation
Figure FDA0003303713950000021
The c is shown as
Figure FDA0003303713950000022
Then, a gate control mechanism is introduced to learn the nonlinear interaction between channels; here, the choice is to use a simple sigmoid function as the door mechanism: s ═ f (W)AR(WBz)) where f (·) and R are sigmoid and ReLU activation functions, WAAnd WBIs the weight after convolution; finally, the final channel statistics s are obtained for rescaling the input X, and finally
Figure FDA0003303713950000023
That is, the characteristic information between each channel is adaptively scaled by the Attention mechanism by the adaptive rescaling of the original characteristic information
Figure FDA0003303713950000024
CN202010479734.8A 2020-05-29 2020-05-29 Image compressed sensing reconstruction method based on Attention multi-feature fusion Active CN111667445B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202010479734.8A CN111667445B (en) 2020-05-29 2020-05-29 Image compressed sensing reconstruction method based on Attention multi-feature fusion

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202010479734.8A CN111667445B (en) 2020-05-29 2020-05-29 Image compressed sensing reconstruction method based on Attention multi-feature fusion

Publications (2)

Publication Number Publication Date
CN111667445A CN111667445A (en) 2020-09-15
CN111667445B true CN111667445B (en) 2021-11-16

Family

ID=72385377

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202010479734.8A Active CN111667445B (en) 2020-05-29 2020-05-29 Image compressed sensing reconstruction method based on Attention multi-feature fusion

Country Status (1)

Country Link
CN (1) CN111667445B (en)

Families Citing this family (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114549676B (en) * 2020-11-26 2025-08-26 南京大学 Arbitrary scale adaptive compressed sensing reconstruction method based on convolutional neural network
CN113034381B (en) * 2021-02-08 2022-06-21 浙江大学 Single image denoising method and device based on cavitated kernel prediction network
CN113098664B (en) * 2021-03-31 2022-10-11 中国人民解放军海军航空大学航空作战勤务学院 MDMSFN-based space-time block code automatic identification method and device
CN114359419B (en) * 2021-11-02 2024-05-17 上海大学 An image compressed sensing reconstruction method based on attention mechanism
CN115330901B (en) * 2022-10-17 2023-01-17 广东工业大学 A method and device for image reconstruction based on compressed sensing network
CN115712065B (en) * 2023-01-05 2023-04-07 湖南大学 Motor fault diagnosis method and system based on perceptual matching of time-frequency revolving door and convolution kernel
CN116228912B (en) * 2023-05-06 2023-07-25 南京信息工程大学 Image compressed sensing reconstruction method based on U-Net multi-scale neural network
CN119379550B (en) * 2024-10-30 2025-11-14 南京航空航天大学 A method for enhancing image quality in gamma-photon industrial reconstruction based on parallel deformable convolutional networks.

Family Cites Families (7)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20120071405A (en) * 2009-10-20 2012-07-02 더 리전트 오브 더 유니버시티 오브 캘리포니아 Incoherent lensfree cell holography and microscopy on a chip
US10825219B2 (en) * 2018-03-22 2020-11-03 Northeastern University Segmentation guided image generation with adversarial networks
CN108765296B (en) * 2018-06-12 2022-04-12 桂林电子科技大学 An Image Super-Resolution Reconstruction Method Based on Recurrent Residual Attention Network
CN110084862B (en) * 2019-04-04 2022-11-15 湖北工业大学 Image compression sensing algorithm based on multi-scale wavelet transform and deep learning
CN110827200B (en) * 2019-11-04 2023-04-07 Oppo广东移动通信有限公司 Image super-resolution reconstruction method, image super-resolution reconstruction device and mobile terminal
CN111028150B (en) * 2019-11-28 2022-04-15 武汉大学 Rapid space-time residual attention video super-resolution reconstruction method
CN110992270A (en) * 2019-12-19 2020-04-10 西南石油大学 Multi-scale residual attention network image super-resolution reconstruction method based on attention

Also Published As

Publication number Publication date
CN111667445A (en) 2020-09-15

Similar Documents

Publication Publication Date Title
CN111667445B (en) Image compressed sensing reconstruction method based on Attention multi-feature fusion
CN113011499B (en) Hyperspectral remote sensing image classification method based on double-attention machine system
CN109410114B (en) Compressed Sensing Image Reconstruction Algorithm Based on Deep Learning
CN113344933B (en) Glandular cell segmentation method based on multi-level feature fusion network
CN111667444B (en) Image compressed sensing reconstruction method based on multi-channel residual error network
CN114943721A (en) Neck ultrasonic image segmentation method based on improved U-Net network
CN111260556B (en) Fourier laminated microscopic reconstruction method based on deep convolutional neural network
CN110675326A (en) Reconstruction and Restoration Method of Computational Ghost Imaging Based on U-Net Network
CN112818764A (en) Low-resolution image facial expression recognition method based on feature reconstruction model
CN113160057B (en) RPGAN image super-resolution reconstruction method based on generative confrontation network
CN110956575B (en) Method and device for converting image style and convolution neural network processor
CN117975086A (en) A few-sample image classification method and system based on metric meta-learning
CN116597203A (en) Knowledge distillation-based anomaly detection method for asymmetric self-encoder
CN117745735A (en) Medical image segmentation method based on adaptive coding and comprehensive attention decoding
CN118505506A (en) Super-resolution reconstruction method, device and medium for low-resolution hyperspectral image
CN117876842A (en) A method and system for detecting anomalies of industrial products based on generative adversarial networks
CN103077408A (en) Method for converting seabed sonar image into acoustic substrate classification based on wavelet neutral network
CN104408697B (en) Image Super-resolution Reconstruction method based on genetic algorithm and canonical prior model
CN113327205B (en) Phase denoising method based on convolutional neural network
CN114581963A (en) Authenticity detection method of OCT fingerprint slice image based on reconstruction difference
Li et al. TPFR-Net: U-shaped model for lung nodule segmentation based on transformer pooling and dual-attention feature reorganization
CN115346091B (en) Method and device for generating Mura defect image data set
Al-Barazanchi et al. Performance evaluation of hybrid CNN for SIPPER plankton image calssification
CN119169289A (en) A semantic segmentation method for crop leaf disease images
CN115222600B (en) Multispectral remote sensing image super-resolution reconstruction method for contrast learning

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant