CN116563409A - Multi-scale space-frequency domain feature information guided MRI (magnetic resonance imaging) acceleration reconstruction system - Google Patents

Abstract

The invention discloses an MRI accelerated reconstruction system guided by multi-scale space-frequency domain feature information, comprising: an image acquisition module, an image accelerated reconstruction module and a prediction output module, the image acquisition module is used to acquire original MRI images; the The image acceleration reconstruction module is used to extract a feature map from the original MRI image, perform energy spectrum weighting and implicit feature alignment on the feature map, and predict and reconstruct the MRI image; the output module is used to output the predicted and reconstructed MRI image. The invention proposes a Fourier attention mechanism to extract spectral features that are more conducive to MRI reconstruction tasks in the Fourier domain; the invention realizes feature alignment through implicit neural expression, so that features from different network depths are well aggregated , to avoid feature blurring, so as to obtain a better MRI reconstruction effect.

Description

A MRI Accelerated Reconstruction System Guided by Multi-Scale Space-Frequency Domain Feature Information

technical field

The invention belongs to the technical field of medical equipment, and in particular relates to an MRI accelerated reconstruction system guided by multi-scale space-frequency domain feature information.

Background technique

MRI Accelerated Reconstruction Magnetic resonance imaging (MRI) is an important clinical medical examination method. However, the MRI scanning time is too long and the imaging is slow, which cannot meet the requirements of real-time dynamic high-precision imaging, which seriously affects the promotion and development of MRI. In recent years, a variety of MRI accelerated reconstruction methods based on deep learning have emerged, mainly U-net network, DeepResidual network (ResNet), generative confrontation network (GAN), deep cascaded convolutional neural network (Cascade Net), etc. Such methods usually use a large number of pairs of low-quality and high-quality MRI images as training samples, extract shallow and deep features in MRI images, and establish a mapping from under-sampled artifact images to artifact-free images through network parameter adjustment. .

These mainstream deep learning MRI reconstruction algorithms are all based on convolutional neural networks. In a convolutional neural network, usually as the network deepens, the feature map shrinks exponentially and the number of channels increases. The shallow features usually represent shallow texture information, and the deep features carry high-level semantic information. Current methods usually directly deconvolve and upsample feature maps, so that feature maps of different scales are aligned at the same resolution. This approach often makes some precise information in the feature map blurred, resulting in artificial artifacts in MRI reconstruction results, and at the same time increasing the amount of calculation. Therefore, it is crucial to aggregate features at different scales to efficiently and accurately guide MRI reconstruction.

The essence of the attention mechanism is to enable the network to locate the information of interest, and the key is to learn a weight distribution and apply it to the features. The attention mechanism is mainly divided into spatial attention and channel attention. Among them, channel attention models the importance of each feature channel, and then enhances or suppresses different channels for different tasks, so as to perform feature assignment according to the input, which is simple and effective.

The powerful feature selection and feature localization capabilities of the attention mechanism are also applied to the MRI fast reconstruction problem. However, current methods only perform weight scaling at the feature level in the spatial domain, and essentially only utilize the average intensity of feature maps to calculate the scaling factor, but do not utilize the power spectral features of different feature maps in the Fourier domain.

Implicit Neural Representation The visual world observed by the human eye is continuous. However, due to hardware limitations, images are usually processed as discretized pixel arrays composed of a finite number of pixels, so the accuracy of the image is limited by the resolution. The implicit neural expression proposes a new continuous image solution from the perspective of image representation methods. The key idea is to represent a 3D scene or 2D image as a function, which maps coordinates to corresponding signals, and is controlled by deep neural networks. network to parameterize it. Implicit neural representations were originally proposed for 3D shape and surface modeling, and have also shown great potential in 2D image super-resolution tasks. But so far there is no work to apply implicit neural representation well to the MRI accelerated reconstruction problem.

To sum up, existing studies use deep learning algorithms to establish the mapping relationship between under-sampled MRI images and fully-sampled MRI images to accelerate MRI imaging, but there are still some problems in these studies. First of all, the mainstream convolutional neural network-based method aligns feature maps of different scales through up-down sampling and convolution, which blurs many important precise information and introduces unnecessary artifacts to MRI reconstruction images; secondly, the current The MRI reconstruction method that introduces the attention mechanism only performs weight scaling at the feature level in the spatial domain, and does not utilize the power spectral features of different feature maps in the Fourier domain. Aiming at the above problems, MRI reconstruction guided by multi-feature information is proposed.

Contents of the invention

In order to solve the above technical problems, the present invention proposes a MRI accelerated reconstruction system guided by multi-scale space-frequency domain feature information. By using the method based on implicit neural expression, the feature maps of different scales obtained by encoding are interpolated to complete the spatial feature Alignment; at the same time, K-space restoration is performed to improve reconstruction performance.

To achieve the above purpose, the present invention provides an MRI accelerated reconstruction system guided by multi-scale space-frequency domain feature information, including:

Image acquisition module, image acceleration reconstruction module and prediction output module,

The image acquisition module is used to acquire the original MRI image;

The image acceleration reconstruction module is used to extract a feature map from the original MRI image, perform energy spectrum weighting and implicit feature alignment on the feature map, and predict and reconstruct the MRI image;

The output module is used to output the predicted reconstructed MRI image.

Optionally, the image acceleration reconstruction module includes:

The feature extraction submodule is used to extract the features of the original MRI image and obtain the feature map;

The Fourier channel attention submodule is used to perform energy spectrum weighting on the feature map, and obtain the weighted feature map;

The implicit feature function sub-module performs implicit feature alignment on the weighted feature map, and obtains the aligned feature map;

The encoder predicts and reconstructs the MRI image based on the aligned feature map.

Optionally, in the feature extraction submodule, obtaining the feature map includes:

Inputting the original MRI image into the feature extraction sub-module to obtain the feature maps with different depth information and different scales.

Optionally, the Fourier channel attention submodule includes: a Fourier transform layer, where the Fourier transform layer is used to perform fast Fourier transform on the spectrogram.

Optionally, in the Fourier channel attention submodule, obtaining the weighted feature map includes:

Based on the Fourier transform layer, transforming the feature map into a spectrogram, performing convolution kernel globalization processing on the spectrogram, and obtaining an attention coefficient;

Multiplying the attention coefficient and the feature map to obtain the weighted feature map.

Optionally, in the implicit feature function submodule, before obtaining the aligned feature map, include:

Using a position encoding function to encode the relative two-dimensional coordinates of the position to be queried in the MRI image and the corresponding feature map;

The position encoding function is:

in, is the position coding function, ω ₁ , ω ₂ , ω _L are frequency parameters, and x is the coordinate.

Optionally, in the implicit feature function sub-module, implicit feature alignment is performed on the weighted feature map through an implicit feature function, and the aligned feature map is obtained;

Wherein, the implicit feature function is:

Among them, M(x _q ) is the implicit eigenvalue, f _θ is the decoding function, x _q is the coordinate of the eigenvalue M to be obtained, z ^* is the eigenvector closest to M, x ^* is the coordinate of z ^* , x _q -x ^* is the relative coordinate, Encode the location.

Optionally, the predictive reconstructed MRI image includes:

Based on the aligned feature maps, obtaining corresponding feature vectors, original relative coordinates, and encoded relative coordinates in different feature maps;

Obtaining corresponding predicted intensity values based on the corresponding feature vectors, original relative coordinates and encoded relative coordinates in different feature maps;

The predicted intensity values are decoded to predict a reconstructed MRI image.

Optionally, the MRI accelerated reconstruction system further includes: a K-space correction module, which accelerates the MRI by calculating the output loss between the predicted and reconstructed MRI image and the high-quality image corresponding to the predicted and reconstructed MRI image. Rebuild the system for optimization.

Compared with the prior art, the present invention has the following advantages and technical effects:

The present invention proposes a Fourier channel attention mechanism, efficiently utilizes the energy spectrum features of different feature maps in the Fourier domain, learns the hierarchical expression of high-frequency information more accurately and effectively, and obtains depth features that are more conducive to MRI reconstruction.

The present invention realizes accurate and efficient feature alignment through implicit neural expression, and realizes accurate MRI reconstruction based on this. The present invention can recover the original image without artifacts for the MRI image containing only 12.5% downsampled information, It can significantly improve the performance of MRI reconstruction.

Description of drawings

The drawings constituting a part of the application are used to provide further understanding of the application, and the schematic embodiments and descriptions of the application are used to explain the application, and do not constitute an improper limitation to the application. In the attached picture:

FIG. 1 is a schematic flow diagram of an MRI accelerated reconstruction system according to an embodiment of the present invention;

FIG. 2 is a schematic structural diagram of an encoder network model according to an embodiment of the present invention;

3 is a schematic structural diagram of a Fourier channel attention network model according to an embodiment of the present invention;

Fig. 4 shows the reconstruction results of applying the Cartesian, random undersampling method and 4× acceleration magnification respectively on the public data sets CC359 and IXI according to the embodiment of the present invention;

Fig. 5 shows the reconstruction results of Cartesian, random subsampling methods and 8× acceleration ratio applied to the public data sets CC359 and IXI respectively according to the embodiment of the present invention.

Detailed ways

It should be noted that, in the case of no conflict, the embodiments in the present application and the features in the embodiments can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and embodiments.

It should be noted that the steps shown in the flowcharts of the accompanying drawings may be performed in a computer system, such as a set of computer-executable instructions, and that although a logical order is shown in the flowcharts, in some cases, The steps shown or described may be performed in an order different than here.

The present invention proposes an MRI accelerated reconstruction system guided by multi-scale space-frequency domain feature information, including: an image acquisition module, an image accelerated reconstruction module and a prediction output module,

An image acquisition module, configured to acquire an original MRI image;

The image acceleration reconstruction module is used to extract the feature map from the original MRI image, perform energy spectrum weighting and implicit feature alignment on the feature map, and predict and reconstruct the MRI image;

The output module is used for predicting and reconstructing MRI images based on the aligned feature maps.

Further, the image acceleration reconstruction module includes:

The feature extraction submodule is used to extract the feature of the original MRI image and obtain the feature map;

The Fourier channel attention sub-module is used to weight the energy spectrum of the feature map and obtain the weighted feature map;

The implicit feature function sub-module performs implicit feature alignment on the weighted feature map to obtain the aligned feature map.

Further, the MRI accelerated reconstruction system further includes: a K-space correction module, which optimizes the MRI accelerated reconstruction system by calculating the loss between the output predicted and reconstructed MRI image and the target image corresponding to the predicted and reconstructed MRI image.

The multi-scale feature information based on implicit neural expression guides the MRI reconstruction network, including five stages: encoding, Fourier channel attention modulation, implicit feature alignment, decoding, and K-space correction. For a spectrogram that has been accelerated and undersampled, it is zero-filled and its corresponding low-quality MRI image is obtained by inverse Fourier transform, and it is used as the input of the encoder to obtain four different levels and different sizes. feature map of . Then, the obtained feature map is weighted and scaled through the Fourier channel attention mechanism, and then the feature map that is more conducive to MRI reconstruction is obtained. Next, the number of channels and the scale of the encoded feature maps are aligned by an implicit feature alignment function. Finally, for the intensity value at a certain coordinate to be queried, query the corresponding feature vectors in the four-stage feature maps after alignment in turn, calculate the relative coordinates of the feature maps and the encoded relative coordinates, and store all the feature maps After connecting, input it into the multi-layer perceptron for decoding. By calculating the intensity values at all pixel points of the input image simultaneously and in parallel, the network prediction output image can be decoded. In addition, in the network verification and testing phase, further k-space correction is performed on the output reconstructed image.

Example

As shown in Figure 1, the MRI accelerated reconstruction system guided by a kind of multi-scale space-frequency domain feature information provided by the present embodiment includes five modules: (1) encoder, which extracts four features with different depth information; ( II) Fourier channel attention mechanism, which performs channel modulation on the features obtained by the encoder; (III) implicit feature function, which implicitly aligns the obtained feature maps; (IV) decoder, which simultaneously queries the obtained image The intensity values corresponding to the coordinates of all pixels are used to obtain the reconstructed image; (V)K space correction is used to correct the output reconstructed image. in:

(I) Encoder: In order to extract features of different levels, an encoder is constructed by learning the backbone structure of ResNet, as shown in Figure 2. The encoder is mainly divided into four stages. When a low-quality MRI image with a size of 256×256 is input, it first passes through the first stage, that is, through the convolutional layer, batch normalization layer, ReLU activation function, and maximum After the pooling layer, the first feature map is obtained. Similarly, the feature maps of the other three stages are obtained in sequence. The specific process is shown in Figure 2. After four stages of encoding, different depth information and different Feature Maps of Size and Scale in is a feature map with C _i channels of size H _i ×W _i , where F is the feature map, C is the number of channels, H is the height of the feature map, W is the width of the feature map, R is the real number space, and i is the feature In the graph stage, as shown in Figure 2, darker colors represent deeper features. Specifically, as the number of network layers deepens, a feature map with deeper high-level semantic information is obtained, which is represented by a darker color in the figure, and the size of the feature map is also reduced. Among them, the output of the shallower network layer contains more low-level spatial details, while the output of the deeper network layer has more high-level semantic information;

(II) Fourier channel attention mechanism: In order to efficiently utilize the energy spectrum features of different feature maps in the Fourier domain, so as to learn the hierarchical expression of high-frequency information more accurately and effectively, a Fourier channel attention mechanism is designed . As shown in Figure 3, this module contains a fast Fourier transform layer, which performs Fourier transform through torch's own function torch.fft.fftn, that is, directly performs Fourier transform on the feature map, so that the feature map becomes The spectrogram, and then perform convolution and global pooling on the spectrogram to obtain the attention coefficient on each channel, and multiply the coefficient with the original feature map to adaptively rescale each feature map;

(III) Implicit feature function: For the encoded discrete feature map F _i , its feature vectors are regarded as latent codes uniformly distributed in two-dimensional space, representing a two-dimensional information field respectively. The implicit feature function defines a decoding function f _θ on the discrete feature map F _i , so as to obtain the continuous feature map M. Specifically, given a discrete feature map, the eigenvalues of M at F _i are defined as:

M(x _q )＝f _θ (z ^* , x _q -x ^* )

Among them, x _q is the coordinate of the eigenvalue M to be obtained, z ^* is the eigenvector closest to M, x ^* is the coordinate of z ^* , and f _θ is the decoding function. In the actual training process, f _θ is trained and learned together with the encoder. Since the high-resolution feature map and the low-resolution feature map are equivalent to the explicit representation of the implicit function f _θ at different sampling rates, when f _θ is well fitted, you can query any position through the coordinates x _q The eigenvector values of , so as to complete the size alignment of feature maps of different sizes.

In addition, the high-frequency component is a measure of the edge and contour of an image, which plays a decisive role in image quality, so learning to reconstruct the high-frequency part of an image is the key to the image generation task. However, recent studies have shown that neural networks tend to learn low-frequency signals and are insensitive to high-frequency signals, and their learning ability is limited when directly operating on two-dimensional coordinates. Therefore, the two-dimensional coordinates are first encoded using the position encoding function φ(x):

φ(x)=(sin(ω ₁ x),cos(ω ₂ x),…,sin(ω _L x),cos(ω _L x0),

Among them, φ(x) is the encoding function, and the frequency parameters ω ₁ , ω ₂ ... are initialized as 2e ⁿ , n∈{1,2,...}, and can be fine-tuned during the training process. Therefore, the final definition of the implicit eigenfunction is:

Among them, the relative coordinate x _q -x ^* and its position code φ(x _q -x ^* ) are input into the implicit function.

(Ⅳ) Decoder: After encoding and aligning implicit features, for an input image with a size of 256×256, four corresponding feature maps with different depth information and the same size are obtained . For each pixel in the input image, its coordinates are queried to obtain the corresponding feature vectors, original relative coordinates, and encoded relative coordinates in different feature maps, and they are connected and input into the multi-layer perceptron for training. , to get its corresponding predicted strength value. And by decoding and outputting the predicted intensity values of all pixels simultaneously and in parallel, the predicted and reconstructed MRI images of the network can be obtained.

(Ⅴ) K-space correction: During actual training, the encoder, the implicit function f _θ and the decoder are trained and optimized together. Here, the network is optimized by computing the loss between the reconstructed image output by the network and its corresponding fully sampled target image.

In addition, when verifying and testing the trained network, in order to maintain the real information in the original frequency domain information, K-space correction is further performed on the network output image. That is, the network reconstructed image is Fourier transformed to obtain its spectrogram, and those parts that are not filled with 0 are replaced with the original k-space data, thereby preserving the original measurement data. Finally, the final output image is obtained by inverse Fourier transform.

The performance of the present invention is tested on CC359 and IXI public data sets. Specifically, the performance of the present invention was tested under different conditions of 1D random mask and 2D random mask, and 4×, 8× accelerated sampling, and compared with UNet and KIKINet, the results are shown in Table 1 CC359 data set The test results are shown in Table 2 IXI data set test results. It can be seen that this embodiment obtains the best reconstruction results in almost all cases. In addition, in order to better qualitatively evaluate the performance of this embodiment, the reconstructed images under different experimental settings are shown, as shown in Fig. 4 and Fig. 5 .

Table 1

Table 2

This embodiment proposes the Fourier attention mechanism for the first time, and extracts spectral features that are more conducive to MRI reconstruction tasks in the Fourier domain; for the first time, feature alignment is achieved through implicit neural expression, so that features from different network depths are well integrated Aggregation avoids feature ambiguity, resulting in better MRI reconstruction results.

The above are only preferred specific implementation methods of the present application, but the scope of protection of the present application is not limited thereto. Anyone skilled in the art can easily think of changes or substitutions within the technical scope disclosed in the present application. All should be covered within the scope of protection of this application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims (9)

1. An MRI accelerated reconstruction system guided by multi-scale space-frequency domain feature information, is characterized in that, comprising: an image acquisition module, an image accelerated reconstruction module and a prediction output module, The image acquisition module is used to acquire the original MRI image; The image acceleration reconstruction module is used to extract a feature map from the original MRI image, perform energy spectrum weighting and implicit feature alignment on the feature map, and predict and reconstruct the MRI image; The output module is used to output the predicted reconstructed MRI image. 2. The MRI accelerated reconstruction system guided by a kind of multi-scale space-frequency domain feature information based on claim 1, wherein the image accelerated reconstruction module comprises: The feature extraction submodule is used to extract the features of the original MRI image and obtain the feature map; The Fourier channel attention submodule is used to perform energy spectrum weighting on the feature map, and obtain the weighted feature map; The implicit feature function sub-module performs implicit feature alignment on the weighted feature map, and obtains the aligned feature map; The encoder predicts and reconstructs the MRI image based on the aligned feature map. 3. The MRI accelerated reconstruction system guided by a kind of multi-scale space-frequency domain feature information based on claim 2, characterized in that, in the feature extraction submodule, obtaining the feature map comprises: Inputting the original MRI image into the feature extraction sub-module to obtain the feature maps with different depth information and different scales. 4. based on the MRI accelerated reconstruction system guided by a kind of multi-scale space-frequency domain feature information according to claim 2, it is characterized in that, the Fourier channel attention sub-module comprises: a Fourier transform layer, the Fourier The Fourier transform layer is used to fast Fourier transform the spectrogram. 5. The MRI accelerated reconstruction system guided by a kind of multi-scale space-frequency domain feature information based on claim 4, characterized in that, in the Fourier channel attention sub-module, obtain the weighted feature map include: Based on the Fourier transform layer, transforming the feature map into a spectrogram, performing convolution kernel globalization processing on the spectrogram, and obtaining an attention coefficient; Multiplying the attention coefficient and the feature map to obtain the weighted feature map. 6. The MRI accelerated reconstruction system guided by a kind of multi-scale space-frequency domain feature information based on claim 2, characterized in that, in the implicit feature function sub-module, before obtaining the aligned feature map, include : Using a position encoding function to encode the relative two-dimensional coordinates of the position to be queried in the MRI image and the corresponding feature map; The position encoding function is: in, is the position coding function, ω ₁ , ω ₂ , ω _L are frequency parameters, and x is the coordinate. 7. Based on the MRI accelerated reconstruction system guided by a kind of multi-scale space-frequency domain feature information according to claim 2, it is characterized in that, in the implicit feature function sub-module, the weighted The feature map performs implicit feature alignment, and obtains the aligned feature map; Wherein, the implicit feature function is: Among them, M(x _q ) is the implicit eigenvalue, f _θ is the decoding function, x _q is the coordinate of the eigenvalue M to be obtained, z ^* is the eigenvector closest to M, x ^* is the coordinate of z ^* , x _q -x ^* is the relative coordinate, Encode the location. 8. The MRI accelerated reconstruction system guided by a kind of multi-scale space-frequency domain feature information based on claim 1, characterized in that, said prediction and reconstruction of MRI images comprises: Based on the aligned feature maps, obtaining corresponding feature vectors, original relative coordinates, and encoded relative coordinates in different feature maps; Obtaining corresponding predicted intensity values based on the corresponding feature vectors, original relative coordinates and encoded relative coordinates in different feature maps; The predicted intensity values are decoded to predict a reconstructed MRI image. 9. The MRI accelerated reconstruction system guided by a kind of multi-scale space-frequency domain feature information based on claim 1, characterized in that, the MRI accelerated reconstruction system further comprises: a K-space correction module, the predicted output by calculation The loss between the reconstructed MRI image and the high-quality image corresponding to the predicted reconstructed MRI image is used to optimize the MRI accelerated reconstruction system.