CN116665906A - Resting state functional magnetic resonance brain age prediction method based on similarity twin network - Google Patents

Abstract

The invention provides a resting state functional magnetic resonance brain age prediction method based on a similarity twin network, which belongs to the technical field of medical image intelligent diagnosis, and solves the technical problem of insufficient accuracy and stability in the traditional brain age prediction method. The technical solution is as follows: S1: collect functional magnetic resonance imaging rs-fMRI data of the subjects; S2: construct twin neural network; S3: design feature similarity and label similarity measurement module; S4: define confidence Evaluate the brain age prediction module; S5: Input the brain image data in the test data set into the model for analysis, so as to obtain the predicted brain age of each test data sample. The invention has the beneficial effects of high prediction accuracy, accurate prediction of brain image data, and helping doctors to more accurately evaluate the brain age of patients.

Description

Resting state fMRI brain age prediction method based on similarity Siamese network

technical field

The invention relates to the technical field of medical image intelligent diagnosis, in particular to a resting state functional magnetic resonance brain age prediction method based on a similarity twin network.

Background technique

Humans also experience cognitive decline with age, with different parts of the brain showing non-linear relationships in the aging process, and the degree of brain aging in traumatized patients is more uncertain. With the emergence of the concept of brain age, research on brain age has also grown exponentially. In addition to predicting brain age, researchers have also explored different patterns of brain aging and their associations with cognitive impairment, mortality, cardiovascular disease, etc. Predictive applications for clinical diagnosis. As humans age, so does the risk of morbidity and mortality. On a macroscopic level, this leads to physiological changes such as enlarged ventricles. At the microscopic level, phenomena such as mitochondrial changes will also appear. Therefore, the study of brain age is important for the development of clinically applicable biological models that can be used to predict brain health in aging or disease.

Traditional brain age prediction methods based on feature engineering have some technical defects. Traditional methods usually use feature engineering-based methods to extract the features of the input image. These methods require artificial selection and design of feature extractors, which are difficult to deal with high-dimensional and complex data, and are susceptible to noise and interference. These limitations lead to traditional methods. The performance of is often unstable and cannot obtain consistent results on a variety of different datasets. Secondly, the traditional method ignores the influence of the neural network structure. In the neural network, the weight and bias of each layer are the key factors of network learning. Different neural network structures have different learning ability and expressive ability, so choose the appropriate neural network Network structure is crucial for brain age prediction tasks.

Contents of the invention

The purpose of the present invention is to provide a resting state functional magnetic resonance brain age prediction method based on the similarity twin network, which aims to improve the accuracy and generalization ability of brain age prediction, and has a wide range of applications in brain science research and clinical medicine. application prospects. The Siamese neural network can automatically learn the features of the input image, and realize the brain age prediction task by comparing the similarity of the input image. The similarity siamese neural network can also handle multimodal input data and has good generalization performance.

In order to achieve the above invention, the present invention adopts the following technical solutions: a resting state fMRI brain age prediction method based on the similarity twin network, comprising the following steps:

S1: Collect the functional magnetic resonance imaging data of the subject to form an original sample set. The samples of the original sample set include the subject's resting state functional magnetic resonance imaging and its corresponding actual age information, and then perform preprocessing. Resting-state fMRI performed operations such as slice timing and head motion correlation, and converted the preprocessed data into three-dimensional image data. Secondly, the preprocessed image data was compared with the corresponding actual age of the subjects One-to-one correspondence to form a sample set, wherein the sample set includes multiple sets of image data and their corresponding actual age information, and finally divides the sample set into a training sample set and a test sample set;

S2: Construct a twin neural network, use a double convolutional neural network based on a fully convolutional neural network as a branch network, and modify it so that the network of the two paths shares parameters. The network consists of two parts: for input from a pair A convolutional neural network backbone for extracting deep features from images and a backbone for fusing the extracted deep features into a similarity measure

S3: Design feature similarity and label similarity measurement module, denoted as Input the known rs-fMRI in the first channel, after convolution and aggregation, get the feature information set Z _x , input the unknown rs-fMRI in the second channel, and get this set of unknown feature vectors Z _y through the same operation , where rs-fMRI means resting-state functional magnetic resonance imaging, and then use the cosine distance CS() to describe the similarity of the obtained feature pairs, learn a transformation matrix through the training data, and finally use the contrastive loss as the loss function to train the similarity Sexual learning, optimize the similarity loss function;

S4: Define the confidence evaluation brain age prediction module, select three groups of known brain age labels with the highest similarity, and the predicted brain age p=mean(p ₁ ,p ₂ ,p ₃ ) is the known brain age label with the highest similarity among the three groups The average of brain age, where p is the predicted brain age, p ₁ , p ₂ , and p ₃ are the three known brain age labels with the highest similarity, and mean() is the average value;

S5: Input the brain image data in the test data set into the model for analysis, so as to obtain the predicted brain age of each test data sample.

As a further optimization scheme of the resting-state fMRI brain age prediction method based on the similarity twin network provided by the present invention, the specific steps of the step S2 are as follows:

Step S2.1: Send rs-fMRI image 1 and rs-fMRI image 2 to a twin convolutional neural network at the same time to obtain the first feature information Z _x and the second feature information Z _y respectively;

Step S2.2: The input image will undergo operations such as convolution and pooling. The dual-channel convolution fusion network architecture adopts parameter sharing, and two convolutional neural networks are built. The number of channels of each convolutional neural network is C. 2C, 4C, 8C, 8C,

4C, where C is the initial number of channels;

Step S2.3: The structure of the feature extraction network consists of six modules, the first five modules are the same, followed by the convolution layer, the normalization layer, the convolution layer, the nonlinear activation function layer, the fifth module and the sixth A dropout layer is added to the module to solve the overfitting problem, and the sixth module contains a convolution layer with a convolution kernel size of 1;

Step S2.3: Define the result E( _xi ,y _i ) of the similarity measure

Among them, Φ() is the feature extraction network, is the similarity measurement module, b is the bias, x _i is the feature tensor of the first track, y _i is the feature tensor of the second track, and i is the ith feature vector;

Step S2.4: Use the contrastive loss function L(Y, _xi ,y _i ) to evaluate the effect of the Siamese neural network on differentiating given multiple images. If the input pairs of samples are not similar, then their The distance will be smaller, which leads to an increase in the loss value:

in, D is the similarity distance between the twin neural network outputs, P is the feature dimension of the input samples, Y is the label of whether the input samples are similar, Y=1 means that the input samples are relatively similar, Y=0 means that the input samples are not similar, m is the set threshold, and N is the number of data.

As a further optimization scheme of the resting state fMRI brain age prediction method based on the similarity twin network provided by the present invention, the specific steps of the step S3 are as follows:

Step S3.1: Given the 4D first-channel feature tensor Z _x and second-channel feature tensor Z _y extracted from the input image pair, where Z _x ={x ₁ ,x ₂ ,..., _xi ,...,x _n } and Z _y ＝{y ₁ ,y ₂ ,...,y _i ,...,y _n } are the age-labeled rs-fMRI image sets in the training samples, and n is the training data The number of concentrated rs-fMRI images, the first age set of rs-MRI images in the training sample is L _x ＝{x' ₁ ,x' ₂ ,...x' _n }, the second age set is L _y ＝{y' ₁ ,y' ₂ ,...y' _n }, where x' _i , y' _i are respectively the age corresponding to the first feature tensor x _i and the second feature tensor y _i Corresponding age;

Step S3.2: Divide the obtained features into two groups, that is, the feature vectors of the training set are as follows, where (x ₁ ,y ₁ ) represents a sample set:

Step S3.3: Use the cosine distance to describe the similarity of each group of images, and the cosine distance CS(xi _, y _i ,A) is given by the following formula:

Among them, the superscript T represents the transpose of the matrix, A is a linear transformation matrix that calculates the cosine similarity in the transformed subspace, and the similarity cosine distance CS(x' _i ,y' _i , A) is given by:

The cosine similarity f(A) between vectors is defined as follows:

Among them, α and β are given parameters about A, α is used to balance the contribution of positive samples and negative samples to the margin, β controls the maximization threshold, A ₀ is a predefined matrix, and ||AA ₀ || represents A and the distance between A ₀ , N is the number of data;

Step S3.4: Divide the objective function into two items: g(A) and h(A):

h(A)＝β||AA ₀ || ² (8)

g(A) is a simple voting scheme for each sample, and the value of α is determined by cross-validation, and h(A) is to regularize the matrix A so that it is as close as possible to the predefined matrix A ₀ ;

Step S3.5: Fuse the similarity CS(xi _, y _i ,A) between the features and the similarity CS(x' _i ,y' _i ,A) of the module sample labels in the similarity measure, and then Then the similarity between the output features and the similarity f(A) between the sample labels, the loss function is calculated and backpropagated according to the similarity to optimize the network and obtain the best network model.

Compared with prior art, the beneficial effect of the present invention is:

1. Automatic learning features: The similarity twin neural network can automatically learn the features of the input image through training, without manual selection and design of feature extraction modules. This automatic feature learning can more accurately capture the information in the image and improve the prediction accuracy In addition, the similarity twin neural network can process high-dimensional data, capture more detailed information, improve the ability of the neural network to express complex data, and is more suitable for tasks that require complex modeling of data.

2. Processing multimodal data: The similarity twinning neural network can process multimodal input data at the same time, and integrate multiple information to improve prediction accuracy. For brain age prediction tasks, the similarity twinning neural network can use both structure and Functional magnetic resonance imaging data, fusion of multiple information to improve prediction accuracy.

3. Improve prediction accuracy: The similarity twinning neural network realizes the brain age prediction task by comparing the similarity of the input images. This method processes the image data more accurately and can better predict the brain age of the subjects; Compared with the traditional method, the similarity twin neural network does not need to manually design a feature extractor, thereby avoiding the uncertainty caused by manual selection of parameters and feature extraction methods, and the prediction effect is more accurate.

4. Strong generalization ability: the similarity twinning neural network has good generalization performance, can handle high-dimensional complex data, and has good robustness to noise and interference; the similarity twinning neural network can learn from samples General rules, so that it has better generalization ability for unseen samples, and effectively avoids overfitting; this makes the similarity twin neural network suitable for processing a wider range of data.

Description of drawings

The accompanying drawings are used to provide a further understanding of the present invention, and constitute a part of the description, and are used to explain the present invention, but not to limit the present invention.

FIG. 1 is an overall framework diagram of the resting-state fMRI brain age prediction method based on the similarity twin network of the present invention.

Fig. 2 is a structural diagram of a similarity twinning convolutional neural network model of the resting-state fMRI brain age prediction method based on the similarity twinning network of the present invention.

Fig. 3 is a brain age prediction module diagram of the resting state fMRI brain age prediction method based on the similarity twin network of the present invention.

Fig. 4 is a twin neural network module diagram of the resting state fMRI brain age prediction method based on the similarity twin network of the present invention.

Detailed ways

In order to make the object, technical solution and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. Of course, the specific embodiments described here are only used to explain the present invention, not to limit the present invention.

Example 1

Referring to Fig. 1 to Fig. 4, the technical solution provided in this embodiment is a method for predicting brain age in resting state fMRI based on similarity twin network, taking the first pair of pictures x ₁ and y ₁ in the data set as an example, starting from From loading data to obtaining classification results, it includes the following steps:

S1: Collect the functional magnetic resonance imaging data of the subject to form an original sample set. The samples of the original sample set include the functional magnetic resonance image data of the subject and its corresponding actual age information, and then perform preprocessing, and the static Slice timing and head motion correlation operations were performed on the breath-state MRI image, and the preprocessed data was converted into three-dimensional image data. Secondly, the preprocessed image data and the corresponding actual age of the subjects were Correspondingly, a sample set is formed, wherein the sample set includes multiple sets of image data and their corresponding actual age information, and finally the sample set is divided into a training sample set and a test sample set;

S2: Construct a twin neural network, use a double convolutional neural network based on a fully convolutional neural network as a branch network, and modify it so that the network of the two paths shares parameters. The network consists of two parts: for input from a pair A convolutional neural network backbone for extracting deep features from images and a backbone for fusing the extracted deep features into a similarity measure

S3: Design feature similarity and label similarity measurement module Input a set of known rs-fMRI in the first channel, after convolution and aggregation, get the feature information Z _x , input a set of unknown rs-fMRI in the second channel, and get this set of unknown feature vectors through the same operation Z _y , where rs-fMRI means resting-state functional magnetic resonance imaging, and then use the cosine distance CS() to describe the similarity of the obtained feature pairs, learn a transformation matrix through the training data, and finally use the contrastive loss as the loss function to Train similarity learning and optimize the similarity loss function;

S4: Define the confidence evaluation brain age prediction module, select three groups of known brain age labels with the highest similarity, and the predicted brain age p=mean(p ₁ ,p ₂ ,p ₃ ) is the known brain age label with the highest similarity among the three groups The average of brain age where p is the predicted brain age, p ₁ , p ₂ , and p ₃ are the three known brain age labels with the highest similarity, and mean() is the average value;

S5: Input the brain image data in the test data set into the model for analysis, so as to obtain the predicted brain age of each test data sample.

Specifically, the specific steps of the step S2 are as follows:

Step S2.1: Send rs-fMRI image 1 and rs-fMRI image 2 to a twin convolutional neural network at the same time to obtain the first feature information Z _x and the second feature information Z _y respectively;

Step S2.2: The input image will undergo operations such as convolution and pooling. The dual-channel convolution fusion network architecture adopts parameter sharing, and two convolutional neural networks are built. The number of channels of each convolutional neural network is 16. 32, 64, 128, 128;

Step S2.3: The structure of the feature extraction network consists of six modules, the first five modules are the same, followed by the convolution layer, the normalization layer, the convolution layer, the nonlinear activation function layer, the fifth module and the sixth A dropout layer is added to the module to solve the overfitting problem, and the sixth module contains a convolution layer with a convolution kernel size of 1;

Step S2.3: Define the result E(x ₁ ,y ₁ ) of the similarity measure for computing the distance between instances x ₁ and y ₁ :

Among them, Φ() is the feature extraction network, is the similarity measurement module, b is the bias, x ₁ is the first feature vector of the first feature tensor, y ₁ is the first feature vector of the second feature tensor, the middle of x ₁ and y ₁ The sections are as follows:

Step S2.4: Use the contrastive loss function L(Y,x ₁ ,y ₁ ) to evaluate the effect of the Siamese neural network on differentiating given multiple images. If the input pairs of samples are not similar, then their The distance will be smaller, which leads to an increase in the loss value:

in, D is the similarity distance between the twin neural network outputs, P is the feature dimension of the input sample, where P=4, Y is the label of whether the input samples are similar, the input samples here are relatively similar, so Y=1, that is get:

Among them, m is the set threshold, which is set to 0.5D here, and N is the number of data.

As the resting-state fMRI brain age prediction method based on the similarity twin network provided in this embodiment, the specific steps of the step S3 are as follows:

Step S3.1: Given the 4D first-channel feature tensor Z _x and second-channel feature tensor Z _y extracted from the input image pair, where Z _x ={x ₁ ,x ₂ ,..., _xi ,...,x _n } and Z _y ＝{y ₁ ,y ₂ ,...,y _i ,...,y _n } are the age-labeled rs-fMRI image sets in the training samples, and n is the training data The number of concentrated rs-fMRI images, the first age set of rs-MRI images in the training sample is L _x ＝{x' ₁ ,x' ₂ ,...x' _n }, the second age set is L _y ＝{y' ₁ ,y' ₂ ,...y' _n }, where x' ₁ , y' ₁ are the age of the first track and the age of the second track respectively, here are 45 and 48 respectively;

Step S3.2: Divide the obtained features into two groups, that is, the feature vectors of the training set are as follows:

Step S3.3: Use the cosine distance to describe the similarity of each group of images, and the cosine distance CS(x ₁ ,y ₁ ,A) is given by the following formula:

Among them, the superscript T represents the transpose of the matrix, and A is a linear transformation matrix that calculates the cosine similarity in the transformed subspace. The matrix is brought in and the result is obtained through matrix operations:

At the same time, the similarity cosine distance CS(x' _i ,y' _i ,A) of each age group is given by the following formula:

The final result CS(x' ₁ ,y' ₁ ,A)=0.786 is obtained, and the cosine similarity f(A) between vectors is defined as follows:

Among them, α and β are given parameters about A, α is used to balance the contribution of positive samples and negative samples to the margin, β controls the maximization threshold and is set to 0.1, A ₀ is a predefined matrix, ||AA ₀ || indicates the distance between A and A ₀ , this function will go through multiple rounds of matrix summation calculations, and finally get the similarity f(A)=0.805;

Step S3.4: Divide the objective function into two items: g(A) and h(A):

h(A)＝β||AA ₀ || ² (20)

g(A) is a simple voting scheme for each sample. The value of α is determined by cross-validation. After the first round of calculation, the value of α is 0.2. h(A) is to regularize the matrix A so that as close as possible to the predefined matrix A ₀ ;

Step S3.5: Fuse the similarity CS(x ₁ ,y ₁ ,A) between the features and the similarity CS(x' ₁ ,y' ₁ ,A) of the module sample labels in the similarity measure, and tightly Then the similarity between the output features and the similarity f(A) between the sample labels, the loss function is calculated and backpropagated according to the similarity to optimize the network and obtain the best network model.

The above descriptions are only preferred embodiments of the present invention, and are not intended to limit the present invention. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present invention shall be included in the protection of the present invention. within range.

Claims (3)

1. The resting state fMRI brain age prediction method based on similarity twin network, is characterized in that, comprises the following steps: S1: Collect the functional magnetic resonance imaging data of the subject to form an original sample set. The samples of the original sample set include the subject's resting-state functional magnetic resonance imaging rs-fMRI and its corresponding actual age information, and then carry out pre-treatment Processing, the slice timing and head motion correlation operations are performed on the resting-state functional magnetic resonance imaging rs-fMRI, and the preprocessed data is converted into three-dimensional image data. Secondly, the preprocessed image data and the corresponding One-to-one correspondence of the actual age of the subjects to form a sample set, wherein the sample set includes multiple sets of image data and their corresponding actual age information, and finally divide the sample set into a training sample set and a test sample set; S2: Construct a twin neural network, use a double convolutional neural network based on a fully convolutional neural network as a branch network, and modify it so that the network of the two paths shares parameters. The network consists of two parts: for input from a pair A convolutional neural network backbone for extracting deep features from images and a backbone for fusing the extracted deep features into a similarity measure S3: Design feature similarity and label similarity measurement module, denoted as Input the known rs-fMRI in the first channel, after convolution and aggregation, get the feature information set Z _x , input the unknown rs-fMRI in the second channel, and get this set of unknown feature vectors Z _y through the same operation , where rs-fMRI means resting-state functional magnetic resonance imaging, and then use the cosine distance CS() to describe the similarity of the obtained feature pairs, learn a transformation matrix through the training data, and finally use the contrastive loss as the loss function to train the similarity Sexual learning, optimize the similarity loss function; S4: Define the confidence evaluation brain age prediction module, select three groups of known brain age labels with the highest similarity, and the predicted brain age p=mean(p ₁ ,p ₂ ,p ₃ ) is the known brain age label with the highest similarity among the three groups The average of brain age, where p is the predicted brain age, p ₁ , p ₂ , and p ₃ are the three known brain age labels with the highest similarity, and mean() is the average value; S5: Input the brain image data in the test data set into the model for analysis, so as to obtain the predicted brain age of each test data sample. 2. the resting state fMRI brain age prediction method based on similarity twin network according to claim 1, is characterized in that, the specific steps of described step S2 are as follows: Step S2.1: Send rs-fMRI image 1 and rs-fMRI image 2 to a twin convolutional neural network at the same time to obtain the first feature information Z _x and the second feature information Z _y respectively; Step S2.2: The input image will undergo operations such as convolution and pooling. The dual-channel convolution fusion network architecture adopts parameter sharing, and two convolutional neural networks are built. The number of channels of each convolutional neural network is C. 2C, 4C, 8C, 8C, 4C, where C is the initial number of channels; Step S2.3: The structure of the feature extraction network consists of six modules, the first five modules are the same, followed by the convolution layer, the normalization layer, the convolution layer, the nonlinear activation function layer, the fifth module and the sixth A dropout layer is added to the module to solve the overfitting problem, and the sixth module contains a convolution layer with a convolution kernel size of 1; Step S2.3: Define the result E( _xi ,y _i ) of the similarity measure Among them, Φ() is the feature extraction network, is the similarity measurement module, b is the bias, x _i is the feature tensor of the first track, y _i is the feature tensor of the second track, and i is the ith feature vector; Step S2.4: Use the contrastive loss function L(Y, _xi ,y _i ) to evaluate the effect of the Siamese neural network on differentiating given multiple images. If the input pairs of samples are not similar, then their The distance will be smaller, which leads to an increase in the loss value: in, D is the similarity distance between the twin neural network outputs, P is the feature dimension of the input samples, Y is the label of whether the input samples are similar, Y=1 means that the input samples are relatively similar, Y=0 means that the input samples are not similar, m is the set threshold, and N is the number of data. 3. the resting state fMRI brain age prediction method based on similarity twin network according to claim 1, is characterized in that, the concrete steps of described step S3 are as follows: Step S3.1: Given the 4D first-channel feature tensor Z _x and second-channel feature tensor Z _y extracted from the input image pair, where Z _x ={x ₁ ,x ₂ ,..., _xi ,...,x _n } and Z _y ＝{y ₁ ,y ₂ ,...,y _i ,...,y _n } are the age-labeled rs-fMRI image sets in the training samples, and n is the training data The number of concentrated rs-fMRI images, the first age set of rs-MRI images in the training sample is L _x ＝{x' ₁ ,x' ₂ ,...x' _n }, the second age set is L _y ＝{y' ₁ ,y' ₂ ,...y' _n }, where x' _i , y' _i are respectively the age corresponding to the first feature tensor x _i and the second feature tensor y _i Corresponding age; Step S3.2: Divide the obtained features into two groups, that is, the feature vectors of the training set are as follows, where (x ₁ ,y ₁ ) represents a sample set: Step S3.3: Use the cosine distance to describe the similarity of each group of images, and the cosine distance CS(xi _, y _i ,A) is given by the following formula: Among them, the superscript T represents the transpose of the matrix, A is a linear transformation matrix that calculates the cosine similarity in the transformed subspace, and the similarity cosine distance CS(x' _i ,y' _i , A) is given by: The cosine similarity f(A) between vectors is defined as follows: Among them, α and β are given parameters about A, α is used to balance the contribution of positive samples and negative samples to the margin, β controls the maximization threshold, A ₀ is a predefined matrix, and ||AA ₀ || represents A and the distance between A ₀ , N is the number of data; Step S3.4: Divide the objective function into two items: g(A) and h(A): h(A)＝β||AA ₀ || ² (8) g(A) is a simple voting scheme for each sample, and the value of α is determined by cross-validation, and h(A) is to regularize the matrix A so that it is as close as possible to the predefined matrix A ₀ ; Step S3.5: Fuse the similarity CS(xi _, y _i ,A) between the features and the similarity CS(x' _i ,y' _i ,A) of the module sample labels in the similarity measure, and then Then, the similarity between the output features and the similarity f(A) between the sample labels, the loss function is calculated and backpropagated according to the similarity to optimize the network and obtain the best network model.