CN111898324B - A segmentation task-assisted three-dimensional dose distribution prediction method for nasopharyngeal carcinoma - Google Patents

Abstract

The invention relates to a three-dimensional dose distribution prediction method for nasopharyngeal carcinoma based on segmentation task assistance, which specifically includes: collecting original nasopharyngeal carcinoma images and labeling them; building a dose distribution prediction model, the prediction model comprising an auxiliary segmentation network, The dose prediction network and the adversarial network; the segmentation network and the prediction network share the encoder network parameters, and the shared representation information between the two is obtained by jointly training the segmentation task and the dose prediction task, enhancing the feature expression ability of the shared encoder, and promoting the network in The essential features of the segmentation task that are helpful for dose prediction are maximized under limited training data. At the same time, in order to effectively utilize the feature information of the prediction decoder at different scales, the present invention proposes a multi-scale iterative fusion IMF strategy at the prediction task decoder side to obtain more accurate prediction results.

Description

A segmentation task-assisted three-dimensional dose distribution prediction method for nasopharyngeal carcinoma

technical field

The invention relates to the field of image processing, in particular to a three-dimensional dose distribution prediction method for nasopharyngeal carcinoma based on segmentation task assistance.

Background technique

Medical image segmentation is an important field of image processing. The purpose of this field is to segment the important information of the patient's condition in medical images and provide a scientific basis for clinical diagnosis and treatment, which is of great value for doctors to judge the condition. In tumor treatment, radiotherapy is the most effective and safest treatment technology for patients' health, and its non-invasive and low-toxicity characteristics have also been recognized by the medical community. Adhering to the prescribed dose of radiation therapy is the key to successful treatment. However, the dose control in the clinical radiotherapy process is a difficult subject, and the existing research results are far from enough for clinical application.

Although it is very difficult to predict the dose distribution of tumor radiotherapy, there are many research results with significant effect. Based on the indications of the crisis organ DVH (dose-volume histogram), Wu et al. proposed a new concept OVH (overlap volume histogram) which has a strong correlation with it. Under the guidance of clinical experience, it is proposed that the further the distance from the target volume, the lower the dose to the voxel should be. Under the premise of obeying this assumption, according to the OVH comparison of a certain organ, the corresponding upper (lower) limit is found, but the model is relatively subjective and rough. Zhu et al. established a multivariate nonlinear correlation model between the DTH (distance-to-target histogram) of a certain organ and the corresponding DVH, and new patients can obtain the correlation model by substituting the DTH of the diseased organ into the correlation model. DVH prediction of organs. The model can obtain better prediction results to a certain extent. However, in addition to DTH, the volume of the organ and the volume of the target volume are also important factors affecting DVH, and the problem of overfitting caused by the small amount of data needs to be solved. In his master's thesis, Kong Fantu proposed a machine learning method based on support vector regression (SVR) and artificial neural network (artificial neural network, ANN) to establish an association model between patient geometry and organ 3D dose distribution. However, the difference between the target volume and the organ at risk (that is, the segmentation of the organ at risk and the target volume) is not considered, and there are few evaluation indicators in the process of dose prediction. Only the difference of the three-dimensional dose distribution error and the DVH of the organ is used.

With the continuous development of medical imaging technology, the research results of algorithms for delineating medical image target areas based on traditional machine learning and deep neural networks are increasingly fruitful. Especially in the processing of tumor images, the techniques of segmenting tumor regions have made remarkable achievements. In terms of tumor treatment, radiotherapy is one of the main methods of current tumor treatment, with the advantages of low toxicity and non-invasiveness. Radiation therapy not only has the greatest possibility of curing or controlling the tumor, but also guarantees the patient's quality of life after recovery. In order to improve the accuracy and safety of clinical radiotherapy, it is of great significance to study the radiation dose distribution of target organs and their surrounding organs at risk (OAR). At present, there are few domestic research results on assisted 3D dose distribution prediction through segmentation tasks. The existing dose distribution prediction algorithms are either based on traditional machine learning, or use the original CT image and the segmented anatomical structure as prior information, and only use depth The neural network does dose distribution prediction without considering the guiding role of the segmentation task on the prediction task.

At present, domestic and foreign scholars mostly use the original CT image and the segmented anatomical structure as prior information to explore the unique internal relationship between the anatomical structure and the dose distribution when studying the related work of 3D dose distribution. However, the two tasks of anatomical structure segmentation and dose distribution prediction are actually interrelated and complementary. At present, the research on the use of segmentation tasks to assist in 3D dose prediction tasks is still blank. Most of the existing research results of 3D dose distribution prediction are based on deep neural networks. Specifically, Dan Nguyen et al. proposed HD U-net (Hierarchically Densely Connected U-net), which inputs the 3D image blocks of the head and neck according to the size of 96 × 96 × 64, and undergoes two dense convolutions (Dense convolution) , Dense convolutions effectively use standard convolutions with ReLU activations, which are then stitched with the previous set of feature maps. Each layer of U-net performs this operation twice in a row, and then densely downsamples the generated feature map set. The dense downsampling operation is calculated by a convolution and ReLU with a stride of 2 to obtain a set of A new feature map with half the resolution of the input feature map, the original feature map set is then max pooled and concatenated with the new feature map set. Repeat the above operations until the last layer of U-net, up-sampling the lowest-level feature map set, and then splicing with the feature map obtained by dense down-sampling of the corresponding layer through skip-connection. Repeat this operation to the first layer of the network, after two dense connections, and finally get the dose distribution prediction result through a 3×3×3 standard convolution. This method has obtained better experimental results. Vasant Kearney et al. proposed a DoseNet prediction model to predict the dose distribution of prostate radiotherapy patients. DoseNet is a fully convolutional neural network based on U-net structure. DoseNet accepts six 3D matrices as input to the first convolutional layer. 3D CT, prostate, bladder, penile bulb, urethra, and rectal body were fed into the network as unique input channels. All input channels were normalized separately as 6 separate groups before training. The input channel size of the first convolutional layer is 128×128×64×6 and the output channel size is 128×128×64×16. The above output is then subjected to a series of convolutional downsampling, at each step convolutional downsampling increases the size of the fourth dimension by a factor of 2 while continuously reducing the size of the first three dimensions. This enables the network to learn hierarchical relationships over a large receptive field of CT images. Correspondingly, the upsampling step in turn restores the original dimensionality and continues to learn nonlinear relationships in the input data. Once the network is restored to its original dimensions, the last convolutional layer predicts a three-dimensional dose distribution matrix for one channel. Both methods utilize deep neural networks to achieve radiation dose distribution prediction for medical images, and achieve significant experimental results. However, these two prediction models also have their own shortcomings. The former does not consider the influence of the organs at risk around the target area, and the latter cannot effectively utilize the anatomical features of tumors and organs at risk.

SUMMARY OF THE INVENTION

In view of the deficiencies of the prior art, the present invention proposes a three-dimensional dose distribution prediction method for nasopharyngeal carcinoma based on segmentation task assistance. The dose distribution prediction is the main task, and the nasopharyngeal carcinoma segmentation task is the auxiliary task. In this way, the segmentation task can guide the prediction task, thereby improving the accuracy of dose distribution prediction. The method includes the following steps:

Step 1: Collect the original nasopharyngeal carcinoma image, and preprocess the original nasopharyngeal carcinoma image, including radiotherapy dose and organ outline annotation;

Step 2: constructing a dose distribution prediction model, the prediction model includes an auxiliary segmentation network, a dose prediction network and an adversarial network;

Step 3: Divide the labeled original nasopharyngeal cancer image into a set of image blocks with a size of 64×64×64, each block is represented by I _i , i=1, 2,...n, and input I _i to the auxiliary segmentation network split training in

Step 4: Divide the original nasopharyngeal carcinoma image into a set of image blocks with a size of 64 × 64 × 64, each block is represented by I _j , j=1, 2, ... n, input I _j into the dose prediction network, and at the same time , share the parameters learned by the auxiliary segmentation network to the dose prediction network, and perform the following specific operations with the assistance of the auxiliary segmentation network:

Step 41: Convolve I _j with a convolution kernel with a size of 5×5×5 and a stride of 1, and then perform an average pooling process on the convolved output image with a stride of 2 to obtain I _j1 ;

Step 42: perform 3×3×3 convolution processing on I _j1 with a step size of 2, and then perform an average pooling processing on the convolved image with a step size of 2 to obtain a feature map I _j2 ;

Step 43: perform convolution processing on I _j2 with a convolution kernel with a size of 3×3×3 and a stride of 1 to obtain a deep feature map I _j3 ;

Step 44: Perform 3×3×3 deconvolution processing on the deep feature map I _j3 with a stride of 1 to obtain the feature map I _D1 . In order to make full use of the spatial domain information, I _D1 and I _j3 are connected by a cross-layer connection get up to get I _E1 ;

Step 45: Upsampling the feature map I _E1 with a sampling scale of 2, and then performing 3×3×3 deconvolution processing with a stride of 1 to obtain I _D2 . In order to make full use of the spatial domain information, use cross-layer connections Connect I _D2 and I _j2 to obtain I _E2 ;

Step 46: Upsampling the feature map I _E2 with a sampling scale of 2, and then performing a deconvolution process of 3×3×3 and a stride of 1 to obtain I _D3 . In order to make full use of the spatial domain information, I _D3 and I _j1 are connected by cross-layer connection to obtain I _E3 ;

Step 5: Integrate _IE1 , _IE2 , _IE3 through a multi-scale iterative fusion strategy to obtain a predicted dose distribution image _IE ;

Step 6: Input the real dose distribution image pair and the predicted dose distribution image pair into the adversarial network from different channels, and use the adversarial network to distinguish the predicted dose distribution map and the real dose distribution map, and train the adversarial network, which includes steps. :

Step 61: Perform convolution processing on the two image pairs with a convolution kernel with a size of 5×5×5 and a stride of 2, and then perform Maxout processing on the output image, and then go through a 2×2×2 pooling;

Step 62: Perform convolution processing with a convolution kernel size of 3 × 3 × 3 and a stride of 2 on the feature map output in Step 62, and then perform Maxout processing on the output image, and then go through 2 × 2 × 2 pooling processing ;

Step 63: Finally, combine the corresponding channels of the two image pairs, perform 1×1×1 convolution processing respectively, and then classify them through sigmoid activation, so as to discriminate the relative authenticity of the images;

Step 7: Repeat steps 3 to 6. When the adversarial network cannot discriminate between true and false image pairs, that is, when the adversarial loss converges to 0.5, the training of the dose distribution prediction model is completed.

According to a preferred embodiment, the segmentation step of step 3 specifically includes:

Step 31: Convolve I _i with a convolution kernel with a size of 5×5×5 and a stride of 1, and then perform an average pooling process on the convolved output image with a stride of 2 to obtain I _i1 ;

Step 32: Perform 3×3×3 convolution processing on I _i1 with a step size of 2, and then perform an average pooling process on the output image with a step size of 2 to obtain I _i2 ;

Step 33: perform convolution processing on I _i2 with a convolution kernel with a size of 3×3×3 and a stride of 1 to obtain a deep feature map I _i3 ;

Step 34: perform deconvolution processing on the deep feature map I _i3 with a size of 3×3×3 and a step size of 1 to obtain the feature map I _F1 ;

Step 35: Upsampling the feature map _IF1 with a sampling scale of 2, and then performing deconvolution processing of 3×3×3 with a step size of 1 to obtain the feature map _IF2 ;

Step 36: Upsampling the feature map _IF2 with a sampling scale of 2, and then performing a deconvolution process of 3×3×3 with a step size of 1 to obtain the feature map _IF3 ;

Step 37: Upsampling the feature map _IF3 with a sampling scale of 2, and then performing 5×5×5 deconvolution processing with a step size of 1 to obtain a segmentation result, and save the network model parameters in the entire segmentation process.

The beneficial effects of the present invention are:

1. Compared with the traditional method of machine learning and artificial neural network for predicting dose distribution, the present invention has stronger ability to extract image features, takes less time, and has better experimental effect.

2. Compared with the existing method of directly predicting the three-dimensional dose distribution of medical images using deep neural networks, the present invention uses an auxiliary segmentation network, which can make full use of the existing differences between the anatomical structure features and dosimetry features of tumors and OARs. correlation, so as to obtain more refined prediction results.

3. By using an adversarial network, the present invention can obtain a dose distribution result whose effect is close to the plan designed by a radiotherapy physicist.

Description of drawings

Fig. 1 is the structure diagram of dose distribution prediction model of the present invention;

Fig. 2 is the experimental result figure of the present invention;

Figure 2(a) is a real dose distribution image; and

Figure 2(b) is a predicted dose distribution image.

Detailed ways

In order to make the objectives, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below with reference to the specific embodiments and the accompanying drawings. It should be understood that these descriptions are exemplary only and are not intended to limit the scope of the invention. Also, in the following description, descriptions of well-known structures and techniques are omitted to avoid unnecessarily obscuring the concepts of the present invention.

The following detailed description is given in conjunction with the accompanying drawings.

The invention solves the problem of dose distribution prediction of radiotherapy for nasopharyngeal carcinoma in the field of medical images. Accurate segmentation of tumor images can help doctors accurately outline the tumor contour to determine the target area of radiotherapy, and the prediction of radiotherapy dose distribution on tumor images can ensure the accuracy and safety of clinical radiotherapy.

In order to make full use of the correlation between the anatomical structure features and dosimetric features of tumors and organs at risk, the present invention proposes a three-dimensional dose distribution prediction method assisted by segmentation tasks. This method takes dose distribution prediction as the main task and nasopharyngeal carcinoma segmentation task as the auxiliary task. The segmentation task guides the prediction task through multi-task joint learning, thereby improving the accuracy of dose distribution prediction. Figure 1 shows the model structure of the proposed method, including three networks: the auxiliary segmentation network SegNet, the dose prediction network PreNet and the adversarial network AdvNet. The segmentation network and the prediction network share the encoder network parameters, and the shared representation information between the two is obtained by jointly training the segmentation task and the dose prediction task, enhancing the feature expression ability of the shared encoder, and promoting the network to maximize the performance under limited training data. Mining essential features that are helpful for dose prediction in segmentation tasks. At the same time, in order to effectively utilize the feature information of the decoder at different scales, the present invention proposes a multi-scale iterative fusion IMF strategy on the decoder side of the prediction task. The present invention proposes a three-dimensional dose distribution prediction method for nasopharyngeal carcinoma based on segmentation task assistance, the method comprising the following steps:

Step 1: Collect the original nasopharyngeal carcinoma image, and preprocess the original nasopharyngeal carcinoma image, including radiotherapy dose and organ outline annotation.

Step 2: Build a dose distribution prediction model, which includes an auxiliary segmentation network, a dose prediction network and an adversarial network.

Step 3: Divide the labeled original nasopharyngeal cancer image into a set of image blocks with a size of 64×64×64, each block is represented by I _i , i=1, 2,...n, and input I _i to the auxiliary segmentation network for segmentation training.

Step 31: Convolve I _i with a convolution kernel with a size of 5×5×5 and a stride of 1, and then perform an average pooling process on the convolved output image with a stride of 2 to obtain I _i1 .

Step 32: Perform 3×3×3 convolution processing on I _i1 with a stride of 2, and then perform an average pooling process on the output image with a stride of 2 to obtain I _i2 .

Step 33: Convolve I _i2 with a convolution kernel with a size of 3×3×3 and a stride of 1 to obtain a deep feature map I _i3 .

Step 34: Perform deconvolution processing on the deep feature map I _i3 with a size of 3×3×3 and a stride of 1 to obtain a feature map I _F1 .

Step 35: Perform up-sampling with a sampling scale of 2 on the feature map _IF1 , and then perform a deconvolution process of 3×3×3 with a stride of 1 to obtain the feature map _IF2 .

Step 36: Perform up-sampling with a sampling scale of 2 on the feature map _IF2 , and then perform deconvolution processing of 3×3×3 with a stride of 1 to obtain the feature map _IF3 .

Step 37: Upsampling the feature map _IF3 with a sampling scale of 2, and then performing 5×5×5 deconvolution processing with a step size of 1 to obtain a segmentation result, and save the network model parameters in the entire segmentation process.

Step 4: Divide the original nasopharyngeal carcinoma image into a set of image blocks with a size of 64×64×64, each block is represented by I _j , j=1, 2,...n, input I _j into the dose prediction network, and at the same time , share the parameters learned by the auxiliary segmentation network to the dose prediction network, and perform the following specific operations with the assistance of the auxiliary segmentation network:

Step 41: Convolve I _j with a convolution kernel with a size of 5×5×5 and a stride of 1, and then perform an average pooling process on the convolved output image with a stride of 2 to obtain I _j1 .

Step 42: Perform 3×3×3 convolution processing with stride 2 on I _j1 , and then perform average pooling processing on the convolved image with stride 2 to obtain the feature map I _j2 .

Step 43: Convolve I _j2 with a convolution kernel with a size of 3×3×3 and a stride of 1 to obtain a deep feature map I _j3 .

Step 44: Perform 3×3×3 deconvolution processing on the deep feature map I _j3 with a stride of 1 to obtain the feature map I _D1 . In order to make full use of the spatial domain information, I _D1 and I _j3 are connected by a cross-layer connection get up to get I _E1 ;

Step 45: Upsampling the feature map I _E1 with a sampling scale of 2, and then performing 3×3×3 deconvolution processing with a stride of 1 to obtain I _D2 . In order to make full use of the spatial domain information, use cross-layer connections Connect I _D2 and I _j2 to get I _E2 .

Step 46: Upsampling the feature map I _E2 with a sampling scale of 2, and then performing a deconvolution process of 3×3×3 and a stride of 1 to obtain I _D3 . In order to make full use of the spatial domain information, I _D3 and I _j1 are connected by a cross-layer connection to obtain I _E3 .

Step 5: Integrate I _E1 , I _E2 , and I _E3 through a multi-scale iterative fusion strategy to obtain a predicted dose distribution map I _E .

Specifically, the prediction results at different scales are obtained at the decoder side of the prediction task, and then the prediction results at the smallest scale are up-sampled to obtain an image size of the same size as the next scale, and the up-sampled prediction results are obtained. The result is fused with the prediction result of the next scale, and iteratively fused layer by layer in this way until the fused image at the largest scale is obtained, which is the final predicted dose distribution. The multi-scale iterative fusion strategy can be expressed as:

为下一尺度融合后的结果；S表示为不同尺度特征图的数量。Among them, p _S is the prediction result at the current scale, p _S+1 is the prediction result of the next scale, Up( ) is the upsampling operation,

is the result of fusion at the next scale; S is the number of feature maps at different scales.

Step 6: Input the real dose distribution image pair and the predicted dose distribution image pair into the adversarial network from different channels respectively, and use the adversarial network to distinguish the predicted dose distribution map and the real dose distribution map to train the adversarial network; The dose distribution image pair includes the original nasopharyngeal carcinoma image and the real dose distribution image. The real dose distribution image is an image manually annotated by an experienced physician. The predicted dose distribution image pair includes the original nasopharyngeal carcinoma image and the predicted dose distribution image. The dose prediction The distribution image is the image output by the dose prediction network.

Step 61: Convolve the two image pairs with a convolution kernel with a size of 5×5×5 and a stride of 2, and then perform Maxout processing on the output image, and then go through 2×2×2 pooling deal with.

Step 62: Perform convolution processing with a convolution kernel size of 3 × 3 × 3 and a stride of 2 on the feature map output in Step 62, and then perform Maxout processing on the output image, and then go through 2 × 2 × 2 pooling processing .

Step 63: Finally, combine the corresponding channels of the two image pairs, perform 1×1×1 convolution processing respectively, and then classify them through Sigmoid activation, so as to discriminate the relative authenticity of the images.

Step 7: Repeat steps 3 to 6. When the adversarial network cannot discriminate between true and false image pairs, that is, when the adversarial loss converges to 0.5, the training of the dose distribution prediction model is completed, and the prediction network can generate a dose distribution prediction image that is close to the real dose distribution image. .

The loss function of the three-dimensional dose distribution prediction model is composed of the auxiliary segmentation network Dice loss function, the adversarial network loss function and the prediction network loss function. They are respectively expressed as follows:

(1) Auxiliary segmentation network Dice loss function

Among them, pred represents the segmentation result obtained by the segmentation network, and gt represents the gold standard.

(2) The loss function of the adversarial network:

Among them, D(Y) represents the probability that the adversarial network discriminates the real data to be true, and D(PreNet(I, S)) represents the probability that the adversarial network discriminates the prediction result to be true.

(3) In order to make the predicted dose distribution map approximate to the real dose distribution map, the loss of the prediction network includes the voxel-level loss function of the L1 norm and the perceptual loss function based on the VGG network:

Among them, Y represents the real data, PreNet() represents the prediction result, VGG(Y) represents the result obtained after the real data is input into the VGG network, and VGG(PreNet(I, S)) represents the result obtained by inputting the prediction result to the VGG network.

其中，λ₁和λ₂为权值参数。Finally, the loss function of the network is:

Among them, λ ₁ and λ ₂ are weight parameters.

By using the shared encoder network to perform feature extraction on the underlying features of the auxiliary segmentation task, the features that are helpful for the dose prediction task are effectively transferred to the prediction task, and the representation extraction ability of the prediction task is improved. In addition, it is expected to obtain more accurate prediction results by fusing the feature information of different levels through a multi-scale weighting strategy.

After the dose distribution prediction results were obtained in the testing phase, the conformity index CI and the heterogeneity index HI were calculated between the prediction results and the gold standard. The gold standard refers to the dose distribution images annotated by professional doctors. The conformal index CI and the quality index HI are numbers between 0 and 1. Among them, the closer the calculated conformity index CI is to 1, the more accurate the prediction result; the closer the quality index HI is to 0, the more accurate the prediction result is. Table 1 shows the calculation method and meaning of the evaluation index adopted in the present invention.

Table 1. Evaluation Metrics

Fig. 2 is a graph of the experimental results of the present invention, wherein Fig. 2(a) is the real dose distribution image, and Fig. 2(b) is the predicted dose distribution image. It can be seen from the comparison between the two that the contour of the predicted dose distribution image and the The details are relatively close to the real dose distribution image, which can effectively predict the three-dimensional dose distribution of medical images.

It should be noted that the above-mentioned specific embodiments are exemplary, and those skilled in the art can come up with various solutions inspired by the disclosure of the present invention, and these solutions also belong to the disclosure scope of the present invention and fall within the scope of the present invention. within the scope of protection of the invention. It should be understood by those skilled in the art that the description of the present invention and the accompanying drawings are illustrative rather than limiting to the claims. The protection scope of the present invention is defined by the claims and their equivalents.

Claims (1)

1. A three-dimensional dose distribution prediction method for nasopharyngeal carcinoma based on segmentation task assistance is characterized in that, the method comprises the following steps: Step 1: Collect the original nasopharyngeal carcinoma image, and preprocess the original nasopharyngeal carcinoma image, including radiotherapy dose and organ outline annotation; Step 2: constructing a dose distribution prediction model, the prediction model includes an auxiliary segmentation network, a dose prediction network and an adversarial network; Step 3: Divide the labeled original nasopharyngeal cancer image into a set of image blocks with a size of 64×64×64, each block is represented by I _i , i=1, 2,...n, and input I _i to the auxiliary segmentation network segmentation training, including: Step 31: Convolve I _i with a convolution kernel with a size of 5×5×5 and a stride of 1, and then perform an average pooling process on the convolved output image with a stride of 2 to obtain I _i1 ; Step 32: Perform 3×3×3 convolution processing on I _i1 with a step size of 2, and then perform an average pooling process on the output image with a step size of 2 to obtain I _i2 ; Step 33: perform convolution processing on I _i2 with a convolution kernel with a size of 3×3×3 and a stride of 1 to obtain a deep feature map I _i3 ; Step 34: perform deconvolution processing on the deep feature map I _i3 with a size of 3×3×3 and a step size of 1 to obtain the feature map I _F1 ; Step 35: perform up-sampling with a sampling scale of 2 on the feature map _IF1 , and then perform deconvolution processing of 3×3×3 with a step size of 1 to obtain the feature map _IF2 ; Step 36: Upsampling the feature map _IF2 with a sampling scale of 2, and then performing a deconvolution process of 3×3×3 with a step size of 1 to obtain the feature map _IF3 ; Step 37: Upsampling the feature map _IF3 with a sampling scale of 2, and then performing 5×5×5 deconvolution processing with a step size of 1 to obtain a segmentation result, and save the network model parameters in the entire segmentation process; Step 4: Divide the original nasopharyngeal carcinoma image into a set of image blocks with a size of 64×64×64, each block is represented by I _j , j=1, 2,...n, input I _j into the dose prediction network, and at the same time , share the parameters learned by the auxiliary segmentation network to the dose prediction network, and perform the following specific operations with the assistance of the auxiliary segmentation network: Step 41: Perform convolution processing on I _j with a convolution kernel with a size of 5×5×5 and a stride of 1, and then perform an average pooling process on the convolved output image with a stride of 2 to obtain I _j1 ; Step 42: perform 3×3×3 convolution processing on I _j1 with a step size of 2, and then perform an average pooling processing on the convolved image with a step size of 2 to obtain a feature map I _j2 ; Step 43: perform convolution processing on I _j2 with a convolution kernel with a size of 3×3×3 and a stride of 1 to obtain a deep feature map I _j3 ; Step 44: Perform 3×3×3 deconvolution processing on the deep feature map I _j3 with a stride of 1 to obtain the feature map I _D1 . In order to make full use of the spatial domain information, I _D1 and I _j3 are connected by a cross-layer connection get up to get I _E1 ; Step 45: Upsampling the feature map I _E1 with a sampling scale of 2, and then performing 3×3×3 deconvolution processing with a stride of 1 to obtain I _D2 . In order to make full use of the spatial domain information, use cross-layer connections Connect I _D2 and I _j2 to obtain I _E2 ; Step 46: Upsampling the feature map I _E2 with a sampling scale of 2, and then performing 3×3×3 deconvolution processing with a stride of 1 to obtain I _D3 . In order to make full use of the spatial domain information, use cross-layer connection Connect I _D3 and I _j1 to get I _E3 ; Step 5: Integrate _IE1 , _IE2 , _IE3 through a multi-scale iterative fusion strategy to obtain a predicted dose distribution image _IE ; Step 6: Input the real dose distribution image pair and the predicted dose distribution image pair into the adversarial network from different channels, and use the adversarial network to distinguish the predicted dose distribution map and the real dose distribution map, and train the adversarial network, which includes steps. : Step 61: Perform convolution processing on the real dose distribution image pair and the predicted dose distribution image pair with a convolution kernel with a size of 5×5×5 and a stride of 2, and then perform Maxout processing on the output image , and then undergo a 2×2×2 pooling process; Step 62: Perform convolution processing with a convolution kernel size of 3 × 3 × 3 and a stride of 2 on the feature map output in Step 61, and then perform Maxout processing on the output image, and then go through 2 × 2 × 2 pooling processing ; Step 63: Finally, combine the corresponding channels of the two image pairs, perform 1×1×1 convolution processing respectively, and then classify them through sigmoid activation, so as to discriminate the relative authenticity of the images; Step 7: Repeat steps 3 to 6. When the adversarial network cannot discriminate between true and false image pairs, that is, when the adversarial loss converges to 0.5, the training of the dose distribution prediction model is completed.

Images