[go: up one dir, main page]

CN117636076B - Prostate MRI image classification method based on deep learning image model - Google Patents

Prostate MRI image classification method based on deep learning image model Download PDF

Info

Publication number
CN117636076B
CN117636076B CN202410101922.5A CN202410101922A CN117636076B CN 117636076 B CN117636076 B CN 117636076B CN 202410101922 A CN202410101922 A CN 202410101922A CN 117636076 B CN117636076 B CN 117636076B
Authority
CN
China
Prior art keywords
image
dimensional
swin
module
output
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Active
Application number
CN202410101922.5A
Other languages
Chinese (zh)
Other versions
CN117636076A (en
Inventor
刘建刚
赵立涛
傅旭
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Beihang University
Original Assignee
Beihang University
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Beihang University filed Critical Beihang University
Priority to CN202410101922.5A priority Critical patent/CN117636076B/en
Publication of CN117636076A publication Critical patent/CN117636076A/en
Application granted granted Critical
Publication of CN117636076B publication Critical patent/CN117636076B/en
Active legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/764—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using classification, e.g. of video objects
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/045—Combinations of networks
    • G06N3/0455—Auto-encoder networks; Encoder-decoder networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/20—Image preprocessing
    • G06V10/25—Determination of region of interest [ROI] or a volume of interest [VOI]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/80—Fusion, i.e. combining data from various sources at the sensor level, preprocessing level, feature extraction level or classification level
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images

Landscapes

  • Engineering & Computer Science (AREA)
  • Theoretical Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • General Physics & Mathematics (AREA)
  • Computing Systems (AREA)
  • Health & Medical Sciences (AREA)
  • Artificial Intelligence (AREA)
  • Software Systems (AREA)
  • General Health & Medical Sciences (AREA)
  • Multimedia (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Biomedical Technology (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Mathematical Physics (AREA)
  • Data Mining & Analysis (AREA)
  • Computational Linguistics (AREA)
  • Biophysics (AREA)
  • Databases & Information Systems (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Image Processing (AREA)

Abstract

The invention belongs to the technical field of image processing, in particular relates to a prostate MRI image classification method based on a deep learning image model, and aims to solve the problems of poor classification accuracy and lower robustness of the existing MRI image classification method. The method comprises the following steps: acquiring a prostate MRI image of a target object as an input image; preprocessing an input image to obtain a preprocessed image; extracting three-dimensional lesion level ROI (region of interest) of the preprocessed image, and performing data coordination and data enhancement processing on the extracted ROI region; and inputting the ROI area subjected to the data enhancement treatment into a pre-constructed prostate MRI image classification model, and further obtaining a classification result. The MRI image classification method improves the classification accuracy and the robustness of the MRI image classification method.

Description

Prostate MRI image classification method based on deep learning image model
Technical Field
The invention belongs to the technical field of image processing, and particularly relates to a prostate MRI image classification method, system and equipment based on a deep learning image model.
Background
Multiparameter magnetic resonance imaging (mpMRI), including T2 weighted imaging (T2 WI), diffusion Weighted Imaging (DWI) and dynamic contrast enhanced imaging (DCE). The imaging mode can describe the anatomical form of the tumor, presents the association of the tumor with surrounding tissues, and is widely applied to the diagnosis and treatment decision of the prostate cancer at present. Clinically, MRI assessment is based on visual assessment of distinct focal features in the image (such as tumor size, location and intensity), which is highly dependent on the radiologist's high level of expertise and thus easily leads to discrepancies between observers. In addition, some subtle, even macroscopic features (e.g., texture features, high-level features) associated with prostate cancer aggressiveness and tumor progression are missed in visual assessment. However, incorporating such imperceptible information into the clinical assessment can greatly improve the accuracy of prostate MRI image classification.
Deep learning is one of the typical artificial intelligence methods that automatically learns task-specific high-dimensional, excavatable and quantifiable features (e.g., tumor anatomy, neurovascular bundles, and other invisible features) that are closely related to the aggressiveness of prostate cancer. Using these high-throughput and depth-mined features as input information, the deep-learning image model can output quantitative scores for image classification. Deep learning has been widely used to aid in diagnosis and treatment decisions of prostate cancer. However, in the field of prediction (two classification problems: whether to relapse or not to metastasize) of a prostate cancer patient such as prediction of postoperative local relapse or metastasis by using MRI image information of the prostate, a study of a deep learning method has not been reported yet.
Based on the method, the prostate MRI image classification method based on the deep learning image model is extracted.
Disclosure of Invention
In order to solve the above problems in the prior art, that is, in order to solve the problems of poor classification accuracy and low robustness of the existing MRI image classification method, the first aspect of the present invention provides a deep learning image model-based prostate MRI image classification method, which includes:
s100, acquiring a prostate MRI image of a target object as an input image; the input image comprises a T2WI image, a DWI image and an ADC image;
s200, preprocessing the input image to obtain a preprocessed image; the preprocessing comprises data de-identification and data registration;
s300, extracting a three-dimensional lesion level ROI from the preprocessed image, and performing data coordination and data enhancement processing on the extracted ROI region; the data coordination comprises bias field correction, resampling and normalization;
s400, respectively inputting the ROI areas subjected to the data enhancement treatment into a pre-constructed prostate MRI image classification model, and further obtaining classification results;
the prostate MRI image classification model is a deep-learning image model comprising a fine-tuned three-dimensional Swin-transducer.
In some preferred embodiments, the input image is subjected to data de-identification and data registration by the following methods:
converting the prostate MRI image from DICOM format to nifi format;
taking the format-converted T2WI image as a reference image, and taking the format-converted DWI image and the format-converted ADC image as floating images; registering the floating image onto the reference image by a three-dimensional rigid transformation.
In some preferred embodiments, the preprocessed image is subjected to three-dimensional lesion level ROI extraction by the method of:
acquiring a lesion area of the preprocessed image; expanding and setting pixel values in the upper direction, the lower direction, the left direction and the right direction by taking the longest boundary of the lesion area as a standard, and then intercepting a two-dimensional square area; the two-dimensional square region containing the tumor and the peritumor region are merged into a three-dimensional lesion-level ROI region.
In some preferred embodiments, the extracted ROI area is subjected to data coordination processing, which includes:
performing bias field correction on the three-dimensional lesion level ROI region by using N4ITK operation;
interpolation processing is carried out on the three-dimensional lesion level ROI area after the offset field correction, and the interpolation is carried out to a set resolution;
and carrying out normalization processing on the three-dimensional lesion level ROI area after interpolation processing.
In some preferred embodiments, the prostate MRI image classification model has the structure:
the prostate MRI image classification model comprises a lesion input module, a three-branch network module with an edge attention mechanism, a soft attention fusion module and an integration module;
the lesion input module is used for inputting the extracted ROI areas corresponding to the T2WI image, the DWI image and the ADC image to the three-branch network module with the edge attention mechanism;
the three-branch network module with the edge attention mechanism comprises three parallel and independent fine-tuned three-dimensional Swin-converter branch networks which are respectively used for inputting the extracted ROI areas corresponding to the T2WI image, the DWI image and the ADC image;
the fine-tuned three-dimensional Swin-converter branch network comprises a 3D patch segmentation layer, four stages, a normalization layer, a first one-dimensional self-adaptive average pooling layer and a first full-connection layer which are connected in sequence; the four phases are divided into a phase 1, a phase 2, a phase 3 and a phase 4 according to the sequence; the stage 1 comprises a linear dimension reduction layer and 2 first 3D Swin-transducer blocks; the stage 2 includes a patch combining layer and 2 second 3D Swin-fransformer blocks; the stage 3 comprises a patch merging layer and 6 third 3D Swin-fransformer blocks; said stage 4 comprises a patch combining layer and 2 fourth 3D Swin-fransformer blocks;
each 3D Swin-fransformer block is embedded with an edge attention mechanism that is entered into each 3D Swin-fransformer block along with the original input of each 3D Swin-fransformer block;
the method for acquiring the edge attention mechanism comprises the following steps: firstly, contracting the boundary of the prostatic lesions inwards by using a convolution kernel of 3*3 through a corrosion algorithm, and subtracting the prostatic lesion region from the prostatic lesion contracted region to obtain prostatic lesion edge information, thereby setting edge attention weight according to the distance from the pixel value in each ROI region to the prostatic lesion edge from small to large;
the output of the stage 1 sequentially passes through a second one-dimensional self-adaptive average pooling layer and a second full-connection layer and then is fused with the output of the first full-connection layer; the output of the stage 2 sequentially passes through a third one-dimensional self-adaptive average pooling layer and a third full-connection layer and then is fused with the output of the first full-connection layer; the output of the stage 3 sequentially passes through a fourth one-dimensional self-adaptive average pooling layer and a fourth full-connection layer and then is fused with the output of the first full-connection layer; the characteristics of the output fusion of the first full-connection layer, the second full-connection layer, the third full-connection layer and the fourth full-connection layer are processed through a fifth full-connection layer and then output, and the processed characteristics are used as the output of the fine-tuned three-dimensional Swin-transducer branch network; the output of the fine-tuned three-dimensional Swin-converter branch network is processed by a space attention module and a channel attention module to obtain the output fraction of the fine-tuned three-dimensional Swin-converter branch network;
respectively carrying out dot product weighting processing on the output scores of three finely-adjusted three-dimensional Swin-transducer branch networks in the three-branch network module with the edge attention mechanism through the soft attention fusion module, carrying out fusion after dot product weighting, and carrying out scaling processing through softmax after fusion to obtain classification scores, namely primary classification probability, which are used as deep learning labels;
the integration module is used for combining the classification score with the pre-acquired clinical features through a logistic regression model so as to obtain a classification result; the clinical features included biopsy Gleason panel, PI-RADS score, PSA level, ADC value, and D-max.
In some preferred embodiments, the outputs of the three fine-tuned three-dimensional Swin-transducer branch networks are respectively subjected to dot product weighting processing by the soft attention fusion module, and the method comprises the following steps:
the soft attention fusion module respectively encodes lesion positions in the extracted ROI areas corresponding to the T2WI image, the DWI image and the ADC image to obtain attention weights;
and carrying out dot product weighting processing on the output scores of the three fine-tuned three-dimensional Swin-transducer branch networks and the corresponding attention weights.
In some preferred embodiments, the data enhancement includes translation, rotation.
In a second aspect of the present invention, a deep learning image model-based prostate MRI image classification system is provided, the system comprising:
an image acquisition module configured to acquire a prostate MRI image of a target object as an input image; the input image comprises a T2WI image, a DWI image and an ADC image;
the preprocessing module is configured to preprocess the input image to obtain a preprocessed image; the preprocessing comprises data de-identification and data registration;
the ROI extraction module is configured to extract the three-dimensional lesion level ROI of the preprocessed image and perform data coordination and data enhancement processing on the extracted ROI region; the data coordination comprises bias field correction, resampling and normalization;
the image classification module is configured to input the ROI area subjected to the data enhancement processing into a pre-constructed prostate MRI image classification model so as to obtain a classification result;
the prostate MRI image classification model is a deep-learning image model comprising a fine-tuned three-dimensional Swin-transducer.
In a third aspect of the present invention, a prostate MRI image classification apparatus based on a deep learning image model is provided, comprising; at least one processor; and a memory communicatively coupled to at least one of the processors; the memory stores instructions executable by the processor for execution by the processor to implement the deep learning image model-based prostate MRI image classification method described above.
The invention has the beneficial effects that:
aiming at the problem of prostate MRI image classification, the invention extracts the multiscale ROI by utilizing the improved Swin-transducer method, can automatically extract effective information of the inner tumor periphery of the automatic learning prostate tumor, excavates multiscale information, integrates multiscale information, and combines clinical characteristic information to construct an accurate MRI image classification model, thereby improving the classification precision of prostate MRI images and effectively improving the accuracy, stability and robustness of image classification.
Drawings
Other features, objects and advantages of the present application will become more apparent upon reading of the detailed description of non-limiting embodiments made with reference to the following drawings.
FIG. 1 is a flow chart of a method for classifying MRI images of the prostate based on a deep-learning image model according to an embodiment of the present invention;
FIG. 2 is a simplified schematic diagram of a method for classifying MRI images of the prostate based on a deep-learning image model according to an embodiment of the present invention;
FIG. 3 is a schematic structural diagram of a prostate MRI image classification model in accordance with an embodiment of the present invention;
fig. 4 is a schematic diagram of a three-branch network module with an edge attention mechanism according to one embodiment of the invention.
Detailed Description
For the purpose of making the objects, technical solutions and advantages of the present invention more apparent, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings, and it is apparent that the described embodiments are some embodiments of the present invention, but not all embodiments. All other embodiments, which can be made by those skilled in the art based on the embodiments of the invention without making any inventive effort, are intended to be within the scope of the invention.
The present application is described in further detail below with reference to the drawings and examples. It is to be understood that the specific embodiments described herein are merely illustrative of the invention and are not limiting of the invention. It should be noted that, for convenience of description, only the portions related to the present invention are shown in the drawings.
It should be noted that, in the case of no conflict, the embodiments and features in the embodiments may be combined with each other.
The prostate MRI image classification method based on the deep learning image model, as shown in figure 1, comprises the following steps:
s100, acquiring a prostate MRI image of a target object as an input image; the input image comprises a T2WI image, a DWI image and an ADC image;
s200, preprocessing the input image to obtain a preprocessed image; the preprocessing comprises data de-identification and data registration;
s300, extracting a three-dimensional lesion level ROI from the preprocessed image, and performing data coordination and data enhancement processing on the extracted ROI region; the data coordination comprises bias field correction, resampling and normalization;
s400, inputting the ROI area subjected to the data enhancement treatment into a pre-constructed prostate MRI image classification model, and further obtaining a classification result;
the prostate MRI image classification model is a deep-learning image model comprising a fine-tuned three-dimensional Swin-transducer.
In order to more clearly describe the prostate MRI image classification method based on the deep learning image model of the present invention, each step of an embodiment of the method of the present invention will be described in detail below with reference to the accompanying drawings.
The prostate MRI image classification method based on the fine-tuning Swin-transducer deep learning label and clinical feature fusion can automatically learn the features around the prostate tumor from the MRI image, and is a method capable of acquiring and fusing sample features with different scales. In addition, the method can be used for more comprehensively describing different mode information provided by combining the magnetic resonance image and clinical characteristics, and more accurate quantitative description is carried out on the sample, so that accurate classification of the prostate MRI image is realized. The method comprises the following steps:
s100, acquiring a prostate MRI image of a target object as an input image; the input image comprises a T2WI image, a DWI image and an ADC image;
in this embodiment, a prostate MRI image is acquired first, and the preferred prostate MRI image of the present invention includes a T2WI image, a DWI image, an ADC image, and the like;
s200, preprocessing the input image to obtain a preprocessed image; the preprocessing comprises data de-identification and data registration;
in this embodiment, to protect specific sensitive information (e.g., name, address, and contact information) of a target object (e.g., patient), all images are converted from digital imaging and communications in medicine (DICOM) to neuroimaging information technology planning (nifi) format, as shown in fig. 2.
To eliminate distortion due to target object motion during MRI acquisition, the images of each patient are registered using an open toolkit of SimpleElastix (see links for details: http:// SimpleElastix. Gitsub. Io /). Wherein the T2WI image serves as a reference image. The DWI image and the ADC image were registered as floating images onto the T2WI image by a three-dimensional rigid transformation.
S300, extracting a three-dimensional lesion level ROI from the preprocessed image, and performing data coordination and data enhancement processing on the extracted ROI region;
in this embodiment, the three-dimensional lesion level ROI extraction method is:
taking the longest boundary of lesions in each target object image (i.e. the preprocessed image) as a standard, expanding and setting pixel values (preferably 5 in the invention) in the upper, lower, left and right directions, and intercepting a two-dimensional square area. The two-dimensional region containing the tumor and the peri-tumor region are then merged into a three-dimensional lesion-level ROI.
And then carrying out data coordination and data enhancement processing on the extracted ROI area.
The data coordination comprises offset field correction, resampling and normalization;
bias field correction: MRI images may show variations in brightness for the same tissue due to bias fields caused by inconsistencies in the scanning equipment, differences in the position of the target object in the scanner, and other deviations in the scanning process. Therefore, offset field correction is performed using the N4ITK operation to reduce intensity non-uniformity, thereby improving accuracy of subsequent predictions.
Resampling is: to meet the input requirements of the deep learning mode, the three-dimensional lesion-level ROI areas of all target objects are interpolated to a set resolution. The present invention preferably interpolates the three-dimensional lesion-level ROI area to 64×64×16.
And then respectively carrying out normalization processing on the three-dimensional lesion level ROI areas after the interpolation processing. The normalized formula is:wherein, the method comprises the steps of, wherein,x i is the ROI area image firstiGray value of each voxel +.>The pixel mean value of the ROI area image is represented, and σ represents the standard deviation of the pixels of the ROI area image.
Data enhancement: in order to alleviate the over-fitting and weak generalization problems of the model, data enhancement operations are performed on the training images. Data enhancement operations include translation and rotation. The model is trained using the enhanced training data set. The performance of the optimal super-parameters and test model are selected using the non-enhanced validation set and the external test set, respectively.
S400, inputting the ROI areas subjected to the data enhancement processing into a pre-constructed prostate MRI image classification model respectively, and further obtaining classification results.
In this embodiment, a classification model of prostate MRI images is constructed based on the prostate MRI images using a Swin-transducer deep learning method of fine tuning, and a deep learning label is generated. As shown in fig. 3 and 4, the prostate MRI image classification model has the following structure:
the prostate MRI image classification model comprises a lesion input module, a three-branch network module with an edge attention mechanism, a soft attention fusion module and an integration module;
the lesion input module is used for inputting the extracted ROI areas corresponding to the T2WI image, the DWI image and the ADC image to the three-branch network module with the edge attention mechanism;
the three-branch network module with the edge attention mechanism comprises three parallel and independent fine-tuned three-dimensional Swin-converter branch networks which are respectively used for inputting the extracted ROI areas corresponding to the T2WI image, the DWI image and the ADC image;
the fine-tuned three-dimensional Swin-converter branch network comprises a 3D patch segmentation layer (namely 3D Patch Partition), four stages, a normalization layer, a first one-dimensional self-adaptive average pooling layer and a first full-connection layer which are connected in sequence; the four phases are divided into a phase 1, a phase 2, a phase 3 and a phase 4 according to the sequence; the stage 1 comprises a Linear dimension reduction layer (namely Linear dimension), and 2 first 3D Swin-transducer blocks; the stage 2 includes a patch combining layer and 2 second 3D Swin-fransformer blocks; the stage 3 includes a Patch merge layer (i.e., patch merge) and 6 third 3D Swin-transform blocks; the stage 4 comprises a patch merging layer and 2 fourth 3D Swin-transducer blocks (the module can integrate the information of the first, second and third stages into a final output through a feature multiplexing operation so as to combine the global information of the large receptive field and the local information of the small receptive field, thereby being beneficial to improving the performance of the deep learning image model); in fig. 4, D represents depth, W represents width, H represents height, and C represents the number of channels.
Each 3D Swin-fransformer block is embedded with an edge attention mechanism that is entered into each 3D Swin-fransformer block along with the original input of each 3D Swin-fransformer block;
the method for acquiring the edge attention mechanism comprises the following steps: firstly, contracting a prostate lesion boundary inwards by a convolution kernel of 3*3 through a corrosion algorithm, and obtaining the information of the edge of the prostate lesion by subtracting a prostate lesion region from the prostate lesion contraction region, so as to set an edge attention weight (the weight is specifically set according to specific conditions and is not exemplified here) according to the distance from a pixel value in each ROI region to the edge of the prostate lesion from small to large; the invention embeds the edge information into each convolution layer of the three branch networks, so that the weight of the pixel value at the position closer to the center of the edge is larger, the position of the edge is indicated, and the network is facilitated to find the edge and define the edge characteristics.
The output of the stage 1 sequentially passes through a second one-dimensional self-adaptive average pooling layer and a second full-connection layer and then is fused with the output of the first full-connection layer; the output of the stage 2 sequentially passes through a third one-dimensional self-adaptive average pooling layer and a third full-connection layer and then is fused with the output of the first full-connection layer; the output of the stage 3 sequentially passes through a fourth one-dimensional self-adaptive average pooling layer and a fourth full-connection layer and then is fused with the output of the first full-connection layer; the characteristics of the output fusion of the first full-connection layer, the second full-connection layer, the third full-connection layer and the fourth full-connection layer are processed through a fifth full-connection layer and then output, and the characteristics are used as the output of the fine-tuning three-dimensional Swin-transducer branch network; the output of the fine-tuned three-dimensional Swin-converter branch network is processed by a spatial attention module and a channel attention module, the characteristics related to classification tasks are enhanced, the characteristics unrelated to the classification tasks are restrained, and finally the characteristics of the full-connection layer are redefined, so that the output score (namely the output of the prediction risk score) of the fine-tuned three-dimensional Swin-converter branch network is obtained;
respectively carrying out dot product weighting processing on the output scores of three finely-adjusted three-dimensional Swin-transducer branch networks (shown in figure 3) in the three-branch network module with the edge attention mechanism through the soft attention fusion module, carrying out fusion after dot product weighting, and carrying out scaling processing through softmax after fusion to obtain classification scores; i.e., the primary classification probability, as a deep learning label;
i.e. the three branches are integrated by one soft attention fusion module, as shown in fig. 2. The soft attention fusion module consists of three Embeddding (i.e., lambda TZ 、λ PZ Or lambda TZ+PZ ) The composition encodes location information of the lesion (i.e., located only in the Transition Zone (TZ), only in the Peripheral Zone (PZ), or both TZ and PZ), respectively. To achieve integration of the three branches, the outputs of the three branches (i.e., y T2WI, lesions ，y DWI, lesions ，y ADC, lesions The lesions in FIG. 3) and an Embedding selected according to the actual location of the lesionsDot product rows. The dot product is then scaled by Softmax to a value in the domain from 0 to 1.
The integration module is used for combining the first classification score with the pre-acquired clinical features through a logistic regression model so as to obtain a classification result; the clinical features included biopsy Gleason panel, PI-RADS score, PSA level, ADC value, and D-max. In addition, during the training process, the class weight of the loss function is set to [0.8,1], and the weight of the positive class patient is increased.
And the integration module is used for fusing the first classification score (namely the deep learning label) with clinical characteristics and combining information of different modes to realize accurate diagnosis of the prostate MRI image. Prostate MRI image classification is performed using logistic regression methods in combination with deep learning labels (i.e., final risk probabilities of the image model, otherwise known as initial classification probabilities) and clinical features (including biopsy Gleason panel, PI-RADS score, PSA level, ADC values, D-max, etc.).
A prostate MRI image classification system based on a deep-learning image model according to a second embodiment of the present invention includes:
an image acquisition module configured to acquire a prostate MRI image of a target object as an input image; the input image comprises a T2WI image, a DWI image and an ADC image;
in the present embodiment, a prostate MRI image is acquired by a magnetic particle imaging apparatus;
the magnetic particle imaging device is in communication connection with the control processor; the control processor generates scanning parameters of the magnetic particle imaging equipment and sets the parameters of the magnetic particle imaging equipment through cables or wireless communication; the method comprises the steps that an object to be imaged is arranged in the center of an imaging view field of magnetic particle imaging equipment, when the magnetic particle imaging equipment receives an imaging instruction sent by a control processor, the object to be imaged is scanned, a prostate MRI image corresponding to the object to be imaged is obtained, and a voltage response signal is sent to the control processor through a cable or wireless communication;
the control processor comprises a preprocessing module, an ROI extraction module and an image classification module.
The preprocessing module is configured to preprocess the input image to obtain a preprocessed image; the preprocessing comprises data de-identification and data registration;
the ROI extraction module is configured to extract the three-dimensional lesion level ROI of the preprocessed image and perform data coordination and data enhancement processing on the extracted ROI region; the data coordination comprises bias field correction, resampling and normalization;
the image classification module is configured to input the ROI area subjected to the data enhancement processing into a pre-constructed prostate MRI image classification model so as to obtain a classification result;
the prostate MRI image classification model is a deep-learning image model comprising a fine-tuned three-dimensional Swin-transducer.
It will be clear to those skilled in the art that, for convenience and brevity of description, specific working processes and related descriptions of the above-described system may refer to corresponding processes in the foregoing method embodiments, which are not repeated herein.
It should be noted that, in the prostate MRI image classification system based on the deep learning image model provided in the foregoing embodiment, only the division of the foregoing functional modules is illustrated, in practical application, the foregoing functional allocation may be performed by different functional modules according to needs, that is, the modules or steps in the foregoing embodiment of the present invention are further decomposed or combined, for example, the modules in the foregoing embodiment may be combined into one module, or may be further split into a plurality of sub-modules, so as to complete all or part of the functions described above. The names of the modules and steps related to the embodiments of the present invention are merely for distinguishing the respective modules or steps, and are not to be construed as unduly limiting the present invention.
A prostate MRI image classification apparatus based on a deep learning image model according to a third embodiment of the present invention includes; at least one processor; and a memory communicatively coupled to at least one of the processors; the memory stores instructions executable by the processor for execution by the processor to implement the deep learning image model-based prostate MRI image classification method described above.
A computer-readable storage medium of a fourth embodiment of the present invention stores computer instructions for execution by the computer to implement the above-described deep-learning image model-based prostate MRI image classification method.
It can be clearly understood by those skilled in the art that, for convenience and brevity of description, the specific working process of the deep learning image model-based prostate MRI image classification device, the computer-readable storage medium and the related description described above may refer to the corresponding process in the foregoing method example, and will not be repeated herein.
Those of skill in the art will appreciate that the various illustrative modules, method steps described in connection with the embodiments disclosed herein may be implemented as electronic hardware, computer software, or combinations of both, and that the program(s) corresponding to the software modules, method steps, may be embodied in Random Access Memory (RAM), memory, read Only Memory (ROM), electrically programmable ROM, electrically erasable programmable ROM, registers, hard disk, removable disk, CD-ROM, or any other form of storage medium known in the art. To clearly illustrate this interchangeability of electronic hardware and software, various illustrative components and steps have been described above generally in terms of their functionality. Whether such functionality is implemented as electronic hardware or software depends upon the particular application and design constraints imposed on the solution. Those skilled in the art may implement the described functionality using different approaches for each particular application, but such implementation is not intended to be limiting.
The terms "first," "second," "third," and the like, are used for distinguishing between similar objects and not for describing a particular sequential or chronological order.
Thus far, the technical solution of the present invention has been described in connection with the preferred embodiments shown in the drawings, but it is easily understood by those skilled in the art that the scope of protection of the present invention is not limited to these specific embodiments. Equivalent modifications and substitutions for related technical features may be made by those skilled in the art without departing from the principles of the present invention, and such modifications and substitutions will be within the scope of the present invention.

Claims (6)

1. A prostate MRI image classification method based on a deep learning image model, the method comprising:
s100, acquiring a prostate MRI image of a target object as an input image; the input image comprises a T2WI image, a DWI image and an ADC image;
s200, preprocessing the input image to obtain a preprocessed image; the preprocessing comprises data de-identification and data registration;
s300, extracting a three-dimensional lesion level ROI from the preprocessed image, and performing data coordination and data enhancement processing on the extracted ROI region; the data coordination comprises bias field correction, resampling and normalization;
the method for extracting the three-dimensional lesion level ROI from the preprocessed image comprises the following steps:
acquiring a lesion area of the preprocessed image; expanding and setting pixel values in the upper direction, the lower direction, the left direction and the right direction by taking the longest boundary of the lesion area as a standard, and then intercepting a two-dimensional square area; combining a two-dimensional square region containing the tumor and a peritumor region into a three-dimensional lesion level ROI region;
s400, inputting the ROI area subjected to the data enhancement treatment into a pre-constructed prostate MRI image classification model, and further obtaining a classification result;
the prostate MRI image classification model is a deep learning image model comprising a fine-tuned three-dimensional Swin-transducer;
the prostate MRI image classification model has the structure that:
the prostate MRI image classification model comprises a lesion input module, a three-branch network module with an edge attention mechanism, a soft attention fusion module and an integration module;
the lesion input module is used for inputting the extracted ROI areas corresponding to the T2WI image, the DWI image and the ADC image to the three-branch network module with the edge attention mechanism;
the three-branch network module with the edge attention mechanism comprises three parallel and independent fine-tuned three-dimensional Swin-converter branch networks which are respectively used for inputting the extracted ROI areas corresponding to the T2WI image, the DWI image and the ADC image;
the fine-tuned three-dimensional Swin-converter branch network comprises a 3D patch segmentation layer, four stages, a normalization layer, a first one-dimensional self-adaptive average pooling layer and a first full-connection layer which are connected in sequence; the four phases are divided into a phase 1, a phase 2, a phase 3 and a phase 4 according to the sequence; the stage 1 comprises a linear dimension reduction layer and 2 first 3D Swin-transducer blocks; the stage 2 includes a patch combining layer and 2 second 3D Swin-fransformer blocks; the stage 3 comprises a patch merging layer and 6 third 3D Swin-fransformer blocks; said stage 4 comprises a patch combining layer and 2 fourth 3D Swin-fransformer blocks;
each 3D Swin-fransformer block is embedded with an edge attention mechanism that is entered into each 3D Swin-fransformer block along with the original input of each 3D Swin-fransformer block;
the method for acquiring the edge attention mechanism comprises the following steps: firstly, contracting the boundary of the prostatic lesions inwards by using a convolution kernel of 3*3 through a corrosion algorithm, and subtracting the prostatic lesion region from the prostatic lesion contracted region to obtain prostatic lesion edge information, thereby setting edge attention weight according to the distance from the pixel value in each ROI region to the prostatic lesion edge from small to large;
the output of the stage 1 sequentially passes through a second one-dimensional self-adaptive average pooling layer and a second full-connection layer and then is fused with the output of the first full-connection layer; the output of the stage 2 sequentially passes through a third one-dimensional self-adaptive average pooling layer and a third full-connection layer and then is fused with the output of the first full-connection layer; the output of the stage 3 sequentially passes through a fourth one-dimensional self-adaptive average pooling layer and a fourth full-connection layer and then is fused with the output of the first full-connection layer; the characteristics of the output fusion of the first full-connection layer, the second full-connection layer, the third full-connection layer and the fourth full-connection layer are processed through a fifth full-connection layer and then output, and the processed characteristics are used as the output of the fine-tuned three-dimensional Swin-transducer branch network; the output of the fine-tuned three-dimensional Swin-converter branch network is processed by a space attention module and a channel attention module to obtain the output fraction of the fine-tuned three-dimensional Swin-converter branch network;
respectively carrying out dot product weighting processing on the output scores of three finely-adjusted three-dimensional Swin-transducer branch networks in the three-branch network module with the edge attention mechanism through the soft attention fusion module, carrying out fusion after dot product weighting, and carrying out scaling processing through softmax after fusion to obtain classification scores, namely primary classification probability, which are used as deep learning labels;
the output of the three fine-tuned three-dimensional Swin-transducer branch networks is respectively subjected to dot product weighting processing through the soft attention fusion module, and the method comprises the following steps:
the soft attention fusion module respectively encodes lesion positions in the extracted ROI areas corresponding to the T2WI image, the DWI image and the ADC image to obtain attention weights;
carrying out dot product weighting processing on the output scores of the three fine-tuned three-dimensional Swin-transducer branch networks and the corresponding attention weights;
the integration module is used for combining the classification score with the pre-acquired clinical features through a logistic regression model so as to obtain a classification result; the clinical features included biopsy Gleason panel, PI-RADS score, PSA level, ADC value, and D-max.
2. The method for classifying prostate MRI images based on a deep-learning image model according to claim 1, wherein said input image is subjected to data de-recognition and data registration, and the method comprises:
converting the prostate MRI image from DICOM format to nifi format;
taking the format-converted T2WI image as a reference image, and taking the format-converted DWI image and the format-converted ADC image as floating images; registering the floating image onto the reference image by a three-dimensional rigid transformation.
3. The method for classifying prostate MRI images based on a deep learning image model according to claim 1, wherein the extracted ROI area is subjected to data coordination processing, and the method comprises the steps of:
performing bias field correction on the three-dimensional lesion level ROI region by using N4ITK operation;
interpolation processing is carried out on the three-dimensional lesion level ROI area after the offset field correction, and the interpolation is carried out to a set resolution;
and carrying out normalization processing on the three-dimensional lesion level ROI area after interpolation processing.
4. The method of classifying prostate MRI images based on a deep-learning image model of claim 1, wherein said data enhancement comprises translation, rotation.
5. A deep-learning image model-based prostate MRI image classification system, the system comprising:
an image acquisition module configured to acquire a prostate MRI image of a target object as an input image; the input image comprises a T2WI image, a DWI image and an ADC image;
the preprocessing module is configured to preprocess the input image to obtain a preprocessed image; the preprocessing comprises data de-identification and data registration;
the method for extracting the three-dimensional lesion level ROI from the preprocessed image comprises the following steps:
acquiring a lesion area of the preprocessed image; expanding and setting pixel values in the upper direction, the lower direction, the left direction and the right direction by taking the longest boundary of the lesion area as a standard, and then intercepting a two-dimensional square area; combining a two-dimensional square region containing the tumor and a peritumor region into a three-dimensional lesion level ROI region;
the ROI extraction module is configured to extract the three-dimensional lesion level ROI of the preprocessed image and perform data coordination and data enhancement processing on the extracted ROI region; the data coordination comprises bias field correction, resampling and normalization;
the image classification module is configured to input the ROI area subjected to the data enhancement processing into a pre-constructed prostate MRI image classification model so as to obtain a classification result;
the prostate MRI image classification model is a deep learning image model comprising a fine-tuned three-dimensional Swin-transducer;
the prostate MRI image classification model has the structure that:
the prostate MRI image classification model comprises a lesion input module, a three-branch network module with an edge attention mechanism, a soft attention fusion module and an integration module;
the lesion input module is used for inputting the extracted ROI areas corresponding to the T2WI image, the DWI image and the ADC image to the three-branch network module with the edge attention mechanism;
the three-branch network module with the edge attention mechanism comprises three parallel and independent fine-tuned three-dimensional Swin-converter branch networks which are respectively used for inputting the extracted ROI areas corresponding to the T2WI image, the DWI image and the ADC image;
the fine-tuned three-dimensional Swin-converter branch network comprises a 3D patch segmentation layer, four stages, a normalization layer, a first one-dimensional self-adaptive average pooling layer and a first full-connection layer which are connected in sequence; the four phases are divided into a phase 1, a phase 2, a phase 3 and a phase 4 according to the sequence; the stage 1 comprises a linear dimension reduction layer and 2 first 3D Swin-transducer blocks; the stage 2 includes a patch combining layer and 2 second 3D Swin-fransformer blocks; the stage 3 comprises a patch merging layer and 6 third 3D Swin-fransformer blocks; said stage 4 comprises a patch combining layer and 2 fourth 3D Swin-fransformer blocks;
each 3D Swin-fransformer block is embedded with an edge attention mechanism that is entered into each 3D Swin-fransformer block along with the original input of each 3D Swin-fransformer block;
the method for acquiring the edge attention mechanism comprises the following steps: firstly, contracting the boundary of the prostatic lesions inwards by using a convolution kernel of 3*3 through a corrosion algorithm, and subtracting the prostatic lesion region from the prostatic lesion contracted region to obtain prostatic lesion edge information, thereby setting edge attention weight according to the distance from the pixel value in each ROI region to the prostatic lesion edge from small to large;
the output of the stage 1 sequentially passes through a second one-dimensional self-adaptive average pooling layer and a second full-connection layer and then is fused with the output of the first full-connection layer; the output of the stage 2 sequentially passes through a third one-dimensional self-adaptive average pooling layer and a third full-connection layer and then is fused with the output of the first full-connection layer; the output of the stage 3 sequentially passes through a fourth one-dimensional self-adaptive average pooling layer and a fourth full-connection layer and then is fused with the output of the first full-connection layer; the characteristics of the output fusion of the first full-connection layer, the second full-connection layer, the third full-connection layer and the fourth full-connection layer are processed through a fifth full-connection layer and then output, and the processed characteristics are used as the output of the fine-tuned three-dimensional Swin-transducer branch network; the output of the fine-tuned three-dimensional Swin-converter branch network is processed by a space attention module and a channel attention module to obtain the output fraction of the fine-tuned three-dimensional Swin-converter branch network;
respectively carrying out dot product weighting processing on the output scores of three finely-adjusted three-dimensional Swin-transducer branch networks in the three-branch network module with the edge attention mechanism through the soft attention fusion module, carrying out fusion after dot product weighting, and carrying out scaling processing through softmax after fusion to obtain classification scores, namely primary classification probability, which are used as deep learning labels;
the output of the three fine-tuned three-dimensional Swin-transducer branch networks is respectively subjected to dot product weighting processing through the soft attention fusion module, and the method comprises the following steps:
the soft attention fusion module respectively encodes lesion positions in the extracted ROI areas corresponding to the T2WI image, the DWI image and the ADC image to obtain attention weights;
carrying out dot product weighting processing on the output scores of the three fine-tuned three-dimensional Swin-transducer branch networks and the corresponding attention weights;
the integration module is used for combining the classification score with the pre-acquired clinical features through a logistic regression model so as to obtain a classification result; the clinical features included biopsy Gleason panel, PI-RADS score, PSA level, ADC value, and D-max.
6. A deep learning image model-based prostate MRI image classification apparatus, comprising:
at least one processor; and a memory communicatively coupled to at least one of the processors;
wherein the memory stores instructions executable by the processor for execution by the processor to implement the deep learning image model-based prostate MRI image classification method of any one of claims 1-4.
CN202410101922.5A 2024-01-25 2024-01-25 Prostate MRI image classification method based on deep learning image model Active CN117636076B (en)

Priority Applications (1)

Application Number Priority Date Filing Date Title
CN202410101922.5A CN117636076B (en) 2024-01-25 2024-01-25 Prostate MRI image classification method based on deep learning image model

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
CN202410101922.5A CN117636076B (en) 2024-01-25 2024-01-25 Prostate MRI image classification method based on deep learning image model

Publications (2)

Publication Number Publication Date
CN117636076A CN117636076A (en) 2024-03-01
CN117636076B true CN117636076B (en) 2024-04-12

Family

ID=90035777

Family Applications (1)

Application Number Title Priority Date Filing Date
CN202410101922.5A Active CN117636076B (en) 2024-01-25 2024-01-25 Prostate MRI image classification method based on deep learning image model

Country Status (1)

Country Link
CN (1) CN117636076B (en)

Families Citing this family (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN119942327B (en) * 2025-01-03 2025-09-26 广州大学 Automatic identification method for remote sensing image of solid waste landfill based on deep learning

Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN114693933A (en) * 2022-04-07 2022-07-01 天津大学 Medical image segmentation device based on generative adversarial network and multi-scale feature fusion
CN114820520A (en) * 2022-04-24 2022-07-29 广东工业大学 Prostate image segmentation method and intelligent prostate cancer auxiliary diagnosis system
CN114943688A (en) * 2022-04-27 2022-08-26 江苏婷灏健康科技有限公司 Method for extracting interest region in mammary gland image based on palpation and ultrasonic data
WO2023098289A1 (en) * 2021-12-01 2023-06-08 浙江大学 Automatic unlabeled pancreas image segmentation system based on adversarial learning
CN116452619A (en) * 2023-04-10 2023-07-18 云南大学 MRI image segmentation method based on high-resolution network and boundary enhancement
KR20230114893A (en) * 2022-01-26 2023-08-02 서강대학교산학협력단 Self-supervised Swin transformer model structure and method of learning the self-supervised Swin transformer model
CN116664590A (en) * 2023-08-02 2023-08-29 中日友好医院(中日友好临床医学研究所) Automatic Segmentation Method and Device Based on Dynamic Contrast Enhanced Magnetic Resonance Image
CN117058448A (en) * 2023-08-10 2023-11-14 太原理工大学 Pulmonary CT image classification system based on domain knowledge and parallel separable convolution Swin transducer
WO2023231329A1 (en) * 2022-05-30 2023-12-07 湖南大学 Medical image semantic segmentation method and apparatus

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2017165801A1 (en) * 2016-03-24 2017-09-28 The Regents Of The University Of California Deep-learning-based cancer classification using a hierarchical classification framework
WO2019210292A1 (en) * 2018-04-27 2019-10-31 Delphinus Medical Technologies, Inc. System and method for feature extraction and classification on ultrasound tomography images

Patent Citations (9)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2023098289A1 (en) * 2021-12-01 2023-06-08 浙江大学 Automatic unlabeled pancreas image segmentation system based on adversarial learning
KR20230114893A (en) * 2022-01-26 2023-08-02 서강대학교산학협력단 Self-supervised Swin transformer model structure and method of learning the self-supervised Swin transformer model
CN114693933A (en) * 2022-04-07 2022-07-01 天津大学 Medical image segmentation device based on generative adversarial network and multi-scale feature fusion
CN114820520A (en) * 2022-04-24 2022-07-29 广东工业大学 Prostate image segmentation method and intelligent prostate cancer auxiliary diagnosis system
CN114943688A (en) * 2022-04-27 2022-08-26 江苏婷灏健康科技有限公司 Method for extracting interest region in mammary gland image based on palpation and ultrasonic data
WO2023231329A1 (en) * 2022-05-30 2023-12-07 湖南大学 Medical image semantic segmentation method and apparatus
CN116452619A (en) * 2023-04-10 2023-07-18 云南大学 MRI image segmentation method based on high-resolution network and boundary enhancement
CN116664590A (en) * 2023-08-02 2023-08-29 中日友好医院(中日友好临床医学研究所) Automatic Segmentation Method and Device Based on Dynamic Contrast Enhanced Magnetic Resonance Image
CN117058448A (en) * 2023-08-10 2023-11-14 太原理工大学 Pulmonary CT image classification system based on domain knowledge and parallel separable convolution Swin transducer

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
MRI-derived radiomics models for diagnosis, aggressiveness, and prognosis evaluation in prostate cancer;Xuehua Zhu et al.;《Journal of Zhejiang University-SCIENCE B (Biomedicine & Biotechnology)》;20230815;全文 *

Also Published As

Publication number Publication date
CN117636076A (en) 2024-03-01

Similar Documents

Publication Publication Date Title
CN106056595B (en) Based on the pernicious assistant diagnosis system of depth convolutional neural networks automatic identification Benign Thyroid Nodules
EP1922999B1 (en) Image processing method and image processing device
JP2023540910A (en) Connected Machine Learning Model with Collaborative Training for Lesion Detection
CN112862805B (en) Acoustic neuroma image automatic segmentation method and system
CN111383759A (en) Automatic pneumonia diagnosis system
CN115131289A (en) Training method of image processing model
CN113962957A (en) Medical image processing method, bone image processing method, device and equipment
CN117636076B (en) Prostate MRI image classification method based on deep learning image model
CN116485853A (en) A medical image registration method and device based on deep learning neural network
CN119180954A (en) Cervical vertebra segmentation and key point detection method based on diffusion model
CN118823344A (en) Medical image semantic segmentation method and system based on channel and spatial attention mechanism
Godla et al. An ensemble learning approach for multi-modal medical image fusion using deep convolutional neural networks
Shi et al. Segment anything model for few-shot medical image segmentation with domain tuning
CN119722671B (en) Cervical cancer pathological image analysis method based on two-stage generation countermeasure network guidance
CN119785038A (en) A method for automatic segmentation and recognition of ultrasound images based on deep learning
CN117911432B (en) Image segmentation method, device and storage medium
CN113362350A (en) Segmentation method and device for cancer medical record image, terminal device and storage medium
CN120198387A (en) A method and system for automatically delineating high-risk clinical target areas and organs at risk in brachytherapy for cervical cancer based on deep learning
CN119964782A (en) B-ultrasound auxiliary diagnosis system based on artificial intelligence
CN119517323A (en) Three-dimensional reconstruction method of human tissue CT images based on SAM series models
CN119180956A (en) WMH image segmentation system of mixed scale attention
Kumar et al. RETRACTED ARTICLE: Medical image fusion based on type-2 fuzzy sets with teaching learning based optimization
CN117152581A (en) A deep learning-based MRI image recognition method and device
Juwita et al. MMPU-Net: A parameter-efficient network for fine-stage of pancreas and pancreas-tumor segmentation on CT scans
Subasi Medical image segmentation using artificial intelligence

Legal Events

Date Code Title Description
PB01 Publication
PB01 Publication
SE01 Entry into force of request for substantive examination
SE01 Entry into force of request for substantive examination
GR01 Patent grant
GR01 Patent grant