CN111815569A - Image segmentation method, device, device and storage medium based on deep learning - Google Patents

Abstract

Embodiments of the present invention disclose a deep learning-based image segmentation method, device, terminal device and storage medium. A training set is obtained based on the segmentation and marking operation on the sample image data in the organ image data, and the organ image data is obtained by scanning the organs; an initial image segmentation model is generated, and the training set is input into the initial image segmentation model Model training is performed to obtain an image segmentation model for image structure segmentation, and the initial image segmentation model is obtained by adding a regular constraint branch parallel to the decoder branch of the U-Net frame at the central layer of the U-Net frame, so The regular constraint branch is used to perform regular constraints on the encoder branch of the U-Net framework; the organ image data to be segmented is input into the image segmentation model, to segment the organ image data to be segmented mark. By placing additional constraints on the encoder weights in the U‑Net network, more robust features can be extracted from the image data.

Description

Image segmentation method, device, device and storage medium based on deep learning

technical field

Embodiments of the present invention relate to the technical field of image processing, and in particular, to a deep learning-based image segmentation method, apparatus, terminal device, and storage medium.

Background technique

In recent years, non-invasive imaging technology can easily obtain brain image data, and through imaging analysis, the anatomical structure and internal lesions of the brain can be quantitatively analyzed, which can be used as an effective basis for disease diagnosis and treatment. Magnetic resonance imaging (Magnetic Resonance Imaging, MRI) technology is an effective technical means for brain image analysis. The automatic segmentation of various anatomical structures (such as gray matter, white matter, cerebrospinal fluid, etc.) in brain MRI images is the basis for quantitative analysis of various brain tissue regions, and is also a key basic step for lesion analysis, 3D visualization, and surgical navigation. Therefore, it is of great significance to study accurate, fast and efficient automatic segmentation methods of brain structures.

MRI images of the brain can show the complex anatomy inside the brain. Due to certain problems such as blurred boundaries, low contrast, low spatial resolution, partial volume effect, and huge differences in the structure and morphology of various tissues, the difficulty of structural segmentation of brain MRI images is greatly increased.

Traditional brain structure segmentation methods are mainly based on manual segmentation and semi-automatic segmentation methods. Manual segmentation methods are time-consuming, tedious, and expensive. The segmentation results are seriously affected by personal experience, and the segmentation results are difficult to reproduce. Semi-automatic segmentation methods mainly include: gray-based methods (threshold method, region growing, fuzzy clustering, etc.), probability map-based methods, active contour-based methods (geodesic method and level set method, etc.), these methods still Relying on the prior interaction of the operator, the segmentation accuracy and efficiency are difficult to reach the practical application level.

At present, in the field of medical image segmentation, image segmentation methods based on deep learning have developed rapidly. Compared with traditional methods, deep learning-based image segmentation methods have great advantages in segmentation accuracy and fully automatic segmentation.

However, the inventor found that when the image segmentation method based on deep learning is used to segment the brain MRI image, the existing image segmentation method based on deep learning is due to the design of the neural network structure. The accuracy of the results will be reduced, and the robustness of the entire segmentation system will be less. Moreover, this phenomenon is relatively common. In addition to brain MRI image segmentation, there is also a problem of insufficient robustness in data segmentation of other organs and modalities.

SUMMARY OF THE INVENTION

The present invention provides an image segmentation method, device, terminal device and storage medium based on deep learning, so as to solve the technical problem of insufficient robustness in the prior art for data segmentation with obvious area division in an image.

In a first aspect, an embodiment of the present invention provides an image segmentation method based on deep learning, including:

A training set is obtained based on the segmentation and labeling operation on the sample image data in the organ image data, the organ image data is obtained by scanning the organ;

An initial image segmentation model is generated, and the training set is input into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation. The initial image segmentation model is added to the central layer of the U-Net framework with Obtained from the parallel regular constraint branch of the decoder branch of the U-Net framework, and the regular constraint branch is used to perform regular constraints on the encoder branch of the U-Net framework;

The organ image data to be segmented is input into the image segmentation model, so as to perform segmentation marking on the organ image data to be segmented.

In a second aspect, an embodiment of the present invention further provides an apparatus for image segmentation based on deep learning, including:

a training set generation unit, configured to obtain a training set based on the segmentation and labeling operation on the sample image data in the organ image data, the organ image data is obtained by scanning the organ;

A model training unit for generating an initial image segmentation model, and inputting the training set into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation, the initial image segmentation model in U-Net The central layer of the framework is obtained by adding a regular constraint branch parallel to the decoder branch of the U-Net framework, and the regular constraint branch is used to perform regular constraints on the encoder branch of the U-Net framework;

The segmentation marking unit is configured to input the organ image data to be segmented into the image segmentation model, so as to perform segmentation and marking on the organ image data to be segmented.

In a third aspect, an embodiment of the present invention further provides a terminal device, including:

one or more processors;

memory for storing one or more programs;

When the one or more programs are executed by the one or more processors, the one or more processors implement the deep learning-based image segmentation method as described in the first aspect.

In a fourth aspect, an embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the program is executed by a processor, implements the deep learning-based image segmentation method described in the first aspect.

The above-mentioned deep learning-based image segmentation method, device, terminal device and storage medium obtain a training set based on a segmentation and labeling operation on sample image data in the organ image data, which is obtained by scanning the organ; generating The initial image segmentation model, the training set is input into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation, the initial image segmentation model is added in the central layer of the U-Net framework and all The decoder branch of the U-Net framework is obtained from the parallel regular constraint branch, and the regular constraint branch is used to perform regular constraints on the encoder branch of the U-Net framework; the organ image data to be segmented is input into the An image segmentation model is used to segment and mark the organ image data to be segmented. By adding a regular constraint branch in parallel with the decoder branch in the central layer of the U-Net framework, the regular constraint branch and the encoder of the U-Net framework are combined into a variational autoencoder. In the overall model, the variational autoencoder and U- Net framework shares the weights of the encoder, and under the action of the loss function of the variational autoencoder, additional constraints can be placed on the encoder weights in the U-Net network, so that the overall model can be extracted into image data more robustly Characteristics.

Description of drawings

1 is a flowchart of a deep learning-based image segmentation method provided in Embodiment 1 of the present invention;

FIG. 2 is a schematic diagram of brain MRI image data;

3 is a schematic diagram of manual segmentation and marking of brain MRI image data;

Fig. 4 is the structural schematic diagram of the image segmentation model of this scheme;

Figure 5 is a schematic diagram of the multi-scale structure set up in the center of the scheme;

Figure 6 is the result of segmenting the brain MRI image data in the MRBrainS18 dataset based on this scheme;

Fig. 7 is the result of segmentation of brain MRI image data in IBSR based on this scheme;

8 is a schematic structural diagram of an apparatus for image segmentation based on deep learning according to Embodiment 2 of the present invention;

FIG. 9 is a schematic structural diagram of a terminal device according to Embodiment 3 of the present invention.

Detailed ways

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are used to explain the present invention, but not to limit the present invention. In addition, it should be noted that, for the convenience of description, the drawings only show some but not all structures related to the present invention.

It should be noted that, due to space limitations, the description of this application does not exhaustively list all optional implementations. After reading the description of this application, those skilled in the art should be able to imagine that as long as the technical features are not contradictory, any Combinations can constitute alternative embodiments.

For example, in an implementation of Embodiment 1, a technical feature is described: the preprocessing of organ image data is mainly to perform preprocessing related to grayscale density and/or grayscale range to obtain grayscale density and/or grayscale range. Or organ image data with the same grayscale range, in another implementation of the first embodiment, another technical feature is recorded: for the content feature of the brain MRI image data, the number of downsampling times of the encoder is 3 Second-rate. Since the above two technical features are not contradictory to each other, those skilled in the art should be able to think after reading the specification of this application that the embodiment with both features is also an optional embodiment, that is, graying out the organ image After preprocessing related to intensity density and grayscale range, downsampling is performed 3 times during training.

In addition, it should be noted that the implementation form of this solution is not a collection of all the technical features described in the first embodiment, and the description of some technical features is for the optimal realization of the solution, and the combination of several technical features described in the first embodiment If the original design intention of this solution can be realized, it can be used as a non-independent implementation, and of course it can also be used as a specific product form.

Each embodiment will be described in detail below.

Example 1

FIG. 1 is a flowchart of an image segmentation method based on deep learning according to Embodiment 1 of the present invention. The deep learning-based image segmentation method provided in the embodiments can be performed by various operating devices for image segmentation, the operating devices can be implemented by means of software and/or hardware, and the operating devices can be two or more physical devices. Entity composition, which can also be a physical entity composition

Specifically, referring to FIG. 1 , the deep learning-based image segmentation method specifically includes:

Step S101 : obtaining a training set based on a segmentation and labeling operation on sample image data in the organ image data obtained by scanning the organ.

The organ image data applicable in this scheme are mainly images of the same type of images with similar regional distribution corresponding to the tissue structure, such as fundus images and brain MRI image data. In this scheme, the brain MRI image data is used as an example for illustration. . The brain MRI image data is obtained by scanning the brains of many people with an MRI scanner. When generating brain MRI image data, there can be various data mode requirements, such as T1-weighted imaging (as shown in Figure 2), T1lR-weighted imaging, T2-weighted imaging, etc. For imaging under different data mode requirements, additional registration operations are required during image preprocessing. In the scheme, only T1-weighted imaging data is used to improve segmentation efficiency and reduce segmentation costs. Of course, in the case where the data division processing capability is sufficient, the comprehensive processing of multiple data modes can also be adopted.

In the case of using only T1-weighted imaging data and no registration operation required, the preprocessing of organ image data is mainly to perform preprocessing related to grayscale density and/or grayscale range to obtain grayscale density and/or grayscale Organ image data with the same degree range. During the imaging process of brain MRI image data, affected by factors such as coils, the final collected brain MRI image data will show irregular and slow gray density changes, which will interfere with digital image segmentation and quantitative analysis. In this solution, brain MRI image data with consistent grayscale density is obtained by performing bias field correction on brain MRI image data. Specifically, the N4 Bias Field Correction method can be used to perform bias field correction. In addition, since the grayscale range of each image slice obtained by MRI is inconsistent, a grayscale normalization operation is required. Specifically, the grayscale distribution of the image can be normalized to (0,1) by the following method:

I=(II _min )/(I _max -I _min )

Wherein, I represents the grayscale matrix of the current sequence image, and _Imin and _Imax represent the minimum and maximum values of the grayscale matrix of the current image, respectively.

On the basis of the organ image data obtained by preprocessing, a plurality of preprocessed organ image data are selected as sample image data to receive segmentation and marking operations to obtain a training set. The specific labeling operation is done by manual operation, and the labeling content includes the target area for segmentation. If it is brain MRI image data, the target area such as Cortical Gray Matter (GM), White Matter (White Matter), Cerebrospinal Fluid (Cerebrospinal Fluid), By manually labeling the segmented images, a gold standard image (Ground Truth) corresponding to the organ image data can be obtained. Figure 3 is the gold standard image, where 10 is the cerebrospinal fluid area, 11 is the gray matter area, 12 is the white matter area, and the black area outside the brain structure area is the background area.

Step S102: Generate an initial image segmentation model, input the training set into the initial image segmentation model for model training, to obtain an image segmentation model for image structure segmentation, and the initial image segmentation model is in the center of the U-Net frame. The layer is obtained by adding a canonical constraint branch parallel to the decoder branch of the U-Net framework, and the canonical constraint branch is used to perform canonical constraints on the encoder branch of the U-Net framework.

As shown in Figure 4, the main body of the image segmentation model in this scheme is the U-Net framework, that is, a U-shaped network composed of two branches on the left in Figure 4. The left branch in the U-shaped network is the encoder branch, and the right branch is the encoder branch. The branch is the decoder branch, and the bottom is the central layer of the U-shaped network. In this scheme, the architecture of the overall neural network is based on the U-shaped network, and a regular constraint branch (the rightmost branch in Figure 4) is further added to the central layer of the U-shaped network. The regular constraint branch is parallel to the decoder branch, and the regular constraint Branches are used to regularize the encoder branches.

In the image segmentation model constructed by this scheme, it can be regarded as a comprehensive realization of U-shaped network and variational autoencoder. The network parameters of the encoder branch are shared by the U-shaped network and the variational autoencoder, that is, the encoder branch and the decoder branch form a complete U-shaped network, and the encoder branch is constrained by the loss function of the decoder branch. The encoder branch and the regular constraint branch form a complete variational autoencoder, and the encoder branch is constrained by the reconstruction error of the variational autoencoder. Thus, the encoder branch is constrained by two aspects, where the regularization constraint branch is able to capture the features of the image through reconstruction errors. In addition, the regular constraint branch only works during the training process of the image segmentation model. When the organ image data is segmented after the model training is completed, the regular constraint branch does not participate in the data processing process of the segmentation mark. The regular constraint branch is equivalent to adding the idea of image generation confrontation to the image segmentation model, which prompts the encoder branch to extract more effective information.

Overall, since the U-shaped network and the variational autoencoder share the weight of the encoder branch, under the action of the loss function of the regular constraint branch, additional constraints can be placed on the weight of the encoder branch, so that the encoder branch can extract More robust features, so that the overall model has better robustness when segmenting organ image data after training.

On the basis of the overall architecture of the above image segmentation model, the training process of the image segmentation model in step S102 can be further implemented through steps S1021-S1026.

Step S1021: Generate an initial image segmentation model.

When initializing the image segmentation model, based on the overall model architecture described above and adapting to different types of organ image data, the parameter details of the image segmentation model can be further initialized.

For the loss function of the regular constraint branch, it can be calculated by the KL divergence between the posterior probability distribution q _φ (z|x) and the likelihood function p _θ (z|x), where x is the input image and z is the The hidden variables in the variational autoencoder composed of the regular constraint branch and the encoder branch, φ and θ correspond to the parameters to be learned by the network in the encoder branch and the decoder branch, respectively.

Specifically, the loss function L _VAE (θ, φ) is calculated by the following formula:

表示重建均方误差，D_KL(q_φ(z|x)||p_θ(z))表示先验分布p_θ(z)与后验概率分布q_φ(z|x)的反向KL散度，在模型中，希望两个分布尽可能接近，所以用该约束。由于变分自编码器和U型网络共享编码器分支的权重，在变分自编码器的损失函数(即正则约束分支的损失函数)的作用下，能够对编码器分支的权重进行额外的约束，使得编码器分支提取更加鲁棒的特征，也即提高整体模型的鲁棒性。in,

represents the reconstruction mean square error, D _KL (q _φ (z|x)||p _θ (z)) represents the inverse KL dispersion of the prior distribution p _θ (z) and the posterior probability distribution q _φ (z|x) Degree, in the model, we want the two distributions to be as close as possible, so use this constraint. Since the variational autoencoder and the U-shaped network share the weight of the encoder branch, under the action of the loss function of the variational autoencoder (that is, the loss function of the regular constraint branch), additional constraints can be placed on the weight of the encoder branch. , so that the encoder branch extracts more robust features, that is, improves the robustness of the overall model.

In addition, Gaussian noise is added between the central layer and the regular constraint branch. The regular constraint branch adds Gaussian noise to the hidden layer, which improves the anti-interference performance of the model, improves the generalization performance, and alleviates the problem of unbalanced samples. The hidden layer refers to the low-dimensional feature expression of the network, that is, the central part of the network. As shown in Figure 4, Gaussian noise is added between the center of the network and the regular constraint branch. Mathematically, the Gaussian noise passes through the N(μ,σ ² ) (ie, normal distribution) constraints are implemented.

In a specific implementation process, the loss function of the decoder may be a binary cross-entropy loss. which is:

In the formula, y represents the Ground Truth, and p _i represents the output value of the logic layer.

When necessary, a multi-scale structure can also be set in the central layer. The multi-scale structure is the atrous convolution module (AtrousConv Module), and its structure is shown in Figure 5. Atrous convolution can increase the receptive field of the extracted features and improve the segmentation performance of the algorithm for objects at different scales. In this scheme, by setting different hole convolution step sizes (r=1, r=2, r=5, r=7), object features at different scales can be obtained.

The central layer can also include a Dropout operation to prevent overfitting and increase the generalization performance of the model.

For the content characteristics of the brain MRI image data, the number of neurons in each convolutional layer of the encoder branch is the same. Unlike most U-shaped networks, this scheme increases the number of neurons in the initial layers and reduces the number of neurons in the deep layers. At the same time, considering that the deep neurons are relatively small in terms of memory occupation and calculation amount, the same number of neurons is set in the encoder branch of the network. When analyzing brain MRI image data, it was found that high-resolution images contain richer image details than low-resolution images. Similarly, in feature maps, high-resolution feature maps contain richer and more complex information, and more neurons are needed to learn these features; in low-resolution feature maps, they contain less information and can be learned by more few neurons to learn effective features.

In addition, according to the content feature of the brain MRI image data, the number of times of downsampling by the encoder is 3 times. With the deepening of the network layer, the parameters, training difficulty, and training cost of the deep neural network will increase significantly. In the brain structure segmentation task, although there are large differences in brain structure between people, the overall structural features are relatively consistent, and shallow networks can already learn these salient features. Compared with the deep network, the gradient backhaul of the shallow network is more effective, which helps to improve the segmentation accuracy. Of course, for other types of organ image data, a more appropriate number of downsampling times can be set after specific analysis of the image features.

The special treatment of this scheme in terms of the depth of the network layer and the number of neurons, through the above two strategies, can reduce the network parameters to 20% of the traditional U-shaped network, reduce the amount of calculation and the running time of the algorithm, improve the segmentation accuracy and speed, which is of great significance for practical applications.

Step S1022: Input the training set into the initial image segmentation model for model training to obtain a transition image segmentation model.

Generally speaking, multiple trainings are required on the basis of the initial image segmentation model to complete the generation of the final image segmentation model. In this scheme, the image segmentation model obtained after each training is defined as a transitional segmentation model. That is to say, the training process of the initial image segmentation model and the transition image segmentation model in this scheme is the same, and the difference is mainly in the update and change of the training set. The two definitions are only for the scheme to express the clear distinction between the names of different stages.

Step S1023: Input the organ image data that is not segmented and marked into the transition image segmentation model for segmentation and marking, and correct and determine the segmentation and marking result.

In the actual model training process, the sample data in the training set constructed based on the sample image data may not be too much. Segmentation markers are both a test of the transitional image segmentation model and an addition to the samples in the training set. Specifically, there are two results for the correction and determination of the segmentation marking result, the first result is that it is determined that no correction is required, and the second result is that it is determined that correction is required. If it is the first result, it can generally be determined that the transition image segmentation model has passed the test, and step S1026 can be executed; if it is the second result, the segmentation and labeling results are corrected, and the corrected segmentation and labeling results can be added to the training set for further For training, step S1024 is executed. Through this scheme of correcting segmentation and labeling results, through manual confirmation and a small amount of correction, reliable segmentation and labeling of organ image data can be quickly obtained, and the training set can be rapidly increased.

Step S1024: Add and update the corrected segmentation and labeling result to the training set.

Step S1025: Train and update the transition image segmentation model according to the added and updated training set.

The updating of the training set and the updating of the transitional image segmentation model are similar to the aforementioned training process of the construction of the training set and the training of the initial image data segmentation model, and will not be repeated here.

Step S1026: When the segmentation mark result is determined to be accurate, the current transition image segmentation model is used as the image segmentation model.

It should be noted that steps S1021 to S1026 are the overall description of the model training process, and the numbers of the steps do not represent absolute constraints on the execution order. Each step can be specifically adjusted according to the execution progress of the model training; for example, in step S1023, if the segmentation mark result is determined to be accurate and does not require correction, it can be considered that the test result of the current transition image segmentation model has reached the preset performance test standard, and The transition image segmentation model is used as the final image segmentation model, that is, step S1026 is performed after step S1023. There may also be cycles between multiple steps in each step; for example, in steps S1023-S1025, it may happen that the unsegmented and marked organ image data is input into the transition image segmentation model multiple times in a row, and the obtained segmentation and marking results all require In this case, steps 1023-S1025 need to be performed cyclically to complete the segmentation labeling corresponding to each unsegmented and labelled organ image, segmentation labeling result correction, training set update and transition image segmentation model update. If the model training process is progressing smoothly, the judgment operation is determined based on the correction in step S1023, and some of the steps may not even be performed; If it is determined that the correction of the result of the segmentation marking of the data does not require correction, step S1026 may be directly performed without performing steps S1024 and S1025.

In addition, in the specific implementation process, in order to improve the segmentation accuracy and precision of the image segmentation model, it can be further limited that the segmentation labeling result is determined to be accurate only when the segmentation labeling results of several consecutive unsegmented image data do not need to be corrected, and the corresponding confirmation image segmentation model.

Step S103: Input the organ image data to be segmented into the image segmentation model, so as to mark the organ image data to be segmented for segmentation.

In practical application, accurate segmentation results can be obtained quickly by inputting the organ image data to be segmented into the image segmentation model and automatically predicted by the image segmentation model. Figures 6 and 7 respectively show the results of segmenting the brain MRI image data in the MRBrainS18 dataset and the brain MRI image data in the IBSR (Internet Brain Segmentation Repository, International Brain Segmentation Database) based on this scheme. Image represents the organ image data to be segmented, Ground Truth represents the result of manually segmenting and labeling the organ image data to be segmented, and Predict represents the segmentation and labeling result of the organ image data to be segmented by the image segmentation model.

In the above, the deep learning-based image segmentation method, device, terminal device and storage medium obtain a training set based on the segmentation and marking operation of sample image data in the organ image data, and the organ image data is obtained by scanning the organ; An initial image segmentation model is generated, and the training set is input into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation. The initial image segmentation model is added to the central layer of the U-Net framework with The decoder branch of the U-Net framework is obtained from the parallel regular constraint branch, and the regular constraint branch is used to perform regular constraints on the encoder branch of the U-Net framework; the organ image data to be segmented is input into the The image segmentation model is used to segment and mark the organ image data to be segmented. By adding a regular constraint branch in parallel with the decoder branch in the central layer of the U-Net framework, the regular constraint branch and the encoder of the U-Net framework are combined into a variational autoencoder. In the overall model, the variational autoencoder and U- Net framework shares the weights of the encoder, and under the action of the loss function of the variational autoencoder, additional constraints can be placed on the encoder weights in the U-Net network, so that the overall model can be extracted into image data more robustly Characteristics.

Embodiment 2

FIG. 8 is a schematic structural diagram of an apparatus for image segmentation based on deep learning according to Embodiment 2 of the present invention. Referring to FIG. 8 , the deep learning-based image segmentation apparatus includes: a training set generating unit 201 , a model training unit 202 and a segmentation marking unit 203 .

Among them, the training set generation unit 201 is used to obtain a training set based on the segmentation and marking operation on the sample image data in the organ image data, and the organ image data is obtained by scanning the organs; the model training unit 202 is used to generate The initial image segmentation model, the training set is input into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation, the initial image segmentation model is added in the central layer of the U-Net framework and all The parallel regular constraint branch of the decoder branch of the U-Net framework is obtained, and the regular constraint branch is used to perform regular constraints on the encoder branch of the U-Net framework; the segmentation marking unit 203 is used to divide the dirty The organ image data is input into the image segmentation model, so as to perform segmentation marking on the organ image data to be segmented.

On the basis of the above embodiment, the model training unit 202 includes:

The model initialization module is used to generate the initial image segmentation model;

an initial training module for inputting the training set into the initial image segmentation model for model training to obtain a transitional image segmentation model;

an intermediate test module, used for inputting the unsegmented and marked organ image data into the transition image segmentation model for segmentation and marking, and correcting and determining the segmentation and marking results;

A training set update module, for adding and updating the revised segmentation mark result to the training set;

A model updating module, used for training and updating the transition image segmentation model according to the added and updated training set;

The model determination module is configured to use the current transition image segmentation model as the image segmentation model when the segmentation marking result is determined to be accurate.

On the basis of the above embodiment, the loss function of the regular constraint branch is calculated by the KL divergence between the posterior probability distribution q _φ (z|x) and the likelihood function p _θ (z|x), where x is the input image, z is the latent variable in the variational autoencoder composed of the regular constraint branch and the encoder branch, φ and θ correspond to the parameters to be learned by the network in the encoder branch and the decoder branch, respectively.

On the basis of the above embodiment, the loss function L _VAE (θ, φ) is calculated by the following formula:

L _VAE (θ,φ)＝-Ε _z～qφ(z|x) logp _θ (x|z)+D _KL (q _φ (z|x)||p _θ (z))

表示重建均方误差，D_KL(q_φ(z|x)||p_θ(z))表示先验分布p_θ(z)与后验概率分布q_φ(z|x)的反向KL散度。in,

represents the reconstruction mean square error, D _KL (q _φ (z|x)||p _θ (z)) represents the inverse KL dispersion of the prior distribution p _θ (z) and the posterior probability distribution q _φ (z|x) Spend.

On the basis of the above embodiment, Gaussian noise is added between the central layer and the regular constraint branch.

Based on the above embodiment, the loss function of the decoder is a binary cross-entropy loss.

On the basis of the above embodiment, the organ image data is brain MRI image data;

Correspondingly, the training set generating unit 201 includes:

The image preprocessing module is used to preprocess the organ image data to obtain organ image data with consistent grayscale density and/or grayscale range;

The segmentation and marking module is used to select a plurality of preprocessed organ image data as sample image data and receive a segmentation and marking operation to obtain a training set.

On the basis of the above embodiment, the grayscale density is preprocessed by bias field correction; the grayscale range is preprocessed by grayscale normalization.

On the basis of the above embodiments, the central layer includes a multi-scale structure.

On the basis of the above embodiment, the central layer includes a Dropout operation.

On the basis of the above embodiment, the number of neurons in each convolutional layer of the encoder is the same.

On the basis of the above embodiment, the number of downsampling times of the encoder is 3 times.

The deep learning-based image segmentation apparatus provided in the embodiment of the present invention is included in a living body detection device, and can be used to execute any of the deep learning-based image segmentation methods provided in the above-mentioned first embodiment, and has corresponding functions and beneficial effects.

Embodiment 3

FIG. 9 is a schematic structural diagram of a terminal device according to Embodiment 3 of the present invention, where the terminal device is a specific hardware presentation solution of the aforementioned deep learning-based image segmentation device. As shown in FIG. 9 , the terminal device includes a processor 310, a memory 320, an input device 330, an output device 340, and a communication device 350; the number of processors 310 in the terminal device may be one or more, and in FIG. Take the processor 310 as an example; the processor 310, the memory 320, the input device 330, the output device 340, and the communication device 350 in the terminal device can be connected by a bus or in other ways.

As a computer-readable storage medium, the memory 320 can be used to store software programs, computer-executable programs, and modules, such as program instructions/modules corresponding to the deep learning-based image segmentation method in the embodiments of the present invention (for example, deep learning-based image segmentation methods). The training set generation unit 201, the model training unit 202 and the segmentation labeling unit 203) in the image segmentation device. The processor 310 executes various functional applications and data processing of the terminal device by running the software programs, instructions and modules stored in the memory 320 , that is, to implement the above-mentioned deep learning-based image segmentation method.

The memory 320 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function; the storage data area may store data created according to the use of the terminal device, and the like. Additionally, memory 320 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other non-volatile solid-state storage device. In some instances, the memory 320 may further include memory located remotely from the processor 310, and these remote memories may be connected to the terminal device through a network. Examples of such networks include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

The input device 330 may be used to receive input numerical or character information, and generate key signal input related to user setting and function control of the terminal device. The output device 340 may include a display device such as a display screen.

The above-mentioned terminal equipment includes an image segmentation device based on deep learning, which can be used to execute any image segmentation method based on deep learning, and has corresponding functions and beneficial effects.

Embodiment 4

Embodiments of the present invention further provide a storage medium containing computer-executable instructions, when executed by a computer processor, the computer-executable instructions are used to execute the image segmentation method based on deep learning provided in any embodiment of the present application. related operations, and have corresponding functions and beneficial effects.

As will be appreciated by those skilled in the art, the embodiments of the present application may be provided as a method, a system, or a computer program product.

Accordingly, the present application may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present application may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein. The present application is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the present application. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram. These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flows of the flowcharts and/or the block or blocks of the block diagrams. These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

In a typical configuration, a computing device includes one or more processors (CPUs), input/output interfaces, network interfaces, and memory. Memory may include non-persistent memory in computer readable media, random access memory (RAM) and/or non-volatile memory in the form of, for example, read only memory (ROM) or flash memory (flash RAM). Memory is an example of a computer-readable medium.

Computer-readable media includes both persistent and non-permanent, removable and non-removable media, and storage of information may be implemented by any method or technology. Information may be computer readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read only memory (ROM), Electrically Erasable Programmable Read Only Memory (EEPROM), Flash Memory or other memory technology, Compact Disc Read Only Memory (CD-ROM), Digital Versatile Disc (DVD) or other optical storage, Magnetic tape cassettes, magnetic tape magnetic disk storage or other magnetic storage devices or any other non-transmission medium that can be used to store information that can be accessed by a computing device. Computer-readable media, as defined herein, excludes transitory computer-readable media, such as modulated data signals and carrier waves.

It should also be noted that the terms "comprising", "comprising" or any other variation thereof are intended to encompass a non-exclusive inclusion such that a process, method, article or device comprising a series of elements includes not only those elements, but also Other elements not expressly listed or inherent to such a process, method, article of manufacture or apparatus are also included. Without further limitation, an element qualified by the phrase "comprising a..." does not preclude the presence of additional identical elements in the process, method, article of manufacture or apparatus that includes the element.

Note that the above are only preferred embodiments of the present invention and applied technical principles. Those skilled in the art will understand that the present invention is not limited to the specific embodiments described herein, and various obvious changes, readjustments and substitutions can be made by those skilled in the art without departing from the protection scope of the present invention. Therefore, although the present invention has been described in detail through the above embodiments, the present invention is not limited to the above embodiments, and can also include more other equivalent embodiments without departing from the concept of the present invention. The scope is determined by the scope of the appended claims.

Claims (10)

1. An image segmentation method based on deep learning, characterized in that, comprising: A training set is obtained based on the segmentation and labeling operation on the sample image data in the organ image data, the organ image data is obtained by scanning the organ; An initial image segmentation model is generated, and the training set is input into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation. The initial image segmentation model is added to the central layer of the U-Net framework with Obtained from the parallel regular constraint branch of the decoder branch of the U-Net framework, and the regular constraint branch is used to perform regular constraints on the encoder branch of the U-Net framework; The organ image data to be segmented is input into the image segmentation model, so as to perform segmentation marking on the organ image data to be segmented. 2. The method according to claim 1, wherein, in the step of generating an initial image segmentation model, the training set is input into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation ,include: Generate an initial image segmentation model; Inputting the training set into the initial image segmentation model for model training to obtain a transition image segmentation model; Inputting the unsegmented and marked organ image data into the transition image segmentation model for segmentation and marking, and correcting and determining the segmentation and marking results; adding and updating the revised segmentation and labeling results to the training set; The transition image segmentation model is trained and updated according to the added and updated training set; When the segmentation marking result is determined to be accurate, the current transition image segmentation model is used as the image segmentation model. 3. The method according to claim 1, wherein the loss function of the regular constraint branch passes the KL between the posterior probability distribution q _φ (z|x) and the likelihood function p _θ (z|x) Divergence is calculated, where x is the input image, z is the latent variable in the variational autoencoder composed of the regular constraint branch and the encoder branch, z obeys the standard normal distribution, φ, θ correspond to the encoder respectively The parameters to be learned by the network in the branch and decoder branch. 4. The method according to claim 3, wherein the loss function L _VAE (θ, φ) is calculated by the following formula:

in,

represents the reconstruction mean square error, D _KL (q _φ (z|x)||p _θ (z)) represents the inverse KL dispersion of the prior distribution p _θ (z) and the posterior probability distribution q _φ (z|x) Spend. 5. The method of claim 1, wherein the loss function of the decoder is a binary cross-entropy loss. 6. The method according to claim 1, wherein the organ image data is brain MRI image data; Correspondingly, the training set obtained based on the segmentation and marking operation on the sample image data in the organ image data includes: Preprocess the organ image data to obtain organ image data with consistent grayscale density and/or grayscale range; Select multiple pre-processed organ image data as sample image data to receive segmentation and labeling operations to obtain a training set. 7. The method according to claim 6, wherein the grayscale density is preprocessed by bias field correction; the grayscale range is preprocessed by grayscale normalization. 8. An image segmentation device based on deep learning, characterized in that, comprising: a training set generation unit, configured to obtain a training set based on the segmentation and labeling operation on the sample image data in the organ image data, the organ image data is obtained by scanning the organ; A model training unit is used to generate an initial image segmentation model, and input the training set into the initial image segmentation model for model training to obtain an image segmentation model for image structure segmentation, the initial image segmentation model is in U-Net The central layer of the framework is obtained by adding a regular constraint branch parallel to the decoder branch of the U-Net framework, and the regular constraint branch is used to perform regular constraints on the encoder branch of the U-Net framework; The segmentation marking unit is used for inputting the organ image data to be segmented into the image segmentation model, so as to perform segmentation and marking on the organ image data to be segmented. 9. A terminal device, comprising: one or more processors; memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the deep learning-based image segmentation method according to any one of claims 1-7. 10. A computer-readable storage medium on which a computer program is stored, characterized in that, when the program is executed by a processor, the deep learning-based image segmentation method according to any one of claims 1-7 is implemented.

Images