WO2020238043A1 - Unet network-based lung lobe segmentation method and apparatus, computer readable storage medium - Google Patents
Unet network-based lung lobe segmentation method and apparatus, computer readable storage medium Download PDFInfo
- Publication number
- WO2020238043A1 WO2020238043A1 PCT/CN2019/117321 CN2019117321W WO2020238043A1 WO 2020238043 A1 WO2020238043 A1 WO 2020238043A1 CN 2019117321 W CN2019117321 W CN 2019117321W WO 2020238043 A1 WO2020238043 A1 WO 2020238043A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- lung
- area
- lobe
- image
- image data
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/11—Region-based segmentation
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/10—Segmentation; Edge detection
- G06T7/187—Segmentation; Edge detection involving region growing; involving region merging; involving connected component labelling
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10072—Tomographic images
- G06T2207/10081—Computed x-ray tomography [CT]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20081—Training; Learning
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/20—Special algorithmic details
- G06T2207/20084—Artificial neural networks [ANN]
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30061—Lung
Definitions
- the present invention relates to the technical field of lung lobe image processing, in particular to a method, device and computer-readable storage medium for lung lobe segmentation based on UNet network.
- the main purpose of the present invention is to provide a lung lobe segmentation method, device and computer readable storage medium based on UNet network, aiming to solve the problem that the existing lung lobe segmentation method is limited by individual lung morphological differences resulting in low efficiency and low accuracy of lung lobe segmentation Technical issues.
- the present invention provides a lung lobe segmentation device based on UNet network, including a processor suitable for implementing various computer program instructions and a memory suitable for storing multiple computer program instructions, characterized in that the computer program
- the instructions are loaded by the processor and execute the following steps: obtain lung CT image data from the image input device; normalize the input lung CT image data; use 2D UNet network to filter the processed lung CT image data Intrapulmonary area and extrapulmonary area, and the intrapulmonary area is regarded as the lung area candidate area; 3D is used for the lung area candidate area
- the UNet network segmented five lung lobe mask regions to obtain the upper left lobe, lower left lobe, upper right lobe, middle right lobe, and lower right lobe. Morphological processing was performed on the five lung lobe mask regions to obtain the final lung lobe segmentation results; The results of lung lobes segmentation are stored in the memory, or output and displayed on the monitor screen.
- the step of normalizing the input lung CT image data includes: preprocessing the input lung CT image data to limit the lung CT image data to [-1000, 400] The interval is then normalized to the interval [0,1] to exclude non-lung areas with higher brightness.
- the step of performing morphological processing on the mask regions of the five lung lobes respectively includes the following steps: removing the area outside the mask regions of the five lung lobes and filling the holes in the mask regions of the five lung lobes.
- the step of using the 2D UNet network to filter the intra-pulmonary area and the extra-pulmonary area on the processed image data, and using the intra-pulmonary area as the candidate lung area includes the following steps: combining the lung CT image and the lung area gold Standard images are all scaled to the size [256, 256]; input the scaled lung CT image and lung area gold standard image into the 2D UNet network to segment the lung area image including the lung area and the lung area; the 2D The lung area image obtained by UNet output is then scaled to the original image size, and each layer of lung area is expanded with a radius of 5 pixels; the bounding box of the lung area is extracted according to the size of the lung area, and the lungs are extracted from the bounding box Area candidate area.
- the step of segmenting five lung lobe mask regions using a 3D UNet network for the lung region candidate region includes the following steps: randomly cropping the lung CT images to obtain a size of [128, 128, 64], and After random change, rotation, upside down, and size change for image enhancement processing; the lung area image data is multiplied by the processed lung CT image data and the lung area image data is multiplied by the lung leaf gold standard image data and input into the 3D UNet network. Lung lobe segmentation is performed on the CT image of the lung; the segmented lung lobe image is restored to the original size to obtain 5 different lung lobe mask regions.
- the present invention also provides a lung lobe segmentation method based on UNet network, which is applied to a computer device.
- the lung lobe segmentation method based on UNet network includes the following steps: acquiring lung CT image data from an image input device; Normalize the lung CT image data; use 2D for the processed lung CT image data UNet network screens out the lung area and the lung area, and uses the lung area as the lung area candidate area; 3D is used for the lung area candidate area
- the UNet network segmented five lung lobe mask regions to obtain the upper left lobe, lower left lobe, upper right lobe, middle right lobe, and lower right lobe. Morphological processing was performed on the five lung lobe mask regions to obtain the final lung lobe segmentation results; The results of lung lobes segmentation are stored in the memory, or output and displayed on the monitor screen.
- the step of normalizing the input lung CT image data includes the following steps: preprocessing the input lung CT image data to limit the lung CT image data to [-1000, 400 ] This interval is then normalized to the interval [0,1] to exclude non-lung areas with higher brightness.
- the step of performing morphological processing on the mask regions of the five lung lobes respectively includes the following steps: removing the area outside the mask regions of the five lung lobes and filling the holes in the mask regions of the five lung lobes.
- the step of using the 2D UNet network to filter the intra-pulmonary area and the extra-pulmonary area on the processed image data, and using the intra-pulmonary area as the candidate lung area includes the following steps: combining the lung CT image and the lung area gold Standard images are all scaled to the size [256, 256]; input the scaled lung CT image and lung area gold standard image into the 2D UNet network to segment the lung area image including the lung area and the lung area; the 2D The lung area image obtained by UNet output is then scaled to the original image size, and each layer of lung area is expanded with a radius of 5 pixels; the bounding box of the lung area is extracted according to the size of the lung area, and the lungs are extracted from the bounding box Area candidate area.
- the step of segmenting five lung lobe mask regions using a 3D UNet network for the lung region candidate region includes the following steps: randomly cropping the lung CT images to obtain a size of [128, 128, 64], and After random change, rotation, upside down, and size change for image enhancement processing; the lung area image data is multiplied by the processed lung CT image data and the lung area image data is multiplied by the lung leaf gold standard image data and input into the 3D UNet network. Lung lobe segmentation is performed on the CT image of the lung; the segmented lung lobe image is restored to the original size to obtain 5 different lung lobe mask regions.
- the present invention also provides a computer-readable storage medium that stores a plurality of computer program instructions that are loaded by a processor of a computer device and execute the UNet network-based lung lobe The method steps of the segmentation method.
- the lung lobe segmentation method, device and computer-readable storage medium based on UNet network of the present invention can normalize lung CT images and use 2D UNet network to quickly obtain lung CT images. Accurately extract the lung area, remove the influence of noise outside the lung area, and then use the 3D UNet network to segment the lung lobes in the lung CT image to obtain five lung lobes mask areas, thereby efficiently controlling the accuracy and speed of lung lobes segmentation, and improving The efficiency of lung lobes segmentation is not limited to individual lung morphological differences.
- the invention realizes fast and accurate extraction of lung lobes through UNet network, locates the position of lung cancer, and provides medical guidance for the diagnosis and treatment of lung cancer for doctors.
- FIG. 1 is a block diagram of the structure of a preferred embodiment of a lung lobe segmentation device based on UNet network of the present invention
- FIG. 2 is a method flowchart of a preferred embodiment of a lung lobe segmentation method based on UNet network of the present invention
- Figure 3 is a schematic diagram of extracting lung regions from lung CT images using 2D UNet network
- Figure 4 is a schematic diagram of segmenting five lung lobes from a lung CT image using a 3D UNet network.
- Fig. 1 is a schematic structural diagram of a preferred embodiment of a lung lobe segmentation device based on UNet network of the present invention.
- the lung lobe segmentation device 1 based on the UNet network includes, but is not limited to, a memory 11 suitable for storing various computer program instructions, a processor 12 that executes various computer program instructions, and a display 13. Both the memory 11 and the display 13 are electrically connected to the processor 12 through an electrical connection line, and are connected to the processor 12 through a data bus for data transmission.
- the processor 12 can call the lung lobe segmentation program 10 based on the UNet network stored in the memory 11, and execute the lung CT image data input by the lung lobe segmentation program 10 from the image input device 2, and use the UNet network based on lung Partial CT image data segmented the lung lobes.
- the lung lobe segmentation device 1 may be a personal computer, a notebook computer, a server, and other computer devices installed with the lung lobe segmentation program 10 based on the UNet network of the present invention.
- the lung lobe segmentation device 1 is connected with an image input device 2.
- the image input device 2 can be a CT scanner, which can scan the human lungs to obtain lung CT images; the image input device 2 can also be a medical
- the image database stores CT images of the lungs of the human body.
- the lung lobe segmentation device 1 can obtain lung CT images from the image input device 2, and execute the lung lobe segmentation program 10 through the processor 12 to process the lung CT images, and use the UNet network to quickly and accurately segment the lung CT images. Lung area.
- the memory 11 includes at least one type of readable storage medium.
- the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), and magnetic memory. , Disks, CDs, etc.
- the memory 11 may be an internal storage unit of the lung lobe segmentation device 1 based on the UNet network, for example, the hard disk, read-only memory ROM, random access memory RAM, and electronic memory of the lung lobe segmentation device 1 based on UNet network. Erase and write memory EEPROM, flash memory FLASH or CD, etc.
- the memory 11 may also be an external storage device of the lung lobe segmentation device 1 based on the UNet network.
- a plug-in hard disk equipped on the lung lobe segmentation device 1 based on the UNet network a smart memory card (Smart Media Card, SMC), Secure Digital (Secure Digital, SD) card, flash card (Flash Card), etc.
- the memory 11 may also include both the internal storage unit of the lung lobe segmentation device 1 based on the UNet network and an external storage device.
- the memory 11 can be used not only to store application software and various data installed in the lung lobe segmentation device 1 based on the UNet network, for example, to store the program code of the lung lobe segmentation program 10 based on the UNet network, etc., but also to temporarily store Data to be output or to be output.
- the processor 12 may be a central processing unit (Central Processing Unit) in some embodiments.
- Central Processing Unit CPU
- controller a controller
- microcontroller a microprocessor
- the display 13 may be a touch display screen or a general LED display screen, which can display the results of lung lobe segmentation and the segmented lung lobe regions in different parts.
- the lung lobe segmentation program 10 based on the UNet network may also be divided into one or more modules, and one or more modules are stored in the memory 11 and processed by one or more
- the present invention is executed by the processor (the processor 12 in this embodiment).
- the module referred to in the present invention refers to a series of computer program instruction segments that can complete specific functions, and is used to describe the lung lobe segmentation program 10 based on the UNet network. The execution process in the lung lobe segmentation device 1 based on UNet network is described.
- the lung lobe segmentation program 10 based on the UNet network is composed of program modules composed of multiple computer program instructions, including, but not limited to, the lung image input module 101, the image data processing module 102, and the lung area
- the module referred to in the present invention refers to a series of computer program instruction segments that can be executed by the processor 12 of the lung lobe segmentation device 1 and can complete fixed functions, and are stored in the memory 11.
- the lung image input module 101 is used to obtain lung CT image data from the image input device 2.
- the image input device 2 may be a CT scanner, which can scan human lungs to obtain lung CT images; the image input device 2 may also be a medical image database, storing CT images of human lungs to be segmented .
- the image data processing module 102 is used for normalizing the input lung CT image data to exclude non-pulmonary regions with higher brightness.
- the image data processing module 102 preprocesses the input lung CT image data, limits the image data to the interval [-1000, 400], and then normalizes it to [0, 1] This interval is to exclude non-lung areas with higher brightness.
- the intra-pulmonary area screening module 103 is used to segment the intra-pulmonary area and the extra-pulmonary area using a 2D UNet network on the processed image data, and use the intra-pulmonary area as a candidate lung area.
- the 2D UNet network uses a multi-layer 2D convolutional layer as the main structure of the encoding-decoding network for detection and output.
- 2D The size used for UNet network input is [256, 256].
- the images input by the 2D UNet network are the lung CT image (a1) processed in step S12 and the gold standard image (a2) of the lung area outlined by the corresponding doctor, the lung CT image (a1) and the lung All the regional gold standard images (a2) need to be scaled to the size of [256, 256], and then input to the 2D UNet network to segment the lung region and the lung region segmentation results (a3).
- the lung area image obtained during UNet output is scaled back to the original image size, and each layer of lung area is expanded with a radius of 5 pixels, and the bounding box of the lung area is extracted according to the size of the lung area, and extracted from the bounding box The candidate area of the lung area (a3).
- the lung lobe segmentation module 104 is configured to use a 3D UNet network to screen and segment five lung lobe mask regions for the lung area candidate area to obtain the upper left lobe, the lower left lobe, the upper right lobe, the middle right lobe, and the lower right lobe area; in this embodiment
- the 3D UNet network adopts an encoding-decoding network with a multi-layer 3D convolutional layer as the main structure for detection and output.
- random cropping is used in the training process to obtain an input image of [128, 128, 64] size, which undergoes random change, rotation, upside down, and scale changes for image enhancement processing.
- the input image of the 3D UNet network uses the lung area image data output in step S23 multiplied by the lung CT image processed in step S12 (b1) and the lung area image data output in step S23 multiplied by the doctor's outline Gold standard image of lung leaf (b2); mixed loss function: Dice loss function and Focal are used during model training Loss function, where the Dice Loss function is used to measure the quality of the result in image segmentation, Focal The loss function is used to deal with category imbalance.
- a moving sliding window method can be used to obtain an image with a size of [128, 128, 64]. After the trained model is predicted, the original image can be restored to obtain 5 different lung lobe mask regions (b3).
- the lung lobe processing module 105 is used to perform morphological processing on the five lung lobes mask regions to obtain the final lung lobe segmentation results; in this embodiment, the five lung lobes mask regions are respectively subjected to morphological processing
- the steps include: removing the area outside the mask area of the five lung lobes and filling the holes in the mask area of the five lung lobes to obtain the final lung lobe segmentation result.
- the lung lobe segmentation result output module 106 is used to store the lung lobe segmentation result in the memory 11 or output and display it on the screen of the display 13.
- the segmented images of different lung lobes are displayed on the screen of the display 13 for doctors to provide a more comprehensive reference in disease diagnosis and treatment.
- FIG. 2 it is a flowchart of a preferred embodiment of a lung lobe segmentation method based on UNet network of the present invention.
- the various method steps of the lung lobe segmentation method based on the UNet network are implemented by a computer software program, which is stored in a computer-readable storage medium in the form of computer program instructions (for example, in this embodiment)
- the computer-readable storage medium may include: read-only memory, random access memory, magnetic or optical disk, etc., and the computer program instructions can be loaded by a processor (for example, the processor 12 in this embodiment) and execute the following steps:
- Step S21 Acquire CT image data of the lung from the image input device 2.
- the image input device 2 may be a CT scanner, which can scan human lungs to obtain lung CT images; it may also be a medical image database that stores CT images of human lungs to be segmented.
- Step S22 Perform normalization processing on the input lung CT image data to exclude non-pulmonary regions with higher brightness.
- the image data processing module 102 preprocesses the input lung CT image data, limits the image data to the interval [-1000, 400], and then normalizes it to [0, 1] This interval is to exclude non-lung areas with higher brightness.
- Step S23 Use a 2D UNet network to segment the processed image data into the lung area and the lung area, and use the lung area as a candidate lung area.
- the 2D UNet network uses a multi-layer 2D convolutional layer as the main structure of the encoding-decoding network for detection and output.
- 2D The size used for UNet network input is [256, 256].
- the images input by the 2D UNet network are the lung CT image (a1) processed in step S12 and the gold standard image (a2) of the lung area outlined by the corresponding doctor, the lung CT image (a1) and the lung area All the gold standard images (a2) need to be scaled to the size of [256, 256], and then input into the 2D UNet network to segment the lung region and the lung region segmentation results (a3).
- the lung area image obtained during UNet output is scaled back to the original image size, and each layer of lung area is expanded with a radius of 5 pixels, and the bounding box of the lung area is extracted according to the size of the lung area, and extracted from the bounding box The candidate area of the lung area (a3).
- Step S24 Use the 3D UNet network to screen and segment the five lung lobe mask regions for the lung area candidate region to obtain the regions of the upper left lobe, the lower left lobe, the upper right lobe, the middle right lobe, and the lower right lobe; in this embodiment, the 3D UNet
- the network uses a multi-layer 3D convolutional layer as the main structure of the encoding-decoding network for detection and output.
- random cropping is used in the training process to obtain an input image of [128, 128, 64] size, which undergoes random change, rotation, upside down, and scale changes for image enhancement processing. Input to the 3D UNet network for network model training.
- the input image of the 3D UNet network uses the lung area image data output in step S23 multiplied by the lung CT image processed in step S12 (b1) and the lung area image data output in step S23 multiplied by the doctor's outline Gold standard image of lung leaf (b2); mixed loss function: Dice loss function and Focal are used during model training Loss function, where the Dice Loss function is used to measure the quality of the result in image segmentation, Focal The loss function is used to deal with category imbalance.
- a moving sliding window method can be used to obtain an image with a size of [128, 128, 64]. After the trained model is predicted, the original image can be restored to obtain 5 different lung lobe mask regions (b3).
- Step S25 Perform morphological processing on the five lung lobes mask regions to obtain the final lung lobe segmentation results; in this embodiment, the step of performing morphological processing on the mask regions of the five lung lobes includes: removing five The area outside the mask area of the lung lobes, and the holes that fill the mask area of the five lung lobes, obtain the final lung lobe segmentation result.
- Step S26 is to store the result of the lung lobes segmentation in the memory 11 or output and display it on the screen of the display 13.
- the segmented images of different lung lobes are displayed on the screen of the display 13 for doctors to provide a more comprehensive reference in disease diagnosis and treatment.
- the present invention also provides a computer-readable storage medium that stores a plurality of computer program instructions, and the computer program instructions are loaded by a processor of a computer device and execute the method of lung lobe segmentation based on the UNet network of the present invention.
- Various steps Those skilled in the art can understand that all or part of the steps of the various methods in the foregoing embodiments can be completed by related program instructions.
- the program can be stored in a computer-readable storage medium.
- the storage medium may include: read-only memory, random access memory, Disk or CD, etc.
- the lung lobe segmentation method, device and computer-readable storage medium based on UNet network of the present invention can normalize lung CT images, and use 2D UNet network to quickly and accurately extract lung regions from lung CT images, and remove After the influence of noise outside the lung area, the 3D UNet network is used to segment the lung lobes in the lung CT image to obtain five lung lobes mask regions, thereby efficiently controlling the accuracy and speed of lung lobes segmentation, improving the efficiency of segmentation and not being affected. Limited to differences in individual lung morphology.
- the invention realizes fast and accurate extraction of lung lobes through UNet network, locates the position of lung cancer, and provides medical guidance for the diagnosis and treatment of lung cancer for doctors.
- the lung lobe segmentation method, device and computer-readable storage medium based on UNet network of the present invention can normalize lung CT images and use 2D UNet network to quickly obtain lung CT images. Accurately extract the lung area, remove the influence of noise outside the lung area, and then use the 3D UNet network to segment the lung lobes in the lung CT image to obtain five lung lobes mask areas, thereby efficiently controlling the accuracy and speed of lung lobes segmentation, and improving The efficiency of lung lobes segmentation is not limited to individual lung morphological differences.
- the invention realizes fast and accurate extraction of lung lobes through UNet network, locates the position of lung cancer, and provides medical guidance for the diagnosis and treatment of lung cancer for doctors.
Landscapes
- Engineering & Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Physics & Mathematics (AREA)
- General Physics & Mathematics (AREA)
- Theoretical Computer Science (AREA)
- Apparatus For Radiation Diagnosis (AREA)
- Image Processing (AREA)
Abstract
Description
本发明涉及肺叶影像处理的技术领域,尤其涉及一种基于UNet网络的肺叶分割方法、装置及计算机可读存储介质。The present invention relates to the technical field of lung lobe image processing, in particular to a method, device and computer-readable storage medium for lung lobe segmentation based on UNet network.
随着CT的应用普及,为肺癌的早期筛查提供了便利。近年来的统计发现,肺癌的发病率越来越高,同时也是癌症致死率的首要原因,外科切除肺叶是治疗肺癌的首要方法。传统的肺叶分割算法一般是利用灰度信息寻找肺裂,再根据肺裂和其他的解剖学信息(气管和血管)提取出肺叶,这种方法不仅计算量大,而且很多肺裂由于疾病的影响,模糊不清,很难找全所有切片中的肺裂,造成肺叶提取困难。如何通过UNet网络实现快速、准确地提取肺叶,定位肺癌位置,为医生的诊断治疗提供指导成为业界的研究重点。With the popularization of CT applications, it facilitates the early screening of lung cancer. Statistics in recent years have found that the incidence of lung cancer is getting higher and higher, and it is also the primary cause of cancer mortality. Surgical removal of lung lobes is the primary method of treatment for lung cancer. Traditional lung lobes segmentation algorithms generally use gray information to find lung fissures, and then extract lung lobes based on lung fissures and other anatomical information (trachea and blood vessels). This method is not only computationally expensive, but many lung fissures are affected by diseases. , Illegible, it is difficult to find the lung fissures in all the slices, making it difficult to extract the lung lobe. How to extract lung lobes quickly and accurately through UNet network, locate lung cancer location, and provide guidance for doctors' diagnosis and treatment has become the research focus of the industry.
本发明的主要目的在于提供一种基于UNet网络的肺叶分割方法、装置及计算机可读存储介质,旨在解决现有肺叶分割方法受限于个体肺部形态差异造成肺叶分割效率低且准确度低的技术问题。The main purpose of the present invention is to provide a lung lobe segmentation method, device and computer readable storage medium based on UNet network, aiming to solve the problem that the existing lung lobe segmentation method is limited by individual lung morphological differences resulting in low efficiency and low accuracy of lung lobe segmentation Technical issues.
为实现上述目的,本发明提供一种基于UNet网络的肺叶分割装置,包括适于实现各种计算机程序指令的处理器以及适于存储多条计算机程序指令的存储器,其特征在于,所述计算机程序指令由处理器加载并执行如下步骤:从影像输入设备获取肺部CT影像数据;对输入的肺部CT影像数据进行归一化处理;对处理后的肺部CT影像数据利用2D UNet网络筛选出肺内区域和肺外区域,并将肺内区域作为肺区候选区域;对肺区候选区域利用3D UNet网络分割出五个肺叶掩膜区域,得到左上叶、左下叶、右上叶、右中叶以及右下叶的区域;对五个肺叶掩膜区域分别进行形态学处理,得到最终的肺叶分割结果;将肺叶分割结果存储在存储器中,或者输出并显示在显示器的屏幕上。In order to achieve the above objective, the present invention provides a lung lobe segmentation device based on UNet network, including a processor suitable for implementing various computer program instructions and a memory suitable for storing multiple computer program instructions, characterized in that the computer program The instructions are loaded by the processor and execute the following steps: obtain lung CT image data from the image input device; normalize the input lung CT image data; use 2D UNet network to filter the processed lung CT image data Intrapulmonary area and extrapulmonary area, and the intrapulmonary area is regarded as the lung area candidate area; 3D is used for the lung area candidate area The UNet network segmented five lung lobe mask regions to obtain the upper left lobe, lower left lobe, upper right lobe, middle right lobe, and lower right lobe. Morphological processing was performed on the five lung lobe mask regions to obtain the final lung lobe segmentation results; The results of lung lobes segmentation are stored in the memory, or output and displayed on the monitor screen.
优选地,所述对输入的肺部CT影像数据进行归一化处理的步骤包括:对输入的肺部CT影像数据进行预处理,将肺部CT影像数据的限定到[-1000,400]这个区间,然后归一化到[0,1]这个区间,以排除亮度较高的非肺部区域。Preferably, the step of normalizing the input lung CT image data includes: preprocessing the input lung CT image data to limit the lung CT image data to [-1000, 400] The interval is then normalized to the interval [0,1] to exclude non-lung areas with higher brightness.
优选地,所述对五个肺叶的掩膜区域分别进行形态学处理的步骤包括如下步骤:去除五个肺叶掩膜区域外的区域,并填充五个肺叶掩膜区域的孔洞。Preferably, the step of performing morphological processing on the mask regions of the five lung lobes respectively includes the following steps: removing the area outside the mask regions of the five lung lobes and filling the holes in the mask regions of the five lung lobes.
优选地,所述对处理后的影像数据利用2D UNet网络筛选出肺内区域和肺外区域,并将肺内区域作为肺区候选区域的步骤包括如下步骤:将肺部CT影像和肺区金标准图像均缩放到尺寸大小为[256,256];将缩放后的肺部CT影像和肺区金标准图像输入到2D UNet网络分割出包括肺内区域和肺外区域的肺区图像;将2D UNet输出得到的肺区图像再缩放至原图大小,并对每层肺区利用半径为5个像素的大小进行膨胀;根据肺区大小提取肺部区域的边界框,并从边界框中提取肺区候选区域。Preferably, the step of using the 2D UNet network to filter the intra-pulmonary area and the extra-pulmonary area on the processed image data, and using the intra-pulmonary area as the candidate lung area includes the following steps: combining the lung CT image and the lung area gold Standard images are all scaled to the size [256, 256]; input the scaled lung CT image and lung area gold standard image into the 2D UNet network to segment the lung area image including the lung area and the lung area; the 2D The lung area image obtained by UNet output is then scaled to the original image size, and each layer of lung area is expanded with a radius of 5 pixels; the bounding box of the lung area is extracted according to the size of the lung area, and the lungs are extracted from the bounding box Area candidate area.
优选地,所述对肺区候选区域利用3D UNet网络分割出五个肺叶掩膜区域的步骤包括如下步骤:对肺部CT影像采用随机裁剪法得到尺寸大小为[128,128,64],并经过随机变化、旋转、上下翻转、尺寸变化做影像增强处理;将肺区影像数据乘以处理过的肺部CT影像数据和肺区影像数据乘以肺叶金标准图像数据输入到3D UNet网络中对肺部CT影像进行肺叶分割;将分割出的肺叶图像还原至原图大小得到5个不同的肺叶掩膜区域。Preferably, the step of segmenting five lung lobe mask regions using a 3D UNet network for the lung region candidate region includes the following steps: randomly cropping the lung CT images to obtain a size of [128, 128, 64], and After random change, rotation, upside down, and size change for image enhancement processing; the lung area image data is multiplied by the processed lung CT image data and the lung area image data is multiplied by the lung leaf gold standard image data and input into the 3D UNet network. Lung lobe segmentation is performed on the CT image of the lung; the segmented lung lobe image is restored to the original size to obtain 5 different lung lobe mask regions.
另一方面,本发明还提供一种基于UNet网络的肺叶分割方法,应用于计算机装置中,所述基于UNet网络的肺叶分割方法包括如下步骤:从影像输入设备获取肺部CT影像数据;对输入的肺部CT影像数据进行归一化处理;对处理后的肺部CT影像数据利用2D UNet网络筛选出肺内区域和肺外区域,并将肺内区域作为肺区候选区域;对肺区候选区域利用3D UNet网络分割出五个肺叶掩膜区域,得到左上叶、左下叶、右上叶、右中叶以及右下叶的区域;对五个肺叶掩膜区域分别进行形态学处理,得到最终的肺叶分割结果;将肺叶分割结果存储在存储器中,或者输出并显示在显示器的屏幕上。On the other hand, the present invention also provides a lung lobe segmentation method based on UNet network, which is applied to a computer device. The lung lobe segmentation method based on UNet network includes the following steps: acquiring lung CT image data from an image input device; Normalize the lung CT image data; use 2D for the processed lung CT image data UNet network screens out the lung area and the lung area, and uses the lung area as the lung area candidate area; 3D is used for the lung area candidate area The UNet network segmented five lung lobe mask regions to obtain the upper left lobe, lower left lobe, upper right lobe, middle right lobe, and lower right lobe. Morphological processing was performed on the five lung lobe mask regions to obtain the final lung lobe segmentation results; The results of lung lobes segmentation are stored in the memory, or output and displayed on the monitor screen.
优选地,所述对输入的肺部CT影像数据进行归一化处理的步骤包括如下步骤:对输入的肺部CT影像数据进行预处理,将肺部CT影像数据的限定到[-1000,400]这个区间,然后归一化到[0,1]这个区间,以排除亮度较高的非肺部区域。Preferably, the step of normalizing the input lung CT image data includes the following steps: preprocessing the input lung CT image data to limit the lung CT image data to [-1000, 400 ] This interval is then normalized to the interval [0,1] to exclude non-lung areas with higher brightness.
优选地,所述对五个肺叶的掩膜区域分别进行形态学处理的步骤包括如下步骤:去除五个肺叶掩膜区域外的区域,并填充五个肺叶掩膜区域的孔洞。Preferably, the step of performing morphological processing on the mask regions of the five lung lobes respectively includes the following steps: removing the area outside the mask regions of the five lung lobes and filling the holes in the mask regions of the five lung lobes.
优选地,所述对处理后的影像数据利用2D UNet网络筛选出肺内区域和肺外区域,并将肺内区域作为肺区候选区域的步骤包括如下步骤:将肺部CT影像和肺区金标准图像均缩放到尺寸大小为[256,256];将缩放后的肺部CT影像和肺区金标准图像输入到2D UNet网络分割出包括肺内区域和肺外区域的肺区图像;将2D UNet输出得到的肺区图像再缩放至原图大小,并对每层肺区利用半径为5个像素的大小进行膨胀;根据肺区大小提取肺部区域的边界框,并从边界框中提取肺区候选区域。Preferably, the step of using the 2D UNet network to filter the intra-pulmonary area and the extra-pulmonary area on the processed image data, and using the intra-pulmonary area as the candidate lung area includes the following steps: combining the lung CT image and the lung area gold Standard images are all scaled to the size [256, 256]; input the scaled lung CT image and lung area gold standard image into the 2D UNet network to segment the lung area image including the lung area and the lung area; the 2D The lung area image obtained by UNet output is then scaled to the original image size, and each layer of lung area is expanded with a radius of 5 pixels; the bounding box of the lung area is extracted according to the size of the lung area, and the lungs are extracted from the bounding box Area candidate area.
优选地,所述对肺区候选区域利用3D UNet网络分割出五个肺叶掩膜区域的步骤包括如下步骤:对肺部CT影像采用随机裁剪法得到尺寸大小为[128,128,64],并经过随机变化、旋转、上下翻转、尺寸变化做影像增强处理;将肺区影像数据乘以处理过的肺部CT影像数据和肺区影像数据乘以肺叶金标准图像数据输入到3D UNet网络中对肺部CT影像进行肺叶分割;将分割出的肺叶图像还原至原图大小得到5个不同的肺叶掩膜区域。Preferably, the step of segmenting five lung lobe mask regions using a 3D UNet network for the lung region candidate region includes the following steps: randomly cropping the lung CT images to obtain a size of [128, 128, 64], and After random change, rotation, upside down, and size change for image enhancement processing; the lung area image data is multiplied by the processed lung CT image data and the lung area image data is multiplied by the lung leaf gold standard image data and input into the 3D UNet network. Lung lobe segmentation is performed on the CT image of the lung; the segmented lung lobe image is restored to the original size to obtain 5 different lung lobe mask regions.
另一方面,本发明还提供一种计算机可读存储介质,该计算机可读存储介质存储多条计算机程序指令,所述计算机程序指令由计算机装置的处理器加载并执行所述基于UNet网络的肺叶分割方法的各项方法步骤。In another aspect, the present invention also provides a computer-readable storage medium that stores a plurality of computer program instructions that are loaded by a processor of a computer device and execute the UNet network-based lung lobe The method steps of the segmentation method.
相较于现有技术,本发明所述基于UNet网络的肺叶分割方法、装置及计算机可读存储介质,能够通过对肺部CT图像进行归一化处理,采用2D UNet网络从肺部CT图像快速准确地提取肺区,去除肺区外噪声的影响,然后采用3D UNet网络对肺部CT图像中的肺叶进行分割,得到五个肺叶掩膜区域,从而高效地控制肺叶分割准确度和速度,提高了肺叶分割的效率且不受限于个体肺部形态差异。本发明通过UNet网络实现快速、准确地提取肺叶,定位肺癌位置,为医生对肺癌的诊断治疗提供医学指导。Compared with the prior art, the lung lobe segmentation method, device and computer-readable storage medium based on UNet network of the present invention can normalize lung CT images and use 2D UNet network to quickly obtain lung CT images. Accurately extract the lung area, remove the influence of noise outside the lung area, and then use the 3D UNet network to segment the lung lobes in the lung CT image to obtain five lung lobes mask areas, thereby efficiently controlling the accuracy and speed of lung lobes segmentation, and improving The efficiency of lung lobes segmentation is not limited to individual lung morphological differences. The invention realizes fast and accurate extraction of lung lobes through UNet network, locates the position of lung cancer, and provides medical guidance for the diagnosis and treatment of lung cancer for doctors.
图1是本发明基于UNet网络的肺叶分割装置的较佳实施例的结构方框示意图;1 is a block diagram of the structure of a preferred embodiment of a lung lobe segmentation device based on UNet network of the present invention;
图2是本发明基于UNet网络的肺叶分割方法较佳实施例的方法流程图;2 is a method flowchart of a preferred embodiment of a lung lobe segmentation method based on UNet network of the present invention;
图3是采用2D UNet网络从肺部CT图像提取肺区的示意图;Figure 3 is a schematic diagram of extracting lung regions from lung CT images using 2D UNet network;
图4是采用3D UNet网络从肺部CT图像分割出五个肺叶的示意图。Figure 4 is a schematic diagram of segmenting five lung lobes from a lung CT image using a 3D UNet network.
本发明目的的实现、功能特点及优点将结合实施例,参照附图做进一步说明。The realization of the objectives, functional characteristics and advantages of the present invention will be further described in conjunction with the embodiments and with reference to the accompanying drawings.
为更进一步阐述本发明为达成预定发明目的所采取的技术手段及功效,以下结合附图及较佳实施例,对本发明的具体实施方式、结构、特征及其功效,详细说明如下。应当理解,此处所描述的具体实施例仅仅用以解释本发明,并不用于限定本发明。In order to further explain the technical means and effects of the present invention to achieve the intended purpose of the invention, the specific implementation, structure, features and effects of the present invention will be described in detail below with reference to the drawings and preferred embodiments. It should be understood that the specific embodiments described here are only used to explain the present invention, but not to limit the present invention.
参照图1所示,图1是本发明基于UNet网络的肺叶分割装置的较佳实施例的结构示意图。在本实施例中,所述基于UNet网络的肺叶分割装置1包括,但不仅限于,适于存储各种计算机程序指令的存储器11、执行各种计算机程序指令的处理器12以及显示器13。所述存储器11和显示器13均通过电连接线与所述处理器12进行电气连接,并通过数据总线与处理器12进行数据传输连接。所述处理器12能够调用存储在所述存储器11中的基于UNet网络的肺叶分割程序10,并执行该肺叶分割程序10从影像输入设备2输入的肺部CT影像数据,并利用UNet网络基于肺部CT影像数据对肺叶进行分割。所述肺叶分割装置1可以为安装有本发明所述基于UNet网络的肺叶分割程序10的个人计算机、笔记本电脑、服务器等计算机装置。Referring to Fig. 1, Fig. 1 is a schematic structural diagram of a preferred embodiment of a lung lobe segmentation device based on UNet network of the present invention. In this embodiment, the lung lobe segmentation device 1 based on the UNet network includes, but is not limited to, a memory 11 suitable for storing various computer program instructions, a processor 12 that executes various computer program instructions, and a display 13. Both the memory 11 and the display 13 are electrically connected to the processor 12 through an electrical connection line, and are connected to the processor 12 through a data bus for data transmission. The processor 12 can call the lung lobe segmentation program 10 based on the UNet network stored in the memory 11, and execute the lung CT image data input by the lung lobe segmentation program 10 from the image input device 2, and use the UNet network based on lung Partial CT image data segmented the lung lobes. The lung lobe segmentation device 1 may be a personal computer, a notebook computer, a server, and other computer devices installed with the lung lobe segmentation program 10 based on the UNet network of the present invention.
在本实施例中,所述肺叶分割装置1连接有影像输入设备2,该影像输入设备2可以为CT扫描仪,能够扫描人体肺部得到肺部CT影像;该影像输入设备2也可以是医疗影像数据库,存储有人体的肺部CT影像。所述肺叶分割装置1能够从影像输入设备2获取肺部CT影像,并通过处理器12执行肺叶分割程序10对肺部CT影像进行处理,利用UNet网络对肺部CT影像快速准确地分割出不同的肺区。In this embodiment, the lung lobe segmentation device 1 is connected with an image input device 2. The image input device 2 can be a CT scanner, which can scan the human lungs to obtain lung CT images; the image input device 2 can also be a medical The image database stores CT images of the lungs of the human body. The lung lobe segmentation device 1 can obtain lung CT images from the image input device 2, and execute the lung lobe segmentation program 10 through the processor 12 to process the lung CT images, and use the UNet network to quickly and accurately segment the lung CT images. Lung area.
在本实施例中,所述存储器11至少包括一种类型的可读存储介质,所述可读存储介质包括闪存、硬盘、多媒体卡、卡型存储器(例如,SD或DX存储器等)、磁性存储器、磁盘、光盘等。所述存储器11在一些实施例中可以是所述基于UNet网络的肺叶分割装置1的内部存储单元,例如该基于UNet网络的肺叶分割装置1的硬盘、只读存储器ROM,随机存储器RAM、电可擦写存储器EEPROM、快闪存储器FLASH或光盘等。所述存储器11在另一些实施例中也可以是基于UNet网络的肺叶分割装置1的外部存储设备,例如该基于UNet网络的肺叶分割装置1上配备的插接式硬盘,智能存储卡(Smart Media Card, SMC),安全数字(Secure Digital, SD)卡,闪存卡(Flash Card)等。进一步地,所述存储器11还可以既包括基于UNet网络的肺叶分割装置1的内部存储单元也包括外部存储设备。所述存储器11不仅可以用于存储安装于基于UNet网络的肺叶分割装置1的应用软件及各类数据,例如存储基于UNet网络的肺叶分割程序10的程序代码等,还可以用于暂时地存储已经输出或者将要输出的数据。In this embodiment, the memory 11 includes at least one type of readable storage medium. The readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or DX memory, etc.), and magnetic memory. , Disks, CDs, etc. In some embodiments, the memory 11 may be an internal storage unit of the lung lobe segmentation device 1 based on the UNet network, for example, the hard disk, read-only memory ROM, random access memory RAM, and electronic memory of the lung lobe segmentation device 1 based on UNet network. Erase and write memory EEPROM, flash memory FLASH or CD, etc. In other embodiments, the memory 11 may also be an external storage device of the lung lobe segmentation device 1 based on the UNet network. For example, a plug-in hard disk equipped on the lung lobe segmentation device 1 based on the UNet network, a smart memory card (Smart Media Card, SMC), Secure Digital (Secure Digital, SD) card, flash card (Flash Card), etc. Further, the memory 11 may also include both the internal storage unit of the lung lobe segmentation device 1 based on the UNet network and an external storage device. The memory 11 can be used not only to store application software and various data installed in the lung lobe segmentation device 1 based on the UNet network, for example, to store the program code of the lung lobe segmentation program 10 based on the UNet network, etc., but also to temporarily store Data to be output or to be output.
在本实施例中,所述处理器12在一些实施例中可以是一中央处理器(Central Processing Unit, CPU)、控制器、微控制器、微处理器或其他数据处理芯片,用于调用并运行存储器11中存储的程序代码或处理数据,例如执行基于UNet网络的肺叶分割程序10等。所述显示器13可以为触摸显示屏也可以为通用的LED显示屏,能够显示肺叶分割结果,分割出的不同部位的肺叶区域。In this embodiment, the processor 12 may be a central processing unit (Central Processing Unit) in some embodiments. Processing Unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips are used to call and run the program code or processing data stored in the memory 11, such as executing the lung lobe segmentation program 10 based on the UNet network. The display 13 may be a touch display screen or a general LED display screen, which can display the results of lung lobe segmentation and the segmented lung lobe regions in different parts.
可选地,在其他实施例中,所述基于UNet网络的肺叶分割程序10还可以被分割为一个或者多个模块,一个或者多个模块被存储于存储器11中,并由一个或多个处理器(本实施例为处理器12)所执行以完成本发明,本发明所称的模块是指能够完成特定功能的一系列计算机程序指令段,用于描述基于UNet网络的肺叶分割程序10在所述基于UNet网络的肺叶分割装置1中的执行过程。Optionally, in other embodiments, the lung lobe segmentation program 10 based on the UNet network may also be divided into one or more modules, and one or more modules are stored in the memory 11 and processed by one or more The present invention is executed by the processor (the processor 12 in this embodiment). The module referred to in the present invention refers to a series of computer program instruction segments that can complete specific functions, and is used to describe the lung lobe segmentation program 10 based on the UNet network. The execution process in the lung lobe segmentation device 1 based on UNet network is described.
在本实施例中,所述基于UNet网络的肺叶分割程序10由多条计算机程序指令组成的程序模块组成,包括但不局限于,肺部影像输入模块101、影像数据处理模块102、肺内区域筛选模块103、肺叶分割模块104、肺叶处理模块105以及肺叶分割结果输出模块106。本发明所称的模块是指一种能够被肺叶分割装置1的处理器12执行并且能够完成固定功能的一系列计算机程序指令段,其存储在存储器11中。In this embodiment, the lung lobe segmentation program 10 based on the UNet network is composed of program modules composed of multiple computer program instructions, including, but not limited to, the lung image input module 101, the image data processing module 102, and the lung area The screening module 103, the lung lobe segmentation module 104, the lung lobe processing module 105, and the lung lobe segmentation result output module 106. The module referred to in the present invention refers to a series of computer program instruction segments that can be executed by the processor 12 of the lung lobe segmentation device 1 and can complete fixed functions, and are stored in the memory 11.
所述肺部影像输入模块101用于从影像输入设备2获取肺部CT影像数据。在本实施例中,所述影像输入设备2可以为CT扫描仪,能够扫描人体肺部得到肺部CT影像;该影像输入设备2也可以为医疗影像数据库,存储有待分割的人体肺部CT影像。The lung image input module 101 is used to obtain lung CT image data from the image input device 2. In this embodiment, the image input device 2 may be a CT scanner, which can scan human lungs to obtain lung CT images; the image input device 2 may also be a medical image database, storing CT images of human lungs to be segmented .
所述影像数据处理模块102用于对输入的肺部CT影像数据进行归一化处理,以排除亮度较高的非肺部区域。在本实施例中,所述影像数据处理模块102对输入的肺部CT影像数据进行预处理,将影像数据的限定到[-1000,400]这个区间,然后归一化到[0,1]这个区间,以排除亮度较高的非肺部区域。The image data processing module 102 is used for normalizing the input lung CT image data to exclude non-pulmonary regions with higher brightness. In this embodiment, the image data processing module 102 preprocesses the input lung CT image data, limits the image data to the interval [-1000, 400], and then normalizes it to [0, 1] This interval is to exclude non-lung areas with higher brightness.
所述肺内区域筛选模块103用于对处理后的影像数据利用2D UNet网络分割出肺内区域和肺外区域,并将肺内区域作为肺区候选区域。在本实施例中,所述2D UNet网络采用的是多层2D卷积层为主体结构的编码-解码网络进行检测和输出,为了保证图像能正确输入到2D UNet网络中,2D UNet网络输入使用的尺寸是[256,256]。如图3所示,2D UNet网络输入的图像为经步骤S12处理过的肺部CT影像(a1)和对应的医生勾画的肺区金标准图像(a2),肺部CT影像(a1)和肺区金标准图像(a2)均需要全部缩放到[256,256]这个尺寸,再输入到2D UNet网络分割出肺内区域和肺外区域的肺区分割结果(a3)。在2D UNet输出时得到的肺部区域图像再缩放回原来图像的大小,对每层肺区利用半径为5个像素的大小进行膨胀,根据肺区大小提取肺部区域的边界框,从边界框中提取出肺区候选区域(a3)。The intra-pulmonary area screening module 103 is used to segment the intra-pulmonary area and the extra-pulmonary area using a 2D UNet network on the processed image data, and use the intra-pulmonary area as a candidate lung area. In this embodiment, the 2D UNet network uses a multi-layer 2D convolutional layer as the main structure of the encoding-decoding network for detection and output. In order to ensure that the image can be correctly input into the 2D UNet network, 2D The size used for UNet network input is [256, 256]. As shown in Figure 3, the images input by the 2D UNet network are the lung CT image (a1) processed in step S12 and the gold standard image (a2) of the lung area outlined by the corresponding doctor, the lung CT image (a1) and the lung All the regional gold standard images (a2) need to be scaled to the size of [256, 256], and then input to the 2D UNet network to segment the lung region and the lung region segmentation results (a3). In 2D The lung area image obtained during UNet output is scaled back to the original image size, and each layer of lung area is expanded with a radius of 5 pixels, and the bounding box of the lung area is extracted according to the size of the lung area, and extracted from the bounding box The candidate area of the lung area (a3).
所述肺叶分割模块104用于对肺区候选区域利用3D UNet网络筛分割五个肺叶掩膜区域,得到左上叶、左下叶、右上叶、右中叶以及右下叶的区域;在本实施例中,所述3D UNet网络采用的是多层3D卷积层为主体结构的编码-解码网络进行检测和输出。为了保证图像能正确输入到3D UNet网络中,训练过程中采用随机裁剪的方法得到[128,128,64]大小的输入图像,经过随机变化、旋转、上下翻转、尺度变化等做影像增强处理,输入到3D UNet网络中进行网络模型训练。如图4所示,3D UNet网络的输入图像使用的是S23步骤输出的肺区影像数据乘以S12步骤处理过的肺部CT影像(b1)和S23步骤输出的肺区影像数据乘以医生勾画的肺叶金标准图像(b2);模型训练时使用了混合损失函数:Dice loss函数和Focal loss函数,其中,Dice Loss函数用于图像分割中衡量结果的好坏,Focal loss函数用于处理类别不均衡。测试阶段可以采用移动滑窗的方法得到[128,128,64]大小的图像,经过训练出的模型预测后,还原回原图可得到5个不同的肺叶掩膜区域(b3)。The lung lobe segmentation module 104 is configured to use a 3D UNet network to screen and segment five lung lobe mask regions for the lung area candidate area to obtain the upper left lobe, the lower left lobe, the upper right lobe, the middle right lobe, and the lower right lobe area; in this embodiment The 3D UNet network adopts an encoding-decoding network with a multi-layer 3D convolutional layer as the main structure for detection and output. In order to ensure that the image can be correctly input into the 3D UNet network, random cropping is used in the training process to obtain an input image of [128, 128, 64] size, which undergoes random change, rotation, upside down, and scale changes for image enhancement processing. Input to the 3D UNet network for network model training. As shown in Figure 4, the input image of the 3D UNet network uses the lung area image data output in step S23 multiplied by the lung CT image processed in step S12 (b1) and the lung area image data output in step S23 multiplied by the doctor's outline Gold standard image of lung leaf (b2); mixed loss function: Dice loss function and Focal are used during model training Loss function, where the Dice Loss function is used to measure the quality of the result in image segmentation, Focal The loss function is used to deal with category imbalance. In the test phase, a moving sliding window method can be used to obtain an image with a size of [128, 128, 64]. After the trained model is predicted, the original image can be restored to obtain 5 different lung lobe mask regions (b3).
所述肺叶处理模块105用于对五个肺叶掩膜区域分别进行形态学处理,得到最终的肺叶分割结果;在本实施例中,所述对五个肺叶的掩膜区域分别进行形态学处理的步骤包括:去除五个肺叶掩膜区域外的区域,以及填充五个肺叶掩膜区域的孔洞,得到最终的肺叶分割结果。The lung lobe processing module 105 is used to perform morphological processing on the five lung lobes mask regions to obtain the final lung lobe segmentation results; in this embodiment, the five lung lobes mask regions are respectively subjected to morphological processing The steps include: removing the area outside the mask area of the five lung lobes and filling the holes in the mask area of the five lung lobes to obtain the final lung lobe segmentation result.
所述肺叶分割结果输出模块106用于将肺叶分割结果存储在存储器11中,或者输出并显示在显示器13的屏幕上。在本实施例中,将分割出的不同的肺叶影像显示在显示器13的屏幕上,以供医生在疾病诊断治疗方面提供更加全面的参考。The lung lobe segmentation result output module 106 is used to store the lung lobe segmentation result in the memory 11 or output and display it on the screen of the display 13. In this embodiment, the segmented images of different lung lobes are displayed on the screen of the display 13 for doctors to provide a more comprehensive reference in disease diagnosis and treatment.
参考图2所示,是本发明基于UNet网络的肺叶分割方法较佳实施例的流程图。在本实施例中,所述基于UNet网络的肺叶分割方法的各种方法步骤通过计算机软件程序来实现,该计算机软件程序以计算机程序指令的形式存储于计算机可读存储介质(例如本实施例的存储器11)中,计算机可读存储介质可以包括:只读存储器、随机存储器、磁盘或光盘等,所述计算机程序指令能够被处理器(例如本实施例的处理器12)加载并执行如下步骤:Referring to FIG. 2, it is a flowchart of a preferred embodiment of a lung lobe segmentation method based on UNet network of the present invention. In this embodiment, the various method steps of the lung lobe segmentation method based on the UNet network are implemented by a computer software program, which is stored in a computer-readable storage medium in the form of computer program instructions (for example, in this embodiment) In the memory 11), the computer-readable storage medium may include: read-only memory, random access memory, magnetic or optical disk, etc., and the computer program instructions can be loaded by a processor (for example, the processor 12 in this embodiment) and execute the following steps:
步骤S21,从影像输入设备2获取肺部CT影像数据。在本实施例中,所述影像输入设备2可以为CT扫描仪,能够扫描人体肺部得到肺部CT影像;也可以为医疗影像数据库,存储有待分割的人体肺部CT影像。Step S21: Acquire CT image data of the lung from the image input device 2. In this embodiment, the image input device 2 may be a CT scanner, which can scan human lungs to obtain lung CT images; it may also be a medical image database that stores CT images of human lungs to be segmented.
步骤S22,对输入的肺部CT影像数据进行归一化处理,以排除亮度较高的非肺部区域。在本实施例中,所述影像数据处理模块102对输入的肺部CT影像数据进行预处理,将影像数据的限定到[-1000,400]这个区间,然后归一化到[0,1]这个区间,以排除亮度较高的非肺部区域。Step S22: Perform normalization processing on the input lung CT image data to exclude non-pulmonary regions with higher brightness. In this embodiment, the image data processing module 102 preprocesses the input lung CT image data, limits the image data to the interval [-1000, 400], and then normalizes it to [0, 1] This interval is to exclude non-lung areas with higher brightness.
步骤S23,对处理后的影像数据利用2D UNet网络分割出肺内区域和肺外区域,并将肺内区域作为肺区候选区域。在本实施例中,所述2D UNet网络采用的是多层2D卷积层为主体结构的编码-解码网络进行检测和输出,为了保证图像能正确输入到2D UNet网络中,2D UNet网络输入使用的尺寸是[256,256]。如图3所示2D UNet网络输入的图像为经步骤S12处理过的肺部CT影像(a1)和对应的医生勾画的肺区金标准图像(a2),肺部CT影像(a1)和肺区金标准图像(a2)均需要全部缩放到[256,256]这个尺寸,再输入到2D UNet网络分割出肺内区域和肺外区域的肺区分割结果(a3)。在2D UNet输出时得到的肺部区域图像再缩放回原来图像的大小,对每层肺区利用半径为5个像素的大小进行膨胀,根据肺区大小提取肺部区域的边界框,从边界框中提取出肺区候选区域(a3)。Step S23: Use a 2D UNet network to segment the processed image data into the lung area and the lung area, and use the lung area as a candidate lung area. In this embodiment, the 2D UNet network uses a multi-layer 2D convolutional layer as the main structure of the encoding-decoding network for detection and output. In order to ensure that the image can be correctly input into the 2D UNet network, 2D The size used for UNet network input is [256, 256]. As shown in Figure 3, the images input by the 2D UNet network are the lung CT image (a1) processed in step S12 and the gold standard image (a2) of the lung area outlined by the corresponding doctor, the lung CT image (a1) and the lung area All the gold standard images (a2) need to be scaled to the size of [256, 256], and then input into the 2D UNet network to segment the lung region and the lung region segmentation results (a3). In 2D The lung area image obtained during UNet output is scaled back to the original image size, and each layer of lung area is expanded with a radius of 5 pixels, and the bounding box of the lung area is extracted according to the size of the lung area, and extracted from the bounding box The candidate area of the lung area (a3).
步骤S24,对肺区候选区域利用3D UNet网络筛分割五个肺叶掩膜区域,得到左上叶、左下叶、右上叶、右中叶以及右下叶的区域;在本实施例中,所述3D UNet网络采用的是多层3D卷积层为主体结构的编码-解码网络进行检测和输出。为了保证图像能正确输入到3D UNet网络中,训练过程中采用随机裁剪的方法得到[128,128,64]大小的输入图像,经过随机变化、旋转、上下翻转、尺度变化等做影像增强处理,输入到3D UNet网络中进行网络模型训练。如图4所示,3D UNet网络的输入图像使用的是S23步骤输出的肺区影像数据乘以S12步骤处理过的肺部CT影像(b1)和S23步骤输出的肺区影像数据乘以医生勾画的肺叶金标准图像(b2);模型训练时使用了混合损失函数:Dice loss函数和Focal loss函数,其中,Dice Loss函数用于图像分割中衡量结果的好坏,Focal loss函数用于处理类别不均衡。测试阶段可以采用移动滑窗的方法得到[128,128,64]大小的图像,经过训练出的模型预测后,还原回原图可得到5个不同的肺叶掩膜区域(b3)。Step S24: Use the 3D UNet network to screen and segment the five lung lobe mask regions for the lung area candidate region to obtain the regions of the upper left lobe, the lower left lobe, the upper right lobe, the middle right lobe, and the lower right lobe; in this embodiment, the 3D UNet The network uses a multi-layer 3D convolutional layer as the main structure of the encoding-decoding network for detection and output. In order to ensure that the image can be correctly input into the 3D UNet network, random cropping is used in the training process to obtain an input image of [128, 128, 64] size, which undergoes random change, rotation, upside down, and scale changes for image enhancement processing. Input to the 3D UNet network for network model training. As shown in Figure 4, the input image of the 3D UNet network uses the lung area image data output in step S23 multiplied by the lung CT image processed in step S12 (b1) and the lung area image data output in step S23 multiplied by the doctor's outline Gold standard image of lung leaf (b2); mixed loss function: Dice loss function and Focal are used during model training Loss function, where the Dice Loss function is used to measure the quality of the result in image segmentation, Focal The loss function is used to deal with category imbalance. In the test phase, a moving sliding window method can be used to obtain an image with a size of [128, 128, 64]. After the trained model is predicted, the original image can be restored to obtain 5 different lung lobe mask regions (b3).
步骤S25,对五个肺叶掩膜区域分别进行形态学处理,得到最终的肺叶分割结果;在本实施例中,所述对五个肺叶的掩膜区域分别进行形态学处理的步骤包括:去除五个肺叶掩膜区域外的区域,以及填充五个肺叶掩膜区域的孔洞,得到最终的肺叶分割结果。Step S25: Perform morphological processing on the five lung lobes mask regions to obtain the final lung lobe segmentation results; in this embodiment, the step of performing morphological processing on the mask regions of the five lung lobes includes: removing five The area outside the mask area of the lung lobes, and the holes that fill the mask area of the five lung lobes, obtain the final lung lobe segmentation result.
步骤S26,于将肺叶分割结果存储在存储器11中,或者输出并显示在显示器13的屏幕上。在本实施例中,将分割出的不同的肺叶影像显示在显示器13的屏幕上,以供医生在疾病诊断治疗方面提供更加全面的参考。Step S26 is to store the result of the lung lobes segmentation in the memory 11 or output and display it on the screen of the display 13. In this embodiment, the segmented images of different lung lobes are displayed on the screen of the display 13 for doctors to provide a more comprehensive reference in disease diagnosis and treatment.
本发明还一种计算机可读存储介质,该计算机可读存储介质存储多条计算机程序指令,所述计算机程序指令由计算机装置的处理器加载并执行本发明所述基于UNet网络的肺叶分割方法的各个步骤。本领域技术人员可以理解,上述实施方式中各种方法的全部或部分步骤可以通过相关程序指令完成,该程序可以存储于计算机可读存储介质中,存储介质可以包括:只读存储器、随机存储器、磁盘或光盘等。The present invention also provides a computer-readable storage medium that stores a plurality of computer program instructions, and the computer program instructions are loaded by a processor of a computer device and execute the method of lung lobe segmentation based on the UNet network of the present invention. Various steps. Those skilled in the art can understand that all or part of the steps of the various methods in the foregoing embodiments can be completed by related program instructions. The program can be stored in a computer-readable storage medium. The storage medium may include: read-only memory, random access memory, Disk or CD, etc.
本发明所述基于UNet网络的肺叶分割方法、装置及计算机可读存储介质,能够通过对肺部CT图像进行归一化处理,采用2D UNet网络从肺部CT图像快速准确地提取肺区,去除肺区外噪声的影响,然后采用3D UNet网络对肺部CT图像中的肺叶进行分割,得到五个肺叶掩膜区域,从而高效地控制肺叶分割准确度和速度,提高了分割的效率且不受限于个体肺部形态差异。本发明通过UNet网络实现快速、准确地提取肺叶,定位肺癌位置,为医生对肺癌的诊断治疗提供医学指导。The lung lobe segmentation method, device and computer-readable storage medium based on UNet network of the present invention can normalize lung CT images, and use 2D UNet network to quickly and accurately extract lung regions from lung CT images, and remove After the influence of noise outside the lung area, the 3D UNet network is used to segment the lung lobes in the lung CT image to obtain five lung lobes mask regions, thereby efficiently controlling the accuracy and speed of lung lobes segmentation, improving the efficiency of segmentation and not being affected. Limited to differences in individual lung morphology. The invention realizes fast and accurate extraction of lung lobes through UNet network, locates the position of lung cancer, and provides medical guidance for the diagnosis and treatment of lung cancer for doctors.
以上仅为本发明的较佳实施例,并非因此限制本发明的专利范围,凡是利用本发明说明书及附图内容所作的等效结构或等效流程变换,或直接或间接运用在其他相关的技术领域,均同理包括在本发明的专利保护范围内。The above are only preferred embodiments of the present invention, and do not limit the scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the description and drawings of the present invention, or directly or indirectly applied to other related technologies In the same way, all fields are included in the scope of patent protection of the present invention.
相较于现有技术,本发明所述基于UNet网络的肺叶分割方法、装置及计算机可读存储介质,能够通过对肺部CT图像进行归一化处理,采用2D UNet网络从肺部CT图像快速准确地提取肺区,去除肺区外噪声的影响,然后采用3D UNet网络对肺部CT图像中的肺叶进行分割,得到五个肺叶掩膜区域,从而高效地控制肺叶分割准确度和速度,提高了肺叶分割的效率且不受限于个体肺部形态差异。本发明通过UNet网络实现快速、准确地提取肺叶,定位肺癌位置,为医生对肺癌的诊断治疗提供医学指导。Compared with the prior art, the lung lobe segmentation method, device and computer-readable storage medium based on UNet network of the present invention can normalize lung CT images and use 2D UNet network to quickly obtain lung CT images. Accurately extract the lung area, remove the influence of noise outside the lung area, and then use the 3D UNet network to segment the lung lobes in the lung CT image to obtain five lung lobes mask areas, thereby efficiently controlling the accuracy and speed of lung lobes segmentation, and improving The efficiency of lung lobes segmentation is not limited to individual lung morphological differences. The invention realizes fast and accurate extraction of lung lobes through UNet network, locates the position of lung cancer, and provides medical guidance for the diagnosis and treatment of lung cancer for doctors.
Claims (10)
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201910437417.7A CN111986206A (en) | 2019-05-24 | 2019-05-24 | Lung lobe segmentation method and device based on UNet network and computer-readable storage medium |
| CN201910437417.7 | 2019-05-24 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2020238043A1 true WO2020238043A1 (en) | 2020-12-03 |
Family
ID=73436885
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/CN2019/117321 Ceased WO2020238043A1 (en) | 2019-05-24 | 2019-11-12 | Unet network-based lung lobe segmentation method and apparatus, computer readable storage medium |
Country Status (2)
| Country | Link |
|---|---|
| CN (1) | CN111986206A (en) |
| WO (1) | WO2020238043A1 (en) |
Families Citing this family (10)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN112349429B (en) * | 2020-12-01 | 2021-09-21 | 苏州体素信息科技有限公司 | Disease prediction method, disease prediction model training method and device, and storage medium |
| CN112884792B (en) * | 2021-02-02 | 2022-10-25 | 青岛海信医疗设备股份有限公司 | Lung image segmentation method and device, electronic equipment and storage medium |
| CN112950554B (en) * | 2021-02-05 | 2021-12-21 | 慧影医疗科技(北京)有限公司 | Lung lobe segmentation optimization method and system based on lung segmentation |
| CN112734755A (en) * | 2021-03-15 | 2021-04-30 | 四川大学 | Lung lobe segmentation method based on 3D full convolution neural network and multitask learning |
| CN113344881A (en) * | 2021-06-09 | 2021-09-03 | 西安智诊智能科技有限公司 | Lung lobe image segmentation method based on deep learning |
| CN113576487B (en) * | 2021-06-18 | 2023-06-13 | 深圳技术大学 | Feature determination method, electrocardio prediction device, electronic equipment and storage medium |
| CN115147359B (en) * | 2022-06-06 | 2023-04-07 | 北京医准智能科技有限公司 | Lung lobe segmentation network model training method and device, electronic equipment and storage medium |
| CN117523207B (en) * | 2024-01-04 | 2024-04-09 | 广东欧谱曼迪科技股份有限公司 | Method, device, electronic equipment and storage medium for lung lobe segmentation correction processing |
| CN117576126B (en) * | 2024-01-16 | 2024-04-09 | 广东欧谱曼迪科技股份有限公司 | Optimization method and device for lung lobe segmentation, electronic equipment and storage medium |
| CN120163938A (en) * | 2025-04-28 | 2025-06-17 | 南方医科大学南方医院 | Method for constructing dynamic fluoroscopic model of pulmonary vascular and method for dynamic fluoroscopic model of pulmonary vascular |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180137656A1 (en) * | 2016-11-11 | 2018-05-17 | Shanghai Neusoft Medical Technology Co., Ltd. | Correcting pet image attenuation |
| CN108133476A (en) * | 2017-12-26 | 2018-06-08 | 安徽科大讯飞医疗信息技术有限公司 | A kind of Lung neoplasm automatic testing method and system |
| CN108389210A (en) * | 2018-02-28 | 2018-08-10 | 深圳天琴医疗科技有限公司 | A kind of medical image cutting method and device |
| CN109685787A (en) * | 2018-12-21 | 2019-04-26 | 杭州依图医疗技术有限公司 | Output method, device in the lobe of the lung section segmentation of CT images |
-
2019
- 2019-05-24 CN CN201910437417.7A patent/CN111986206A/en active Pending
- 2019-11-12 WO PCT/CN2019/117321 patent/WO2020238043A1/en not_active Ceased
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20180137656A1 (en) * | 2016-11-11 | 2018-05-17 | Shanghai Neusoft Medical Technology Co., Ltd. | Correcting pet image attenuation |
| CN108133476A (en) * | 2017-12-26 | 2018-06-08 | 安徽科大讯飞医疗信息技术有限公司 | A kind of Lung neoplasm automatic testing method and system |
| CN108389210A (en) * | 2018-02-28 | 2018-08-10 | 深圳天琴医疗科技有限公司 | A kind of medical image cutting method and device |
| CN109685787A (en) * | 2018-12-21 | 2019-04-26 | 杭州依图医疗技术有限公司 | Output method, device in the lobe of the lung section segmentation of CT images |
Non-Patent Citations (1)
| Title |
|---|
| HUANG, SHAOHUI ET AL.: "Application Research of 2D/3D Cascade Convolution in Segmentation of CT Pulmonary Artery", CHINA NUMERAL MEDICAL SCIENCE JOURNAL, vol. 14, no. 5, 15 May 2019 (2019-05-15), pages 1 - 5, XP055762277 * |
Also Published As
| Publication number | Publication date |
|---|---|
| CN111986206A (en) | 2020-11-24 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2020238043A1 (en) | Unet network-based lung lobe segmentation method and apparatus, computer readable storage medium | |
| WO2022063199A1 (en) | Pulmonary nodule automatic detection method, apparatus and computer system | |
| CN108133476B (en) | Method and system for automatically detecting pulmonary nodules | |
| CN110033456A (en) | Medical image processing method, device, equipment and system | |
| CN111462049B (en) | A method for automatic labeling of the shape of the lesion area in the breast contrast-enhanced ultrasound video | |
| CN114092450B (en) | Real-time image segmentation method, system and device based on gastroscopy video | |
| CN110930414B (en) | Method, device, server and storage medium for marking shadows of lung regions in medical images | |
| CN110276408B (en) | 3D image classification method, device, equipment and storage medium | |
| CN115115657B (en) | Lesion segmentation method and device, electronic device and storage medium | |
| CN110021021B (en) | Head image analysis device and image analysis method | |
| CN111738975B (en) | Image recognition method and image recognition device | |
| CN114332132A (en) | Image segmentation method and device and computer equipment | |
| JP6735340B2 (en) | Automatic removal of physiological accumulation from nuclear medicine images and automatic segmentation of CT images | |
| CN117315378B (en) | Grading judgment method for pneumoconiosis and related equipment | |
| CN113393445A (en) | Breast cancer image determination method and system | |
| CN117765330A (en) | MRI image-based data labeling method and system | |
| CN115359060B (en) | Focus example segmentation method and device for pneumonia CT image | |
| CN116862872A (en) | Mammary gland focus ultrasonic image processing model training method, related method and device | |
| Ghorakavi | TBNet: Pulmonary tuberculosis diagnosing system using deep neural networks | |
| CN114757908A (en) | Image processing method, device, device and storage medium based on CT image | |
| CN116309647B (en) | Brain lesion image segmentation model construction method, image segmentation method and equipment | |
| CN112633405A (en) | Model training method, medical image analysis device, medical image analysis equipment and medical image analysis medium | |
| US20210383547A1 (en) | Medical image processing | |
| CN118447063A (en) | A retinal image registration method based on dilated vessel constraints and metric learning | |
| Merzoug et al. | A deep system for breast tumor classification from mammograms |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 19931315 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 19931315 Country of ref document: EP Kind code of ref document: A1 |