[go: up one dir, main page]

WO2024080393A1 - Système et procédé de détection d'objet dans une image médicale thoracique - Google Patents

Système et procédé de détection d'objet dans une image médicale thoracique Download PDF

Info

Publication number
WO2024080393A1
WO2024080393A1 PCT/KR2022/015341 KR2022015341W WO2024080393A1 WO 2024080393 A1 WO2024080393 A1 WO 2024080393A1 KR 2022015341 W KR2022015341 W KR 2022015341W WO 2024080393 A1 WO2024080393 A1 WO 2024080393A1
Authority
WO
WIPO (PCT)
Prior art keywords
chest
object detection
bounding box
image
ray image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/KR2022/015341
Other languages
English (en)
Korean (ko)
Inventor
명준표
안태준
한성원
강민지
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Industry Academic Cooperation Foundation of Catholic University of Korea
Original Assignee
Industry Academic Cooperation Foundation of Catholic University of Korea
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Industry Academic Cooperation Foundation of Catholic University of Korea filed Critical Industry Academic Cooperation Foundation of Catholic University of Korea
Priority to PCT/KR2022/015341 priority Critical patent/WO2024080393A1/fr
Publication of WO2024080393A1 publication Critical patent/WO2024080393A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/02—Arrangements for diagnosis sequentially in different planes; Stereoscopic radiation diagnosis
    • A61B6/03—Computed tomography [CT]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B6/00—Apparatus or devices for radiation diagnosis; Apparatus or devices for radiation diagnosis combined with radiation therapy equipment
    • A61B6/12—Arrangements for detecting or locating foreign bodies
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems

Definitions

  • the present invention provides a chest medical image object detection system and method.
  • Chest X-ray examination is a test method using X-rays, and is the simplest and most basic test to diagnose chest diseases. It is the easiest and most essential test to determine the size and shape of the heart, expansion of each part of the heart and large blood vessels, size of pulmonary blood vessels, and pulmonary edema.
  • Computed tomography is an imaging diagnostic method that transmits X-rays to the target area and reconstructs the difference in absorption through a computer to obtain a cross-sectional image or three-dimensional image of the human body.
  • CT Computer Tomography
  • Chest computed tomography can be performed to diagnose diseases within the lung parenchyma, small pulmonary nodules, and bronchial areas. In addition, it is possible to determine the presence or absence of lung cancer, stage nodules, and metastasis from other organs.
  • Object detection technology is a technology related to computer vision and image processing, and deals with detecting objects in digital images.
  • object detection algorithms based on convolutional neural networks are being used in various fields.
  • Chest Therefore, there are limitations in diagnosing microscopic lesions or early lesions.
  • the quality of reading is uneven due to the lack of professional manpower compared to the large reading volume.
  • since chest pressure is uneven due to the lack of professional manpower compared to the large reading volume.
  • Chest computed tomography allows for a wealth of diagnoses because it can be judged three-dimensionally through image sequences. However, in some underdeveloped regions or countries, chest computed tomography examination itself may be difficult.
  • Embodiments of the present invention can match a chest X-ray image with an object identified in a chest computed tomography image read by a corresponding specialist.
  • an object detection model can be created by learning the chest The reading result can be obtained.
  • embodiments of the present invention include a collection unit that collects a plurality of chest X-ray images and a reading of a chest computed tomography image corresponding to the chest X-ray image; Receive a readout of the chest X-ray image and a chest computed tomography image, match one or more objects recorded in the readout of the chest computed tomography image to the chest A matching unit that generates a bounding box annotation corresponding to the X-ray image; a model generator that receives a chest X-ray image and a bounding box annotation and generates an object detection model by applying an object detection algorithm to the chest X-ray image and the bounding box annotation; an object detection unit that detects objects in the patient's chest X-ray image by inputting the patient's chest X-ray image into an object detection model; and an output unit that outputs the object detected by the detection unit, wherein the interpretation text of the chest computed tomography image is written according to the opinion of a specialist.
  • the system provides a
  • embodiments of the present invention provide a method for detecting a chest medical image object, comprising: collecting a plurality of chest X-ray images and a readout of a chest computed tomography image corresponding to the chest X-ray image; A matching step of receiving a readout of the chest X-ray image and a chest computed tomography image, and matching one or more objects recorded in the readout of the chest X-ray image to the chest An annotation generation step of generating a bounding box annotation corresponding to the chest X-ray image and including bounding box information corresponding to the object; A model generation step of receiving a chest X-ray image and a bounding box annotation, and applying an object detection algorithm to the chest An object detection step of detecting an object in the patient's chest X-ray image by inputting the patient's chest X-ray image into an object detection model; and an output step of outputting the object detected in the detection step.
  • an object to be used for diagnosis can be detected from a patient's chest X-ray image by designing an object detection model using an object detection algorithm.
  • a chest medical image object detection system can detect one or more objects at a time from a chest X-ray image and display the detected object on the chest X-ray image. Beyond limitations such as resolution resulting from the human eye observation of the person reading the chest
  • objects can be automatically detected by a chest medical image object detection system, thereby reducing the work burden of radiologists and reducing X-ray reading costs.
  • FIG. 1 is a schematic configuration diagram of a chest medical image object detection system according to embodiments of the present invention.
  • Figure 2 is a flowchart showing an operation in which a matching unit according to embodiments of the present invention determines an object to be matched to a chest X-ray image among objects recorded in a readout of a chest computed tomography image.
  • FIG. 3 is a diagram illustrating an example of an operation in which a matching unit according to embodiments of the present invention matches an object recorded in a readout of a chest computed tomography image with a bounding box in a chest X-ray image.
  • Figure 4 is a diagram illustrating an example of an operation in which a matching unit generates a bounding box annotation according to embodiments of the present invention.
  • Figure 5 is a diagram showing types of objects according to embodiments of the present invention.
  • Figure 6 is a diagram illustrating an example of bounding box information for an object according to embodiments of the present invention.
  • Figure 7 is a diagram showing an example of a bounding box annotation according to embodiments of the present invention.
  • Figure 8 is a flowchart showing the operation of the model generator according to embodiments of the present invention to generate an object detection model by applying the Faster R-CNN algorithm to the chest X-ray image and bounding box annotation.
  • Figure 9 is a diagram showing the operation of the object detection unit according to embodiments of the present invention to detect an object by inputting a chest X-ray image into an object detection model that learned the Faster R-CNN algorithm.
  • Figure 10 is a diagram showing a method for detecting a chest medical image object according to embodiments of the present invention.
  • 'part' includes a unit realized by hardware, a unit realized by software, and a unit realized using both. Additionally, one unit may be realized using two or more pieces of hardware, and two or more units may be realized using one piece of hardware.
  • FIG. 1 is a schematic configuration diagram of a chest medical image object detection system according to embodiments of the present invention.
  • the chest medical image object detection system 100 includes a collection unit 110, a matching unit 120, a model creation unit 130, an object detection unit 140, and It may include an output unit 150.
  • the collection unit 110 may collect a plurality of chest X-ray images and interpretations of chest computed tomography images corresponding to the chest X-ray images.
  • Chest computed tomography enables three-dimensional diagnosis using image sequences. Therefore, compared to chest X-ray images, which are two-dimensional flat images, relatively precise diagnosis is possible and various diseases can be identified.
  • the matching unit 120 receives a readout of the chest You can. And the matching unit 120 includes bounding box information and can generate a bounding box annotation corresponding to the chest X-ray image.
  • Bounding box information refers to information about the area where an object is located and the type of object in a chest X-ray image.
  • Bounding Box Annotation refers to a label containing bounding box information for all objects matched to one chest X-ray image. Bounding box annotation can be treated as an integral part of the corresponding chest X-ray image, or it can be handled and managed separately from the chest X-ray image.
  • the model generator 130 may receive a chest X-ray image and a bounding box annotation, and apply an object detection algorithm to the chest .
  • Object detection algorithms include the traditional Canny Edge detection algorithm, Harris corner detection algorithm, Haar-Like feature selection algorithm, HOG (Histogram of Oriented Gradient) algorithm, and SIFT algorithm (Scale Invariant). Classification algorithms including Feature Transform, SVM, and Adaboost can be used.
  • the model generator 130 may use a convolutional neural network (CNN), which is a class of deep learning algorithms among object detection algorithms, to learn an object detection model.
  • CNN convolutional neural network
  • a convolutional neural network (CNN) is a neural network used to analyze visual images. It is specialized in processing multidimensional arrays and can extract and process features from images.
  • a convolutional neural network (CNN) can be composed of a convolutional layer and a pooling layer.
  • the convolution layer can perform the role of extracting image features through convolution operations.
  • the convolution layer can generate a feature map by performing a convolution operation on the input image while going through a filter of a specific size.
  • a convolutional neural network may perform padding to keep the size of the feature map the same as the size of the input image while performing a convolution operation.
  • Padding means adding rows and columns of a certain number of widths to the edges of a convolutional neural network (CNN) input.
  • Zero padding which fills the added columns and rows with zeros, can be used as padding.
  • a pooling layer can be added after the convolution layer.
  • the pooling layer can perform a pooling operation to reduce the size of the feature map by downsampling the feature map.
  • Pooling methods include Max Pooling, which extracts the maximum value for a specific area, and Average Pooling, which extracts the average for a specific area. When pooling is used, the size of the feature map is reduced, so the number of weights in the feature map can be reduced.
  • CNN convolutional neural network
  • Object detection algorithms using convolutional neural networks include the R-CNN algorithm, Fast R-CNN algorithm, and Faster R-CNN algorithm.
  • the object detection unit 140 may detect an object in the patient's chest X-ray image by inputting the patient's chest X-ray image into an object detection model.
  • the type of object can be identified as a detection result.
  • the output unit 150 may output the object detected by the object detection unit 140.
  • the output unit 150 may output the object detection result generated by the object detection unit 140. At this time, the output unit 150 may output the detection result through a method such as screen output through a display or printing of the detection result using a printer. The output unit 150 may output the detection result in a way that a bounding box of the detected object is displayed on the patient's chest X-ray image.
  • FIG. 2 is a flowchart illustrating an operation of the matching unit 120 according to embodiments of the present invention to determine an object to be matched to a chest X-ray image among objects recorded in a readout of a chest computed tomography image.
  • the matching unit 120 can confirm the operation of determining whether the object recorded in the readout of the chest computed tomography image can be matched to the chest X-ray image.
  • images from chest computed tomography allow for three-dimensional judgment through image sequences, enabling more precise diagnosis than chest X-ray images and identifying various diseases. Since the chest X-ray image is a two-dimensional planar image, there may be cases where the object cannot be identified in the chest
  • the matching unit 120 can determine whether the object recorded in the readout of the chest computed tomography image can be reflected in the chest X-ray image.
  • the matching unit 120 may determine whether the object identified in the analysis of the chest computed tomography image can be reflected by matching it to the chest X-ray image as a bounding box (S220).
  • the matching unit 120 can match the object with a bounding box on the chest There is (S230).
  • the matching unit 120 uses a bounding box on the chest It can be judged that matching is possible.
  • the matching unit 120 may terminate the matching process without matching the object to the chest there is.
  • the matching unit 120 detects 4) cirrhosis with ascites and spleen enlargement, 5) a small attenuated lesion less than 2.1 cm in the left hemisphere suspected to be a liver cyst. In this case, it can be determined that matching is not possible using the bounding box on the chest X-ray image.
  • the matching unit 120 matches the object with a bounding box on the chest X-ray image for 1) cardiac hypertrophy, 2) linear/small segment atelectasis, and 3) non-specific bilateral pleural effusion (S230).
  • the corresponding bounding box information can be saved in the bounding box annotation, 4) cirrhosis with ascites and splenomegaly, 5) cyst less than 2.1 cm in the left hemisphere suspected to be a liver cyst. Small attenuated lesions can be excluded from learning by terminating the process without marking them on the chest X-ray image.
  • FIG. 3 is a diagram illustrating an example of an operation in which a matching unit according to embodiments of the present invention matches an object recorded in a readout of a chest computed tomography image to a chest X-ray image as a bounding box.
  • the matching unit 120 may match the object recorded in the readout of the chest computed tomography image with a bounding box on the chest X-ray image.
  • the matching unit 120 provides information about the object and the area where the object exists with respect to 1) cardiac hypertrophy, 2) linear/small-segment atelectasis, and 3) non-specific bilateral pleural effusion identified in the interpretation of the chest computed tomography image. can be displayed.
  • Figure 4 is a diagram illustrating an example of an operation in which a matching unit generates a bounding box annotation according to embodiments of the present invention.
  • the matching unit 120 includes bounding box information and can generate a bounding box annotation corresponding to the chest X-ray image.
  • the bounding box annotation generated by the matching unit 120 contains bounding box information corresponding to each object.
  • Bounding Box Annotation can be used when learning an object detection model by applying an object detection algorithm with chest X-ray images.
  • Bounding Box Annotation can be used as ground truth when training an object detection model.
  • Ground truth is an expression of the original or actual value of the data to be learned and can be used as a standard when learning an object detection model using object detection or verifying the performance of the learned object detection model. .
  • the objects of cardiac hypertrophy, pulmonary cirrhosis, pleural effusion_left, and pleural effusion_right are matched to the chest It may include bounding box information of all objects included in .
  • Figure 5 is a diagram showing types of objects according to embodiments of the present invention.
  • the objects matched to the chest X-ray image are pleural effusion, atelectasis, pulmonary nodule, pulmonary consolidation, emphysema, pneumothorax, and cardiac hypertrophy
  • Types of objects including Cardiomegaly, Chemoport, Pacemaker, Bronchial wall thickening, Reticular opacities, Pleural thickening, and Bronchiectasis. It may apply to any one of
  • Objects matched to the chest X-ray image may include a diagnosis and a medical device.
  • the diagnosis name of the object may include a diagnosis name that can be identified from traditional chest X-ray images and a diagnosis name specialized for chest computed tomography images.
  • Diagnoses that can be identified from traditional chest X-ray images include Pleural effusion, Atelectasis, Pulmonary nodule, Consolidation, Emphysema, Pneumothorax, Cardiomegaly, etc. may include.
  • Pleural effusion is the accumulation of plasmatic fluid or exudate in the chest cavity and can be caused by inflammation, tumor, or heart failure. There may be symptoms of respiratory failure. It can be treated by removing fluid accumulated in the chest cavity using a tube or syringe.
  • Atelectasis refers to the loss of air in the alveoli of the entire or part of the lung. This refers to a condition in which the lung acini cannot fully unfold and is shrunken, so the air it contains is lost, and gas exchange, which is the function of the lung acini, does not occur. Bronchial obstruction caused by lung cancer, foreign bodies, or excessive secretions can cause atelectasis.
  • Pulmonary nodules are abnormal growths that form in the lungs and may contain one or more nodules. It may appear as spots, coins, or shadows in the lungs.
  • Consolidation also called pulmonary fibrosis
  • pulmonary fibrosis is a disease in which the hardened part of the lung expands, the volume of the lung decreases, and symptoms such as shortness of breath, cough, and phlegm worsen due to the hardened lung.
  • Emphysema is a pathological accumulation of air in tissues or organs, and is a disease in which the respiratory tract in the terminal bronchioles is abnormally expanded due to destruction of the acillary wall.
  • Pneumothorax refers to the abnormal accumulation of air or gas in the pleural space, often causing severe pain, shortness of breath, sneezing, vomiting, coughing, and difficulty breathing.
  • Cardiomegaly refers to a condition in which excessive stress is placed on the heart, causing the myocardium to thicken and the heart to become enlarged. It may be caused by various heart diseases such as heart malformation, heart valve disease, and hypertension, or it may be a healthy condition found in athletes. In particular, patients with high blood pressure can have their hearts enlarged in response to the high pressure in their arteries.
  • Diagnoses specific to chest computed tomography imaging may include bronchial wall thickening, reticular shadows, pleural thickening, and bronchiectasis.
  • Bronchial wall thickening is the most common abnormal finding found on chest CT in asthma patients, meaning that the bronchial walls are thickened. In case of bronchial inflammation, it may appear as bronchial wall thickening.
  • Reticular opacities are reticular shadows that may appear on chest computed tomography images when inflammation invades the alveoli.
  • Pleural thickening is also called pleural thickening, and thickening of the pleura is called pleural thickening. After suffering from pleurisy, the pleura may become thick. Pleural thickening can be confirmed on chest computed tomography images.
  • Bronchiectasis is a disease in which part of the lumen of the bronchi is expanded and deformed, and is characterized by foul-smelling breath, coughing fits, and discharge of mucus and purulent substances.
  • the enlarged lumen makes it easier for phlegm to accumulate, and bacteria can cause inflammation.
  • Medical devices that can be identified in chest X-ray images may include chemoports and pacemakers.
  • Chemoport is a device that is inserted periodically to safely administer anti-inflammatory drugs. It is located under the skin near the heart. Only the drug inlet protrudes beyond the skin and is not visible from the outside. It may appear as a circular structure on a chest X-ray image.
  • a pacemaker is a device implanted in cases where the heart rhythm is abnormal due to cardiomyopathy or bradycardia due to arrhythmia. It is a medical device that coordinates the rhythm of the atria and ventricles through electrode wires inserted into the heart. It consists of a pulse generator and electrode wires, and the metal part makes it visible on chest X-ray images.
  • Figure 6 is a diagram showing an example of bounding box information for an object according to embodiments of the present invention.
  • Bounding Box Information displays the center X coordinate, center Y coordinate, horizontal length, vertical length, and Bounding Box of the bounding box matched to the chest It can contain the object name of the object being used.
  • the center X coordinate of the bounding box of the cardiac hypertrophy object is is H1, and the object name may correspond to cardiac hypertrophy.
  • the bounding box information of the cardiac hypertrophy object may include “X1, Y1, W1, H1, cardiac hypertrophy.”
  • Figure 7 is a diagram showing an example of a bounding box annotation according to embodiments of the present invention.
  • the bounding box annotation may include bounding box information of all objects included in the chest X-ray image.
  • the bounding box information includes the center X coordinate, center Y coordinate, horizontal length, vertical length, and object name of the object displayed by the bounding box. You can do this, but Bounding Box Annotation is the center It can contain the object name of the object displayed. For example, for cardiac hypertrophy, it can be stored as "X1, Y1, W1, H1, cardiac hypertrophy", and for pulmonary cirrhosis, it can be stored as "X2, Y2, W2, H2, pulmonary cirrhosis", and for pleural effusion on both sides. It can be saved as “X3, Y3, W3, H3, pleural effusion_left” and “X4, Y4, W4, H4, pleural effusion_right”.
  • Bounding Box Annotation can perform the role of ground truth, and in learning an object detection model, when determining whether an object is included in a specific area and when determining whether an object is included in a specific area. It can be used for accurate area detection through bounding box regression.
  • Figure 8 is a flow chart showing the operation of the model generator according to embodiments of the present invention to generate an object detection model by applying the Faster R-CNN algorithm to the chest X-ray image and bounding box annotation.
  • the model generator 130 may generate an object detection model by applying the Faster R-CNN algorithm to the chest X-ray image and bounding box annotation.
  • the model generator 130 may use region-based convolutional neural networks as an object detection algorithm used when generating an object detection model. Among them, the Faster R-CNN algorithm can be used.
  • Faster R-CNN inherits the structure of Fast R-CNN, a previously proposed algorithm. Instead of the selective search part, which was pointed out as the cause of the bottleneck, Faster R-CNN uses a regional proposal network (RPN). The region of interest (RoI) can be calculated. Through this, region of interest (RoI) calculation using GPU has become possible, and higher accuracy can be secured while calculating fewer regions of interest (RoI) than Fast R-CNN, a structure proposed before Faster R-CNN. In addition, Faster R-CNN inherits the structure of Fast R-CNN and can classify objects by employing Fast R-CNN's Object Detection Network.
  • RPN regional proposal network
  • the model generator 130 When generating an object detection model by applying Faster R-CNN to the chest X-ray image and bounding box annotation, the model generator 130 performs alternating training and approximate joint learning. , methods such as non-approximate training can be used. As an example, it can be assumed that an object detection model is created using alternating training.
  • the model generator 130 can learn a region proposal network (RPN) using a previously learned convolutional neural network (CNN) model (S810). At this time, the model generator 130 inputs the chest (RPN) can be learned. In this process, the previously learned convolutional neural network (CNN) model and region proposal network (RPN) can be updated.
  • RPN region proposal network
  • CNN convolutional neural network
  • the model generator 130 can learn an object detection network of Fast R-CNN by importing only the region proposal layer from the learned region proposal network (RPN) (S820). In this process, the detection network object detection network of convolutional neural network (CNN) and Fast R-CNN can be updated.
  • RPN learned region proposal network
  • the model generator 130 can fine tune only the layers corresponding to the region proposal network (RPN) while fixing the convolutional neural network (CNN) (S830). In this process, only the regional proposal layer of the regional proposal network (RPN) can be updated.
  • RPN region proposal network
  • CNN convolutional neural network
  • the model generator 130 fixes the layers included in the convolutional neural network (CNN) and the region proposal network (RPN), and layers included only in the detection network object detection network of Fast R-CNN. You can make fine adjustments (S840). During this process, only the Object Detection Network of Fast R-CNN can be updated.
  • CNN convolutional neural network
  • RPN region proposal network
  • the model generator 130 performs alternating learning to obtain a region proposal network (RPN) and a detection network object detection network of Fast R-CNN that share a convolutional neural network (CNN). .
  • steps S830 and S840 while fixing the convolutional neural network (CNN), the region proposal network (RPN) and the detection network of Fast R-CNN and the object detection network were obtained, so the convolutional neural network (CNN) can be seen as sharing.
  • Figure 9 is a diagram showing the operation of the object detection unit 140 according to embodiments of the present invention to detect an object by inputting a chest X-ray image into an object detection model that learned the Faster R-CNN algorithm.
  • the object detection unit 140 can detect an object in the patient's chest X-ray image by inputting the patient's chest X-ray image into an object detection model to which the Faster R-CNN algorithm is applied.
  • the object detection unit 140 may generate a feature map through a convolutional neural network (CNN) of an object detection model for the patient's chest RPN).
  • CNN convolutional neural network
  • the region proposal network (RPN) of the object detector 140 can calculate an intermediate layer process that performs as many 3X3 channels as previously learned on the feature map to generate a region proposal.
  • the object detector 140 can obtain a second feature map by passing the intermediate layer process.
  • the object detector 140 may use the 2nd feature map to calculate classification and bounding box regression prediction values.
  • a result having a depth of 18 (2X9) can be obtained in the width of the input feature map, which means that anchor boxes corresponding to each coordinate are It contains a predicted value as to whether it corresponds to an object.
  • the object detector 140 may perform 1X1 convolution as many as 4X (the number of anchor boxes) channels in bounding box regression. For example, if there are 9 types of anchor boxes, a depth of 36 can be obtained, which may correspond to predicted values for the X coordinate, Y coordinate, area, and height for each anchor. The value obtained as a result of the convolution operation can be used as is in Bounding Box Regression.
  • the object detection unit 140 can sort the probability values of objects obtained through classification and select a certain number of anchors in descending order. Bounding box regression can be applied to each selected anchor. Afterwards, the region of interest (RoI) can be calculated by applying IoU to a plurality of detected bounding boxes and applying Non-Maximum Suppression, an algorithm that selects boxes that are judged to overlap. there is.
  • RoI region of interest
  • the object detection unit 140 projects the region of interest (RoI) onto the first feature map and then applies RoI pooling to generate a new feature map, with respect to which Fast R- By entering the CNN detection network object detection network, you can obtain object prediction results.
  • RoI region of interest
  • CNN convolutional neural network
  • a convolutional neural network may experience a gradient loss problem in which the gradient gradually decreases as it moves toward the input layer.
  • reLU rectified linear units
  • batch normalization can be used.
  • Batch normalization means normalizing in batches input at one time. Batch normalization is performed before passing the activation function at each layer. When batch normalization is implemented, there is a normalization part for each layer, and it can be adjusted to prevent a distorted distribution.
  • Batch normalization can be performed using the average and division of the mini-batch, and then adjusting the scale and shift values using the gamma and beta values.
  • the object detection unit 140 may evaluate the performance of the object detection model generated by the model creation unit 130.
  • mAP mean average precision
  • the object detection unit 140 calculates the Intersection over Union (IoU) of the predicted area and the actual area of the ground truth, and if the IoU is greater than or equal to the threshold, it may be determined that the object has been properly detected.
  • IoU Intersection over Union
  • IoU can be calculated as follows:
  • Precision refers to the proportion of correctly detected positives among those determined to be positive.
  • the object detection unit 140 determines precision and recall according to the accumulated number of TPs and FPs. can be calculated.
  • the object detector 140 calculates the AP by calculating the area under the PR curve (Precision-Recall curve), and can check the mAP by calculating the average for all classes.
  • Figure 10 is a diagram showing a method for detecting a chest medical image object according to embodiments of the present invention.
  • the chest medical image object detection method may include a collection step (S1010) of collecting a plurality of chest X-ray images and readouts of chest computed tomography images corresponding to the chest X-ray images. Meanwhile, the collection step (S1010) may be executed by the collection unit 110 described above.
  • the chest medical image object detection method receives a readout of a chest X-ray image and a chest computed tomography image, and matches one or more objects recorded in the readout of the chest computed tomography image to the chest It may include a matching step (S1020).
  • the chest medical image object detection method may include an annotation generation step (S1030) of generating a bounding box annotation that includes bounding box information and corresponds to the chest X-ray image.
  • the matching step (S1020) and the annotation generating step (S1030) may be executed by the matching unit 120 described above.
  • the chest medical image object detection method is to receive a chest X-ray image and bounding box annotation, and apply an object detection algorithm to the chest It may include a model creation step (S1040). Meanwhile, the model creation step (S1040) may be executed by the model creation unit 130 described above.
  • the chest medical image object detection method may include an object detection step (S1050) of detecting an object in the patient's chest X-ray image by inputting the patient's chest X-ray image into an object detection model. Meanwhile, the object detection step (S1050) may be executed by the object detection unit 140.
  • the chest medical image object detection method may include an output step (S1060) of outputting the object detected in the detection step. Meanwhile, the output step (S1060) may be executed by the output unit 150 described above.
  • the object is Pleural effusion, Atelectasis, Pulmonary nodule, Consolidation, Emphysema, Pneumothorax, Cardiomegaly, Chemoport, It may correspond to any one of the types of objects including pacemaker, bronchial wall thickening, reticular opacities, pleural thickening, and bronchiectasis.
  • Bounding Box Information is the center May include names.
  • bounding box annotation may include bounding box information of all objects included in the chest X-ray image.
  • the object detection algorithm may be the Faster R-CNN algorithm.
  • the chest medical image object detection system 100 described above may be implemented by a computing device including at least some of a processor, memory, user input device, and presentation device.
  • Memory is a medium that stores computer-readable software, applications, program modules, routines, instructions, and/or data that are coded to perform specific tasks when executed by a processor.
  • the processor may read and execute computer-readable software, applications, program modules, routines, instructions, and/or data stored in memory.
  • a user input device may be a means for allowing a user to input a command that causes a processor to execute a specific task or to input data required to execute a specific task.
  • User input devices may include a physical or virtual keyboard, keypad, key buttons, mouse, joystick, trackball, touch-sensitive input means, or microphone.
  • Presentation devices may include displays, printers, speakers, or vibrating devices.
  • Computing devices may include a variety of devices such as smartphones, tablets, laptops, desktops, servers, and clients.
  • a computing device may be a single stand-alone device or may include multiple computing devices operating in a distributed environment comprised of multiple computing devices cooperating with each other through a communication network.
  • the above-described chest medical image object detection method includes a processor, and is coded as computer-readable software, applications, program modules, and routines that, when executed by the processor, can perform an image diagnosis method using a deep learning model. , instructions, and/or data structures, etc. may be executed by a computing device having a memory.
  • the above-described embodiments can be implemented through various means.
  • the present embodiments may be implemented by hardware, firmware, software, or a combination thereof.
  • the present embodiments include one or more Application Specific Integrated Circuits (ASICs), Digital Signal Processors (DSPs), Digital Signal Processing Devices (DSPDs), Programmable Logic Devices (PLDs), and Field Programmable Gates (FPGAs).
  • ASICs Application Specific Integrated Circuits
  • DSPs Digital Signal Processors
  • DSPDs Digital Signal Processing Devices
  • PLDs Programmable Logic Devices
  • FPGAs Field Programmable Gates
  • Arrays processors, controllers, microcontrollers, or microprocessors.
  • the chest medical image object detection method may be implemented using an artificial intelligence semiconductor device in which neurons and synapses of a deep neural network are implemented with semiconductor devices.
  • the semiconductor device may be currently used semiconductor devices, such as SRAM, DRAM, or NAND, or may be next-generation semiconductor devices such as RRAM, STT MRAM, or PRAM, or a combination thereof.
  • the results (weights) of learning the deep learning model using software are transferred to a synapse-mimicking element arranged in an array or an artificial intelligence semiconductor device. You can also learn on your device.
  • the above-described chest medical image object detection method may be implemented in the form of a device, procedure, or function that performs the functions or operations described above.
  • Software code can be stored in a memory unit and run by a processor.
  • the memory unit is located inside or outside the processor and can exchange data with the processor through various known means.
  • system generally refer to computer-related entities hardware, hardware and software. It may refer to a combination of, software, or running software.
  • the foregoing components may be a process, processor, controller, control processor, object, thread of execution, program, and/or computer run by a processor.
  • an application running on a controller or processor and the controller or processor can be a component.
  • One or more components may reside within a process and/or thread of execution, and the components may be located on a single device (e.g., system, computing device, etc.) or distributed across two or more devices.
  • another embodiment provides a computer program stored in a computer recording medium that performs the above-described chest medical image object detection method.
  • Another embodiment also provides a computer-readable recording medium on which a program for realizing the above-described chest medical image object detection method is recorded.
  • the program recorded on the recording medium can be read, installed, and executed on the computer to execute the above-described steps.
  • the above-mentioned program is a C, C++ program that the computer's processor (CPU) can read through the computer's device interface (Interface).
  • the computer's processor CPU
  • the computer's device interface Interface
  • code coded in computer languages such as JAVA and machine language.
  • These codes may include functional codes related to functions that define the above-described functions, and may also include control codes related to execution procedures necessary for the computer's processor to execute the above-described functions according to predetermined procedures.
  • these codes may further include memory reference-related codes that determine which location (address address) in the computer's internal or external memory the additional information or media required for the computer's processor to execute the above-mentioned functions should be referenced. .
  • the code is It may further include communication-related codes for how to communicate with other computers, servers, etc., and what information or media should be transmitted and received during communication.
  • Recording media that can be read by a computer recording the above-described program include, for example, ROM, RAM, CD-ROM, magnetic tape, floppy disk, optical media storage, etc., and also include carrier wave (e.g. , transmission via the Internet) may also be implemented.
  • carrier wave e.g. , transmission via the Internet
  • computer-readable recording media can be distributed across computer systems connected to a network, so that computer-readable code can be stored and executed in a distributed manner.
  • the functional program for implementing the present invention and the code and code segments related thereto are designed by programmers in the technical field to which the present invention belongs, taking into account the system environment of the computer that reads the recording medium and executes the program. It can also be easily inferred or changed by .
  • the chest medical image object detection method may also be implemented in the form of a recording medium containing instructions executable by a computer, such as an application or program module executed by a computer.
  • Computer-readable media can be any available media that can be accessed by a computer and includes both volatile and non-volatile media, removable and non-removable media. Additionally, computer-readable media may include all computer storage media. Computer storage media includes both volatile and non-volatile, removable and non-removable media implemented in any method or technology for storage of information such as computer-readable instructions, data structures, program modules or other data.
  • the chest medical image object detection method may be executed by an application installed by default on the terminal (this may include programs included in the platform or operating system, etc. installed by default on the terminal), and the user may use the application store server, application, or It may also be executed by an application (i.e. program) installed directly on the master terminal through an application providing server such as a service-related web server.
  • an application i.e., program
  • the above-described chest medical image object detection method is implemented as an application (i.e., program) installed by default in the terminal or directly installed by the user and can be recorded on a computer-readable recording medium such as the terminal.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Medical Informatics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Public Health (AREA)
  • General Health & Medical Sciences (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biomedical Technology (AREA)
  • Pathology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biophysics (AREA)
  • Physics & Mathematics (AREA)
  • Molecular Biology (AREA)
  • Surgery (AREA)
  • Animal Behavior & Ethology (AREA)
  • High Energy & Nuclear Physics (AREA)
  • Optics & Photonics (AREA)
  • Veterinary Medicine (AREA)
  • Epidemiology (AREA)
  • Primary Health Care (AREA)
  • Data Mining & Analysis (AREA)
  • Databases & Information Systems (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

Des modes de réalisation de la présente invention peuvent fournir un système de détection d'un objet dans une image médicale , comprenant : une unité de collecte qui collecte une pluralité d'images radiologiques thoraciques et des interprétations d'images thoraciques de tomographie assistée par ordinateur correspondant aux images radiologiques thoraciques ; une unité de mise en correspondance qui reçoit les images radiologiques thoraciques et les interprétations des images thoraciques de tomographie assistée par ordinateur, met en correspondance un ou plusieurs objets enregistrés dans les interprétations des images thoraciques de tomographie assistée par ordinateur avec un rectangle englobant sur les images radiologiques thoraciques, et génère une annotation de rectangle englobant correspondant aux images radiologiques thoraciques ; une unité de génération de modèle qui génère un modèle de détection d'objet par l'application d'un algorithme de détection d'objet aux images radiologiques thoraciques reçues et à l'annotation de rectangle englobant ; une unité de détection d'objet qui détecte un objet par l'entrée d'une image radiologique thoracique d'un patient dans le modèle de détection d'objet ; et une unité de sortie qui délivre l'objet détecté.
PCT/KR2022/015341 2022-10-12 2022-10-12 Système et procédé de détection d'objet dans une image médicale thoracique Ceased WO2024080393A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/KR2022/015341 WO2024080393A1 (fr) 2022-10-12 2022-10-12 Système et procédé de détection d'objet dans une image médicale thoracique

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/KR2022/015341 WO2024080393A1 (fr) 2022-10-12 2022-10-12 Système et procédé de détection d'objet dans une image médicale thoracique

Publications (1)

Publication Number Publication Date
WO2024080393A1 true WO2024080393A1 (fr) 2024-04-18

Family

ID=90669833

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/KR2022/015341 Ceased WO2024080393A1 (fr) 2022-10-12 2022-10-12 Système et procédé de détection d'objet dans une image médicale thoracique

Country Status (1)

Country Link
WO (1) WO2024080393A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130057282A (ko) * 2011-11-23 2013-05-31 삼성전자주식회사 컴퓨터 기반 진단 방법 및 그에 따른 컴퓨터 기반 진단 장치
KR20200091329A (ko) * 2019-01-22 2020-07-30 주식회사 스트라드비젼 타겟 객체 예측 네트워크 및 타겟 객체 통합 네트워크를 이용하여 핵심성과지표와 같은 사용자 요구 사항에 따른 최적화를 위해 재구성 가능한 네트워크 기반의 객체 검출기를 학습하는 방법 및 학습 장치, 그리고 이를 이용한 테스팅 방법 및 테스팅 장치
US20200372998A1 (en) * 2019-05-21 2020-11-26 Verb Surgical Inc. Method and system for combining video, image, and audio data with text data for comprehensive data analytics
KR20230040484A (ko) * 2021-09-16 2023-03-23 가톨릭대학교 산학협력단 흉부 의료 영상 객체 검출 시스템 및 그 방법

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
KR20130057282A (ko) * 2011-11-23 2013-05-31 삼성전자주식회사 컴퓨터 기반 진단 방법 및 그에 따른 컴퓨터 기반 진단 장치
KR20200091329A (ko) * 2019-01-22 2020-07-30 주식회사 스트라드비젼 타겟 객체 예측 네트워크 및 타겟 객체 통합 네트워크를 이용하여 핵심성과지표와 같은 사용자 요구 사항에 따른 최적화를 위해 재구성 가능한 네트워크 기반의 객체 검출기를 학습하는 방법 및 학습 장치, 그리고 이를 이용한 테스팅 방법 및 테스팅 장치
US20200372998A1 (en) * 2019-05-21 2020-11-26 Verb Surgical Inc. Method and system for combining video, image, and audio data with text data for comprehensive data analytics
KR20230040484A (ko) * 2021-09-16 2023-03-23 가톨릭대학교 산학협력단 흉부 의료 영상 객체 검출 시스템 및 그 방법

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
WANG BOWEN; TAKEDA TOSHIHIRO; SUGIMOTO KENTO; ZHANG JIAHAO; WADA SHOYA; KONISHI SHOZO; MANABE SHIROU; OKADA KATSUKI; MATSUMURA YAS: "Automatic creation of annotations for chest radiographs based on the positional information extracted from radiographic image reports", COMPUTER METHODS AND PROGRAMS IN BIOMEDICINE, ELSEVIER, AMSTERDAM, NL, vol. 209, 4 August 2021 (2021-08-04), NL , XP086768011, ISSN: 0169-2607, DOI: 10.1016/j.cmpb.2021.106331 *

Similar Documents

Publication Publication Date Title
Wu et al. Development and evaluation of a deep learning algorithm for rib segmentation and fracture detection from multicenter chest CT images
US11972571B2 (en) Method for image segmentation, method for training image segmentation model
Tian et al. Deep multi‐instance transfer learning for pneumothorax classification in chest X‐ray images
WO2020122606A1 (fr) Procédé de mesure du volume d'un organe au moyen d'un réseau neuronal artificiel, et appareil associé
EP3971830B1 (fr) Procédé et appareil de segmentation de signe de pneumonie, support et dispositif électronique
CN111754453A (zh) 基于胸透图像的肺结核检测方法、系统和存储介质
CN110930414B (zh) 医学影像的肺部区域阴影标记方法、装置、服务器及存储介质
Eroğlu et al. Is it useful to use computerized tomography image-based artificial intelligence modelling in the differential diagnosis of chronic otitis media with and without cholesteatoma?
WO2019172498A1 (fr) Système de diagnostic assisté par ordinateur pour indiquer la malignité d'une tumeur, et base de déduction de la malignité et procédé associés
WO2021261808A1 (fr) Procédé permettant d'afficher un résultat de lecture de lésion
WO2021235804A1 (fr) Procédé et système de détermination d'anomalie de dispositif médical
WO2019143021A1 (fr) Procédé de prise en charge de visualisation d'images et appareil l'utilisant
CN114092475B (zh) 病灶长径确定方法、图像标注方法、装置及计算机设备
Pavlov et al. Application of deep learning techniques for detection of COVID-19 using lung CT scans: model development and validation
WO2025084546A1 (fr) Procédé et système d'analyse quantitative d'image cérébrale sur la base d'une image de tomodensitométrie
Yao et al. A deep adversarial model for segmentation-assisted COVID-19 diagnosis using CT images
Hamad et al. A deep learning model for segmentation of covid-19 infections using CT scans
KR102710082B1 (ko) 흉부 의료 영상 객체 검출 시스템 및 그 방법
WO2024080393A1 (fr) Système et procédé de détection d'objet dans une image médicale thoracique
Salama et al. Machine learning and deep learning covid-19 diagnosis system: key achievements, lessons learned, and a transfer learning algorithm: GM Salama et al.
WO2021246625A1 (fr) Système de plateforme en nuage à base d'intelligence artificielle permettant de lire une image médicale où un temps d'exécution attendu d'une couche individuelle est affiché
CN111507964A (zh) 新冠肺炎的集成学习定级方法、电子设备及存储介质
Bhagavan et al. A compressive survey on different image processing techniques to identify the brain tumor
Fukano et al. Recognition method of lung nodules using blood vessel extraction techniques and 3D object models
Miao et al. EVOKE: Elevating Chest X-ray Report Generation via Multi-View Contrastive Learning and Patient-Specific Knowledge

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22962144

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22962144

Country of ref document: EP

Kind code of ref document: A1