[go: up one dir, main page]

WO2024036374A1 - Procédés et systèmes d'analyse automatisée d'images médicales - Google Patents

Procédés et systèmes d'analyse automatisée d'images médicales Download PDF

Info

Publication number
WO2024036374A1
WO2024036374A1 PCT/AU2023/050778 AU2023050778W WO2024036374A1 WO 2024036374 A1 WO2024036374 A1 WO 2024036374A1 AU 2023050778 W AU2023050778 W AU 2023050778W WO 2024036374 A1 WO2024036374 A1 WO 2024036374A1
Authority
WO
WIPO (PCT)
Prior art keywords
none
visual
user
findings
anatomical images
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/AU2023/050778
Other languages
English (en)
Inventor
Ben Austin
Marc NORTHROP
Aengus TRAN
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Annalise Ai Pty Ltd
Original Assignee
Annalise Ai Pty Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from AU2022902333A external-priority patent/AU2022902333A0/en
Application filed by Annalise Ai Pty Ltd filed Critical Annalise Ai Pty Ltd
Publication of WO2024036374A1 publication Critical patent/WO2024036374A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/82—Arrangements for image or video recognition or understanding using pattern recognition or machine learning using neural networks
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/0464—Convolutional networks [CNN, ConvNet]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/09—Supervised learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T7/00—Image analysis
    • G06T7/0002—Inspection of images, e.g. flaw detection
    • G06T7/0012—Biomedical image inspection
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V10/00—Arrangements for image or video recognition or understanding
    • G06V10/70—Arrangements for image or video recognition or understanding using pattern recognition or machine learning
    • G06V10/77—Processing image or video features in feature spaces; using data integration or data reduction, e.g. principal component analysis [PCA] or independent component analysis [ICA] or self-organising maps [SOM]; Blind source separation
    • G06V10/778—Active pattern-learning, e.g. online learning of image or video features
    • G06V10/7784—Active pattern-learning, e.g. online learning of image or video features based on feedback from supervisors
    • G06V10/7788—Active pattern-learning, e.g. online learning of image or video features based on feedback from supervisors the supervisor being a human, e.g. interactive learning with a human teacher
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/40—ICT specially adapted for the handling or processing of medical images for processing medical images, e.g. editing
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/20—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for computer-aided diagnosis, e.g. based on medical expert systems
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H50/00—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics
    • G16H50/70—ICT specially adapted for medical diagnosis, medical simulation or medical data mining; ICT specially adapted for detecting, monitoring or modelling epidemics or pandemics for mining of medical data, e.g. analysing previous cases of other patients
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/04—Architecture, e.g. interconnection topology
    • G06N3/048—Activation functions
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06N—COMPUTING ARRANGEMENTS BASED ON SPECIFIC COMPUTATIONAL MODELS
    • G06N3/00—Computing arrangements based on biological models
    • G06N3/02—Neural networks
    • G06N3/08—Learning methods
    • G06N3/084—Backpropagation, e.g. using gradient descent
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10072—Tomographic images
    • G06T2207/10081—Computed x-ray tomography [CT]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/10—Image acquisition modality
    • G06T2207/10116—X-ray image
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20081—Training; Learning
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/20—Special algorithmic details
    • G06T2207/20084—Artificial neural networks [ANN]
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
    • G06T2207/00—Indexing scheme for image analysis or image enhancement
    • G06T2207/30—Subject of image; Context of image processing
    • G06T2207/30004—Biomedical image processing
    • G06T2207/30061—Lung
    • G—PHYSICS
    • G06—COMPUTING OR CALCULATING; COUNTING
    • G06V—IMAGE OR VIDEO RECOGNITION OR UNDERSTANDING
    • G06V2201/00—Indexing scheme relating to image or video recognition or understanding
    • G06V2201/03—Recognition of patterns in medical or anatomical images
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H30/00—ICT specially adapted for the handling or processing of medical images
    • G16H30/20—ICT specially adapted for the handling or processing of medical images for handling medical images, e.g. DICOM, HL7 or PACS

Definitions

  • the present invention generally relates to computer-implemented methods for analysing medical images, as well as computing systems, services, and devices implementing the methods.
  • Embodiments of the invention improve on analysis of medical images by allowing for user feedback to be processed during automated analysis of medical images employing machine learning techniques, in particular deep learning networks, such as convolutional neural networks, trained using substratification training, to enable error correction and thereby increase accuracy of the machine learning techniques.
  • machine learning techniques in particular deep learning networks, such as convolutional neural networks, trained using substratification training, to enable error correction and thereby increase accuracy of the machine learning techniques.
  • Methods, systems, services, and devices embodying the invention find applications, amongst others, in the clinical assessment of chest conditions such as pneumothorax and other radiological findings pertaining to the chest or head.
  • CXR chest x-ray
  • CXRs are used for acute triage as well as longitudinal surveillance.
  • a CXR is typically examined for any detectable abnormality in addition to the clinical indication for which it was ordered. This means that radiologists must be alert to identify many different conditions, with a concordant risk that some findings may be missed.
  • CXRs are particularly difficult to interpret.
  • the increasing demand for specialists that are qualified to interpret medical images i.e.
  • Empirical training has been used to assess medical imagery, in which mathematical models are generated by learning a dataset.
  • Deep learning is a particularly data- hungry subset of empirical training that is itself a subset of artificial intelligence (Al).
  • Al artificial intelligence
  • DNNs deep neural networks
  • the present invention seeks to address, individually and/or in combination, one or more of the foregoing needs and limitations of the prior art.
  • a computer implemented method for detecting a plurality of visual findings in one or more anatomical images of a subject comprising: providing one or more anatomical images of the subject; inputting the one or more anatomical images into a convolutional neural network (CNN) component of a neural network to output a feature vector; computing an indication of a plurality of visual findings being present in at least one of the one or more anatomical images by a dense layer of the neural network that takes as input the feature vector and outputs an indication of whether each of the plurality of visual findings is present in at least one of the one or more anatomical images; communicating the plurality of visual findings to a user system configured to receive feedback data associated with the plurality of visual findings; and transmitting the feedback data to the neural network; wherein the neural network is trained on a training dataset including, for each of a plurality of subjects, one or more anatomical images, and a plurality of labels associated with the one or more anatomical images and each of
  • CNN convolutional neural network
  • the visual findings may be radiological findings in anatomical images comprising one or more chest x-ray (CXR) images or computed tomography (CT) images.
  • CXR chest x-ray
  • CT computed tomography
  • embodiments of the invention may employ a deep learning model trained to detect/classify pneumothoraces from a CXR image.
  • AIMS Al model service
  • an Al model service (AIMS) 718 for Al processing and generating predictions may identify and returns the predicted radiological findings generated by the deep learning models executed by a machine learning prediction service.
  • Deep learning models embodying the invention can be trained to detect/classify a very high number of visual findings and then re-trained to improve detection/classification based on user feedback.
  • Such models may have been trained and re-trained using CXR images (pixel data) where, in one example, labels were provided for each of the findings (including corresponding to visual findings and feedback data), enabling the deep learning models to be trained to detect combinations of findings, while preventing the models from learning incorrect correlations and improving the model predictions following feedback data.
  • methods according to embodiments of the invention enable monitoring the performance of the Al model over a long period of time to allow for retraining of the Al model 708 as needed, in situations where the generated visual findings are not accurate and may be improved upon.
  • the method allows for user (e.g. radiologist) feedback to be received, processed and transmitted to a server module for correcting errors with the Al model 708 when the prediction generated was incorrect and the user indicated so, so that the next version of the Al model trained with new data which includes the ground truthed feedback data, is more accurate.
  • Feedback data may indicate for example, a radiological finding being missed from the study, a radiological finding added by a user, or rejecting a predicted radiological finding (i.e. indicating the finding is incorrect), amongst others.
  • the method may further comprise computing a segmentation mask indicating a localisation for at least one of the plurality of visual findings, wherein feedback data comprises an indication of incorrect localisation.
  • the method additionally enables detecting very bad predictions such as 10’s of lungs modules identified instead of labelling a visual finding as cystic fibrosis, or wrongly identifying visual findings which are in fact due to errors in the hardware imaging equipment such as a scratch x-ray hospital plate. This allows for a more accurate, reliable, and secure system.
  • An automated analysis of anatomical images using deep learning models is improved by enabling the user to review the results of such automated analysis and provide feedback/corrective information in relation to a radiological finding that may have been missed or incorrectly predicted by the automated analysis process, and using this information to train one or more improved deep learning model(s).
  • the method may further comprise displaying at least one of the one or more anatomical images of the subject and receiving a user selection of one or more areas of the anatomical image(s) and/or a user-provided indication of a first visual finding.
  • a user-provided indication of a first visual finding may be received by the user selecting a first visual finding from a displayed list of visual findings, or by the user typing or otherwise entering a first visual finding.
  • the method comprises receiving both a user selection of one or more areas of the anatomical image(s) and a user- provided indication of a first visual finding associated with the user-selected one or more areas.
  • the method further comprises recording the user selected one or more areas of the anatomical image(s) and/or the user provided indication of the first visual finding in a memory, associated with the one or more anatomical image(s).
  • the method may further comprise using the user-selected one or more areas of the anatomical image(s) and/or the user-provided indication of the first visual finding to train a deep learning model to detect the presence of at least the first visual finding in anatomical images and/or to train a deep learning model to detect areas showing at least the first visual finding in anatomical images.
  • the deep learning model trained to detect areas showing at least the first visual finding in anatomical images may be different from the deep learning model that trained to detect the presence of at least the first visual finding in anatomical images.
  • Using the user-selected one or more areas of the anatomical image(s) and/or the user- provided indication of the first visual finding to train a deep learning model to detect the presence of at least the first visual finding in anatomical images may comprise at least partially re-training the deep learning model that was used to produce the first value.
  • Using the user-selected one or more areas of the anatomical image(s) and/or the user- provided indication of the first visual finding to train a deep learning model to detect the areas showing at least the first visual finding in anatomical images may comprise at least partially retraining the deep learning model that was used to produce a segmentation map indicating the areas of the anatomical image(s) where the first visual finding has been detected.
  • the method may further comprise displaying a list of visual findings on a user interface, wherein the list of visual findings comprises a first sublist comprising one or more visual findings not present in the one or more anatomical images of a subject, the user interface configured to allow a user selecting a visual finding from the first sublist for a displayed one or more anatomical images, wherein the user feedback comprises the selected visual finding and associated one or more anatomical images.
  • the user may manually input the missing visual finding.
  • transmitting feedback data is carried out by a de-identification module configured to remove identification information from the feedback data thereby providing de-identified feedback data.
  • the plurality of visual findings are provided by a server module to an integration layer module, wherein the integration layer module is configured to transmit the feedback data to the server module.
  • the integration layer module does not play a role in the feedback feature described in at least one embodiment of the present invention.
  • the integration layer module comprises a database for storing the feedback data temporarily and the temporary period of time is user configurable. This improves on data security aspects of the system.
  • the plurality of labels associated with at least a subset of the one or more CXR or CT images and each of the respective visual findings in the training dataset may be derived from the results of review of the one or more anatomical images by at least one expert together with feedback data.
  • the plurality of labels for the subset of the images in the training dataset are advantageously derived from the results of review of the one or more images input as feedback data by at least two experts, preferably at least three experts.
  • the plurality of labels for the subset of the images in the training dataset may be obtained by combining the results of review and feedback data input in respect of the one or more anatomical images by a plurality of experts.
  • the plurality of labels associated with the one or more CXR images in the training dataset represent a probability of each of the respective visual findings being present in the at least one of the one or more CXR images of a subject.
  • Labelling using a plurality of labels organised as a hierarchical ontology tree may be obtained through expert review and feedback as explained above.
  • a plurality of labels associated with at least a subset of the one or more chest x-ray images and each of the respective visual findings in the training dataset may be derived from the results of review and feedback data input in respect of the one or more anatomical images by at least one expert using a labelling tool that allows the expert to select labels presented in a hierarchical object (such as e.g. a hierarchical menu).
  • the indication of whether each of the plurality of visual findings is present in at least one of the one or more CXR images represents a probability of the respective visual finding being present in at least one of the one or more CXR images.
  • the plurality of labels associated with at least a further subset of the one or more CXR images and each of the respective visual findings in the training dataset are derived from an indication of the plurality of visual findings being present in at least one of the one or more CXR images obtained using a previously trained neural network and user feedback.
  • the neural network is trained by evaluating the performance of a plurality of neural networks (the plurality of neural networks being trained from a labelled dataset generated via consensus of radiologists) in detecting the plurality of visual findings and in detecting the localisation of any of the plurality of visual findings that are predicted to be present.
  • a system for receiving, processing and transmitting feedback data for a plurality of visual findings in one or more anatomical images of a subject wherein the plurality of visual findings are generated using a convolutional neural network (CNN) component of a neural network
  • CNN convolutional neural network
  • the system comprising: at least one processor; and at least one computer readable storage medium, accessible by the processor, comprising instructions that, when executed by the processor, cause the processor to execute a method as described above.
  • non-transitory computer readable storage media comprising instructions that, when executed by at least one processor, cause the processor to execute a method as described above.
  • Figure 1 is a block diagram of an exemplary architecture of a medical image analysis system embodying the invention
  • Figure 2 is another block diagram of an exemplary architecture of a medical image analysis system embodying the invention
  • Figure 3A is a signal flow diagram illustrating an exemplary method for processing of imaging study results within the embodiments of Figure 1 or Figure 2;
  • Figure 3B is another signal flow diagram illustrating an exemplary method for processing of imaging study results within the embodiments of Figure 1 or Figure 2;
  • Figures 4A to 4G show exemplary interactive user interface screens of a viewer component embodying the invention;
  • Figures 5A and 5B show further exemplary interactive user interface screens of a viewer component embodying the invention
  • Figure 6 is an exemplary report generated on an interactive user interface embodying the invention.
  • Figure 7 illustrates a computer implemented method for detecting a plurality of visual findings in one or more anatomical images of a subject.
  • a system 10 comprises modular system components in communication with each other, including a server system 70 configured to send predicted radiological findings, and receive feedback data/corrective information associated with the radiological findings, via an integration layer module 702.
  • the integration layer module 702 includes at least one local database (not shown) and a processor 800. The predicted radiological findings and feedback data received by the integration layer 702 and stored in the local database before being queued to the processor 800.
  • the system 10 enables processing of feedback data/corrective information to improve the Al model 708 run by the Al model service (AIMS) 718.
  • AIMS Al model service
  • the modular components make it highly configurable by users and radiologists in contrast to prior art systems which are rigid and inflexible and cannot be optimised for changes in disease prevalence and care settings.
  • Another benefit of a modular systems architecture comprising asynchronous microservices is that it enables better re-usability, workload handling, and easier debugging processes (the separate modules are easier to test, implement or design).
  • the system 10 provides an interface specification that allows external applications (patient worklists) to communicate with the system 10 and receive the predicted radiological findings in a more efficient and safe manner, including providing the functionality to receive and process user feedback/corrective information that enables the retraining of Al models 708.
  • the system 100 further comprises a radiology image analysis server (RIAS) 110.
  • An exemplary RIAS 110 is based on a microservices architecture, and comprises a number of modular software components developed and configured in accordance with principles of the present invention.
  • the RIAS 110 receives anatomical image data that is transmitted from a source of anatomical image data, for example, where the anatomical image data captured and initially stored such as a radiological clinic or its data centre.
  • the transmission may occur in bulk batches of anatomical image data and prior to a user having to provide their decision/clinical report on a study.
  • the transmission may be processed, controlled and managed by an integration layer (comprising integrator services of an integration adapter) installed at the radiological clinic or its data centre, or residing at cloud infrastructure.
  • an integration layer comprising integrator services of an integration adapter
  • the RIAS 110 provides analysis services in relation to anatomical images captured by and/or accessible by user devices, such as radiology termin als/workstations, or other computing devices (e.g. personal computers, tablet computers, and/or other portable devices - not shown).
  • the anatomical image data is analysed by one or more software components of the RIAS 110, including through the execution of machine learning models.
  • the RIAS 110 then makes the results of the analysis available and accessible to one or more user devices.
  • the processor 800 may check if the predicted radiological findings are “white-listed” for the RIAS 110.
  • a white-list may be assigned for example using a Digital Imaging and Communications in Medicine (DICOM) tag for the user institution (RIAS) name.
  • DICOM Digital Imaging and Communications in Medicine
  • the processor received and transmits DICOM data, sitting between a DICOM receiver 8001 and a DICOM transmitter 8002. It is possible to select system functionality by enabling or disabling the feedback functionality; this increases system flexibility and configurability.
  • a user (i.e. radiologist) using the RIAS 110 can flag incorrect studies or provide user feedback/corrective information.
  • a ruleset/model running in the integration layer 702 can also provide feedback before the radiological predictions 702 are transmitted to the RIAS 110.
  • Feedback data can indicate one or more of the following non-exhaustive list:
  • a “radiology image” in this context may be any anatomical image including a chest X-ray image (CXR) or a CT image or the brain and/or head.
  • the integration layer 702 may receive worklist priority data from the RIAS 110 representing a user’s worklist for a radiologist (i.e. a user), along with associated data which includes a patient identification (ID) and customer account number for example.
  • the predicted radiological findings and feedback data are transmitted via the integration layer 702 comprising integrator services, the integration layer 702 connecting to an injection layer, the clinical ranking data being processable by the RIAS 110, advantageously communicating to the RIAS 110 the predicted Al radiology findings, and receiving user feedback data in a timely manner.
  • the integration layer 702 communicates the data to predicted radiological findings to the RIAS 110.
  • the RIAS 110 forwards this data to an interactive viewer component 701 , which communicates the predicted Al radiology findings to the user and receives user feedback associated with the Al radiology findings, to communicate user feedback to the server 70.
  • the system 10 comprises modular components which enable multiple integration and injection pathways to facilitate interoperability and deployment in various existing computing environments such as Radiology Information Systems Picture Archiving and Communication System (RIS-PACS) systems from various vendors and at different integration points such as via APIs or superimposing a virtual user interface element on the display device of the radiology terminals/workstations.
  • a PACS server 111 is shown in Figure 1 .
  • the virtual user interface element may be the interactive viewer component 701 , which has the functionality to allow a user to provide feedback data in respect of the predicted radiological findings.
  • the system further comprises a de-identification module 900 comprising a deidentification processor 90 for data de-identification.
  • Data from all sources including feedback data is de-identified and DICOM tags are removed.
  • Protected health information is removed or irreversibly anonymised from reports and images through an automated de-identification process.
  • Image data is preserved at the original resolution and bit-depth.
  • Patient IDs and Study IDs are anonymised to de-identify them while retaining the temporal and logical association between studies and patients.
  • a distributed message queueing service (DMQS) 710 stores user feedback metadata.
  • User feedback data associated with the results ID is stored in a separate database, in this example a cloud imaging processing service (CIPS) 7060.
  • the primary functions of the CIPS 7060 are to: handle feedback data, handle image storage; handle image conversion; handle image manipulation; store image references and metadata to studies and predicted radiological findings; handle image type conversions (e.g. JPEG2000 to JPEG) and store the different image types, store segmentation image results from the Al model(s) 708; manipulate segmentation PNGs by adding a transparent layer over black pixels; handle and store feedback data, and provide open API endpoints for the viewer component 701 to request segmentation maps and radiological images (in a compatible image format expected by the viewer component 701).
  • handle image type conversions e.g. JPEG2000 to JPEG
  • manipulate segmentation PNGs by adding a transparent layer over black pixels
  • handle and store feedback data and provide open API endpoints for the viewer component 701 to request segment
  • API endpoints there are two API endpoints: one for user feedback, one for study images and associated metadata.
  • Providing multiple endpoints supports granularity of configuration, thereby enhancing flexibility and configurability to save and/or update user feedback for example.
  • Preferably user feedback identified by a user feedback ID may be provided even in situations where a study has an error associated with it.
  • Example code snippets of API settings are provided as follows:
  • feedback data is stored against the columns listed below.
  • feedback data may be used to distinguish between cases where: predicted radiological findings are rejected by a user, predicted radiological findings have been missed by the Al model 708, predicted radiological findings are added by the user, feedback has affected the report/patient management or recommendations.
  • a microservice is responsible for acquiring data from the server 70 (via the integration layer 702 shown in Figure 1) to send the CXR images to the Al model 708 for generating predicted radiological findings and then sending back the prioritised predicted findings to the server 70.
  • the microservice is also responsible for storing study-related information, CXR images, predicted radiological findings and user feedback data including metadata.
  • a gateway service may provide monitoring and security control, to function as the entry point for all interactions with a microservice for communicating with an AIMS 718 within the server system 70.
  • the server 70 sends a “study predict” request comprising an entire study, and which may include associated metadata, i.e. scan, series and CXR images. Additionally, at step 9000, the server sends user feedback metadata. The request, user feedback and other associated data are received by a distributed message queueing service (DMQS) 710.
  • DQS distributed message queueing service
  • the request is stored in the CIPS database 7060 as described above.
  • the DMQS 710 accepts incoming HTTP requests and listens on queues for message from the server 70 (optionally via a gateway) and a model handling service (MHS) 716.
  • the DMQS 710 is configured to pass, at step 7840, CXR images to the MHS 716 for the model prediction pipeline.
  • the DMQS 710 may store studies, CXR images, and deep learning predictions, into a database managed by a database management service (not shown).
  • the DMSQ 710 also manages each study’s model findings state and stores the prioritised predicted radiological findings predicted by the Al models 708, stores errors when they occur in a database, accepts HTTP requests to send study data including model predictions for radiological findings, accepts HTTP requests to send the status of study findings, and forwards CXR images and related metadata to the MHS 716 for processing of the predicted radiological findings.
  • the DMSQ 710 also stores and sends user feedback metadata to the CIPS database 7060 as described above.
  • the MHS 716 is configured to accept DICOM compatible CXR images and metadata from the DMQS 710.
  • the MHS 716 also performs validation, and pre-processing to transform study data into JSON format, which may then be further transformed into a suitable format for efficient communication within the microservice.
  • the MHS 716 sends, at step 7860 the study data to the Al model service (AIMS) 718 for Al processing, which identifies and returns the predicted radiological findings generated by the deep learning models executed by a machine learning prediction service.
  • the study data includes organisation thresholds; thresholds not transmitted to AIMS may be rescheduled (see also Table 1).
  • the MHS 716 accepts the predicted radiological findings generated by the deep learning models which are returned via the AIMS 718.
  • the MHS 716 segments (at step 7920), validates, and transforms the prioritized predicted radiological findings (e.g. including predictor, threshold and segments data) representing including CXR data together with clinical ranking data predicted by the Al model 708 into JSON format and returns these, at step, 7940, to the DMQS 710.
  • each JSON file returned corresponds to a single patient study.
  • the DMQS 710 sends, at step 7960, the CXR data together with the visual findings predicted by the re-trained Al model 708 to the integration layer 702 (e.g. via a dispatch service module, not shown).
  • the system may track:
  • Capture feedback (scoped as part of main enhance feedback feature)
  • example feedback data to be transmitted to the server 70 may comprise:
  • organisation data may comprise:
  • Figure 3B shows exemplary steps of feedback data transmission between a server 70 and a client 7000, in a looped manner.
  • Exemplary code snippets of setting are provided below: enum SettingsType ⁇
  • CxrSettings extends BaseSettings ⁇ type: SettingsType. CXR; version: "1"; assign: ⁇ priorities: ⁇ assignPriorityld: number; rank: number; priority: string;
  • CxrAiSettings extends BaseSettings ⁇ type: SettingsType.CXR_AI; version: "1"; modelVersion: string; labels: ⁇ label: string; predictionThreshold: number;
  • ⁇ type FeedbackT rialQuestionType. CHECKBOX; answer: boolean;
  • FreeFormFeedbackTrialAnswer extends BaseFeedbackT rialAnswer
  • PostFeedbackReq ⁇ predictionld: string; username: string; organizationSettingsId: string; studyAccessId: string; answers: FeedbackAnswerfl; findingsAdded: string ⁇ ]; findingsRejected: string ⁇ ]; findingsInaccurateSegment: string ⁇ ]; findingsValuable: string ⁇ ]; isSubmitted: boolean; flaggedWith: string ⁇ ];
  • viewedld may be a table that links together
  • Users may reject a radiological finding predicted by the Al model 708. Users may add a radiological finding that was not predicted by the Al model 708.
  • An auto-complete feature will enable a user to partially type a substring of a radiological finding in text and a suggestion to auto-complete the radiological finding in text is derived from the text information of the ontology tree for the particular imaging modality/body part. The addition of a new radiological finding will increment a counter of the number of radiological findings in a modality user interface component.
  • Users may manually indicate an incorrect localisation/segmentation predicted by the Al model 708. This indication may be a flag icon that may be pressed via a mouse click. Users may also flag if the image or slice is an eligible series. Users may also indicate if a radiological finding is important, either in terms of clinical significance or otherwise.
  • the feedback mode described may be disabled by the user.
  • User feedback data is able to be extracted from the backend for a customer with the following data: date and time feedback was submitted, accession number, study instance UID, Al prediction UID, Al prediction status, user name, model feedback responses, and whether the submit button was clicked by the user.
  • the feedback data may then be transmitted from the database to Cl PS 7060 for further processing.
  • the processed feedback data is communicated to AIMS 718 in order to retrain a new Al Model 708 including new weights and biases derived from the feedback data.
  • user feedback stored in the system 10 can be extracted out and shared back to a customer (i.e. the radiology clinic) for example for product evaluation and monitoring and customisation of outputs.
  • user feedback can be extracted by the system 10 along with the images.
  • This extraction process may involve use of a de-identification tool to de-identify the data (reports and images have any personal information removed or irreversibly anonymised), and storing the data in databases or file storage systems (such as an Amazon S3 bucket). This then creates a common pool of deidentified data aggregated from all of our customers that can be queried using software to produce analytics data about the system 10 and identify areas for potential performance improvements of the Al model 708.
  • a set of possible visual findings may be determined according to an ontology tree. These may be organised into a hierarchical structure or nested structure. The use of a hierarchical structure for the set of visual findings may lead to an improved accuracy of prediction as various levels of granularity of findings can be simultaneously captured, with increasing confidence when going up the hierarchical structure.
  • a dataset of x-ray images may be used.
  • a sub-dataset consisting solely of anatomical CXR images is preferably used for radiological findings.
  • Each set of anatomical images may include two or more images of a body portion of the respective subject depicting a respective different orientation of at least the body portion of subject.
  • Each of the CXR images is an anatomical x-ray image of a human being’s chest associated with a set of labels manually annotated by expert radiologists for example using a chest x-ray software labelling tool.
  • each label indicates whether a particular radiological finding was identified by one or more expert reviewers.
  • a label derived from a plurality of expert reviews and feedback data may be obtained via algorithms that quantify the performance and/or uncertainty of independent reviews combined, e.g. using a vote aggregation algorithm, for example, the Dawid-Skene algorithm. These labels can then be used to train and re-train a deep neural network for findings within CXR images.
  • the estimated radiological finding probability generated via the Dawid-Skene algorithm is an estimated probability of the presence of each finding rather than a binary label. This is a better reflection of the likelihood of each finding and can be used as additional training signal for the deep learning model. As such the deep learning model is trained to minimise the difference between the predicted score compared to the David-Skene algorithm output directly.
  • CNN convolutional neural network
  • a CNN is a neural network that comprises convolution operators which apply a spatial filter over pixels of the input image to calculate a single output pixel, shift the spatial filter to calculate another output pixel and so on until all output pixels of an output image are calculated.
  • That output image is also referred to as a feature map as it maps the feature against the input image.
  • the step from one input image to multiple feature maps is referred to as one layerof the CNN.
  • the CNN can have multiple further layers where further convolution operators are applied to the feature maps generated by a previous layer.
  • Other types of layers that may aid the prediction process and that may be included between the convolutional layers are pooling layers and fully connected layers (also referred to as “dense” layers).
  • the network comprises multiple layers with at least one hidden layer and potentially including the same type of layer multiple times at different depths (i.e. number of layers from the input), such a neural network is referred to as a “deep” neural network, which is in contrast to a neural network with only a single layer, such as a perceptron.
  • the deep neural network has more than 19 layers, more than 150 layers or more than 1 ,000 layers.
  • the output values of the last layer of the CNN are together referred to as a feature vector.
  • the CNN after the last layerof the CNN there is a dense (e.g., fully connected layer) which takes the feature vector and combines values of the feature vector according to the weights of the dense layer.
  • the dense layer has multiple outputs and each of the outputs is a differently weighted combination of the input values of the dense layer.
  • Each output of the dense layer is associated with a visual finding.
  • the dense layer computes an indication of a plurality of visual findings being present in at least one of the one or more anatomical images.
  • the dense layer of the neural network takes as input the feature vector and outputs an indication of whether each of the plurality of visual findings is present in at least one of the one or more anatomical images.
  • the neural network may be trained using backpropagation, which involves the calculates of the gradient of a loss function with respect to the weights of the network, for a single input-output example. A gradient descent method can then be used to minimise the loss and thereby find the optimal weights.
  • CNNs While specific examples are provided herein, it is to be noted that a wide range of different CNNs can be used. For example, LeNet, GoogleNet, ResNet, Inseption, VGG and AlexNet architectures can be used.
  • a window or dialog box 1800 is provided from which the user is able to select a “feedback” button 1802 located, in this example, at the bottom of the visual findings list 1804.
  • a feedback mode meaning that the feedback functionality of viewer component 701 is enabled and ready to allow a user to input feedback on the Al model 708 performance.
  • This data may then be transmitted to the server 70 to help improve Al model 708 performance.
  • the user feedback is preferably saved automatically when the user selects a new study or exits feedback mode.
  • the user can indicate that there are Al model errors present in the study, and optionally provide more specific feedback.
  • the following categories of specific feedback may be provided:
  • the system may display an error message.
  • Example error messages include for example: incorrect “Not a CXR” error or incorrect “No frontal image” error.
  • the user input may indicate that “This is a CXR”.
  • the user input may indicate that there is “No eligible series available for processing”.
  • this indication may specifically mean “this is a contrast CTB”, “this is a bone window CTB”, “this is not a CT of the brain” or “this is not a CXR”.
  • the image or pixel data displayed does not reflect the user’s expectation of what they thought they would be see (see also “incorrect study flag” button 1906 in Figures 5A, 5B).
  • the system may display an error message.
  • Example error messages include for example “No axial images available” or incorrect “Not a non-contrast CTB” error.
  • Possible exemplary scenarios of user feedback for incorrect CT findings include:
  • missing visual findings i.e. visual findings which have not been generated by the Al model 708.
  • the user may select a “missing finding” button 1810 which results in the viewer displaying a field 1810 comprising a list of visual findings 1812 (see Figure 4D).
  • the user may search the list 1812 and select the missing visual finding to be added to the study. Where the missing visual finding the user is searching for is not present in the list 1812, the user may select an “Add New” study finding button 1814 and manually input the visual finding name.
  • predicted search functionality requires three letters to be entered by the user before any predicted text is displayed (“Acute rib fracture” as shown in Figure 4E).
  • the missing finding is added to the list, it is displayed in a field under a “user added” sublist 1818 of the list 1812.
  • a manually added visual finding can be removed by clicking a “cancel” button 1820 displayed in this example to the right of the added visual finding name.
  • the feedback button 1802 may change colour or brightness for example, when switching between feedback mode off (where no feedback is provided) to feedback model on (e.g. where a study is flagged as containing Al model errors).
  • Figures 5A and 5B display views of another example of a window or dialog box 1900 this time for displaying a CT brain study.
  • a user may input feedback data such as adding or rejecting a visual finding, or indicating incorrect localisation/segmentation.
  • the feedback mode is enabled by default. It will be appreciated that users (organisations) may configure fields displaying questions or checkboxes. In this example, a particular important visual finding (e.g. vasogenic oedema) may be flagged when a user selects an “important finding” button 1902 (star shaped in this example) available for each visual finding listed.
  • a particular important visual finding e.g. vasogenic oedema
  • an “important finding” button 1902 star shaped in this example
  • Figure 6 is an exemplary report 600 generated on an interactive user interface.
  • an additional output may be generated in the form of report.
  • the visual findings present is described in sentences, rather than in a discrete list, such as the visual findings list 1804 shown in Figure 4A.
  • the report may be generated using one or more machine learning models, such as natural language programming (NLP) models.
  • NLP natural language programming
  • the report 600 may be customisable to communicate the plurality of visual findings in a desired manner.
  • the generated report 600 may hyperlink 610 each of the visual findings that are present in at least one of the one or more anatomical image, such that, the associated anatomical image is displayed with a segmentation mask indicating a localisation of the visual finding upon a user interacting with the hyperlink 610.
  • a report 600 enables the detected visual findings to be communicated to the user in a more efficient manner.
  • the user may be a radiologist and a report is generated to model a radiographical report format that is readily understandable by the radiologist.
  • a user may edit the report 600 generated on an interactive user interface to provide feedback data. More specifically, the differences between the initially generated report and the edited report may be used as feedback data to train the neural network.
  • the report 600 shown in Figure 6 may be generated on an interactive user interface, for which the user is a radiologist.
  • the bold text 620 is not be editable by the user, whereas the text 630 is editable, which enables the user to provide a correction and create feedback data.
  • the plurality of visual findings communicated on the generated report are associated with at least one of the one or more anatomical images, which may also be provided on the interactive user interface for the user to view.
  • the user may determine that pneumothorax is present in the associated anatomical image, despite the report stating that pneumothorax is not present.
  • the user may edit the text 630 and write “Pneumothorax is present” or “Pneumothorax is detected”, for example.
  • the user may then submit the edited report, and the user system generates the feedback data as the user system detects a difference between the initially generated report and the edited report submitted by the user.
  • the user system may compare the text 600 from both reports, determine that the visual finding is “Pneumothorax” and determine that the visual finding is present in the associated anatomical image.
  • the user system may use one or more algorithms and/or one or more machine learning models, such as NLP models, to compare the two reports and generate the feedback data.
  • Figure 7 illustrates a computer implemented method 700 for detecting a plurality of visual findings in one or more anatomical images of a subject.
  • the method comprises providing 701 one or more anatomical images of the subject and inputting 702 the one or more anatomical images into a convolutional neural network (CNN) component of a neural network to output a feature vector as disclosed herein.
  • Method 700 further comprises computing 703 an indication of a plurality of visual findings being present in at least one of the one or more anatomical images by a dense layer of the neural network that takes as input the feature vector and outputs an indication of whether each of the plurality of visual findings is present in at least one of the one or more anatomical images.
  • CNN convolutional neural network
  • Method 700 also comprises communicating 704 the plurality of visual findings to a user system configured to receive feedback data associated with the plurality of visual findings; and transmitting the feedback data to the neural network.
  • the neural network is trained on a training dataset including, for each of a plurality of subjects, one or more anatomical images, and a plurality of labels associated with the one or more anatomical images and each of the respective visual findings, the plurality of labels comprising labels obtained using the transmitted feedback data.
  • Method 700 may be implemented in a computer system described below.
  • processors may include general purpose CPUs, digital signal processors, GPUs, and/or other hardware devices suitable for efficient execution of required programs and algorithms.
  • Computing systems may include conventional personal computer architectures, or other general-purpose hardware platforms.
  • Software may include open-source and/or commercially available operating system software in combination with various application and service programs.
  • computing or processing platforms may comprise custom hardware and/or software architectures.
  • computing and processing systems may comprise cloud computing platforms, enabling physical hardware resources, including processing and storage, to be allocated dynamically in response to service demands.
  • processing unit ‘component’, and ‘module’ are used in this specification to refer to any suitable combination of hardware and software configured to perform a particular defined task.
  • a processing unit, components, or modules may comprise executable code executing at a single location on a single processing device, or may comprise cooperating executable code modules executing in multiple locations and/or on multiple processing devices.
  • cooperating service components of the cloud computing architecture described above it will be appreciated that, where appropriate, equivalent functionality may be implemented in other embodiments using alternative architectures.
  • the program code embodied in any of the applications/modules described herein is capable of being individually or collectively distributed as a program product in a variety of different forms.
  • the program code may be distributed using a computer readable storage medium having computer readable program instructions thereon for causing a processor to carry out aspects of the embodiments of the invention.
  • Computer readable storage media may include volatile and non-volatile, and removable and non-removable, tangible media implemented in any method or technology for storage of information, such as computer-readable instructions, data structures, program modules, or other data.
  • Computer readable storage media may further include random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), electrically erasable programmable readonly memory (EEPROM), flash memory or other solid state memory technology, portable compact disc read-only memory (CD-ROM), or other optical storage, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other medium that can be used to store the desired information and which can be read by a computer.
  • Computer readable program instructions may be downloaded via transitory signals to a computer, another type of programmable data processing apparatus, or another device from a computer readable storage medium or to an external computer or external storage device via a network.
  • Computer readable program instructions stored in a computer readable medium may be used to direct a computer, other types of programmable data processing apparatus, or other devices to function in a particular manner, such that the instructions stored in the computer readable medium produce an article of manufacture including instructions that implement the functions, acts, and/or operations specified in the flowcharts, sequence diagrams, and/or block diagrams.
  • the computer program instructions may be provided to one or more processors of a general purpose computer, a special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the one or more processors, cause a series of computations to be performed to implement the functions, acts, and/or operations specified in the flowcharts, sequence diagrams, and/or block diagrams.
  • Software components embodying features of the invention may be developed using any suitable programming language, development environment, or combinations of languages and development environments, as will be familiar to persons skilled in the art of software engineering.
  • suitable software may be developed using the Typescript programming language, the Rust programming language, the Go programming language, the Python programming language, the SQL query language, and/or other languages suitable for implementation of applications, including webbased applications, comprising statistical modelling, machine learning, data analysis, data storage and retrieval, and other algorithms.
  • Implementation of embodiments of the invention may be facilitated by the used of available libraries and frameworks, such as TensorFlow or PyTorch for the development, training and deployment of machine learning models using the Python programming language.
  • embodiments of the invention involve the preparation of training data, as well as the implementation of software structures and code that are not well-understood, routine, or conventional in the art of anatomical image analysis, and that while pre-existing languages, frameworks, platforms, development environments, and code libraries may assist implementation, they require specific configuration and extensive augmentation (i.e. additional code development) in order to realize various benefits and advantages of the invention and implement the specific structures, processing, computations, and algorithms described herein with reference to the drawings.
  • any of the embodiments in the present description is not essential, unless required by context or otherwise specified. Therefore, most steps may be performed in any order.
  • any of the embodiments may include more or fewer steps than those disclosed.
  • TFOpLambda tf.concat (TFOpLambda) (None, 2560) 0 tf. math .red u ce_max[0] [0] tf.math. reduce_mean[0][0] dropout (Dropout) (None, 2560) 0 tf.concat[0][0] tf.reshape_1 (TFOpLambda) (None, None, 512, 512, 16) 0 model_1 [0][0] tf. operators .getitem[0][0] tf.
  • Non-trainable params 43,040 Table 3 - Model example 2 (model_1 above)
  • block2a_expand_activation (None, 512, 512, 96) 0 block2a_expand_bn[0][0] (Activation) block2a_dwconv (None, 256, 256, 96) 864 block2a_expand_activation[0][0]
  • Block2a_bn (None, 256, 256, 96) 384 block2a_dwconv[0][0] (BatchNormalization) block2a_activation (Activation) (None, 256, 256, 96) 0 block2a_bn[0][0] block2a_se_squeeze (None, 96) 0 block2a_activation[0][0]
  • Block2b_activation (Activation) (None, 256, 256, 144) 0 block2b_bn[0][0] block2b_se_squeeze (None, 144) 0 block2b_activation[0][0]
  • block2b_drop (FixedDropout) (None, 256, 256, 24) 0 block2b_project_bn[0][0] block2b_add (Add) (None, 256, 256, 24) 0 block2b_drop[0][0] block2a_project_bn[0][0] block3a_expand_conv (None, 256, 256, 144) 3456 block2b_add[0][0]
  • block3a_bn (DepthwiseConv2D) block3a_bn (None, 128, 128, 144) 576 block3a_dwconv[0][0] (BatchNormalization) block3a_activation (Activation) (None, 128, 128, 144) 0 block3a_bn[0][0] block3a_se_squeeze (None, 144) 0 block3a_activation[0][0]
  • block3b_bn (DepthwiseConv2D) block3b_bn (None, 128, 128, 240) 960 block3b_dwconv[0][0] (BatchNormalization) block3b_activation (Activation) (None, 128, 128, 240) 0 block3b_bn[0][0] block3b_se_squeeze (None, 240) 0 block3b_activation[0][0]
  • block3b_drop (FixedDropout) (None, 128, 128, 40) 0 block3b_project_bn[0][0] block3b_add (Add) (None, 128, 128, 40) 0 block3b_drop[0][0] block3a_project_bn[0][0] block4a_expand_conv (None, 128, 128, 240) 9600 block3b_add[0][0]
  • block4a_bn (DepthwiseConv2D) block4a_bn (None, 64, 64, 240) 960 block4a_dwconv[0][0] (BatchNormalization) block4a_activation (Activation) (None, 64, 64, 240) 0 block4a_bn[0][0] block4a_se_squeeze (None, 240) 0 block4a_activation[0][0]
  • Block4b_activation (Activation) (None, 64, 64, 480) 0 block4b_bn[0][0] block4b_se_squeeze (None, 480) 0 block4b_activation[0][0]
  • block4b_drop (FixedDropout) (None, 64, 64, 80) 0 block4b_project_bn[0][0] block4b_add (Add) (None, 64, 64, 80) 0 block4b_drop[0][0] block4a_project_bn[0][0] block4c_expand_conv (None, 64, 64, 480) 38400 block4b_add[0][0]
  • Block4c_activation (Activation) (None, 64, 64, 480) 0 block4c_bn[0][0] block4c_se_squeeze (None, 480) 0 block4c_activation[0][0]
  • Block4c_drop (FixedDropout) (None, 64, 64, 80) 0 block4c_project_bn[0][0] block4c_add (Add) (None, 64, 64, 80) 0 block4c_drop[0][0] block4b_add[0][0] block5a_expand_conv (None, 64, 64, 480) 38400 block4c_add[0][0] (Conv2D) block5a_expand_bn (None, 64, 64, 480) 1920 block5a_expand_conv[0][0] (BatchNormalization) block5a_expand_activation (None, 64, 64, 480) 0 block5a_expand_bn[0][0]
  • Block5b_dwconv (None, 64, 64, 672) 16800 block5b_expand_activation[0][0] (DepthwiseConv2D) block5b_bn (None, 64, 64, 672) 2688 block5b_dwconv[0][0] (BatchNormalization) block5b_activation (Activation) (None, 64, 64, 672) 0 block5b_bn[0][0] block5b_se_squeeze (None, 672) 0 block5b_activation[0][0] (GlobalAveragePooling2D) block5b_se_reshape (None, 1 , 1 , 672) 0 block5b_se_squeeze[0][0] (Reshape) block5b_se_reduce (Conv2D) (None, 1 , 1 , 28) 18844 block5b_se_reduce (Con
  • Block5c_dwconv (None, 64, 64, 672) 16800 block5c_expand_activation[0][0] (DepthwiseConv2D) block5c_bn (None, 64, 64, 672) 2688 block5c_dwconv[0][0] (BatchNormalization) block5c_activation (Activation) (None, 64, 64, 672) 0 block5c_bn[0][0] block5c_se_squeeze (None, 672) 0 block5c_activation[0][0] (GlobalAveragePooling2D) block5c_se_reshape (None, 1 , 1 , 672) 0 block5c_se_squeeze[0][0] (Reshape) block5c_se_reduce (Conv2D) (None, 1 , 1 , 28) 18844 block5c_se_reduce (Con
  • Block6a_dwconv (None, 32, 32, 672) 16800 block6a_expand_activation[0][0] (DepthwiseConv2D) block6a_bn (None, 32, 32, 672) 2688 block6a_dwconv[0][0] (BatchNormalization) block6a_activation (Activation) (None, 32, 32, 672) 0 block6a_bn[0][0] block6a_se_squeeze (None, 672) 0 block6a_activation[0][0]
  • block6b_expand_activation (None, 32, 32, 1152) 0 block6b_expand_bn[0][0] (Activation) block6b_dwconv (None, 32, 32, 1152) 28800 block6b_expand_activation[0][0] (DepthwiseConv2D) block6b_bn (None, 32, 32, 1152) 4608 block6b_dwconv[0][0]
  • Block6b_activation (Activation) (None, 32, 32, 1152) 0 block6b_bn[0][0] block6b_se_squeeze (None, 1152) 0 block6b_activation[0][0] (GlobalAveragePooling2D) block6b_se_reshape (None, 1 , 1 , 1152) 0 block6b_se_squeeze[0][0] (Reshape) block6b_se_reduce (Conv2D) (None, 1 , 1 , 48) 55344 block6b_se_reshape[0][0] block6b_se_expand (Conv2D) (None, 1 , 1 , 1152) 56448 block6b_se_reduce[0][0] block6b_se_excite (Multiply) (None, 32, 32, 1152) 0 block6b_bn[
  • block6b_drop (FixedDropout) (None, 32, 32, 192) 0 block6b_project_bn[0][0] block6b_add (Add) (None, 32, 32, 192) 0 block6b_drop[0][0] block6a_project_bn[0][0] block6c_expand_conv (None, 32, 32, 1152) 221184 block6b_add[0][0]
  • Block6c_activation (Activation) (None, 32, 32, 1152) 0 block6c_bn[0][0] block6c_se_squeeze (None, 1152) 0 block6c_activation[0][0]
  • block6c_drop (FixedDropout) (None, 32, 32, 192) 0 block6c_project_bn[0][0] block6c_add (Add) (None, 32, 32, 192) 0 block6c_drop[0][0] block6b_add[0][0] block6d_expand_conv (None, 32, 32, 1152) 221184 block6c_add[0][0]
  • Block6d_activation (Activation) (None, 32, 32, 1152) 0 block6d_bn[0][0] block6d_se_squeeze (None, 1152) 0 block6d_activation[0][0]
  • Block6d_drop (FixedDropout) (None, 32, 32, 192) 0 block6d_project_bn[0][0] block6d_add (Add) (None, 32, 32, 192) 0 block6d_drop[0][0] block6c_add[0][0] block? a_expand_conv (None, 32, 32, 1152) 221184 block6d_add[0][0]
  • a_se_reshape (None, 1 , 1 , 1152) 0 block7a_se_squeeze[0][0] (Reshape) block7a_se_reduce (Conv2D) (None, 1 , 1 , 48) 55344 block? a_se_reshape[0][0] block7a_se_expand (Conv2D) (None, 1 , 1 , 1152) 56448 block? a_s e_re d u ce [0] [0 ] block7a_se_excite (Multiply) (None, 32, 32, 1152) 0 block?
  • a_project_bn[0][0] top_bn (BatchNormalization) (None, 32, 32, 1280) 5120 top_conv[0][0] top_activation (Activation) (None, 32, 32, 1280) 0 top_bn[0][0] decoder_stageOa_transpose (None, 64, 64, 64) 1310720 top_activation[0][0] (Con v2DT ranspose) decoder_stageOa_bn (None, 64, 64, 64) 256 decoder_stageOa_transpose[0][ (BatchNormalization) 0] decoder_stageOa_relu (None, 64, 64, 64) 0 decoder_stage0a_bn[0][0] (Activation) decoder_stageO_concat (None, 64, 64, 736) 0 decoder_stage0a_relu[0][0] (Con

Landscapes

  • Engineering & Computer Science (AREA)
  • Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Theoretical Computer Science (AREA)
  • General Health & Medical Sciences (AREA)
  • Physics & Mathematics (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • Databases & Information Systems (AREA)
  • Data Mining & Analysis (AREA)
  • General Physics & Mathematics (AREA)
  • Evolutionary Computation (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • Computing Systems (AREA)
  • Software Systems (AREA)
  • Artificial Intelligence (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Pathology (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Radiology & Medical Imaging (AREA)
  • Computational Linguistics (AREA)
  • Mathematical Physics (AREA)
  • Multimedia (AREA)
  • General Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Biophysics (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Quality & Reliability (AREA)
  • Measuring And Recording Apparatus For Diagnosis (AREA)

Abstract

Procédé mis en œuvre par ordinateur servant à détecter des résultats visuels dans des images anatomiques consistant à : fournir des images anatomiques du sujet ; saisir les images anatomiques dans un composant de réseau neuronal convolutif d'un réseau de neurones artificiels pour générer un vecteur de caractéristiques ; calculer une indication de résultats visuels présents dans les images anatomiques par une couche dense du réseau de neurones artificiels qui considère comme entrée le vecteur de caractéristiques et génère une indication selon laquelle chacun des résultats visuels est présent dans la ou les images anatomiques ; communiquer les résultats visuels à un système utilisateur configuré pour recevoir des données de rétroaction associées aux résultats visuels ; et transmettre les données de rétroaction au réseau de neurones artificiels, le réseau de neurones artificiels étant entraîné sur un jeu de données d'entraînement comprenant des images anatomiques, et des étiquettes associées aux images anatomiques et à chacun des résultats visuels respectifs, les étiquettes comprenant des étiquettes obtenues à l'aide des données de rétroaction transmises.
PCT/AU2023/050778 2022-08-17 2023-08-17 Procédés et systèmes d'analyse automatisée d'images médicales Ceased WO2024036374A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
AU2022902333 2022-08-17
AU2022902333A AU2022902333A0 (en) 2022-08-17 Methods and systems for automated analysis of medical images

Publications (1)

Publication Number Publication Date
WO2024036374A1 true WO2024036374A1 (fr) 2024-02-22

Family

ID=89940255

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/AU2023/050778 Ceased WO2024036374A1 (fr) 2022-08-17 2023-08-17 Procédés et systèmes d'analyse automatisée d'images médicales

Country Status (1)

Country Link
WO (1) WO2024036374A1 (fr)

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240242339A1 (en) * 2023-01-18 2024-07-18 Siemens Healthcare Gmbh Automatic personalization of ai systems for medical imaging analysis

Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150086091A1 (en) * 2013-09-20 2015-03-26 Mckesson Financial Holdings Method and apparatus for detecting anatomical elements
WO2018222755A1 (fr) * 2017-05-30 2018-12-06 Arterys Inc. Détection automatisée de lésion, segmentation et identification longitudinale
US20190236782A1 (en) * 2018-01-30 2019-08-01 International Business Machines Corporation Systems and methods for detecting an indication of malignancy in a sequence of anatomical images
WO2019215605A1 (fr) * 2018-05-07 2019-11-14 Zebra Medical Vision Ltd. Systèmes et procédés d'analyse d'images anatomiques
US20200226746A1 (en) * 2019-01-15 2020-07-16 Nec Corporation Of America Systems and methods for automated analysis of medical images

Patent Citations (5)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20150086091A1 (en) * 2013-09-20 2015-03-26 Mckesson Financial Holdings Method and apparatus for detecting anatomical elements
WO2018222755A1 (fr) * 2017-05-30 2018-12-06 Arterys Inc. Détection automatisée de lésion, segmentation et identification longitudinale
US20190236782A1 (en) * 2018-01-30 2019-08-01 International Business Machines Corporation Systems and methods for detecting an indication of malignancy in a sequence of anatomical images
WO2019215605A1 (fr) * 2018-05-07 2019-11-14 Zebra Medical Vision Ltd. Systèmes et procédés d'analyse d'images anatomiques
US20200226746A1 (en) * 2019-01-15 2020-07-16 Nec Corporation Of America Systems and methods for automated analysis of medical images

Cited By (1)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20240242339A1 (en) * 2023-01-18 2024-07-18 Siemens Healthcare Gmbh Automatic personalization of ai systems for medical imaging analysis

Similar Documents

Publication Publication Date Title
US11423538B2 (en) Computer-implemented machine learning for detection and statistical analysis of errors by healthcare providers
US11646119B2 (en) Systems and methods for automated analysis of medical images
Luo et al. Deep mining external imperfect data for chest X-ray disease screening
Badgeley et al. Deep learning predicts hip fracture using confounding patient and healthcare variables
US10853449B1 (en) Report formatting for automated or assisted analysis of medical imaging data and medical diagnosis
US10499857B1 (en) Medical protocol change in real-time imaging
US10496884B1 (en) Transformation of textbook information
CA3137079A1 (fr) Apprentissage machine mis en ƒuvre par ordinateur pour la detection et l'analyse statistique d'erreurs par des prestataires de soins de sante
US12437390B2 (en) System and method for assessing quality of medical images
US20190279363A1 (en) Medical evaluation machine learning workflows and processes
US20230259852A1 (en) Systems and methods for workflow processing
WO2019215604A1 (fr) Systèmes et procédés de détection d'une indication d'un type de recherche visuelle dans une image anatomique
Mall et al. Modeling visual search behavior of breast radiologists using a deep convolution neural network
US11450434B2 (en) Implementation of machine-learning based query construction and pattern identification through visualization in user interfaces
US20230351204A1 (en) Selecting a training dataset with which to train a model
EP4367634A1 (fr) Procédés et systèmes de dépistage radiologique rapide
Teo et al. Discovering the predictive value of clinical notes: machine learning analysis with text representation
Ojha et al. Navigating uncertainty: A user-perspective survey of trustworthiness of ai in healthcare
WO2024036374A1 (fr) Procédés et systèmes d'analyse automatisée d'images médicales
CN117894485A (zh) 临床路径生成方法、装置、设备及存储介质
US11610150B2 (en) Method for computing performance in multiple machine learning classifiers
Mall et al. Can a machine learn from radiologists’ visual search behaviour and their interpretation of mammograms—a deep-learning study
Hannan et al. Enhancing diabetic retinopathy classification accuracy through dual-attention mechanism in deep learning
CN114822807A (zh) 病种识别方法、设备、系统及存储介质
Demchyna et al. Optimisation of intelligent system algorithms for poorly structured data analysis

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 23853756

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 23853756

Country of ref document: EP

Kind code of ref document: A1