CN118736292A - A method, device and program product for auxiliary diagnosis of ovarian adnexal masses based on ultrasound - Google Patents

Abstract

The present application relates to the field of intelligent medical treatment, and specifically to an ultrasound-based auxiliary diagnosis method, device and program product for ovarian adnexal masses. It includes S1 obtaining an ultrasound image of the patient's gynecological adnexal area; S2 inputting the ultrasound image into a first neural network for classification to obtain the normal ovarian classification probability and the ovarian mass classification probability; S3 when the image is determined to be an ovarian mass, inputting the ultrasound image into a second neural network for classification to obtain the benign mass classification probability and the malignant mass classification probability. By integrating the ultrasound image information and clinical pathological information of ovarian masses, the present application helps to more effectively utilize medical data, save medical resources, assist doctors in accurate diagnosis, and improve diagnostic efficiency, and has great clinical value.

Description

A method, device and program for auxiliary diagnosis of ovarian adnexal masses based on ultrasound Product

Technical Field

The present application relates to the field of intelligent medical treatment, and specifically to an ultrasound-based auxiliary diagnosis method, device, program product and computer-readable storage medium for ovarian adnexal masses.

Background Art

Ovarian cancer is one of the three most common gynecological malignancies, with the highest mortality rate among gynecological malignancies, and has become a major threat to women's life and health. According to the National Cancer Center: 2024 National Cancer Report, the incidence and mortality of ovarian cancer in my country have increased significantly. Ovarian cancer occurs deep in the pelvis, with an insidious onset and lack of specific clinical symptoms, and is easily misdiagnosed or missed. More than 75% of patients are initially diagnosed in the late stage, having missed the best time for treatment, and the 5-year relative survival rate is only 29%. Therefore, accurate identification and diagnosis of ovarian cancer is of great significance to improving the survival rate and prognosis of ovarian cancer patients. Ultrasound has the advantages of convenience, low cost, real-time, non-invasive, and repeatability. It is the preferred imaging method for adnexal masses in medical institutions of different levels in my country. However, due to the complexity of the anatomical location and biological behavior of adnexal masses and the extremely time-consuming and operator-dependent characteristics of ultrasound imaging, the accuracy of ultrasound alone in diagnosing ovarian cancer is still relatively low. Therefore, in order to improve the diagnostic accuracy of adnexal masses, many scholars have constructed various clinical diagnostic models and adnexal mass classification systems based on patients' general clinical data, ultrasound image features, laboratory tests, etc., such as the malignancy risk index (RMI), gynecological ultrasound imaging reporting and data system (GI-RADS), simplified rules (SR), adnexal lesion analysis model (ADNEX) and ovarian adnexal reporting and data system (O-RADS). However, due to the high complexity of ultrasound images of ovarian masses and the anatomical features of the deep pelvic cavity, the current ultrasound diagnostic methods for adnexal masses mostly rely on professional radiologists to manually extract features, and subjective evaluation requires rich experience and skills. It is more challenging to identify ovaries and ovarian masses, and to screen for ovarian cancer. Its diagnostic accuracy and efficiency need to be improved, and it lacks practicality and universality, which is not conducive to its widespread promotion and application in primary medical institutions.

In recent years, with the rapid development of artificial intelligence (AI) technology, "medical artificial intelligence" has shown great advantages. Among them, deep learning can automatically extract complex, massive features from shallow to deep layers from image data, quantitatively analyze, avoid operator-dependent feature selection, adopt an end-to-end learning model, and achieve the effect of intelligent recognition and classification, becoming the core application technology of current artificial intelligence research. It has the advantages of reducing the workload of ultrasound physicians and improving diagnostic efficiency; reducing the subjective bias of manual interpretation and improving diagnostic accuracy; providing reference clinical suggestions and solutions for disease prediction, risk assessment, and treatment plan selection, improving the level of primary medical services, and promoting hierarchical diagnosis and treatment. Therefore, it is very important to develop a fast and accurate automatic detection and diagnosis ultrasound intelligent system for screening and diagnosis of ovarian cancer. However, the existing intelligent diagnosis model still requires manual cropping of ultrasound images, which is inefficient, lacks an integrated ovarian mass ultrasound intelligent diagnosis system for detection, segmentation, classification diagnosis, and interpretability analysis, and lacks clinical practical value.

Summary of the invention

In view of the above problems, the present invention proposes a method for auxiliary diagnosis of ovarian adnexal masses based on ultrasound, which specifically includes:

S1 obtains ultrasound images of the patient's gynecological adnexal area;

S2 inputs the ultrasound image into a first neural network for classification to obtain a normal ovary classification probability and an ovarian mass classification probability;

S3: When the image is determined to be an ovarian mass, the ultrasound image is input into a second neural network for classification to obtain a benign mass classification probability and a malignant mass classification probability.

The classification process includes first performing target detection to determine the target position, then performing target region segmentation based on the target position to obtain a segmented region image, and then performing classification based on the segmented region image to obtain a result;

Optionally, the classification process of the first neural network includes first performing ovarian detection on the ultrasound image to obtain the ovarian position, then performing image segmentation based on the ovarian position to obtain the ovarian image; and then classifying based on the ovarian image to obtain a normal ovary or an ovarian mass;

Optionally, the classification process of the second neural network includes first performing mass detection on the ovarian mass to obtain the mass location, then performing image segmentation based on the mass location to obtain a mass area image, and then performing classification based on the mass area image to obtain a benign mass or a malignant mass;

Optionally, S1 is replaced by S11, obtaining an ultrasound image group of the patient's gynecological adnexal area, performing steps S2-S3 on each image in the ultrasound image group in sequence to obtain a classification probability of each image, and performing a weighted average calculation on the classification probability of each image to obtain a diagnosis prediction result for the patient.

The S1 is replaced by S12, obtaining a real-time ultrasound video of the patient's gynecological adnexa, and converting the ultrasound video into a video frame image set; the S2 is replaced by S21, inputting the video frame image set into a first neural network frame by frame for classification to obtain a frame-by-frame image classification probability, and performing a weighted average calculation on the frame-by-frame image classification probability to obtain a probability that the real-time video is a normal ovary or an ovarian mass;

The S3 is replaced by S31, when the real-time video is determined to be an ovarian mass, the video frame image set is input into the second neural network frame by frame for classification to obtain a frame-by-frame classification probability result, and the classification probability result of each image frame is weighted averaged to obtain the probability of a malignant or benign mass in the real-time video; based on the probability of a malignant or benign mass, a benign or malignant diagnosis result of the patient's real-time video is obtained;

Optionally, S12 is replaced by S13; real-time ultrasound video of the patient's gynecological accessory area is obtained, and the ultrasound video is converted into a video frame image set, the video frame image set is continuously grouped and averaged to obtain N average images, and the video frame image set is input into the first neural network frame by frame to obtain a classification probability result, where N is a natural number greater than or equal to 6.

The training steps of the first neural network are:

Obtaining an image set of ovaries and ovarian masses;

Performing data annotation on the image set to obtain annotated images; the data annotation is performed through an annotation hierarchy of case-mass-single-section image;

Inputting the labeled image into a neural network for training to obtain a first neural network;

Optionally, the case in the case-mass-single-section image includes multiple mass results, and the mass includes multiple-section image results;

Optionally, the annotated labels include normal ovarian labels and ovarian mass labels;

Optionally, the training step of the first neural network further includes data preprocessing, performing data preprocessing on the ovarian and ovarian mass image set to obtain preprocessed data, and then performing data labeling based on the preprocessed data;

Optionally, the data preprocessing includes one or more of the following: data enhancement, image flipping, and image rotation.

The training steps of the second neural network are:

Obtaining an image set of an ovarian mass;

Performing data annotation on the ovarian mass image set to obtain an annotated mass image;

Inputting the labeled image of the mass into a neural network for training to obtain a second neural network;

Optionally, the training step of the second neural network further includes data processing, performing data processing on the ovarian mass to obtain a processed image, and then inputting the processed image into the neural network for training to obtain the second neural network;

Optionally, the data processing includes one or more of the following: image cropping, image normalization, image boundary expansion;

Optionally, the image boundary extension is to extend the ovarian image upward by L pixels in each direction, where L is a natural number greater than or equal to 25;

Optionally, the annotated labels include benign or malignant classification labels;

Optionally, the benign or malignant classification label is annotated by the clinical staging and grading of the mass, the pathological subtype, the characteristics of the mass, and the completeness of the image;

Optionally, the label of the pathological subtype is represented by a three-dimensional one-hot code.

Further, the neural network includes a segmentation module and a classification module;

Optionally, the neural network adopts one or more of the following: MTANet, Mask R-CNN, U-Net withClassifier, Panoptic FPN, DeepLabV3+, SOLO, SOLOv2, HRNet;

Optionally, the neural network includes a detection module, a segmentation module and a classification module;

Optionally, the detection module adopts one or more of the following: YOLO, Faster R-CNN, SSD, DETR, CornerNet, CenterNet;

Optionally, the neural network further includes a visualization module, and the ovarian or ovarian mass image is passed through a detection module or a segmentation module to obtain a feature map of a target region of interest, and the feature map of the target region is visualized by the visualization module to obtain a visualization result.

The ultrasound image is classified by a first neural network to obtain a classification probability of a normal ovary or an ovarian mass, and the classification probability of the ovarian mass is compared with a first preset threshold value, and when the classification probability of the ovarian mass is greater than the first threshold value, it is determined to be an ovarian mass, otherwise, it is determined to be a normal ovary;

Optionally, the second neural network performs classification to obtain classification probabilities of benign masses and malignant masses, and compares the classification probabilities with a second preset threshold value; when the classification probability of the benign mass is greater than the second preset threshold value, it is determined to be a benign mass; otherwise, it is determined to be a malignant mass.

The object of the present invention is to provide a computer program product having a computer program/instruction thereon, wherein the computer program/instruction is executed by a processor to implement the above-mentioned ultrasound-based auxiliary diagnosis method for ovarian adnexal masses.

The object of the present invention is to provide a computer device, comprising a memory and a processor and a computer program/instruction stored in the memory, wherein the computer program/instruction is executed by the processor to implement the above-mentioned ultrasound-based auxiliary diagnosis method for ovarian adnexal masses.

The object of the present invention is to provide a computer-readable storage medium having a computer program/instruction stored thereon, wherein the computer program/instruction is executed by a processor to implement the above-mentioned ultrasound-based auxiliary diagnosis method for ovarian adnexal masses.

Advantages of the present invention:

1. Integrate multiple tasks for integrated automatic diagnosis: real-time detection of tumor and normal ovary location, automatic differentiation of normal ovary and tumor, automatic segmentation of normal ovary and tumor, automatic diagnosis of tumor properties and probability judgment, and automatic interpretability analysis.

2. By inputting the attached ultrasound video into the established model, the screening of ovarian masses can be achieved and applied in the clinical diagnosis of ovarian masses. It can detect the position of the ovaries in the ultrasound video in real time and perform segmentation and classification, and realize video detection of the ovarian position and the nature of ovarian masses.

3. In view of the physiological position of the ovarian structure and based on the process characteristics of clinical examination, the auxiliary diagnosis method proposed in the present invention can effectively complete the detection, segmentation, classification and interpretability analysis of ovaries and ovarian masses according to the process of clinical examination, which is conducive to actual clinical application, assisting the examination of junior or intermediate doctors, and improving the examination efficiency and accuracy of junior or intermediate doctors.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work.

FIG1 is a schematic flow chart of an ultrasound-based auxiliary diagnosis method for ovarian adnexal masses provided by an embodiment of the present invention;

FIG2 is a schematic diagram of an ultrasound-based auxiliary diagnosis system for ovarian adnexal masses provided by an embodiment of the present invention;

FIG3 is a schematic diagram of an ultrasound-based auxiliary diagnosis device for ovarian adnexal masses provided by an embodiment of the present invention;

FIG4 is a structural diagram of the overall design process of the diagnostic system provided by an embodiment of the present invention;

FIG5 shows the diagnostic performance of the auxiliary diagnosis system provided by an embodiment of the present invention in an image-based test;

FIG6 is a comparison result of the diagnostic performance between the auxiliary diagnosis system provided by an embodiment of the present invention, a radiologist, and a non-radiologist general practitioner;

FIG. 7 shows the detection, segmentation, diagnosis and interpretation of two types of ovarian masses by the auxiliary diagnosis system provided by an embodiment of the present invention.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention.

In some of the processes described in the specification and claims of the present invention and the above-mentioned figures, multiple operations that appear in a specific order are included, but it should be clearly understood that these operations may not be executed in the order in which they appear in this article or executed in parallel. The sequence numbers of the operations, such as S101, S102, etc., are only used to distinguish between different operations, and the sequence numbers themselves do not represent any execution order. In addition, these processes may include more or fewer operations, and these operations may be executed in sequence or in parallel. It should be noted that the descriptions of "first", "second", etc. in this article are used to distinguish different messages, devices, modules, etc., do not represent the order of precedence, and do not limit the "first" and "second" to be different types.

FIG1 is a schematic diagram of an ultrasound-based auxiliary diagnosis method for ovarian adnexal masses provided by an embodiment of the present invention, specifically comprising:

S1 obtains ultrasound images of the patient's gynecological adnexal area;

In one embodiment, the ultrasound image reflects the difference in acoustic parameters in the medium, and can obtain information different from optics, X-rays, y-rays, etc. Ultrasound has good resolution for human soft tissues, and can obtain useful signals with a dynamic range of up to 120 dB or more, which is conducive to identifying micro-lesions of biological tissues. When ultrasound images display living tissues, the required images can be obtained without dyeing.

In a specific embodiment, data collection: medical record information of ultrasound images and clinical data of adnexal masses are retrieved and collected from multi-center hospitals at different levels and in different regions.

In a specific embodiment, the present invention continuously included patients who had ovarian masses on ultrasound examination and healthy women who showed no ovarian-related abnormalities. The inclusion criteria for patients with ovarian masses were that the patients underwent surgery and obtained histopathological results, or ultrasound follow-up for at least 6 months, or ultrasound follow-up until the lesion disappeared. The exclusion criteria were patients with non-ovarian tumors confirmed by histopathological analysis, or patients without histopathology and lost ultrasound follow-up, and poor ultrasound image quality. According to the World Health Organization Classification of Female Genital Tumors (2020 edition), pathological diagnoses were benign and malignant tumors (malignant tumors included borderline tumors). The data were collected retrospectively by trained researchers using a unified form and reviewed by two researchers.

S2 inputs the ultrasound image into a first neural network for classification to obtain a normal ovary classification probability and an ovarian mass classification probability;

In one embodiment, the classification process includes first performing target detection to determine the target position, then performing target region segmentation based on the target position to obtain a segmented region image, and then performing classification based on the segmented region image to obtain a result.

In one embodiment, the classification process of the first neural network includes first performing ovarian detection on the ultrasound image to obtain the ovarian position, then performing image segmentation based on the ovarian position to obtain the ovarian image; and then classifying based on the ovarian image to obtain a normal ovary or an ovarian mass.

In one embodiment, the training steps of the first neural network are:

Obtaining an image set of ovaries and ovarian masses;

Performing data annotation on the image set to obtain annotated images; the data annotation is performed through an annotation hierarchy of case-mass-single-section image;

The labeled image is input into a neural network for training to obtain a first neural network.

In one embodiment, the case in the case-mass-single-section image includes multiple mass results, and the mass includes multiple-section image results;

Optionally, the annotated labels include a normal ovary label, an ovarian mass label, and an image integrity label.

In one embodiment, the training step of the first neural network further includes data preprocessing, performing data preprocessing on the ovarian and ovarian mass image set to obtain preprocessed data, and then performing data labeling based on the preprocessed data.

In one embodiment, the data preprocessing includes one or more of the following: data enhancement, image flipping, and image rotation.

In one embodiment, the neural network includes a segmentation module and a classification module;

Optionally, the neural network adopts one or more of the following: MTANet, Mask R-CNN, U-Net withClassifier, Panoptic FPN, DeepLabV3+, SOLO, SOLOv2, HRNet;

In one embodiment, the neural network includes a detection module, a segmentation module, and a classification module;

Optionally, the detection module adopts one or more of the following: YOLO, Faster R-CNN, SSD, DETR, CornerNet, CenterNet.

In one embodiment, the neural network further includes a visualization module, and the ovarian or ovarian mass image is passed through a detection module or a segmentation module to obtain a feature map of a target region of interest, and the feature map of the target region is visualized by the visualization module to obtain a visualization result.

In one embodiment, the ultrasound image is classified by a first neural network to obtain a classification probability of a normal ovary or an ovarian mass, and the classification probability of the ovarian mass is compared with a first preset threshold value. When the classification probability of the ovarian mass is greater than the first threshold value, it is determined to be an ovarian mass, otherwise, it is determined to be a normal ovary.

In one embodiment, the first preset threshold is obtained during the training of the first neural network, and the parameter is set as the preset threshold when the loss function of the first neural network does not change during the training process;

Optionally, the first preset threshold value obtained when the first neural network is trained in a data set of ovaries and ovarian masses is 0.9.

In a specific embodiment, data collation includes steps such as data cleaning, renaming, and format conversion. The pre-processed data is selected into the basic database after evaluation: ① Unqualified images are removed by ultrasound physicians with clinical experience according to image quality requirements and records are formed; ② At the same time, images should be renamed according to naming rules; ③ Different image formats are converted into a unified format. Spot checks should be carried out in proportion and records should be formed; ④ After the data is collated to form a basic database, statistics should be conducted based on the collected information and records should be formed.

In a specific embodiment, the OvaMTA system proposed in the present invention consists of two neural networks: the OvaMTA-Seg network for automatic tumor detection and the OvaMTA-Diagnosis network for predicting malignant tumors. Finally, the present invention uses ROC curves, confusion matrices, accuracy, sensitivity, specificity, and Kappa coefficients to evaluate model performance and compare with doctors in video tests, as shown in Figure 4. The goal of the present invention is to develop and validate a deep learning ovarian multi-task attention network (OvaMTA) for real-time navigation of diagnostic planes during ultrasound scanning to identify and mark healthy ovaries and ovarian masses, and to further screen for ovarian cancer using images from heterogeneous multicenter datasets for retrospective testing and external validation. The scalability of the tool was prospectively evaluated using real-world clinical video datasets. The OvaMTA system performs well in tumor localization and ovarian localization in ultrasound scanning videos. At the same time, the ability to diagnose benign and malignant ovarian masses is excellent, which improves the diagnostic accuracy of doctors, including general practitioners.

In a specific embodiment, a multi-task deep learning model for detecting, segmenting, classifying and diagnosing ovarian masses based on ultrasound images, and interpretability analysis includes two parts. OvaMTA-Seg roughly segments the region of interest (ROIs) in the ultrasound image and outputs the probability of the presence of an ovarian mass. When an ovarian mass is present, our system will crop and normalize all connected domains of the ROI and input the corresponding image patch into OvaMTA-diagnosis to accurately segment the mass area and classify benign and malignant. Finally, for the image, the detection frame generated by segmenting the ovarian mass area and the corresponding malignancy probability of the ovarian mass will be output. This automated intelligent diagnosis system process consisting of OvaMTA-Seg and OvaMTA-diagnosis is called OvaMTA.

In a specific embodiment, the two models are trained separately, and considering the functions of each model, the training set of OvaMTA-Seg includes ultrasound images of all training case ovaries and ovarian masses, while the training set of OvaMTA-diagnosis only includes ultrasound images of all training case ovarian masses. Both models are developed based on MTANet. MTANet is a single-stage multi-task attention network, including a pyramid visual transformer (PVT) for classification and a UNet-like multi-attention decoder for segmentation. The present invention uses it as a framework for OvaMTA-Seg and OvaMTA-diagnosis.

In a specific embodiment, all ovarian ultrasound images are converted to PNG format. Next, all ovarian masses are outlined using a labeling system with software version 4.4.0.1 developed by XXXMedical Technology Co., Ltd. The work of marking ROI in the figure is completed by a radiologist with 8 years of experience in ultrasound, and then reviewed and modified by a radiologist with more than 30 years of ultrasound experience. After generating JSON format files for all data sets, the present invention uses OpenCV-python (version 1564.8.1.78) to calculate the bounding rectangle of the annotated ROI. For OvaMTA-Seg, in order to locate the ovary or nodule in the overall image, the complete ultrasound image, ROI binary mask and pathology label are directly supervised for learning. For OvaMTA-diagnosis, in order to focus on the ROI more accurately, the present invention expands its bounding rectangle by 25 pixels in each direction and crops the ultrasound image to obtain the ovarian mass plaque. The pathology label is processed as a three-dimensional unique hot encoding, and the mask is processed as a binary image proportional to the image.

The images in this study have various sizes, so the present invention uniformly resizes all input images to 352×352 pixels by resizing during training and validation. In order to reduce overfitting and increase the number and diversity of training sets, the following transformations are applied to images and masks as data augmentation: horizontal flip with probability 0.5, vertical flip with probability 0.5, and random rotation of 90 degrees.

In a specific embodiment, a two-step decision method is adopted and two thresholds are selected according to the output probability values of the OvaMTA model on the training set and the validation set. When the probability of the presence of an ovarian mass in the image is less than 0.9, the image contains only healthy ovaries and the segmentation result of the ovaries is output. On the contrary, when the probability is greater than 0.9, it is predicted that the image contains an ovarian mass.

The OvaMTA-diagnosis and OvaMTA-Seg models were implemented on an NVIDIA GeForce RTX3080 GPU using PyTorch. All models were trained for 200 epochs using a batch size of 10 and the AdamW optimizer, with the learning rate decayed at 30 epochs. 10% of the images were randomly selected from the training and validation datasets as the validation set to optimize the network and set hyperparameters. Ultimately, OvaMTA will output the bounding box of the ROIs and its probability of a healthy ovary, benign, or malignant tumor, where the sum of these three probability values is equal to 1 and all probability values are distributed in the range of 0 to 1. The median Dice coefficient (mDSC) was used as a metric to evaluate the performance of the model on segmentation. The model interpretable line analysis uses the last layer feature map method to visualize the model's focus area.

S3: When the image is determined to be an ovarian mass, the ultrasound image is input into a second neural network for classification to obtain a benign mass classification probability and a malignant mass classification probability.

In one embodiment, the ultrasound image input to the second neural network is an ultrasound image that is segmented to obtain a ROI and is determined to be an ovarian mass.

In one embodiment, the classification process of the second neural network includes first performing mass detection on the ovarian mass to obtain the mass location, then performing image segmentation based on the mass location to obtain a mass area image, and then performing classification based on the mass area image to obtain a benign mass or a malignant mass.

In one embodiment, the training steps of the second neural network are:

Obtaining an image set of an ovarian mass;

Performing data annotation on the ovarian mass image set to obtain an annotated mass image;

The labeled image of the mass is input into a neural network for training to obtain a second neural network.

In one embodiment, the training step of the second neural network further includes data processing, performing data processing on the ovarian mass to obtain a processed image, and then inputting the processed image into the neural network for training to obtain the second neural network.

In one embodiment, the data processing includes one or more of the following: image cropping, image normalization, image boundary expansion;

Optionally, the image boundary expansion is to expand the ovarian image upward by L pixels in each direction, where L is a natural number greater than or equal to 25.

In one embodiment, the annotated labels include benign or malignant classification labels.

In one embodiment, the benign and malignant classification labels are annotated by the clinical stage grade of the mass, the pathological subtype, the characteristics of the mass, and the completeness of the image;

In one embodiment, the label of the pathological subtype is represented by a three-dimensional one-hot code.

In one embodiment, the second neural network performs classification to obtain classification probabilities of benign masses and malignant masses, and compares the classification probabilities with a second preset threshold value. When the classification probability of a benign mass is greater than the second preset threshold value, it is determined to be a benign mass, otherwise, it is determined to be a malignant mass.

In one embodiment, the second preset threshold is obtained during the training of the second neural network, and the parameter is set as the preset threshold when the loss function of the second neural network does not change during the training process;

Optionally, the second preset threshold value obtained when the data set is ovarian mass and the second neural network is trained is 0.65.

In a specific embodiment, data annotation in a neural network: ① Annotation process: clarify the annotation object, personnel division of labor, annotation steps, result review, unify the annotation rules, confirm the meaning, form and output format of the annotation results, and describe the decision-making mechanism; ② Annotation objects and annotation tasks: clarify the annotation objects and annotation tasks, outline the normal ovaries and ovarian masses in ultrasound images of different modes, classify and annotate the ovarian masses into benign and malignant based on pathological diagnosis, and annotate the clinical stage, grade, pathological subtype, image integrity, and characteristics of the mass (ORADS standard terminology) of the mass; ③ Annotation rules: Annotation rules should be compliant, and compliance should first consider international guidelines. The ultrasound characteristics of ovarian masses adopt the ACR O-RADS Ovarian-Attachment Ultrasound Reporting Vocabulary White Paper, which is the most commonly used internationally. The pathological classification adopts the World Health Organization's Ovarian Tumor Histological Classification (WHO, 2020), and the clinical staging adopts the International Federation of Gynecology and Obstetrics FIGO Surgical Pathology Staging (2014). For the annotation rules that rely on subjective judgment and clinical experience, higher-level standards are used to verify the effectiveness and consistency assessment to ensure that the annotation rules are portable and can be extrapolated to real clinical application scenarios. The annotation level of "case (multiple mass results)-mass (multiple section image results)-single section image" is included. The results of senior physicians are used as the gold standard for mass delineation. The ovarian structure and mass area are delineated separately, and finally they are reviewed and confirmed by the reviewer before proceeding to the next step. After annotating with the delineation software, the ovarian mass is diagnosed and classified (benign, borderline, malignant) considering the impact of the export format and form of the annotation results on the subsequent import; ④ Annotation quality control: During the annotation process, the annotation quality of the annotator should be supervised, and the consistency index and accuracy index should be considered at the same time. When the performance of the labeling personnel shows a significant decline, the labeling work should be suspended, the labeling personnel should be retrained, and they should return to work after passing the assessment; ⑤ Labeling quality assessment: including assessment of accuracy, consistency, comprehensibility, accessibility, data security, and traceability; ⑥ Data statistics: After the data is labeled, a labeling database is formed, which should be counted based on the collected information and recorded.

In a specific embodiment, when the image is predicted to contain an ovarian mass, the connected domain of the segmented region generated by OvaMTA-Seg is clipped by a bounding rectangle and input into OvaMTA-diagnosis to accurately segment the mass and diagnose benign and malignant. When the benign probability output by OvaMTA-diagnosis is greater than 0.65, the model will diagnose the corresponding patch as benign, otherwise it will be diagnosed as malignant.

In a specific embodiment, the model is clinically validated: ultrasound images and clinical case data of patients with adnexal masses from ultrasound examinations in different centers other than those used for model development are collected to verify the stability and diagnostic efficacy of the model. Data of patients in clinical trials from different centers in the next six months are collected to write a model diagnosis report for each case; diagnostic reports of doctors with different years of experience are compared and analyzed with model diagnostic reports; the model is revised, the system theory is further improved, and the core method of diagnosing adnexal masses is verified and revised.

In a specific embodiment, the present invention tests the clinical diagnostic performance through ultrasound images. As shown in FIG5, the receiver operating characteristic curves and confusion matrices of the internal test (a, b, c in FIG5) and the external test (d, e, f in FIG5) data sets show the diagnostic performance of the method proposed by the present invention in detecting ovarian masses based on images and distinguishing benign and malignant ovarian tumors. For the detection of malignant masses, the model of the present invention achieves an AUC of 0.941 (95% CI: 0.940, 0.942) in the internal test set case and an AUC of 0.941 (95% CI: 0.938, 0.941) in the external test set case. In the case of video, the model of the present invention achieves an AUC of 0.911 (95% CI: 0.909, 0.913) and an accuracy of 86.2 (137/159; 95% CI: 80.5, 91.2) in the benign and malignant classification tasks.

In one embodiment, S1 is replaced by S11, an ultrasound image group of the patient's gynecological adnexal area is obtained, steps S2-S3 are performed on each image in the ultrasound image group in sequence to obtain the classification probability of each image, and the classification probability of each image is weighted averaged to obtain the patient's diagnostic prediction result.

In one embodiment, S1 is replaced by S12, obtaining a real-time ultrasound video of the patient's gynecological adnexa, and converting the ultrasound video into a video frame image set; S2 is replaced by S21, inputting the video frame image set into a first neural network frame by frame for classification to obtain a frame-by-frame image classification probability, and performing a weighted average calculation on the frame-by-frame image classification probability to obtain a probability that the real-time video is a normal ovary or an ovarian mass;

The S3 is replaced by S31, when the real-time video is determined to be an ovarian mass, the video frame image set is input into the second neural network frame by frame for classification to obtain a frame-by-frame classification probability result, and the classification probability result of each image frame is weighted averaged to obtain the probability of a malignant or benign mass in the real-time video; based on the probability of a malignant or benign mass, a benign or malignant diagnosis result of the patient's real-time video is obtained;

In one embodiment, S12 is replaced by S13; real-time ultrasound video of the patient's gynecological accessory area is obtained, and the ultrasound video is converted into a video frame image set, the video frame image set is continuously grouped and averaged to obtain N average images, and the video frame image set is input into the first neural network frame by frame to obtain a classification probability result, where N is a natural number greater than or equal to 6.

In a specific embodiment, the present invention obtains an ultrasound image of the patient's gynecological adnexa, inputs the ultrasound image into a first neural network for classification to obtain a normal ovarian classification probability and an ovarian mass classification probability, compares the ovarian mass classification probability with a first preset threshold, and determines it as an ovarian mass when the ovarian mass classification probability is greater than the first threshold, otherwise, it is determined to be a normal ovary. When the image is determined to be an ovarian mass, the ultrasound image is input into a second neural network for classification to obtain a benign mass classification probability and a malignant mass classification probability, and compares the classification probability with a second preset threshold, and determines it as a benign mass when the benign mass classification probability is greater than the second preset threshold, otherwise, it is determined to be a malignant mass.

In a specific embodiment, the present invention obtains an ultrasound image group of the patient's gynecological adnexal area, inputs the image group into a first neural network to obtain the classification probability of each image, performs weighted average calculation on the classification probability of each image to obtain the classification probability of the patient's individual normal ovary and the classification probability of an ovarian mass, and when it is determined to be an ovarian mass, inputs the ultrasound image group into a second neural network for classification to obtain the classification probability of a benign mass and the classification probability of a malignant mass for each image, performs weighted average calculation on the classification probability of each image to obtain the classification probability of a benign mass and the classification probability of a malignant mass for the patient.

In a specific embodiment, the present invention obtains an ultrasound image group of the gynecological adnexa of a patient, inputs the image group into a first neural network to obtain the classification probability of each image, performs a weighted average calculation on the classification probability of each image to obtain the classification probability of the patient's individual normal ovary and the classification probability of an ovarian mass, compares the classification probability of the ovarian mass with a first preset threshold, and when the classification probability of the ovarian mass is greater than the first threshold, it is determined to be an ovarian mass, otherwise, it is determined to be a normal ovary. When it is determined to be an ovarian mass, the ultrasound image group is input into a second neural network for classification to obtain the classification probability of a benign mass and the classification probability of a malignant mass of each image, performs a weighted average calculation on the classification probability of each image to obtain the classification probability of the patient's individual benign mass and the classification probability of a malignant mass, compares the classification probability with a second preset threshold, and when the classification probability of a benign mass is greater than the second preset threshold, it is determined to be a benign mass, otherwise, it is determined to be a malignant mass.

In a specific embodiment, a real-time ultrasound video of the gynecological adnexa of a patient is obtained, and the ultrasound video is converted into a video frame image set; the video frame image set is input into a first neural network frame by frame for classification to obtain a frame-by-frame image classification probability, and the frame-by-frame image classification probability is weighted averaged to obtain the classification probability of a normal ovary or an ovarian mass in the real-time video; the classification probability of the ovarian mass is compared with a first preset threshold, and when the classification probability of the ovarian mass is greater than the first threshold, it is determined to be an ovarian mass, otherwise, it is determined to be a normal ovary. When the real-time video is determined to be an ovarian mass, the video frame image set is input into a second neural network frame by frame for classification to obtain a frame-by-frame classification probability result, and the classification probability result of each image frame is weighted averaged to obtain the probability of a malignant or benign mass in the real-time video; the classification probability is compared with a second preset threshold, and when the classification probability of a benign mass is greater than the second preset threshold, it is determined to be a benign mass, otherwise, it is determined to be a malignant mass.

In a specific embodiment, a real-time ultrasound video of the gynecological adnexa of a patient is obtained, and the ultrasound video is converted into a video frame image set, the video frame image set is continuously grouped and averaged to obtain N average images, and the N average images are input into the first neural network frame by frame to obtain the classification probability results of each image, where N is a natural number greater than or equal to 6. The classification probability of each image is weighted averaged to obtain the normal ovarian classification probability or the classification probability of an ovarian mass of the real-time video; the classification probability of the ovarian mass is compared with the first preset threshold, and when the classification probability of the ovarian mass is greater than the first threshold, it is determined to be an ovarian mass, otherwise, it is determined to be a normal ovary. When the real-time video is determined to be an ovarian mass, the image is input into the second neural network frame by frame for classification to obtain the frame-by-frame classification probability result, and the classification probability results of each image frame are weighted averaged to obtain the malignant classification probability or the benign mass classification probability of the real-time video; the classification probability is compared with the second preset threshold, and when the classification probability of a benign mass is greater than the second preset threshold, it is determined to be a benign mass, otherwise, it is determined to be a malignant mass.

In one embodiment, the present invention collects ultrasound data and pathological results from 23 hospitals and marks the edges of ovarian masses to form image data sets and video data sets.

In a specific embodiment, a total of 2375 patients (10577 images, 159 videos) were included in this study. The data collected from 21 hospitals were divided into the following parts for model development: training and validation sets (6938 images, 1514 patients); internal test set (1584 images, 363 patients). Data collected from patients from two other hospitals (1896 images, 339 patients) were included in the external test set A, including the demographic and clinical characteristics of the patients, and all patients had age information. In the training, validation, and internal test datasets, the age range of the patients was 12 to 86 years old (median = 41), while in the internal test dataset, the age range of the patients was 12 to 86 years old (median = 45). In the external test image and video datasets, the age range of the patients was 15-81 years old (median = 39 years old) and 17-81 years old (median = 43 years old), respectively.

The Dice coefficient of the segmentation model was tested in the internal and external test sets. The average Dice coefficient of the OvaMTA-Seg model in the internal test set was 0.887±0.34, and the average Dice coefficient in the external test set was 0.819±0.201. The Dice coefficient of benign nodules in the internal test set was 0.912±0.103, and the Dice coefficient in the external test set was 0.843±0.192. The Dice coefficient of malignant tumors in the internal test set was 0.878±0.131, and reached 0.851±0.141 in the external test set.

In one specific embodiment, OvaMTA follows a two-stage decision process, so the present invention performs diagnostic tests on the presence of masses and benign/malignant lesions in the internal and external test sets, respectively. In the test for the presence of masses, the OvaMTA-Seg model had an AUC of 0.970 (95% CI: 0.970, 0.973) in the internal test set and an AUC of 0.877 (95% CI: 0.877, 0.882) in the external test set. In terms of mass detection, considering the successful intersection detection of 0.5>IoU, the detection rate of the internal test set was 94.2, while the image-based detection rate of the external test set was 88.1. In the external test set B, the model achieved a 100% detection rate in the video-based test, that is, all masses in the video were detected.

In a specific embodiment, the present invention recruited 6 radiologists and 2 general practitioners to evaluate ovarian masses, including 2 junior radiologists (≥3 years of clinical experience), 2 senior radiologists (≥5 years of clinical experience), 2 experts (≥10 years of clinical experience) and 2 non-radiologists (≥3 years of clinical experience). The eight doctors independently evaluated all anonymous and random videos at the patient level in the external dataset B and assigned benign and malignant labels to each mass. Two weeks later, the OvaMTA results were provided to the radiologists for re-evaluation. The tumor margins and bounding boxes with the probability of malignant tumors segmented by the OvaMTA system will be displayed to the doctors side by side with the original ultrasound video. The video-level diagnostic results of OvaMTA will also be presented to the doctors at the end of the video, and all readers are unaware of the pathological diagnosis.

In a specific embodiment, for image-based individual-level prediction, the weighted average of the predicted probabilities of multiple images for each person is combined into a single score. For video-based prediction, the video is normalized frame by frame and then input into OvaMTA, which outputs the diagnosis result of the current frame in real time and finally outputs the benign and malignant diagnosis of the entire video. In order to take into account contextual information, the feature maps of the previous 6 frames are averaged as the feature map of the current frame, and the binary segmentation region with a threshold of 0.5 is output. The current output segmentation region is expanded by 25 pixels to obtain a patch, and then the healthy, benign, and malignant probabilities of the corresponding frame are output. After outputting the probability of each frame, the frame-level malignant probability of the video is averaged and weighted to obtain the benign and malignant probability of the entire video.

In a specific embodiment, the present invention compares the diagnostic performance of the method of the present invention, radiologists, and non-radiologists, and the comparison results are shown in Figure 6. Figure 6 (a) is a receiver operating characteristic curve of an external test video showing the diagnostic performance of the method of the present invention. And compared with radiologists with different experience levels. The big red star represents the average value of radiologists. The gray circle represents the performance of a primary radiologist, the gray diamond represents the performance of an intermediate radiologist, the gray triangle represents the performance of an expert, and the gray square represents the performance of a general practitioner. Figure 6 (b) is the performance of 8 doctors, and the inter-group comparison is performed by the p value of the paired Mc Nemar test; Figure 6 (c) is a normalized confusion matrix analysis of the method of the present invention for the discrimination performance of benign and malignant ovarian tumors in external test videos. Figure 6 (d) is a Kappa matrix evaluated by 8 doctors in an external test video. Figure 6 (e) is a Kappa matrix of 8 doctors' evaluation of an external test video based on the system-assisted evaluation of the present invention. Each value in the matrix represents the inter-observer consistency between two doctors. On the coordinate axes of the kappa matrix, non-radiologists 1-2 and radiologists 1-6 are arranged in order from bottom to top and from left to right in (e) and (d), and * represents p < 0.05.

In a specific embodiment, in the external test set B, the accuracy of the evaluation of senior radiologists was 88.1 (95% CI: 84.3, 91.5), the sensitivity was 88.6 (95% CI: 82.9, 93.9), and the specificity was 87.6 (95% CI: 82.6, 92.3). The accuracy of the evaluation of junior radiologists was 74.8 (95% CI: 70.1, 79.6), the sensitivity was 68.9 (95% CI: 61.0, 76.9), and the specificity was 86.0 (95% CI: 80.9, 90.9). The accuracy of the evaluation of the intermediate radiologists was 75.8 (95% CI: 71.1, 80.5), the sensitivity was 80.3 (95% CI: 73.1, 86.9), and the specificity was 72.6 (95% CI: 66.3, 79.2). The OvaMTA system gave an assessment comparable to that of senior radiologists, with an accuracy of 86.2 (137/159; 95% CI: 80.5, 91.2; p = 0.504), a sensitivity of 81.8 (54/66; 95% CI: 72.3, 91.0; p = 0.163), and a specificity of 89.2 (83/93; 95% CI: 72.3, 91.0; p = 0.163). CI: 82.8, 95.2; p = 0.678). The accuracy of the OvaMTA system was better than that of mid-level radiologists (p = 0.0002), junior radiologists (p < 0.0001), and general practitioners (p < 0.0001). The sensitivity of the OvaMTA system was better than that of junior radiologists (p = 0.005). The specificity of the OvaMTA system was higher than that of mid-level radiologists (p < 0.0001), junior radiologists (p = 0.002), and general practitioners (p < 0.0001). Overall, the diagnostic ability of the OvaMTA system was better than that of mid- and low-level doctors, and comparable to that of highly qualified radiologists.

With the assistance of OvaMTA, the accuracy of general practitioners significantly improved from 69.2 to 74.8, p<0.0001. The accuracy of junior radiologists improved from 74.8 to 81.1, p=0.004, and the accuracy of mid-level radiologists improved from 75.8 to 80.5, p=0.024. The OvaMTA system significantly enhanced the specificity of junior physicians, including 59.1 to 81.2 for general practitioners, p<0.0001, and 79.0 to 86.0 for junior radiologists, p=0.024. The specificity of these junior physicians assisted by OvaMTA reached a level comparable to that of senior radiologists, 86.0 vs 87.6, p=0.690. The sensitivity of non-radiologists assisted by OvaMTA reached a level comparable to that of senior physicians, 84.8 vs 88.6, p=0.383. With the assistance of OvaMTA, general practitioners have reached the average level of radiologists (ACC: 82.7 vs 81.8, p = 0.804; sensitivity: 84.8 vs 82.6, p = 0.720; specificity: 81.2 vs 81.2, p = 1.0). From the Kappa matrix calculated and visualized in Figure 6 (d) and (e), it can be seen that with the assistance of OvaMTA, the diagnosis between different doctors has become more consistent. Therefore, the OvaMTA system not only improves the diagnosis level of doctors, but also improves their inter-observer criterion.

In a specific embodiment, the clinical application of the model: by submitting the attached ultrasound video to the established model, the screening of ovarian masses is achieved and applied in the clinical diagnosis of ovarian masses, as shown in Figure 7, which shows an example of the segmentation result of OvaMTA-Seg on the ultrasound image and the comparison with manual annotation. The artificial intelligence framework can perform accurate ultrasound image segmentation. In the video test, the model can complete the entire detection and tracking process from the appearance to the disappearance of the tumor. Therefore, the segmentation system can be used alone as a visualization tool for radiologists to highlight the lesion area.

Specifically, as shown in FIG7 and the attached video, OvaMTA can not only accurately locate the tumor, but also find the marginal ovarian tissue of the tumor, and reasonably analyze each frame of the video. The model of the present invention can help doctors diagnose benign and malignant ovarian masses. As can be seen from Videos 1 and 2, the OvaMTA system stably and continuously segments and diagnoses the results frame by frame. FIG7 (a) shows the key frames of a grayscale ultrasound video (Video 1) of a 64-year-old woman with a solid component in a multilocular cyst. The OvaMTA system can detect and draw the edges of the mass, and perform malignant classification and probability in real time. The final diagnosis provided by the OvaMTA system is consistent with the postoperative pathological results (endometrioid carcinoma). The OvaMTA system provides interpretable heat maps in real time, allowing doctors to pay more attention to irregular septa and solid parts. FIG7 (b) shows the key frames of a grayscale ultrasound video (Video 2) of a 52-year-old woman with a unilocular anechoic cyst. The OvaMTA system can detect and draw the edges of the mass, and perform benign classification and probability in real time. The final diagnosis provided by the OvaMTA system was consistent with the postoperative pathological result (serous cystadenofibroma). The OvaMTA system provides interpretable heat maps in real time, allowing doctors to focus on the anechoic cystic part.

In a specific embodiment, OvaMTA not only assumes the function of improving accuracy, but also improves interpretability to a certain extent. This is an important part of the doctor's cooperation with it. On the other hand, the OvaMTA-Seg module enables the OvaMTA system to generate more accurate regional attention to suspicious lesions and their surrounding background. This branch multi-task model outperforms the use of only one classification model while also ensuring the accuracy of the diagnosis. Finally, the use of healthy ovary labels can also improve the entire system. If only batch images are used to train the DL model and then they are directly used during ultrasound scanning, serious misunderstandings may occur. When a healthy ovary appears in an ultrasound scan, it will be segmented and give an incorrect pathological diagnosis. Therefore, for system development, adding negative and positive samples in training is crucial to the functionality of the model. Models that only distinguish different abnormalities are functionally difficult to generalize in real scenarios.

In summary, the ovarian multi-task attention model (OvaMTA) based on integrated ultrasound images proposed in the present invention can detect and classify normal ovaries, benign and malignant ovarian masses, and its diagnostic performance is comparable to that of expert subjective evaluation. The artificial intelligence system can significantly improve the diagnostic level of junior and mid-level radiologists. General practitioners can reach the average performance of radiologists with the assistance of the AI system. OvaMTA has the potential to become an effective and widely applicable tool to help doctors diagnose ovarian masses and ovarian cancer screening.

The embodiments of the present disclosure also provide a computer program product or system, including a computer program, which, when executed by a processor, implements the steps of the above-mentioned ultrasound-based auxiliary diagnosis method for ovarian adnexal masses.

In one embodiment, the present invention proposes an ultrasound-based auxiliary diagnosis system for adnexal masses, including ultrasound image and clinical data collection, data collation, data annotation, multi-task deep learning model for detecting, segmenting, classifying and diagnosing ovarian masses based on ultrasound images, and interpretability analysis, clinical verification, and step setting for clinical application. Through the comprehensive utilization of historical medical data, the integration of ovarian mass ultrasound image information and clinical pathological information can help to more effectively utilize historical medical data, save medical resources, assist doctors in accurate diagnosis, and improve diagnostic efficiency. Artificial intelligence methods are added to the primary screening and clinical diagnosis of adnexal masses, and an adnexal mass diagnosis model is established using ultrasound images, which helps to improve diagnostic efficiency, save doctors' time, reduce misdiagnosis and missed diagnosis rates, alleviate patients' pain, save medical time, and is more helpful for treatment and prognosis, ultimately achieving screening and non-invasive classification-assisted diagnosis of adnexal masses.

FIG2 is a schematic diagram of an ultrasound-based auxiliary diagnosis system for ovarian adnexal masses provided by an embodiment of the present invention, specifically comprising:

Acquisition unit: acquiring ultrasound images of the patient's gynecological adnexal area;

The first classification unit: inputs the ultrasound image into the first neural network for classification to obtain the normal ovary classification probability and the ovarian mass classification probability;

Second classification unit: when the image is determined to be an ovarian mass, the ultrasound image is input into the second neural network for classification to obtain the classification probability of a benign mass and the classification probability of a malignant mass.

FIG3 is a schematic diagram of an ultrasound-based auxiliary diagnosis device for ovarian adnexal masses provided by an embodiment of the present invention, specifically comprising:

A memory and a processor; the memory is used to store program instructions; the processor is used to call program instructions, and when the program instructions are executed, any one of the above-mentioned ultrasound-based auxiliary diagnosis methods for ovarian adnexal masses is obtained.

The present disclosure also provides a computer-readable storage medium storing a computer program. When the computer program is executed by a processor, it is any one of the above-mentioned ultrasound-based auxiliary diagnosis methods for ovarian adnexal masses.

The verification results of this verification example show that assigning inherent weights to indications can improve the performance of the method relative to the default settings. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. In the several embodiments provided in this application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways.

For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present invention may be integrated into a processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit can be implemented in the form of hardware or in the form of software functional units. A person skilled in the art may understand that all or part of the steps in the various methods of the above embodiments may be completed by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the storage medium may include: a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, etc.

A person skilled in the art will understand that all or part of the steps in the above-mentioned embodiment method can be implemented by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. The above-mentioned medium storage can be a read-only memory, a disk or an optical disk, etc.

The above is a detailed introduction to a computer device provided by the present invention. For a person skilled in the art, according to the concept of the embodiments of the present invention, there may be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as limiting the present invention.

Claims (10)

1. An ultrasound-based auxiliary diagnosis method for ovarian adnexal masses, comprising: S1 obtains ultrasound images of the patient's gynecological adnexal area; S2 inputs the ultrasound image into a first neural network for classification to obtain a normal ovary classification probability and an ovarian mass classification probability; S3: When the image is determined to be an ovarian mass, the ultrasound image is input into a second neural network for classification to obtain a benign mass classification probability and a malignant mass classification probability. 2. The ultrasound-assisted diagnosis method for ovarian adnexal masses according to claim 1, characterized in that the classification process includes first performing target detection to determine the target position, then performing target region segmentation based on the target position to obtain a segmented region image, and then performing classification based on the segmented region image to obtain a result; Optionally, the classification process of the first neural network includes first performing ovarian detection on the ultrasound image to obtain the ovarian position, then performing image segmentation based on the ovarian position to obtain the ovarian image; and then classifying based on the ovarian image to obtain a normal ovary or an ovarian mass; Optionally, the classification process of the second neural network includes first performing mass detection on the ovarian mass to obtain the mass location, then performing image segmentation based on the mass location to obtain a mass area image, and then performing classification based on the mass area image to obtain a benign mass or a malignant mass; Optionally, S1 is replaced by S11, obtaining an ultrasound image group of the patient's gynecological adnexal area, performing steps S2-S3 on each image in the ultrasound image group in sequence to obtain a classification probability of each image, and performing a weighted average calculation on the classification probability of each image to obtain a diagnosis prediction result for the patient. 3. The ultrasound-assisted diagnosis method for ovarian adnexal masses according to claim 1 is characterized in that S1 is replaced by S12, a real-time ultrasound video of the patient's gynecological adnexal area is obtained, and the ultrasound video is converted into a video frame image set; S2 is replaced by S21, the video frame image set is input into the first neural network frame by frame for classification to obtain a frame-by-frame image classification probability, and the frame-by-frame image classification probability is weighted averaged to obtain the probability that the real-time video is a normal ovary or an ovarian mass; The S3 is replaced by S31, when the real-time video is determined to be an ovarian mass, the video frame image set is input into the second neural network frame by frame for classification to obtain a frame-by-frame classification probability result, and the classification probability result of each image frame is weighted averaged to obtain the probability of a malignant or benign mass in the real-time video; Obtaining a benign or malignant diagnosis result of the patient's real-time video based on the probability of the malignant or benign mass; Optionally, S12 is replaced by S13; real-time ultrasound video of the patient's gynecological accessory area is obtained, and the ultrasound video is converted into a video frame image set, the video frame image set is continuously grouped and averaged to obtain N average images, and the video frame image set is input into the first neural network frame by frame to obtain a classification probability result, where N is a natural number greater than or equal to 6. 4. The ultrasound-assisted diagnosis method for ovarian adnexal masses according to claim 1, characterized in that the training steps of the first neural network are: Obtaining an image set of ovaries and ovarian masses; Performing data annotation on the image set to obtain annotated images; the data annotation is performed through an annotation hierarchy of case-mass-single-section image; Inputting the labeled image into a neural network for training to obtain a first neural network; Optionally, the case in the case-mass-single-section image includes multiple mass results, and the mass includes multiple-section image results; Optionally, the annotated labels include normal ovarian labels and ovarian mass labels; Optionally, the training step of the first neural network further includes data preprocessing, performing data preprocessing on the ovarian and ovarian mass image set to obtain preprocessed data, and then performing data labeling based on the preprocessed data; Optionally, the data preprocessing includes one or more of the following: data enhancement, image flipping, and image rotation. 5. The ultrasound-based auxiliary diagnosis method for ovarian adnexal masses according to claim 1, characterized in that the training steps of the second neural network are: Obtaining an image set of an ovarian mass; Performing data annotation on the ovarian mass image set to obtain an annotated mass image; Inputting the labeled image of the mass into a neural network for training to obtain a second neural network; Optionally, the training step of the second neural network further includes data processing, performing data processing on the ovarian mass to obtain a processed image, and then inputting the processed image into the neural network for training to obtain the second neural network; Optionally, the data processing includes one or more of the following: image cropping, image normalization, image boundary expansion; Optionally, the image boundary extension is to extend the ovarian image upward by L pixels in each direction, where L is a natural number greater than or equal to 25; Optionally, the annotated labels include benign or malignant classification labels; Optionally, the benign or malignant classification label is annotated by the clinical staging and grading of the mass, the pathological subtype, the characteristics of the mass, and the completeness of the image; Optionally, the label of the pathological subtype is represented by a three-dimensional one-hot code. 6. The ultrasound-based auxiliary diagnosis method for ovarian adnexal masses according to claim 4 or 5, characterized in that the neural network comprises a segmentation module and a classification module; Optionally, the neural network adopts one or more of the following: MTANet, Mask R-CNN, U-Net withClassifier, Panoptic FPN, DeepLabV3+, SOLO, SOLOv2, HRNet; Optionally, the neural network includes a detection module, a segmentation module and a classification module; Optionally, the detection module adopts one or more of the following: YOLO, Faster R-CNN, SSD, DETR, CornerNet, CenterNet; Optionally, the neural network further includes a visualization module, and the ovarian or ovarian mass image is passed through a detection module or a segmentation module to obtain a feature map of a target region of interest, and the feature map of the target region is visualized by the visualization module to obtain a visualization result. 7. The ultrasound-assisted diagnosis method for ovarian adnexal masses according to claim 1, characterized in that the ultrasound image is classified by a first neural network to obtain a classification probability of a normal ovary or an ovarian mass, and the classification probability of the ovarian mass is compared with a first preset threshold value, and when the classification probability of the ovarian mass is greater than the first threshold value, it is determined to be an ovarian mass, otherwise, it is determined to be a normal ovary; Optionally, the second neural network performs classification to obtain classification probabilities of benign masses and malignant masses, and compares the classification probabilities with a second preset threshold value; when the classification probability of the benign mass is greater than the second preset threshold value, it is determined to be a benign mass; otherwise, it is determined to be a malignant mass. 8. A computer program product having a computer program/instruction thereon, wherein the computer program/instruction is executed by a processor to implement any one of the ultrasound-based auxiliary diagnosis methods for ovarian adnexal masses according to claims 1-7. 9. A computer device, comprising a memory and a processor and a computer program/instruction stored in the memory, wherein the computer program/instruction is executed by the processor to implement any one of the ultrasound-based auxiliary diagnosis methods for ovarian adnexal masses according to claims 1-7. 10. A computer-readable storage medium having a computer program/instruction stored thereon, wherein the computer program/instruction is executed by a processor to implement any one of the ultrasound-based auxiliary diagnosis methods for ovarian adnexal masses according to claims 1-7.