CN118552563A - A breast ultrasound image segmentation method based on window attention semantic stream alignment - Google Patents

Abstract

The present invention discloses a method for segmenting breast ultrasound images based on window attention semantic stream alignment, including: collecting breast ultrasound images for preprocessing to construct a data set, and segmenting the data set into a training data set and a test data set; constructing a model, including an encoding-decoding framework, a window self-attention module, and a stream alignment module; selecting a custom hybrid loss function to optimize the segmentation performance of the model, training the model using the training data set, optimizing the model parameters by back propagation and gradient descent algorithms, and adjusting hyperparameters and model structures; evaluating the model using the test data set, and calculating multiple indicators to measure the performance of the model. The present invention can effectively improve the accuracy and robustness of the model in identifying and segmenting cancer areas in ultrasound images by adopting a window self-attention module and a stream alignment module; and can further improve the performance and generalization ability of the model by using a custom hybrid loss function for model training and optimization.

Description

A breast ultrasound image segmentation method based on window attention semantic stream alignment

Technical Field

The present invention relates to the field of medical ultrasound technology, and in particular to a breast ultrasound image segmentation method based on window attention semantic stream alignment.

Background Art

Breast cancer is the most threatening disease affecting women's health in the world, with the highest incidence and mortality rates in the world. In addition, in 2021, it directly caused the deaths of 43,600 patients in the United States. Despite this, doctors can improve the 5-year survival rate by diagnosing and treating patients with early stage I cancer, with a success rate of 99%, while the late stage IV is only 37%. Therefore, early cancer detection is crucial to reducing mortality. In the diagnosis process, breast ultrasound images are the most commonly used medical imaging method for radiologists, with the advantages of high cost performance, no radioactivity, painlessness and dynamicity. Therefore, ultrasound images are often used in cancer screening and treatment effect monitoring. Specifically, breast cancer images are difficult to identify, especially the boundary between cancer cells and muscle layer. In other words, well-trained and skilled doctors are able to accurately locate cancer. Due to the high dependence on radiologists, many deep learning engineers have proposed artificial intelligence to assist doctors in detecting abnormal tissues and generating quantitative indicators. In particular, segmentation algorithms can automatically generate the distribution of cancer cells and intuitively guide doctors to segment abnormal tissues. In addition, segmentation algorithms can also improve the efficiency and objectivity of radiologists in generating accurate diagnoses. Therefore, ultrasound image segmentation remains challenging and has potential in medical imaging.

At present, the field of medical image segmentation mainly uses convolutional neural networks (CNN) for automated image segmentation, where the network structure design mainly includes: encoder-decoder architecture: the encoder is responsible for extracting low-level and high-level features of the image, and the decoder is responsible for mapping these features back to the original image space to generate pixel-level segmentation results; convolutional layer: in the encoder and decoder, convolutional layers are used to extract image features, including local receptive fields and parameter sharing; pooling layer: in the encoder, pooling layers are used to reduce the size of feature maps and reduce the amount of computation to a certain extent; activation layer: activation layers, such as ReLU, are used to introduce nonlinearity and improve the expressiveness of the network; and fully connected layer: in the decoder, fully connected layers are used to generate the final segmentation results.

However, the above technology still has some defects, including: 1). As the number of network layers increases, the computational complexity of CNN also increases accordingly, requiring more computing resources; 2). CNN is prone to overfitting, especially when trained on small data sets; in order to prevent overfitting, regularization, data enhancement and other techniques are needed; 3). The generalization ability of CNN depends on the quality and diversity of its training data. In practical applications, CNN may not be able to handle new situations that do not appear in the training data well; 4). CNN is sensitive to noise and image distortion, which will affect the precision and accuracy of segmentation.

Summary of the invention

In view of the above defects or improvement needs of the prior art, the purpose of the present invention is to provide a breast ultrasound image cancer segmentation method based on window attention semantic stream alignment. The method constructs a breast ultrasound image segmentation model based on window attention semantic stream alignment, adopts a window self-attention module to extract local features of the image, and combines multi-scale and semantic stream features to obtain the global information dependency of the image, so that the model retains more effective global information and improves the robustness and accuracy of the model.

To achieve this object, the present invention adopts the following technical solutions:

In a first aspect, the present invention provides a breast ultrasound image cancer segmentation method based on window attention semantic stream alignment, comprising:

S100, collecting breast ultrasound images, performing preprocessing to construct a data set, and dividing the data set into a training data set and a test data set;

S200, constructing a breast ultrasound image segmentation model based on window attention semantic stream alignment, the model comprising an encoding-decoding framework, a window self-attention module and a stream alignment module, wherein:

The encoding-decoding framework is used to separate the breast cancer feature extraction and feature recognition processes, facilitating feature information mining and integration;

The window self-attention module is used to mine more effective global features by segmenting the image into a series of tokens under the action of the attention mechanism, avoiding the interference of ultrasound image shadows and noise, and improving the robustness and segmentation accuracy of the model;

The stream alignment module is used to improve characteristics through periodic connections or skip connections, align feature sets in the neural network to facilitate cross-layer communication, allow relevant information to propagate between layers, and enhance the model's cross-layer communication capability, contextual reasoning capability, and prediction capability;

S300 uses a custom BCE-Dice hybrid loss function, combines pixel accuracy and prediction overlap area accuracy, optimizes model segmentation performance, and improves segmentation accuracy;

S400, train the model using the training data set, optimize the model parameters through back propagation and gradient descent algorithms, adjust the hyperparameters and model structure, and improve the performance and generalization ability of the model;

S500, evaluate the model using the test data set, and calculate indicators such as precision, recall, overall accuracy, prediction error, Kappa coefficient, F1 score, and intersection-over-union ratio to measure the performance of the model;

Furthermore, the ultrasound image preprocessing includes removing blank areas of the image and removing sensitive information of the image.

Furthermore, collecting breast ultrasound images in step S100 also includes:

The methods of flipping, rotating, changing brightness, adjusting contrast, and adjusting resolution are used to simulate ultrasound images in clinical scenarios and enrich the diversity of data.

Furthermore, the encoder module in the encoding-decoding framework mainly applies window self-attention to infer and obtain feature information through four consecutive stages.

Furthermore, the decoder module in the encoding-decoding framework also introduces multi-scale feature learning to compensate for the loss of shallow features, and uses hollow spatial pyramid pooling as a connection between the encoder and decoder to obtain more contextual information.

Furthermore, the model in S200 also includes an upper block module, which is used for 3×3 convolution and batch normalization to generate corresponding size distribution results.

Furthermore, the custom hybrid loss function includes a focal loss and a Dice loss function, as shown in formulas (1)-(3):

L _Hybrid = L _BCE + L _Dice (3)

Wherein, L _BCE represents the cross entropy loss function, _Yi represents the true label value at pixel i, _Yi represents the predicted result value at pixel i, N represents the number of pixels in the entire ultrasound image, and i represents the pixel index. L _Dice represents the Dice loss function, ∩ represents the intersection, and ∪ represents the union.

Furthermore, the calculation formulas for precision, recall, overall accuracy, prediction error, Kappa coefficient, F1 score and intersection over union ratio in step S500 include:

Among them, TP, TN, FP and FN represent true positive, true negative, false positive and false negative respectively; Precision represents precision; Recall represents recall; OA represents overall accuracy; Pe represents prediction error; Kappa represents Kappa coefficient; F1 represents F1 score; IOU represents intersection over union.

In a second aspect, the present invention provides an electronic device, comprising:

At least one processor, at least one memory and a communication interface; wherein,

The processor, memory and communication interface communicate with each other;

The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to perform any step in the above method.

In a third aspect, the present invention provides a non-transitory computer-readable storage medium, wherein the non-transitory computer-readable storage medium stores computer instructions, wherein the computer instructions enable the computer to execute any step in the above method.

Beneficial effects of the present invention:

1. The present invention constructs a breast ultrasound image segmentation model based on window attention semantic stream alignment, adopts a window self-attention module to extract local features of the image, and combines multi-scale and semantic stream features to obtain the global information dependency of the image, so that the model retains more effective global information and improves the robustness and accuracy of the model.

2 The present invention uses a window self-attention mechanism to extract ultrasound image information. Compared with traditional convolutional neural networks, the window self-attention mechanism has higher computational efficiency. It reduces the amount of computation by dividing the image into small blocks and only focusing on the interactions between these small blocks. This enables the model to process ultrasound images faster and improves computational efficiency.

3. The present invention uses a custom hybrid loss function to optimize the model. The Dice loss function helps to improve the segmentation accuracy, while the focal loss function can ensure that the model pays attention to cancer features and improves the segmentation accuracy by maximizing the overlap between the predicted results and the true labels. The custom hybrid loss function can combine the Dice loss function and the focal loss function to give full play to the advantages of the two loss functions, thereby improving the performance of the model as a whole.

Additional aspects and advantages of the present application will be partially given in the following description, which will become apparent from the following description, or will be understood through the practice of the present application.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and/or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

FIG1 is a flow chart of breast ultrasound image data acquisition in an embodiment of the present invention;

FIG2 is a schematic diagram of the overall structure of the method according to an embodiment of the present invention;

FIG3 is a result analysis diagram in an embodiment of the present invention;

FIG4 is a diagram showing model results in an embodiment of the present invention;

DETAILED DESCRIPTION

The present invention will be further described in detail below in conjunction with the accompanying drawings and embodiments. It is to be understood that the specific embodiments described herein are only used to explain the present invention, rather than to limit the present invention. It should also be noted that, for ease of description, only parts related to the present invention, rather than all structures, are shown in the accompanying drawings.

Those skilled in the art will appreciate that, unless otherwise stated, the singular forms "a", "an", "said" and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and/or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and/or combinations thereof.

It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical terms and scientific terms) used herein have the same meaning as the general understanding of those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art, and will not be interpreted with an idealized or overly formal meaning unless specifically defined as in the embodiments of this application.

In the first aspect, the present invention provides a method for cancer segmentation of breast ultrasound images based on window attention semantic stream alignment. The method constructs a breast ultrasound image segmentation model based on window attention semantic stream alignment, uses a window self-attention module to extract local features of the image, and combines multi-scale and semantic stream features to obtain the global information dependency of the image, so that the model retains more effective global information and improves the robustness and accuracy of the model.

The method comprises:

S100, collecting breast ultrasound images, performing preprocessing to construct a data set, and dividing the data set into a training data set and a test data set;

S200, constructing a breast ultrasound image segmentation model based on window attention semantic stream alignment, the model comprising an encoding-decoding framework, a window self-attention module and a stream alignment module, wherein:

The encoding-decoding framework is used to separate the breast cancer feature extraction and feature recognition processes, facilitating feature information mining and integration;

The window self-attention module is used to mine more effective global features by segmenting the image into a series of tokens under the action of the attention mechanism, avoiding the interference of ultrasound image shadows and noise, and improving the robustness and segmentation accuracy of the model;

The stream alignment module is used to improve characteristics through periodic connections or skip connections, align feature sets in the neural network to facilitate cross-layer communication, allow relevant information to propagate between layers, and enhance the model's cross-layer communication capability, contextual reasoning capability, and prediction capability;

S300 uses a custom BCE-Dice hybrid loss function, combines pixel accuracy and prediction overlap area accuracy, optimizes model segmentation performance, and improves segmentation accuracy;

S400, train the model using the training data set, optimize the model parameters through back propagation and gradient descent algorithms, adjust the hyperparameters and model structure, and improve the performance and generalization ability of the model;

S500, evaluate the model using the test data set, and calculate indicators such as precision, recall, overall accuracy, prediction error, Kappa coefficient, F1 score, and intersection-over-union ratio to measure the performance of the model;

The specific steps are as follows:

Step 1: Dataset Construction

First, we collected breast cancer ultrasound images from the Department of Radiology of Wuhan University People's Hospital. The patients included patients with benign or malignant tumors aged 17 to 79 years old. In addition, Wuhan University People's Hospital approved the ethical approval, numbered WDRY2022-k217. The entire breast ultrasound image collection includes four steps, as shown in Figure 1: first, patient preparation; second, ultrasound radiologists scan suspicious breast areas to obtain ultrasound information; third, use computers to process ultrasound data to generate ultrasound images; fourth, preprocess ultrasound images, including image blank areas and sensitive information removal, to generate the final available ultrasound image results. In addition, breast cancer ultrasound images were collected during the treatment of some patients, so the dataset contains some cancer images at different stages. Finally, 927 clinical ultrasound images were collected. For annotation, we hired two radiologists and one expert to annotate the dataset to generate labels. First, the two radiologists independently marked the cancer area for the entire dataset. Second, the expert checked the two labeled areas, especially the boundary between cancer and shadow, to ensure the final label accuracy. If the tumor area is confused or there is a disagreement between the two radiologists, the final annotation result uses the result of the last expert. In the end, we collected 927 breast ultrasound images containing benign or malignant tumors and annotated corresponding cancer masks to support the development of segmentation algorithms. At the same time, this method uses data enhancement technology to simulate ultrasound images in clinical scenarios by flipping, rotating, changing brightness, contrast, and resolution, enriching the diversity of data, and helping subsequent models get rid of the interference of the above variables, more effectively extracting cancer features, and improving the robustness and accuracy of the model. In addition, the shapes of images collected clinically are inconsistent. In order to better assist model training, this method scales the ultrasound image to 256×256 pixels and inputs it into the model for further analysis.

Step 2: Model building

As shown in Figure 2, the present invention develops a novel breast cancer ultrasound image segmentation network, which aims to obtain global feature dependencies while reducing clutter noise and blurred cancer boundary interference. In addition, we infer the entire ultrasound image instead of the region of interest (ROI) of the cancer, which is more useful as a potential tool for radiologists. As shown in Figure 3, we adopt an encoder and decoder architecture to extract cancer features and generate target results, and features can be adopted from the encoder without worrying about feature confusion problems. In the encoder module, we extract features through four consecutive stages of feature extraction (as shown in Figure 2 (a)), where window self-attention (WSA) is applied to infer feature information (as shown in Figure 2 (b)). Specifically, WSA extracts sequential features from ultrasound images, which can overcome the interference of local region morphology. WSA extracts features through window self-attention and further processes them using linear normalization and multi-layer perceptron, as shown in Figure 2 (b). WSA mainly uses attention mechanism to extract image features, which is more effective in capturing global information than convolutional neural networks. Precisely, the network divides the entire image into small blocks and embeds them as tokens, which supports the image to maintain global dependencies. In this paper, inspired by the self-attention mechanism, the transformer extracts texture and neighboring context information through the sequence. In addition, WSA achieves more comprehensive information extraction by incorporating the hierarchical relationship between low-level images and high-level images. Therefore, WSA can adaptively focus on the main cancer features regardless of tumor size variation, shape variation, and clutter noise. We set the WSA module as the backbone of our architecture to extract more features from breast cancer. In the decoder module, we also introduce multi-scale feature learning to make up for shallow features, such as atrous spatial pyramid pooling (ASPP), because valuable local features are lost in deeper models. In our algorithm, we use ASPP as a connection between the encoder and decoder to effectively obtain more context information, which can improve the segmentation performance when the decoder generates spatial resolution. Then, in the decoder module, we upsample the features of different stages of the encoder module. At the same time, in the decoder module, we apply feature flow alignment (FAM, as shown in Figure 2(d)) to enhance the network's ability to detect edges, because the boundaries of cancer will disappear under the interference of neighboring shadows. FAM adopts a series of convolutions to capture the difference between two input features. This ultimately enables the network to effectively perform cancer screening and detection. In addition, the upper block module in Figure 2(c) involves 3×3 convolution and batch normalization to generate corresponding size distribution results. After constructing the above modules, this comprehensive architecture can effectively perform cancer screening and detection on breast ultrasound images.

As shown in Figure 2(b), the window self-attention (WSA) mechanism simultaneously obtains a balance between capturing global background and local features, and WSA can greatly reduce the computational cost. According to breast ultrasound images, the input image contains similar information such as cancerous tissue, shadows, and clutter noise, so our network uses WSA to retain more global dependencies to avoid the influence of local changes. Specifically, we conduct the encoder module through multiple stages to do our best to extract more valuable information, including stage 1, stage 2, stage 3, and stage 4. When the original image size prepared for training is 256×256×3, we apply a 7×7 convolution with a stride of 4, resulting in 96 channels. Each stage contains a 3×3 convolution with a stride of 2 to reduce the number of features, and a path embedding module is used to double the number of channels. After embedding, WSA begins to extract useful information according to the attention mechanism, which will save more information. At the same time, we also adopt a multi-head self-attention mechanism to enhance feature representation. The sizes of the generated feature maps are 64×64, 32×32, 16×16, and 8×8, corresponding to the resolutions of 1/4, 1/8, 1/16, and 1/32 of the input image size 256. Our network is shown in Figure 2(a), and because each stage undergoes path embedding, the dimensions of the resulting features become 96, 192, 384, and 768. In particular, for WSA in Figure 2(b), our network linearly normalizes the input features, rescales all values to 0-1, and reduces the undesirable effects of outliers. The features are then self-attentioned by the window to obtain more information on the window scale. Specifically, first, we reshape the features according to the window shape to prepare for subsequent operations. Second, it will utilize a linear layer to calculate the query, key, and value, and then calculate the attention map. Third, we utilize multi-head self-attention to calculate the final attention map to extract the best features. Fourth, the final features will be linearly normalized and reshaped to the original input shape. Our network adopts the WSA mechanism in each stage to expand the attention and achieve global self-attention. In addition, we introduce a window self-attention module in the self-attention branch to enhance the position encoding.

As shown in Figure 2(d), Feature Stream Alignment,Due to the frequent mis-segmentation of cancer boundaries in ultrasound images, which often contain complex boundaries between abundant fat and muscle tissues, we adopt the technique of semantic feature stream alignment module to align the features in the neural network to promote the meaningful flow of semantic information. It integrates features from different layers, as shown in Figure 2(d), enabling the model to obtain local and global context. The technique improves the features through recursive or skip connections, enhancing their discriminative ability. It also promotes cross-layer communication, allowing relevant information to propagate between different layers. The attention mechanism highlights the informative regions or features, enhancing the discriminative ability of the network. The semantic feature stream improves the robustness of representation learning, contextual understanding, and prediction. By aligning feature aggregation, it enables the network to exploit meaningful semantic information, resulting in more accurate predictions in tasks such as object recognition and segmentation. In summary, the semantic feature stream enhances the network's understanding, contextual reasoning, and prediction capabilities by promoting the integration and improvement of cross-layer features. Therefore, we introduce the semantic feature stream alignment module in our model.

Step 3: Segmentation model loss function design

In the training phase, the loss function has a crucial impact on the optimization process because the loss function directly affects the optimal performance of the model and the task potential. Therefore, using an appropriate loss function plays a key role in the training strategy to achieve the best algorithm performance. Specifically, the selection of the loss function needs to consider the characteristics of the dataset and the task. According to our breast cancer ultrasound images, we customized a hybrid loss function consisting of a focal loss and a Dice loss function, as shown in formulas (1)-(3). In these formulas, represents the true label (GT), and represents the prediction result of the model. In addition, represents the number of pixels in the entire ultrasound image and represents the pixel index. The designed hybrid loss function has three advantages. First, the Dice loss function improves the segmentation accuracy by maximizing the overlap between the predicted and true labels, which matches our breast cancer ultrasound image segmentation task. Second, according to our dataset, the area of the cancer region is always smaller than that of the normal tissue, which causes the model to focus more on the normal tissue rather than the cancer. Although the Dice loss can alleviate the impact of class imbalance to a certain extent, the focal loss function can solve this problem by emphasizing the cancer characteristics. Third, the hybrid loss function can combine the advantages of the Dice and focal loss functions, thereby performing better than a single loss function. Finally, we adopt a hybrid loss function to segment cancer in ultrasound images while optimizing the performance of the model.

L _Hybrid = L _BCE + L _Dice (3)

Among them, represents the cross entropy loss function, represents the true label value at pixel point i, represents the predicted result value at pixel point i, N represents the number of pixels in the entire ultrasound image, and i represents the pixel index. represents the Dice loss function, represents intersection, and represents union.

Step 4: Test model evaluation metrics

To quantitatively analyze our study, we introduced a number of evaluation metrics to represent the performance of our model in detecting cancer regions in breast cancer ultrasound images. As we know, the selection of evaluation metrics is crucial to show the detection efficiency compared with other segmentation architectures. According to the characteristics of breast cancer ultrasound images, we use precision, recall, overall accuracy (OA), prediction error (Pe), Kappa, F1 and intersection over union (IOU) to represent the segmentation performance of the model, as shown in formulas (4)-(10). Specifically, precision represents the correct identification of cancer regions. Recall, i.e., sensitivity, represents the ability to identify cancer, while specificity emphasizes the ability to identify normal tissues. Pe and OA show the overall performance of the network in segmenting cancer. As for the F1 score, it represents the harmonic mean of precision and recall, which balances the evaluation of precision and recall and represents the overall performance of the method. At the same time, IOU represents the intersection between cancer and normal tissues. In summary, the values of the above 7 statistical indicators range from 0 to 1, and the larger the value of each indicator, the better the performance of detecting abnormal regions. To demonstrate the detection performance for cancer and normal, we use the above 7 metrics to demonstrate the overall segmentation capability including ground truth and breast cancer. In the following formulas (4)-(10), TP, TF, FP, and FN represent true positives, true negatives, false positives, and false negatives, respectively. In addition, the calculation of all metrics is as follows:

Among them, TP, TN, FP and FN stand for true positive, true negative, false positive and false negative respectively; represents precision; represents recall; represents overall accuracy; represents prediction error; Kappa represents Kappa coefficient; F1 represents F1 score; represents intersection over union ratio.

In order to verify the performance of the overlapping chromosome method proposed in the present invention, the present invention gives the final experimental results. As shown in Figure 3, our proposed model conducts some experiments to visualize the cancer detection performance in ultrasound images. On the one hand, we use PR curves and ROC curves to show the performance of the model on the dataset. On the other hand, we visualize the performance of the model through t-SNE3 and related graphs to show the ability of the model in different image features. Our model shows robustness in accurately detecting cancer regions, even in the presence of speckle noise and markers. As shown in Figure 3(a), the AUC (area under the curve) of the PR curve is 0.965, while the AUC of the ROC curve is 0.996. Both AUC values exceed 0.95, indicating that the network has high accuracy in segmenting breast ultrasound images. At the same time, we quantitatively calculate the Euclidean distance and Hu moment similarity of the contours to demonstrate the segmentation ability of the model. It can be seen from the results that the Euclidean distance of most images is small, while the similarity of Hu moment is high. In addition, in Figure 3(b), we present the predicted ground-truth labels (GT) and the area of IOU, showing that these areas are similar, with most IOU values exceeding 80%. By observing the distribution of image areas and area proportions, the generated segmentation results are very close to the ground-truth. These results demonstrate the effectiveness of our model in improving cancer detection in breast ultrasound images. Figure 3(c) shows the square centimeter size of the predicted GT and ground-truth labels, arranged in order of the area size of the ground-truth labels. It also illustrates the proportion of cancer areas within the predicted GT. In addition, we calculate the shortest and longest diameters of the contours in the test set, as well as the coordinates of the centroid in Figure 3(d). In addition, we employ t-SNE in Figure 3(e) for dimensionality reduction visualization, so that we can observe significant differences between cancer areas and normal tissues. This shows that our model accurately captures the significant features between the two and segments cancer. Finally, we randomly sampled 50 images to verify their correlation. As shown in Figure 3(f), we can observe that the similarities between images vary, indicating that the test set has a high degree of diversity. Figure 4 is a performance graph of the test set, where the Grad_CAM graph shows that the model detects cancer areas and gives a distribution result graph of cancer. Table 1 shows the quantitative indicators of different methods.

Table 1 Various indicators of the present invention

The present invention also provides an electronic device, comprising:

At least one processor, at least one memory and a communication interface; wherein,

The processor, memory and communication interface communicate with each other;

The memory stores program instructions that can be executed by the processor, and the processor calls the program instructions to perform any step in the above method.

The present invention also provides a computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, and the computer instructions enable the computer to execute any step in the above method.

It should be understood that, although the steps in the flowchart of the accompanying drawings are displayed in sequence as indicated by the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least a part of the steps in the flowchart of the accompanying drawings may include multiple sub-steps or multiple stages, and these sub-steps or stages are not necessarily executed at the same time, but can be executed at different times, and their execution order is not necessarily sequential, but can be executed in turn or alternately with other steps or at least a part of the sub-steps or stages of other steps.

The above is only a partial implementation method of the present application. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present application. These improvements and modifications should also be regarded as the scope of protection of the present application.

Claims (10)

1. A breast ultrasound image segmentation method based on window attention semantic stream alignment, comprising:

s100, collecting a mammary gland ultrasonic image, preprocessing to construct a data set, and dividing the data set into a training data set and a test data set;

S200, constructing a mammary gland ultrasonic image segmentation model based on window attention semantic flow alignment, wherein the model comprises an encoding-decoding frame, a window self-attention module and a flow alignment module, and the method comprises the following steps of:

the coding-decoding framework is used for separating the breast cancer feature extraction and feature recognition processes, so that feature information can be conveniently mined and integrated;

The window self-attention module is used for excavating more effective global features under the action of an attention mechanism by dividing an image into a series of tokens, avoiding the interference of ultrasonic image shadows and noise points and improving the robustness and the segmentation precision of the model;

the flow alignment module is used for improving characteristics through periodical connection or skip connection, aligning feature sets in the neural network to promote cross-layer communication, allowing related information to spread among layers, and enhancing cross-layer communication capacity, context reasoning capacity and prediction capacity of the model;

S300, adopting a self-defined BCE-Dice mixed loss function, and optimizing the segmentation performance of the model by combining the pixel accuracy and the accuracy of predicting the superposition area, so as to improve the segmentation precision;

S400, training the model by using a training data set, optimizing model parameters through a back propagation and gradient descent algorithm, adjusting super parameters and model structures, and improving the performance and generalization capability of the model;

s500, evaluating the model by using the test data set, and measuring the performance of the model by calculating accuracy, recall, overall accuracy, prediction error, kappa coefficient, F1 score and cross ratio.

2. The method of breast ultrasound image segmentation based on window attention semantic stream alignment according to claim 1, wherein the preprocessing in S100 includes image blank region removal and image sensitive information removal.

3. The method for breast ultrasound image segmentation based on window attention semantic stream alignment according to claim 1, wherein the step S100 further comprises:

the ultrasound images in the clinical scene are simulated by adopting the methods of turning, rotating, changing brightness, adjusting contrast and adjusting resolution, so that the diversity of data is enriched.

4. The method of claim 1, wherein the encoder module in the encoding-decoding framework uses window self-attention to infer acquisition feature information through mainly four consecutive phases.

5. The method for breast ultrasound image segmentation based on window attention semantic stream alignment according to claim 4, wherein a decoder module in the encoding-decoding framework introduces a multi-scale feature learning method to compensate for the loss of shallow features and uses hole space pyramid pooling as a connection between encoder and decoder to obtain more context information.

6. The method according to claim 1, wherein the breast ultrasound image segmentation model based on the window attention semantic flow alignment in step S200 further comprises an upper block module, the upper block module is used for 3×3 convolution and batch normalization, and a corresponding size distribution result is generated.

7. The method for breast ultrasound image segmentation based on window attention semantic stream alignment according to claim 1, wherein the custom mixing loss function comprises a focus loss and a Dice loss function, comprising:

L_Hybrid＝L_BCE+L_Dice (3)

Where L _BCE represents the cross entropy loss function, Y _i represents the true label value at pixel i, Y _i represents the predicted result value at pixel i, N represents the number of pixels of the entire ultrasound image, and i represents the pixel index. L _Dice represents the Dice loss function, U represents the intersection, U represents the union.

8. The method for breast ultrasound image segmentation based on window attention semantic stream alignment according to any one of claims 1 to 7, wherein the calculation of the accuracy, recall, overall accuracy, prediction error, kappa coefficient, F1 score and cross-over ratio in step S500 is:

TP, TN, FP and FN are real examples, real negative examples, false positive examples and false negative examples respectively; precision stands for accuracy; recall represents Recall; OA represents overall accuracy; pe represents a prediction error; kappa stands for Kappa coefficient; f1 represents a fraction F1; IOU stands for cross-over ratio.

9. An electronic device, comprising:

at least one processor, at least one memory, and a communication interface; wherein,

The processor, the memory and the communication interface are communicated with each other;

the memory stores program instructions executable by the processor, the processor invoking the program instructions to perform the method of any of claims 1-8.

10. A computer-readable storage medium, wherein the computer-readable storage medium stores computer instructions, the computer instructions cause the computer to perform the method of any one of claims 1 to 8.