CN109363699B - A method and device for identifying breast imaging lesions - Google Patents

Abstract

The embodiment of the invention provides a method and a device for identifying a breast image focus, which relate to the technical field of machine learning, and the method comprises the following steps: acquiring a mammary gland image; the breast images include breast images of different projection positions of different ipsilateral breasts; inputting the mammary gland image into a feature extraction module to obtain feature images of different sizes of the mammary gland image; inputting the breast image of the breast on the other side of the same projection position of the breast image as a reference image of the breast image into the feature extraction module to obtain reference feature images with different sizes; determining a breast lesion identification frame according to the characteristic image and the reference characteristic image; and determining the breast focus of the breast image according to the breast focus identification frame determined from each characteristic image.

Description

Method and device for identifying breast imaging lesions

technical field

Embodiments of the present invention relate to the technical field of machine learning, and in particular, to a method and device for identifying breast imaging lesions.

Background technique

At present, breast imaging can use low-dose X-rays to examine human breasts, which can detect various breast tumors, cysts and other lesions, help early detection of breast cancer, and reduce its mortality. Breast imaging is an effective test that can be used to diagnose a variety of breast-related diseases in women. Of course, the most important use is in the screening of breast cancer, especially early breast cancer. Therefore, if the early manifestations of breast cancer on breast images can be effectively detected, it will be of great help to doctors.

After a patient takes a breast image, the doctor judges the lesions in the breast image through personal experience. This method is inefficient and has greater subjectivity.

SUMMARY OF THE INVENTION

Embodiments of the present invention provide a method and device for identifying breast lesions in breast images, which are used to solve the problem of low efficiency in the prior art method for judging breast lesions in breast images based on doctor's experience.

An embodiment of the present invention provides a method for identifying breast imaging lesions, including:

acquiring breast images; the breast images include breast images of different projection positions of different breasts;

Inputting the breast image into a feature extraction module to obtain feature images of different sizes of the breast image;

Taking the breast image of the other side of the breast at the same projection position of the breast image as the reference image of the breast image, and inputting it to the feature extraction module to obtain reference feature images of different sizes;

determining a breast lesion identification frame according to the feature image and the reference feature image;

According to the identification frame of breast lesions determined from each feature image, the breast lesions of the breast image are determined.

A possible implementation manner, the determining a breast lesion identification frame according to the feature image and the reference feature image includes:

determining a first breast lesion identification frame in the feature image and a second breast lesion identification frame in the reference feature image;

If it is determined that the positions and/or sizes of the first breast lesion identification frame and the second breast lesion identification frame are the same, the first breast lesion identification frame is deleted.

A possible implementation, the feature extraction module includes N convolution modules; the N convolution modules are down-sampling convolution blocks or up-sampling convolution blocks; each down-sampling convolution block or up-sampling volume The size of the feature images extracted by the block is different, and each convolution module of the N convolution modules includes a first convolution layer and a second convolution layer; the feature images output by the first convolution layer are The number is less than the number of feature images input by the first convolution layer; the number of feature images output by the second convolution layer is greater than the number of feature images input by the first convolution layer; N is greater than 0;

For any one of the feature images of different sizes of the breast image, a breast lesion identification frame is determined from the feature image.

A possible implementation manner, the acquiring feature images of different sizes of the breast image includes:

extracting N/2 first feature images of the breast images through N/2 down-sampling convolution blocks in sequence from the breast images;

Extracting N/2 second feature images of the mammary image from the first feature image output by the N/2th downsampling convolution block sequentially through N/2 upsampling convolution blocks, each upsampling convolution block The sizes of the extracted second feature images are all different;

After merging the first feature image and the second feature image with the same size, N feature images of different sizes of the breast images are determined.

A possible implementation manner, the neural convolutional network model further includes a feature preprocessing module, the feature preprocessing module is located before the N convolution modules; the breast image is input into the feature extraction module ,include:

Input the breast image into the feature preprocessing module, the feature preprocessing module includes a convolution layer, a BN layer, a Relu layer and a pooling layer; the convolution kernel of the feature preprocessing module The size is greater than the size of the convolution sum in the N convolution modules;

Alternatively, the feature preprocessing module includes a plurality of consecutive convolutional layers, a BN layer, a Relu layer and a pooling layer; the size of the convolution kernel of the feature preprocessing module is the same as that of the N convolutional modules. The size of the largest convolution kernel is equal.

A possible implementation manner, before the breast image is input to the feature extraction module, further comprising:

obtaining the original file of the breast image;

Selecting at least one group of window widths and window levels in the original file of the breast image, and acquiring a breast image in a picture format corresponding to the at least one group of window widths and window levels;

The mammary images in the picture format corresponding to the at least one group of window widths and window levels are used as the mammary images input to the feature extraction module.

An embodiment of the present invention provides a device for identifying breast imaging lesions, including:

an acquisition unit, configured to acquire breast images; the breast images include breast images of different projection positions of different breasts;

a processing unit, configured to input the breast image into a feature extraction module to obtain characteristic images of different sizes of the breast image; use the breast image of the other side of the breast at the same projection position of the breast image as the breast image The reference image of the image is input to the feature extraction module to obtain reference feature images of different sizes; according to the feature image and the reference feature image, the identification frame of breast lesions is determined; according to the breast lesions determined from each feature image Recognition box to identify breast lesions on breast images.

A possible implementation manner, the processing unit is specifically used for:

Determine the first breast lesion identification frame in the feature image and the second breast lesion identification frame in the reference feature image; if the position and the position of the first breast lesion identification frame and the second breast lesion identification frame are determined or the same size, delete the first breast lesion identification frame.

On the other hand, an embodiment of the present invention provides a computing device, including at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the program causes the The processing unit performs the steps of any of the methods described above.

In another aspect, an embodiment of the present invention provides a computer-readable storage medium, which stores a computer program executable by a computing device, and when the program runs on the computing device, causes the computing device to execute any of the above A step of the method.

In the embodiment of the present invention, since the characteristic images of breast images are extracted and the breast in each characteristic image is identified, breast lesions can be quickly identified, and the efficiency of breast lesion identification is improved.

In addition, in the convolutional neural network model, the number of channels output by the first convolution layer is set to decrease, and the number of channels output by the second convolution layer is increased to the number of channels input by the first convolution layer, so that during the convolution process , effectively retaining the effective information in the image, while reducing the amount of parameters, improving the effectiveness of feature image extraction, thereby improving the detection accuracy of breast lesions in breast images.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present invention more clearly, the following briefly introduces the accompanying drawings used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can also be obtained from these drawings without any creative effort.

1a is a schematic diagram of a breast image provided by an embodiment of the present invention;

FIG. 1b is a schematic diagram of a breast image provided by an embodiment of the present invention;

1c is a schematic diagram of a breast image provided by an embodiment of the present invention;

1d is a schematic diagram of a breast image provided by an embodiment of the present invention;

FIG. 2 is a schematic flowchart of a method for identifying breast imaging lesions according to an embodiment of the present invention;

3a is a schematic structural diagram of a feature extraction module provided by an embodiment of the present invention;

3b is a schematic structural diagram of a feature extraction module provided by an embodiment of the present invention;

3c is a schematic structural diagram of a feature extraction module provided by an embodiment of the present invention;

3 is a schematic flowchart of a breast imaging lesion identification according to an embodiment of the present invention;

4 is a schematic flowchart of a breast imaging lesion identification according to an embodiment of the present invention;

FIG. 5 is a schematic structural diagram of a device for identifying breast imaging lesions according to an embodiment of the present invention;

FIG. 6 is a schematic structural diagram of a computing device according to an embodiment of the present invention.

Detailed ways

In order to make the objectives, technical solutions and beneficial effects of the present invention clearer, the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention, but not to limit the present invention.

In this embodiment of the present invention, an X-ray mammogram is used as an example for exemplary description, and other images are not described in detail here. Breast imaging can use low-dose (about 0.7 millisievert) X-rays to examine human (mainly female) breasts, which can detect various breast tumors, cysts and other lesions, help early detection of breast cancer, and reduce its incidence. mortality rate. In some countries, older women (generally over 45 years of age) are encouraged to undergo regular mammography (at intervals ranging from one to five years) to screen for early breast cancer. Breast images generally include four X-ray images, which are four breast images of two projection positions (cephalo-caudal CC, medial-lateral oblique MLO) of two breasts, as shown in Figure 1a-d.

The existing technology often only detects a single type of lesions such as calcification or mass, and cannot detect multiple lesions at the same time, and has a narrow application range. At the same time, for these lesions of calcification, the primary image-based feature method is used, which is relatively simple, and the detection accuracy is also relatively poor. For calcification, mass, asymmetry, structural distortion and other types that may exist in the same lesion, the detection accuracy is poor and cannot meet the application requirements.

In view of the above problems, an embodiment of the present invention provides a method for identifying breast imaging lesions, as shown in FIG. 2 , including:

Step 201: acquiring breast images;

Step 202: Input the breast image into a feature extraction module, and obtain characteristic images of different sizes of the breast image;

A possible implementation, the feature extraction module includes N convolution modules; the N convolution modules are down-sampling convolution blocks and/or up-sampling convolution blocks; each down-sampling convolution block or up-sampling convolution block The sizes of the feature images extracted by the sampling convolution blocks are all different, and each convolution module of the N convolution modules includes a first convolution layer and a second convolution layer; the features output by the first convolution layer The number of images is less than the number of feature images input by the first convolution layer; the number of feature images output by the second convolution layer is greater than the number of feature images input by the second convolution layer; N is greater than 0;

For example, the feature extraction module may include three downsampling convolution blocks. Each convolutional module may include a first convolutional layer and a second convolutional layer, the first convolutional layer includes a convolutional layer, a normalization (BatchNormalization, BN) layer connected with the convolutional layer, and a BN layer connected with the BN layer. The activation function layer, the convolution module shown in Figure 3a, includes a first convolution layer and a second convolution layer.

In order to increase the depth of the feature extraction module, as shown in Figure 3b, a possible implementation manner, the steps of the feature image passing through the convolution module may include:

Step 1: Input the feature image input by the convolution module into the first convolution layer to obtain the first feature image; the convolution kernel of the first convolution layer can be N1*m*m*N2; The number of channels of the feature image input by the convolution module, N2 is the number of channels of the first feature image; N1>N2;

Step 2: Input the first feature image into the second convolution layer to obtain the second feature image; the convolution kernel of the first convolution layer can be N2*m*m*N3; N3 is the channel of the second feature image number; N3>N2;

Step 3: After combining the feature image input by the convolution module and the second feature image, determine the feature image output by the convolution module.

In a specific embodiment, the number of feature images output by the second convolution layer may be equal to the number of feature images input by the first convolution layer. That is, N1=N2.

The method for determining the characteristic image corresponding to the breast image described above is only one possible implementation manner, and in other possible implementation manners, the characteristic image corresponding to the breast image may also be determined in other manners, which is not specifically limited.

It should be noted that the activation function in the embodiment of the present invention may be various types of activation functions, for example, may be a linear rectification function (Rectified Linear Unit, ReLU), which is not specifically limited;

Since the input image in the embodiment of the present invention is a two-dimensional image, the feature extraction module in the embodiment of the present invention may be a feature extraction module in a (2Dimensions, 2D) convolutional neural network. Correspondingly, the first convolutional layer The size of the convolution kernel can be m*m, and the size of the convolution kernel of the second convolution layer can be n*n; m and n can be the same or different, which is not limited here; where m, n are greater than or An integer equal to 1. The number of feature images output by the first convolution layer is less than the number of feature images input by the first convolution layer; the number of feature images output by the second convolution layer is greater than the number of feature images output by the second convolution layer The number of input feature images.

Further, in order to optimize the feature extraction module, a possible implementation manner, as shown in Figure 3c, further includes a third convolution layer between the first convolution layer and the second convolution layer; The feature image input by the third convolution layer is the image output by the first convolution layer, and the feature image output by the third convolution layer is the image input by the second convolution layer.

The size of the convolution kernel of the third convolution layer may be k*k, and k may be the same as m, and n may be different, which is not limited here.

In a specific embodiment, the size of the convolution kernel of the first convolutional layer is 3*3; the size of the convolutional kernel of the second convolutional layer is 3*3; the size of the third convolutional layer The size of the convolution kernel is 1*1.

By setting the above convolution kernel, the perceptual field of feature extraction can be effectively improved, which is beneficial to improve the accuracy of breast lesion identification.

Feature images of different sizes may be feature images of different pixels, for example, a feature image with pixels of 500×500 and a feature image with pixels of 1000×1000 are feature images of different sizes.

Optionally, a pre-trained breast lesion detection model is used to extract feature images of different sizes of breast images, and the model is determined after training multiple labeled breast images by using a 2D convolutional neural network.

Optionally, before extracting characteristic images of different sizes of the mammary image, the images are scaled to a specific size, so that the scale between the pixels in each direction and the actual length is constant.

In another possible implementation manner, the feature extraction module includes N/2 down-sampling convolution blocks and N/2 up-sampling convolution blocks; the acquiring feature images of different sizes of the breast image includes:

extracting N/2 first feature images of the breast images through N/2 down-sampling convolution blocks in sequence from the breast images;

Extracting N/2 second feature images of the mammary image from the first feature image output by the N/2th downsampling convolution block sequentially through N/2 upsampling convolution blocks, each upsampling convolution block The sizes of the extracted second feature images are all different;

After merging the first feature image and the second feature image with the same size, N feature images of different sizes of the breast images are determined.

In order to improve the perceptual field of feature extraction and improve the performance of feature extraction, a possible implementation manner, the feature extraction module further includes a feature preprocessing module; the feature preprocessing module includes a convolution layer, a BN layer, One Relu layer and one pooling layer; the size of the convolution kernel of the feature preprocessing module is larger than the size of the convolution kernel of any one of the N convolution modules.

Preferably, the size of the convolution kernel of the convolution layer may be 7*7, and the interval is 2 pixels. The pooling layer is 2*2 max pooling. Through the feature preprocessing module, the image area can be quickly reduced, and the side length can be reduced to 1/4 of the original, which can effectively improve the perceptual field of the feature image, quickly extract shallow features, and effectively reduce the loss of original information.

A possible implementation, the feature preprocessing module includes a plurality of consecutive convolutional layers, a BN layer, a Relu layer and a pooling layer; the size of the convolution kernel of the feature preprocessing module is the same as the N layer. The largest convolution kernels in each convolution module are of equal size.

The step of passing the feature image through the feature preprocessing module may include: inputting the breast image to the feature preprocessing module to obtain a preprocessed feature image; and using the preprocessed feature image as an input to the feature extraction module.

Step 203 : for any feature image in the feature images of different sizes of the breast image, determine a breast lesion identification frame from the feature image.

Optionally, a pre-trained breast lesion detection model is used to determine a breast lesion identification frame from the feature image, and the breast lesion detection model is determined after training multiple breast images of marked breast lesions by using a 2D convolutional neural network. . The area selected by the breast lesion identification frame determined from the feature image does not necessarily contain breast lesions. Therefore, it is necessary to screen each breast lesion identification frame according to the breast lesion probability of the breast lesion identification frame. The breast lesion probability is smaller than the preset threshold. The breast lesion identification box of is deleted, wherein the breast lesion probability is the probability that the area selected by the breast lesion identification frame is a breast lesion.

Step 204: Determine the breast lesions of the breast image according to the breast lesion identification frame determined from each feature image.

Specifically, after the identification frame of the breast lesions is determined, the identification frame is output as the breast lesions in the breast image, and the output parameters of the breast lesions include the central coordinates of the breast lesions and the diameter of the breast lesions, wherein the central coordinates of the breast lesions are the identification of the breast lesions. The center coordinate of the box, the diameter of the breast lesion is the distance from the center of the breast lesion identification box to one of the faces.

Since the feature images of different sizes of breast images are extracted and the breast lesions in each feature image are identified, both large-sized breast lesions and small-sized breast lesions can be detected, which improves the detection accuracy of breast lesions. precision. Secondly, compared with the method of manually judging whether there are breast lesions in a breast image, the method for automatically detecting breast lesions in the present application effectively improves the detection efficiency of breast lesions.

Since the identification frame of breast lesions determined from each feature image may have multiple identification frames corresponding to one breast lesion, if the number of breast lesions in the breast image is directly determined according to the number of identification frames of breast lesions, the number of breast lesions detected will result in the detection of the number of breast lesions. There is a large deviation, so it is necessary to convert each feature image into a feature image of the same size and align it, and then screen the breast lesion identification frame determined from each feature image, and determine the screened breast lesion identification frame as the breast lesion identification frame. Breast lesions in imaging.

In order to further improve the recognition accuracy of breast lesions, a possible implementation manner, the breast images include breast images of different projection positions of different breasts; the inputting the breast images to the feature extraction module includes:

Taking the breast image of the other side of the breast at the same projection position of the breast image as a reference image of the breast image, inputting it to the feature extraction module to obtain a reference feature image;

For any one of the feature images of different sizes of the breast image, determining a breast lesion identification frame from the feature image; including:

determining a first breast lesion identification frame in the feature image and a second breast lesion identification frame in the reference feature image;

If it is determined that the positions and/or sizes of the first breast lesion identification frame and the second breast lesion identification frame are the same, the first breast lesion identification frame is deleted.

In order to further improve the accuracy of breast lesion identification, as shown in FIG. 3 , an embodiment of the present invention provides a method for breast imaging lesion identification, including:

Step 301 : acquiring a breast image; the breast image includes breast images of different projection positions of different breasts;

Step 302: Input the breast image into a feature extraction module to obtain feature images of different sizes of the breast image;

Step 403: Use the breast image of the other side of the breast at the same projection position of the breast image as the reference image of the breast image, and input it to the feature extraction module to obtain reference feature images of different sizes;

Step 304: Determine a breast lesion identification frame according to the feature image and the reference feature image;

Step 305 : Determine the breast lesions of the breast image according to the breast lesion identification frames determined from each feature image.

Through the identification of the reference feature image, the recognition accuracy of the lesion identification frame in the feature image is further improved, the interference of the normal glands of the breast is avoided, and the detection rate of the lesion is improved.

In a possible implementation, the screening process of the breast lesion identification frame includes:

Step 1. Determine the first breast lesion identification frame in the feature image and the second breast lesion identification frame in the reference feature image;

Step 2: If it is determined that the position and/or size of the first breast lesion identification frame and the second breast lesion identification frame are the same, delete the first breast lesion identification frame.

Optionally, the screening process of the breast lesion identification frame may further include the following steps:

Step 1: Determine the breast lesion identification frame with the largest breast lesion probability from the breast lesion identification frame of each feature image.

Step 2: Calculate the intersection ratio of the breast lesion identification frame with the largest breast lesion probability and other breast lesion identification frames.

Step 3: Delete the identification frames of other breast lesions whose intersection ratio is greater than a preset threshold.

Step 4: Determine the breast lesion identification frame with the largest breast lesion probability from the remaining other breast lesion identification frames, and repeat the screening process of the breast lesion identification frame until there are no remaining breast lesion identification frames.

The following describes the screening process of the above-mentioned breast lesion identification frame with specific examples. The breast lesion identification frames determined in each feature image are set to be A, B, C, D, E, and F respectively. The above-mentioned breast lesion identification frames The probabilities of breast lesions were: P(A)=0.9, P(B)=0.85, P(C)=0.95, P(D)=0.75, P(E)=0.96, P(F)=0.65. The above breast lesion identification frames are sorted according to the probability of breast lesions from large to small as: E, C, A, B, D, F. After sorting, it can be known that the breast lesions with the highest probability of breast lesions in the breast lesion identification frame of each feature image The identification frame is E, and then the intersection ratio IOU between the breast lesion identification frame E and other breast lesion identification frames is calculated respectively, and the calculation method of the intersection ratio is shown in formula (1):

Among them, m is the breast lesion identification frame with the largest breast lesion probability, n is the breast lesion identification frame compared with the breast lesion identification frame m, and IOU is the intersection ratio between the breast lesion identification frame m and the breast lesion identification frame n.

The preset threshold is set to 0.5. If the intersection ratio between the breast lesion identification frame C and the breast lesion identification frame E is greater than 0.5, the intersection ratio between the breast lesion identification frame A and the breast lesion identification frame E is greater than 0.5, and the breast lesion identification frame E is greater than 0.5. If the intersection ratio between the lesion identification frame B, the breast lesion identification frame D, the breast lesion identification frame F, and the breast lesion identification frame E is all less than 0.5, the breast lesion identification frame C and the breast lesion identification frame A are deleted, and the breast lesions are identified. Box E identifies breast lesions in breast imaging.

Further, the remaining other breast lesion identification frames B, D, and F are sorted according to the breast lesion probability, and the breast lesion identification frame with the largest breast lesion probability is determined as the breast lesion identification frame B, and then the breast lesion identification frame B and the breast lesion are calculated. The intersection ratio between the identification boxes D and the intersection ratio between the breast lesion identification frame B and the breast lesion identification frame F. If the intersection ratio between the breast lesion identification frame B and the breast lesion identification frame D is greater than 0.5, and the intersection ratio between the breast lesion identification frame B and the breast lesion identification frame F is less than 0.5, the breast lesion identification frame D is deleted, and the breast lesion identification frame D is deleted. The breast lesion identification frame B and the breast lesion identification frame F are determined as breast lesions in the breast image. Since the breast lesion identification frame determined in each feature image is screened according to the breast lesion probability of the breast lesion identification frame and the intersection between the breast lesion identification frames, it is avoided to repeatedly detect and output the same breast lesion in the breast image. Improve the accuracy of detecting the number of breast lesions in breast images.

The following is a detailed introduction to the process of determining a breast lesion detection model by training multiple breast images of marked breast lesions through a convolutional neural network, as shown in Figure 4, including the following steps:

Step 401, acquiring breast images as training samples.

Specifically, the acquired multiple breast images can be directly used as training samples, or an enhancement operation can be performed on the acquired multiple breast images to expand the data volume of the training samples. The enhancement operations include but are not limited to: random up, down, left, and right translation to set pixels (such as 0 to 20 pixels), random rotation to set the angle (such as -15 to 15 degrees), and random zoom to set the multiple (such as 0.85 to 1.15 times).

Step 402, manually label the breast lesions in the training sample.

The breast lesions in the training samples can be marked by professionals such as doctors, and the marked content includes the center coordinates of the breast lesions and the diameter of the breast lesions. Specifically, a plurality of doctors can mark the breast lesions, and determine the final breast lesions and parameters of the breast lesions through a multi-person voting method, and save the results in the form of a mask map. It should be noted that the breast lesions in the manually marked training samples and the enhancement operations of the training samples are in no particular order. The breast lesions in the training samples can be manually marked first, and then the training samples marked with breast lesions can be enhanced. The training samples are augmented, and then the augmented training samples are manually labeled.

Step 403 , input the training samples into the convolutional neural network for training to determine a breast lesion identification model.

The structure of the convolutional neural network includes an input layer, a down-sampling convolution block, an up-sampling convolution block, a target detection network, and an output layer. The training samples are preprocessed and input into the above-mentioned convolutional neural network, the loss function is calculated between the output breast lesions and the mask map of the pre-labeled training samples, and then the back propagation algorithm and the sgd optimization algorithm are used iteratively to determine the breast lesions. Detection model.

Further, the process of using the breast lesion detection model determined by the above training to extract characteristic images of different sizes of breast images includes the following steps:

In step 1, the mammary images are sequentially passed through N/2 down-sampling convolution blocks to extract the first characteristic images of the N mammary images.

The size of the first feature image extracted by each down-sampling convolution block is different, and N/2 is greater than 0.

Optionally, the downsampling convolutional block includes a first convolutional layer and a second convolutional layer, a group connection layer, a front-to-back connection layer, and a downsampling layer.

Step 2: Extract the second feature images of N/2 mammary images from the first feature images output by the N/2 th down-sampling convolution block through N/2 up-sampling convolution blocks in sequence.

The size of the second feature image extracted by each upsampling convolution block is different.

Optionally, the upsampling convolutional block includes a convolutional layer, a group connection layer, a front-to-back connection layer, an upsampling layer, and a synthetic connection layer. The convolution layer includes convolution operation, batch normalization layer and RELU layer.

Step 3: After merging the first feature image and the second feature image with the same size, the feature images of different sizes of N/2 breast images are determined.

The first feature image and the second feature image with the same size are combined by the synthetic connection layer in the up-sampling convolution block to determine the feature images of different sizes. Optionally, when merging, the channel numbers of the first feature image and the second feature image are combined, and the size of the feature image obtained after merging is the same as the size of the first feature image and the second feature image.

Further, the process of using the breast lesion detection model determined by the above training to determine the breast lesion identification frame from the characteristic image includes the following steps:

Step 1, for any pixel in the feature image, take the pixel as the center, and diffuse to the surrounding to determine the first area.

In step 2, a plurality of preset frames are set in the first area according to preset rules.

Due to the different shapes of breast lesions, the preset frame can be set to various shapes. The preset rule may be that the center of the preset frame coincides with the center of the first area, or the corners of the preset frame and the corners of the first area overlap, and so on.

In a specific embodiment, the method for selecting the breast lesion preset frame is to consider each pixel of each feature map as an anchor point. Set multiple preset boxes with different aspect ratios on each anchor point. For each preset frame, the feature map is convolved to predict the offset of a coordinate and size, and the confidence, and the preset frame is determined according to the offset of the coordinate and size, and the confidence.

Step 3, for any preset frame, predict the positional deviation between the preset frame and the first area.

Step 4: Determine the breast lesion identification frame after adjusting the preset frame according to the position deviation, and predict the breast lesion probability of the breast lesion identification frame.

Among them, the breast lesion probability is the probability that the area selected by the breast lesion identification box is a breast lesion. By predicting the positional deviation between the preset frame and the first area, and then adjusting the preset frame by using the position deviation to determine the identification frame, so that the identification frame selects more breast lesion areas in the feature map and improves the accuracy of breast lesion detection.

The specific training process may include: inputting the training data image into the above-mentioned convolutional neural network for calculation. When passing in, multiple images with different window widths and window levels of the lesion are passed in. During training, in the prediction frame output by the network, select the prediction frame set with the highest confidence and the prediction frame set with the largest coincidence with the training samples. The weighted sum of the prediction box confidence and the cross-entropy of the sample annotation, and the cross-entropy of the labeled lesions of the training sample and the offset of the prediction box, is used as the loss function. Trained by the method of backpropagation, the trained optimization algorithm uses the sgd algorithm with momentum and step decay.

In the process of using the algorithm, the input image is preprocessed through the preprocessing module to improve the effect of feature extraction.

A possible implementation manner, the acquiring a breast image includes:

Step 1: Determine the binarized image of the breast image image according to Gaussian filtering of the captured breast image image;

Step 2: Obtain the connected regions of the binarized image, and use the largest region in the connected regions corresponding to the region of the breast imaging image as the segmented breast image;

Step 3: Add the segmented breast image to a preset image template to generate a pre-processed breast image; and use the pre-processed breast image as a breast image input to the feature extraction module.

Specifically, the input of the preprocessing module is breast images saved in Dicom format. Preprocessing can include gland segmentation and image normalization; the main purpose of gland segmentation is to extract the breast part of the input breast image and remove other irrelevant images; image normalization is to classify the image as Uniform format images, specifically, include:

In step 1, the specific binarization threshold can be obtained by calculating the maximum class spacing method of the grayscale histogram of the image.

In step 2, the binarized result can be obtained by the flood fill method to obtain independent area blocks, and the area of each area block can be counted; the area on the image corresponding to the area block with the largest area is taken as Segmented breast images.

In step 3, the preset image template may be a square image with a black bottom plate; specifically, the obtained segmented breast image may be expanded into a 1:1 square image by filling with black borders.

In addition, the output mammogram can be scaled by pixels, for example, the image difference can be scaled to a size of 4096 pixels×4096 pixels.

For mammary glands, due to the irradiation dose of mammary glands and external factors of shooting, the window width and window level of mammary glands can be adjusted to obtain better recognition effect of mammary gland lesion recognition. A possible implementation manner, before the breast image is input to the feature extraction module, further comprising:

obtaining the original file of the breast image;

Selecting at least one group of window widths and window levels in the original file of the breast image, and acquiring a breast image in a picture format corresponding to the at least one group of window widths and window levels;

The mammary images in the picture format corresponding to the at least one group of window widths and window levels are used as the mammary images input to the feature extraction module.

In a specific embodiment, the dicom image can be converted into a png image through three groups of window widths and levels, for example, the first group of window widths is 4000 and the window level is 2000; the second group of window widths is 1000; the window level is 2000 ; The third group of window width is 1500, and the window level is 1500.

Based on the same technical concept, an embodiment of the present invention provides an apparatus for identifying breast lesions. As shown in FIG. 5 , the apparatus can execute the flow of the method for identifying breast lesions.

an acquiring unit 501, configured to acquire a breast image; the breast image includes breast images of different projection positions of different breasts;

The processing unit 502 is configured to input the breast image into a feature extraction module, and obtain characteristic images of different sizes of the breast image; and use the breast image of the other side of the breast at the same projection position of the breast image as the breast image. The reference image of the breast image is input to the feature extraction module to obtain reference feature images of different sizes; according to the feature image and the reference feature image, the identification frame of breast lesions is determined; according to the breast lesions determined from each feature image The lesion identification box identifies the breast lesions of the breast image.

A possible implementation manner, the processing unit 502 is specifically used for:

Determine the first breast lesion identification frame in the feature image and the second breast lesion identification frame in the reference feature image; if the position and the position of the first breast lesion identification frame and the second breast lesion identification frame are determined or the same size, delete the first breast lesion identification frame.

A possible implementation manner, the processing unit 502 is specifically used for:

The breast images are sequentially passed through N/2 downsampling convolution blocks to extract N/2 first feature images of the breast images; the first feature images output by the N/2th downsampling convolution blocks are sequentially passed through N/2 upsampling convolution blocks extract N/2 second feature images of the breast image, and the sizes of the second feature images extracted by each upsampling convolution block are different; After merging with the second feature image, N feature images of different sizes of the breast images are determined.

A possible implementation manner, the feature processing module further includes a feature preprocessing module; the processing unit 502 is specifically used for:

Input the breast image into the feature preprocessing module, the feature preprocessing module includes a convolution layer, a BN layer, a Relu layer and a pooling layer; the convolution kernel of the feature preprocessing module The size is greater than the size of the convolution sum in the N convolution modules;

Alternatively, the feature preprocessing module includes a plurality of consecutive convolutional layers, a BN layer, a Relu layer and a pooling layer; the size of the convolution kernel of the feature preprocessing module is the same as that of the N convolutional modules. The size of the largest convolution kernel is equal.

A possible implementation manner, the obtaining unit 501 is used for:

obtaining the original file of the breast image;

The processing unit 502 is specifically used for:

Select at least one group of window widths and window levels in the original file of the mammary image, and acquire mammary images in the picture format corresponding to the at least one group of window widths and window levels; according to the pictures corresponding to the at least one group of window widths and window levels Format of breast images as input to the feature extraction module.

In a possible implementation manner, the breast images include breast images of different projection positions of different breasts; the acquiring unit 501 is configured to:

The breast image of the other side of the breast at the same projection position of the breast image is used as the reference image of the breast image, and input to the feature extraction module to obtain a reference feature image; determine the first breast in the feature image The lesion identification frame and the second breast lesion identification frame in the reference feature image; if it is determined that the position and/or size of the first breast lesion identification frame and the second breast lesion identification frame are the same, delete the First breast lesion identification box.

An embodiment of the present invention provides a computing device, including at least one processing unit and at least one storage unit, wherein the storage unit stores a computer program, and when the program is executed by the processing unit, the processing unit causes the processing unit to Steps for performing a method of breast lesion identification. As shown in FIG. 6 , it is a schematic diagram of the hardware structure of the computing device described in the embodiment of the present invention, and the computing device may specifically be a desktop computer, a portable computer, a smart phone, a tablet computer, or the like. Specifically, the computing device may include a memory 801, a processor 802, and a computer program stored on the memory. When the processor 802 executes the program, the steps of any of the methods for identifying a breast lesion in the foregoing embodiments are implemented. Wherein, the memory 801 may include read only memory (ROM) and random access memory (RAM), and provide the processor 802 with program instructions and data stored in the memory 801 .

Further, the computing device described in the embodiments of the present application may further include an input device 803, an output device 804, and the like. The input device 803 may include a keyboard, a mouse, a touch screen, etc.; the output device 804 may include a display device, such as a liquid crystal display (LCD), a cathode ray tube (CRT), a touch screen, and the like. The memory 801 , the processor 802 , the input device 803 and the output device 804 may be connected by a bus or in other ways, and the connection by a bus is taken as an example in FIG. 6 . The processor 802 invokes the program instructions stored in the memory 801 and executes the method for identifying a breast lesion provided by the foregoing embodiment according to the obtained program instructions.

Embodiments of the present invention further provide a computer-readable storage medium, which stores a computer program executable by a computing device, and when the program runs on the computing device, causes the computing device to perform steps of the method for identifying breast lesions .

As will be appreciated by one skilled in the art, embodiments of the present invention may be provided as a method, or as a computer program product. Accordingly, the present invention may take the form of an entirely hardware embodiment, an entirely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) having computer-usable program code embodied therein.

The present invention is described with reference to flowchart illustrations and/or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each flow and/or block in the flowchart illustrations and/or block diagrams, and combinations of flows and/or blocks in the flowchart illustrations and/or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to the processor of a general purpose computer, special purpose computer, embedded processor or other programmable data processing device to produce a machine such that the instructions executed by the processor of the computer or other programmable data processing device produce Means for implementing the functions specified in a flow or flow of a flowchart and/or a block or blocks of a block diagram.

These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing apparatus to function in a particular manner, such that the instructions stored in the computer-readable memory result in an article of manufacture comprising instruction means, the instructions The apparatus implements the functions specified in the flow or flow of the flowcharts and/or the block or blocks of the block diagrams.

These computer program instructions can also be loaded on a computer or other programmable data processing device to cause a series of operational steps to be performed on the computer or other programmable device to produce a computer-implemented process such that The instructions provide steps for implementing the functions specified in the flow or blocks of the flowcharts and/or the block or blocks of the block diagrams.

Although preferred embodiments of the present invention have been described, additional changes and modifications to these embodiments may occur to those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be construed to include the preferred embodiment and all changes and modifications that fall within the scope of the present invention.

It will be apparent to those skilled in the art that various modifications and variations can be made in the present invention without departing from the spirit and scope of the invention. Thus, provided that these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims (6)

1. A device for identifying breast imaging lesions, comprising: an acquisition unit, configured to acquire breast images; the breast images include breast images of different projection positions of different breasts; a processing unit, configured to input the breast image into a feature extraction module to obtain characteristic images of different sizes of the breast image; use the breast image of the other side of the breast at the same projection position of the breast image as the breast image The reference image of the image is input to the feature extraction module to obtain reference feature images of different sizes; according to the feature image and the reference feature image, the identification frame of breast lesions is determined; according to the breast lesions determined from each feature image Recognition box to determine breast lesions in breast images; According to the feature image and the reference feature image, determining a breast lesion identification frame includes: determining a first breast lesion identification frame in the feature image and a second breast lesion identification frame in the reference feature image; if determining If the positions and/or sizes of the first breast lesion identification frame and the second breast lesion identification frame are the same, the first breast lesion identification frame is deleted; Determining the breast lesions of the breast image according to the identification frames of breast lesions determined from each feature image specifically includes: after determining the identification frames of breast lesions, outputting the identification frames as breast lesions in the breast images, and outputting the breast lesions of the breast The parameters include the center coordinates of the breast lesions and the diameter of the breast lesions; The feature extraction module includes N convolution modules; the N convolution modules are down-sampling convolution blocks or up-sampling convolution blocks; The sizes are all different, and each convolution module of the N convolution modules includes a first convolution layer and a second convolution layer; the number of feature images output by the first convolution layer is smaller than that of the first convolution layer. The number of feature images input by the convolution layer; the number of feature images output by the second convolution layer is greater than the number of feature images input by the first convolution layer; N is greater than 0; for the breast image Any one of the feature images of different sizes, and a breast lesion identification frame is determined from the any one of the feature images. 2 . The apparatus according to claim 1 , wherein the acquiring characteristic images of different sizes of the breast image comprises: 2 . extracting N/2 first feature images of the breast images through N/2 down-sampling convolution blocks in sequence from the breast images; Extracting N/2 second feature images of the mammary image from the first feature image output by the N/2th downsampling convolution block sequentially through N/2 upsampling convolution blocks, each upsampling convolution block The sizes of the extracted second feature images are all different; After merging the first feature image and the second feature image with the same size, N feature images of different sizes of the breast images are determined. 3. The apparatus according to claim 1, characterized in that, before the feature extraction module, it further comprises a feature preprocessing module, and the feature preprocessing module comprises a convolutional layer, a BN layer, a Relu layer and a pool The size of the convolution kernel of the feature preprocessing module is larger than the size of the convolution sum in the N convolution modules; Alternatively, the feature preprocessing module includes a plurality of consecutive convolutional layers, a BN layer, a Relu layer and a pooling layer; the size of the convolution kernel of the feature preprocessing module is the same as that of the N convolutional modules. The size of the largest convolution kernel is equal. 4. The apparatus according to claim 1, wherein before inputting the breast image to the feature extraction module, the method further comprises: obtaining the original file of the breast image; Selecting at least one group of window widths and window levels in the original file of the breast image, and acquiring a breast image in a picture format corresponding to the at least one group of window widths and window levels; The mammary images in the picture format corresponding to the at least one group of window widths and window levels are used as the mammary images input to the feature extraction module. 5. A computing device, comprising at least one processing unit and at least one storage unit, wherein, The storage unit stores a computer program that, when executed by the processing unit, causes the processing unit to perform the following steps: acquiring breast images; the breast images include breast images of different projection positions of different breasts; Inputting the breast image into a feature extraction module to obtain feature images of different sizes of the breast image; Taking the breast image of the other side of the breast at the same projection position of the breast image as the reference image of the breast image, and inputting it to the feature extraction module to obtain reference feature images of different sizes; determining a breast lesion identification frame according to the feature image and the reference feature image; According to the identification frame of breast lesions determined from each feature image, determine the breast lesions of the breast image; According to the feature image and the reference feature image, determining a breast lesion identification frame includes: determining a first breast lesion identification frame in the feature image and a second breast lesion identification frame in the reference feature image; if determining If the positions and/or sizes of the first breast lesion identification frame and the second breast lesion identification frame are the same, the first breast lesion identification frame is deleted; Determining the breast lesions of the breast image according to the identification frames of breast lesions determined from each feature image specifically includes: after determining the identification frames of breast lesions, outputting the identification frames as breast lesions in the breast images, and outputting the breast lesions of the breast The parameters include the center coordinates of the breast lesions and the diameter of the breast lesions; The feature extraction module includes N convolution modules; the N convolution modules are down-sampling convolution blocks or up-sampling convolution blocks; The sizes are all different, and each convolution module of the N convolution modules includes a first convolution layer and a second convolution layer; the number of feature images output by the first convolution layer is smaller than that of the first convolution layer. The number of feature images input by the convolution layer; the number of feature images output by the second convolution layer is greater than the number of feature images input by the first convolution layer; N is greater than 0; for the breast image Any one of the feature images of different sizes, and a breast lesion identification frame is determined from the any one of the feature images. 6. A computer-readable storage medium, characterized in that it stores a computer program executable by a computing device, and when the program runs on the computing device, the computing device is made to perform the following steps: acquiring breast images; the breast images include breast images of different projection positions of different breasts; Inputting the breast image into a feature extraction module to obtain feature images of different sizes of the breast image; Taking the breast image of the other side of the breast at the same projection position of the breast image as the reference image of the breast image, and inputting it to the feature extraction module to obtain reference feature images of different sizes; determining a breast lesion identification frame according to the feature image and the reference feature image; According to the identification frame of breast lesions determined from each feature image, determine the breast lesions of the breast image; According to the feature image and the reference feature image, determining a breast lesion identification frame includes: determining a first breast lesion identification frame in the feature image and a second breast lesion identification frame in the reference feature image; if determining If the positions and/or sizes of the first breast lesion identification frame and the second breast lesion identification frame are the same, the first breast lesion identification frame is deleted; Determining the breast lesions of the breast image according to the identification frames of breast lesions determined from each feature image specifically includes: after determining the identification frames of breast lesions, outputting the identification frames as breast lesions in the breast images, and outputting the breast lesions of the breast The parameters include the center coordinates of the breast lesions and the diameter of the breast lesions; The feature extraction module includes N convolution modules; the N convolution modules are down-sampling convolution blocks or up-sampling convolution blocks; The sizes are all different, and each convolution module of the N convolution modules includes a first convolution layer and a second convolution layer; the number of feature images output by the first convolution layer is smaller than that of the first convolution layer. The number of feature images input by the convolution layer; the number of feature images output by the second convolution layer is greater than the number of feature images input by the first convolution layer; N is greater than 0; for the breast image Any one of the feature images of different sizes, and a breast lesion identification frame is determined from the any one of the feature images.

Images