CN112784767A

CN112784767A - Cell example segmentation algorithm based on leukocyte microscopic image

Info

Publication number: CN112784767A
Application number: CN202110108263.4A
Authority: CN
Inventors: 赵萌; 杨虹霞; 敖吉; 石凡; 陈胜勇
Original assignee: Tianjin University of Technology
Current assignee: Tianjin University of Technology
Priority date: 2021-01-27
Filing date: 2021-01-27
Publication date: 2021-05-11

Abstract

A cell instance segmentation algorithm based on white blood cell microscopic images belongs to the field of computer vision and medical image processing. Based on the convolutional neural network, a new white blood cell microscopic image (stained) dataset was created, and the edges and categories of each white blood cell were labeled to train the network model shown in Figure 2. On the basis of completing the basic instance segmentation task, the problem of multi-coloring and multi-scale objects is further solved. As shown in Figure 2, based on a general instance segmentation framework, features are extracted from multiple subspaces and feature enhancement is performed on these diverse features. Then, by increasing the attention to salient objects at different scales, we can achieve the results of multi-scale object detection and achieve better detection, segmentation and classification. The invention is suitable for cell microscopic images under various stainings, such as the detection of leukocytes or cell nuclei under magenta staining, Wright staining, or the classification of normal and diseased cells.

Description

Cell example segmentation algorithm based on leukocyte microscopic image

Technical Field

The invention relates to a cell instance segmentation algorithm based on a leukocyte microscopic image, and belongs to the field of computer vision and medical image processing.

Background

White Blood Cells (WBCs) are the most important immune cells in the human body and play an important role in maintaining the immune function of the human body. The peripheral blood of human body contains five types of leucocytes, which have different shapes, sizes and functions and are respectively as follows according to the content of the peripheral blood of normal human body from high to low: neutrophils (neu), 50-70%; lymphocytes (lym), 20-40%; monocytes (mon), 3-8%; eosinophils (eo), 0.5-5%; basophils (bas), 0.5-1%. The five kinds of leucocytes can eliminate pathogens, eliminate allergens, participate in immune reaction, generate antibodies and the like through different modes and different mechanisms, thereby ensuring the health of organisms. The type and number of leukocytes provide important information about the health of the human body.

In recent years, with the development of medical microscopic imaging technology, a technology for automatically processing microscopic images of a minute structure by using an image processing and pattern recognition method has been developed. Among them, automatic classification and identification of leukocytes in a blood cell image is one of representative problems in microscopic image processing and identification, and this field combines related technologies such as image processing and pattern recognition in medical hematology and computer science. The traditional medical image segmentation technology mainly utilizes segmentation methods on natural images, including segmentation methods based on regions, boundaries, statistics, fuzzy theory, etc., and relies on spatial features (gray/texture) or gradient information of images. The research focus is mainly focused on several aspects of automatic segmentation, feature extraction, leukocyte identification and the like of leukocytes in blood cell images. With the development of computer-aided diagnosis, the deep learning method has a good effect on the application of medical image processing. The traditional image classification method has no steps of preprocessing, feature extraction, feature selection and the like, but in a deep learning method: preprocessing, feature extraction, and feature selection are performed by Convolutional Neural Networks (CNN). Convolutional neural networks are a deep learning architecture that can extract features from a portion of an object and perform object recognition. Methods have been used for the extensive study of cell detection and segmentation in microscopic images. Since the cell forms observed under a microscope are similar, the cells need to be subjected to a color reaction by means of a staining technology, so that different cells or different structures of the cells show different colors, and further, tasks such as feature extraction and classification can be realized on the images by utilizing a deep learning method. And a specific example segmentation framework is provided aiming at specific problems of multiple stains and multiple cell scales.

Disclosure of Invention

The invention aims to provide a cell example segmentation algorithm based on a leukocyte microscopic image so as to detect and classify five kinds of leukocytes in a blood cell microscopic image.

In order to achieve the purpose, the scheme of the invention is as follows:

a cell example segmentation algorithm based on a leukocyte microscopic image is based on a general example segmentation network as a basic framework, and enhances feature expression and attention to salient features so as to eliminate the influence caused by different stains and the challenges brought by various cell sizes. The current concrete steps are as follows:

(1) the doctor finishes the collection of blood cell samples of clinical patients, and carries out magenta staining treatment, makes magenta stained blood cell smears, and obtains white cell microscopic images through microscope examination, wherein the total number of the white cell microscopic images is 302;

(2) labeling of the magenta stained white blood cell images, including class labeling of each white blood cell, and accurate pixel-level labeling, was done by the pathologist. And labeling one common dataset with the help of a pathology specialist, for use as our other partial dataset. 242 microscopic images of leukocytes after the Swiss staining treatment were included in this public data set;

(3) designing a neural network, and realizing a basic example segmentation function;

(4) based on the characteristics of a data set, considering various staining conditions and the problem of multi-scale cells, and designing a special feature extraction network;

(5) carrying out multiple experiments, exploring the optimal network parameters, and recording the average precision and the average recall rate of each time;

(6) designing an ablation experiment, and exploring the specific problem solved by each module;

(7) the feasibility and the superiority of the method are proved, namely the method can realize the correct detection and classification of all white blood cell examples in the blood cell image.

The invention has the beneficial effects that: the method can effectively realize the example segmentation of all the leucocyte examples in the blood cell microscopic image, including the detection and classification of the leucocyte. Based on a neural network, a leukocyte data set is newly established and labeled to train a suitable network model, and two challenging problems are further solved on the basis of completing a basic example segmentation task: multiple staining profiles and multi-scale cellular targets. In order to solve the two problems, a new feature extraction and fusion module is constructed, feature expression is enhanced, different feature channels are weighted, and channels containing more effective information have larger weights. In feature fusion, more attention is paid to white blood cells at a specific scale to realize the detection of multi-scale cells. The method is feasible and can be applied to the field of example segmentation of other cells or medical images through fine adjustment.

Brief description of the drawings

FIG. 1 is a flow chart of the production of a data set.

FIG. 2 is a flow chart of an example cell segmentation algorithm based on a white blood cell microscopic image according to the present invention.

Detailed Description

A cell example segmentation algorithm based on a leukocyte microscopic image solves the problems of classifying and identifying leukocytes in a blood cell microscopic image and provides a slight force for medical research.

The white blood cell microscopic image imaging technology is an application technology combining cytopathology, microscopic analysis technology and computer technology, and because a large amount of red blood cell interference exists in a blood smear, the white blood cells are presented more conveniently for subsequent detection and classification through dyeing treatment. Smears are usually treated with swiss and magenta stains. As shown in fig. 1, a flow of data set production is shown. Firstly, blood cell samples of clinical patients are collected, and then the blood cells are stained in different staining modes to obtain blood cell smears. Among these, we mainly used images after magenta dyeing and swiss dyeing processes. And finally, labeling edge pixel points and categories of the target white blood cells by using a labelme labeling tool to obtain a labeled white blood cell data set. We have 544 images as a data set containing five types of white blood cells: neutrophils, lymphocytes, monocytes, eosinophils, and basophils.

Based on the stained microscopic image, example segmentation is realized by using a Convolutional Neural Network (CNN) and a deep learning method, and feature learning is performed on a leukocyte target in the image. The convolutional neural network is a feedforward neural network which comprises convolutional calculation and has a deep structure, and is one of representative algorithms of deep learning.

As shown in fig. 2, the example split network mainly includes several parts in the figure. 387 leukocyte images are used as a training data set, a feature image is obtained through a feature extraction network, multi-scale feature fusion is carried out through a feature pyramid network, channel attention and space attention are respectively introduced into the two parts, influence caused by different dyeing is avoided, and remarkable features under different scales are obtained. And then generating a candidate region, aligning the region of interest (RoI) with the feature map, and performing subsequent detection and segmentation tasks.

The model performance was evaluated by calculating accuracy (Precision) and Recall (Recall) from the detected confusion matrix.

In the neural network algorithm, a feature extraction network, namely a residual error network ResNeXt50 is firstly utilized, different feature channels of the feature extraction network are weighted differently through convolution operations of 50 layers and some nonlinear operations and an additional channel attention module, so that the network learns more effective white blood cell features, and the influence caused by different dyeing conditions in the feature extraction process is avoided.

And after effective features are extracted, fusing feature graphs of different scales by using a feature pyramid network. By accumulating the feature maps under different scales layer by layer, the network can learn the deep global semantic features and can keep the shallow local profile information. And performing channel attention operation on the feature map subjected to channel normalization again through a channel attention and space attention module, and adding different weights to different spatial positions of each scale feature map.

And (3) evaluating the detection result of the model by adopting the accuracy and the recall rate on the basis of the confusion matrix:

wherein, tp (true positive) represents true positive case, fn (false negative) represents false negative case, fp (false positive) represents false positive case, and they represent different prediction results. The accuracy reflects the accuracy of the model, and the recall rate reflects the comprehensiveness of the model. Both are the higher the results, the better the results.

The method comprises the following specific steps:

(1) collecting a blood cell sample of a clinical patient, carrying out magenta dyeing or Ruhrstan's dyeing, making a blood cell smear, and carrying out microscopic examination by a microscope to obtain a leukocyte microscopic image;

(2) labeling all white blood cell examples in the image by a pathology expert, wherein the labeling comprises the white blood cell types to which the white blood cell examples belong and accurate edge labeling;

(3) preprocessing all images, namely cutting the images to be of a uniform size to be used as input of a neural network, and standardizing and normalizing the images through a preset value;

(4) a convolutional neural network suitable for an existing data set is designed to realize the segmentation of the white blood cell examples, and the method is mainly characterized in that a feature extraction network is used for extracting effective white blood cell feature information so as to better complete subsequent classification and segmentation tasks. The characteristic extraction network can also be other convolutional neural networks, and can also carry out characteristic learning on the leukocyte target in the image;

(5) the network performance is evaluated by calculating the accuracy and the recall rate, and the superiority of classification and segmentation performance is proved.

It should be noted that the above-mentioned embodiments are only examples of the present invention, and are only illustrative of the present invention, and therefore do not limit the scope of the present invention. The technical idea of the invention is that only obvious changes are needed and still fall within the scope of the invention.

Claims

1. a cell instance segmentation algorithm based on leukocyte microscopic image, is characterized in that, comprises the steps:

(1) Collect blood cell samples from clinical patients, stain them, make blood cell smears, and obtain microscopic images of white blood cells by microscopy;

(2) Label all leukocyte instances in the image, including the leukocyte category to which they belong, and their precise edge annotations;

(3) Perform preprocessing operations on all images, including cropping to a uniform size as the input of the neural network, and standardizing and normalizing the images by preset values;

(4) Design a neural network suitable for existing datasets to achieve instance segmentation of white blood cells;

(5) Evaluate the performance of the neural network by calculating precision and recall.

2 . The cell instance segmentation algorithm based on leukocyte microscopic images according to claim 1 , wherein the neural network in step (4) is a feature extraction network to extract effective leukocyte feature information. 3 .

3. cell instance segmentation algorithm based on white blood cell microscopic image as claimed in claim 1, is characterized in that, described neural network in step (4) is convolutional neural network, and described convolutional neural network is to image in image. Leukocyte targets for feature learning.

4. The cell instance segmentation algorithm based on leukocyte microscopic images as claimed in claim 2, wherein in step (4), a feature extraction network, that is, a residual network, is used first, through 50 layers of convolution operations and some nonlinear operation, and different weights are applied to different feature channels of the feature extraction network through an additional channel attention module, so that the network can learn more effective white blood cell features and avoid the influence of different staining conditions during the feature extraction process.

5. The cell instance segmentation algorithm based on white blood cell microscopic images as claimed in claim 4, characterized in that, after the effective features are extracted, the feature of different scales of the feature extraction network in step (4) is obtained by using a feature pyramid network The graph is fused, and the feature maps at different scales are accumulated layer by layer, so that the network not only learns the deep global semantic features, but also retains the shallow local contour information. The channel normalization operation is performed on the feature map of each layer of the network. Using the channel attention module and the spatial attention module, the channel attention operation is first performed on the channel normalized feature map, and then the feature map of each scale is used. Different weights are added to different spatial locations of the graph.

6. the cell instance segmentation algorithm based on white blood cell microscopic image as claimed in claim 1, is characterized in that, in step (5), based on confusion matrix, adopts the detection result of precision and recall rate evaluation model:

Among them, TP represents the true example, FN represents the false negative example, FP represents the false positive example, to represent different prediction results, the precision reflects the accuracy of the model, and the recall rate reflects the comprehensiveness of the model, both of which are The higher the result, the better the effect.