EP1685384A1 - Method and device for detecting different types of cells in a biological sample - Google Patents

Abstract

The invention relates to a method and a device for detecting different types of cells in a biological sample. According to the invention, an image (1201 - 120n) of the biological sample (109) is produced; said image (1201 - 120n) is normalised in terms of a distribution of the image values, and the normalised image is then divided into a plurality of image sections; a pre-determined category is associated with each image section, according to the pre-determined properties of the image section; the individual cells in each image section are detected and characteristics of said individual cells are determined; and the individual cells are then associated with a certain cell type on the basis of the detected characteristics.

Description

 Method and device for the detection of different cell types of cells in a biological sample

description

The present invention relates to a method and a device for the detection of different cell types of cells in a biological sample, and in particular to a method and a device for the automatic creation of a differential blood image based on digitized microscopic images of blood smears with the help of image processing methods.

A blood sample is taken from almost every patient who is admitted to a clinic, which is used to diagnose the patient. Depending on the presumption of diagnosis, the attending doctor will arrange different blood test methods. Such a blood test is the so-called differential blood count. With the help of the differential blood count, different causes of a given disease, such as e.g. Diagnose inflammation, infections, allergic reactions, HIV, leukemia, etc.

The differential blood count serves as the basis for diagnosis, which specifies exactly which subgroup of leukocytes (white blood cells) occur in the blood and how often. Different results that deviate from the normal distribution then indicate the individual causes. The evaluation, i.e. the counting of subgroups of leukocytes, was carried out manually under the microscope by a trained specialist for a long time. To make this tiring work easier, little by little, so-called blood count machines were developed that take over this count. The state of the art in the field of automated blood counting machines is limited to machines based on a chemical-physical principle. According to this principle, a blood sample is diluted by a liquid-based method to such an extent that only a single cell is drawn through a measuring capillary. During the flow through this measuring capillary, characteristic information is obtained from each cell, which allows the cell to be assigned to a specific subgroup. Over the years, flow cytometry has proven itself and established in the creation of an automatic differential blood count in the laboratories of many hospitals and medical practices. Most of these flow cytometric machines allow differential blood images of normal and abnormal blood samples to be created robustly and reproducibly.

However, in addition to the normal, especially complex, blood pictures that are changed by different physiological and biological processes occur in clinics, which cannot be analyzed with sufficient precision by the flow cytometric machines mentioned. As a rule, such a sample is classified as conspicuous by such a known automatic machine and a manual examination of the slide under the microscope is carried out by a trained specialist. The order of magnitude for the samples to be examined manually is about 50% of the samples fed to the machine. The manual counting of the leukocytes under the microscope is a tiring work for the specialist, which, due to the work on the microscope, can also have health effects.

In addition to the above-mentioned generation of a differential blood count, similar steps are known, namely an initial automated sample examination paired with a subsequent manual examination of suspicious samples in other areas as well, for example in the examination of other human cells, especially in the localization and classification or detection of dysplastic (tumor-like pre-stages) cells of the cervix (cervix) of women and other diagnostic methods based on cell analysis.

Proceeding from this prior art, the present invention is based on the object of creating an improved ^method and an improved device for detecting different cell types of cells in a biological sample, which is based on a recording of a sample, for example, which is classified as abnormal a training ertung the same and performs the various Zellty ^¬ pen and outputting their frequency in the sample would be without any further manual steps by a specialist is required.

This object is achieved by a method according to claim 1 and by an apparatus according to claim 13.

The present invention provides a method for the detection of different cell types of cells in a biological sample, with the following steps:

(a) providing an image of the biological sample;

(b) normalizing the image of the biological sample with respect to a distribution of image values in the image to obtain a normalized image;

(c) decomposing the normalized image into a plurality of image sections;

(d) assigning each image section portion of the plurality of image portions to a predetermined class, depending on predetermined characteristics of the corresponding ^¬ Page down;

(e) detecting the individual cells or cell groups by combining image portions of the same Klas ^¬ se; (f) detecting predetermined features from the single cells or the cell groups; (g) assigning the individual cells to different cell types based on the detected characteristics.

In step (e) e.g. the leukocyte plasma and the nucleus of white blood cells are combined into a single cell “white blood cells” or into a cell group “white blood cells”. As an alternative or in addition, the plasma and the nucleus of red blood cells are combined to form the single cell “red blood cell” or to the cell area “red blood cells”. Background is and remains (uninteresting) background.

After step (e), it can be provided according to one exemplary embodiment to detect individual cells from the cell groups, e.g. by dividing the cell group into image section groups. If the combined area is too large (or is subject to other criteria), it is assumed that it is a group of touching cells and then this area is separated into individual cells.

According to a preferred embodiment of the present invention, the image is generated by a multi-channel recording, the channels containing different color information or other multi-spectral information. If the channels contain color information, this may be information related to the color of the image, the luminance and chrominance of the image, or the hue, saturation and value of the image. The multispectral information can based on exposure to IR rays, UV rays and X-rays.

According to a further exemplary embodiment, in the event that the channels contain different color information, it is provided that in step (b) a color information value for each channel is acquired for each image section, an average value for each channel is formed on the basis of the acquired color information values, and the image section is assigned to the class based on the mean value determined for each channel. Provision can furthermore be made to verify the classification or assignment of an image section to a class based on one or more image sections which surround the relevant image section.

The image is preferably provided as a digital image, and the decomposition is carried out by determining the image sections based on a predetermined number of pixels. Furthermore, the color values (RGB) of the image are preferably used for the assignment to the classes.

According to a further preferred exemplary embodiment of the present invention, the 'image is normalized before the image is divided or divided into the image sections based on a statistical distribution of different image values in the image, which is preferably color information of the image. In this case, the summation is based on a histogram of the color information. According to a preferred exemplary embodiment, at least two maxima and a minimum enclosed by the same are assigned to each channel for assigned color information at predetermined locations. The normalization is carried out in such a way that the maxima contained in the histogram of a color channel of the image and the minimum with regard to their location are calculated, and then the calculated locations are shifted to the locations that correspond to the channel under consideration. are ordered. Color information between the extreme values is obtained by interpolation between the maxima and the minimum. With digital recordings of blood cells you get a "typical" histogram with two clear maxima and consequently a minimum in between.

In a further preferred exemplary embodiment, the method according to the invention additionally comprises, prior to the step of detecting the individual cells, the combination of individual image sections into specific classes in order to determine corresponding image areas. Furthermore, the method according to the invention can preferably include the additional steps of determining the number of individual cells per cell type and outputting this number.

The present invention also provides an apparatus for the detection of different cell types of cells in a biological sample

an input for receiving an image of the biological sample;

a signal processing device which is adapted to receive the image of the biological sample which is present at the input, to normalize the received image with respect to a distribution of the image values, to split the normalized image into a plurality of image sections, the image sections being predetermined in each case depending on predetermined properties Assign classes, capture individual cells in the image sections, determine predetermined features of the individual cells, and assign individual cells to different cell types, based on the determined features and the class of the assigned image section in which the individual cell is contained;

an output for providing the cell types defined by the signal processing device. According to a preferred embodiment, the device further comprises a sample input for receiving the biological sample and a microscope with an associated digital camera, for example a CDD camera, for generating a digital image of the biological sample and a section thereof. The signal processing device, for example a personal computer, is also adapted to receive the digital image and to process it accordingly.

According to the present invention, a system is thus created which "imitates" the procedure of the specialist required in the prior art and takes over the analysis of the sample under the microscope by means of digital image processing and automatically classifies and counts the cells.

In accordance with a preferred exemplary embodiment of the present invention, the entire image is divided into blocks for the pixel-block classification, in the preferred exemplary embodiment into blocks with the size 8 × 8 pixels. These blocks are preferably not overlapping, which has the advantage in the higher processing speed that can be achieved thereby, since ultimately fewer pixels are viewed. It also gives the method a certain stability against noise (so-called "pixel noise" caused by the digital camera, whose signals always fluctuate somewhat, even if the recorded scene remains completely the same). Overall, this reduction in the resolution of the image is acceptable, since the magnification of the microscope and the physical pixel resolution of the camera are high enough to be able to recognize the decisive structures (blood cells, white and red) in 8 x 8 pixel blocks (a relevant one Object contains several such pixel blocks). For cases where an even more precise determination the edges of the cells are necessary, the process can be repeated in the original magnification on the border of two differently classified blocks (eg "background" / "leukocyte plasma") in order to delimit within the 8 x 8 blocks. This procedure with the different resolutions is also called "hierarchical approach" in image processing. Basically, the grouping into 8 x 8 pixel blocks is sufficient.

For each of the 8 x 8 blocks, the average color value is calculated from the available color channels, in the preferred exemplary embodiment RGB and on the basis of this each block is classified into the required classes in the preferred exemplary embodiment of the blood cells “background”, “leukocyte plasma”, “leukocyte nucleus”, "Erythrocyte plasma" classified.

According to the invention, a multiple classification is thus carried out: first, pixel blocks are classified, quasi in the background and parts of objects, then the relevant objects (plasma plus nucleus of the white blood cells) are segmented (combination of pixel blocks and, if necessary, division again when cells touch) , are then calculated from these characteristics (e.g. size, shape, color of the objects = cells, area, circumference, roundness, granulation, and / or texturing - each of the cell nucleus and plasma) and then these are classified into the by the medicine prescribed cell types of leukocytes. Their occurrence is counted and presented as a histogram (eg "13 leukocytes of the type promyelocyte, 42 of the type ..."). However, no diagnosis is made. A printout with a histogram is generated, which Cell types of leukocytes how often they occur (considered as a percentage).

In addition to the "white" blood picture described above, there are other "blood pictures" that can be made, e.g. "the red blood picture", "the large blood picture", etc.

Preferred exemplary embodiments of the present invention are explained in more detail below with reference to the accompanying drawings. Show it:

1A to IC a block diagram which explains the device according to the invention and the method according to the invention in greater detail using a preferred exemplary embodiment relating to a differential blood count machine;

2 shows the individual steps for pixel block classification according to a preferred exemplary embodiment of the present invention; and

3 shows the steps for color normalization according to a preferred exemplary embodiment of the present invention.

In FIG. 1, the device according to the invention and the method according to the invention are explained in more detail below using the example of a differential blood count machine.

1A shows a first section 100 for sample separation. Here, the biological sample, a blood sample, is prepared using measures known per se. The sample preparation 100 comprises the blood withdrawal 102. The blood withdrawn is, as shown by the arrow 104, subjected to a further processing 106 provided, according to which a blood smear is carried out and the same is stained. The sample preparation thus includes, for example, the spreading 106 of the venous blood on a slide 108, and the subsequent staining of the slide 108, for example using the known May-Giemsa stain. The smeared and colored blood sample 109 is shown schematically on the slide 108.

1B schematically shows the microscope hardware 110 used according to the invention, which, as indicated schematically by the arrow 112, receives the smeared and stained blood on the slide 108. The hardware of the microscope 110 includes a movable cross table, not shown, on which the slide 108 is arranged, controllable optics and lighting, and a CCD color camera with a frame grabber. The slide 108, which is arranged on the movable cross table, is displaced by means of the same under the microscope, in order to carry out a digitization 114 of the blood sample arranged on the slide 108.

The digitization 114 of the blood sample 109 takes place in such a way that the object is guided past the blood sample 109 in a meandering manner, as is shown schematically at 116 in FIG. 1A. In fact, this run is achieved in that the slide 108 with the blood sample 109 arranged thereon is correspondingly moved past the objective by means of the movable cross table. As a rule, different sections of the blood sample 109 are digitized during the digitization and, as is shown schematically by the arrow 118, a plurality of individual images 120χ to 120 _{n are} output. In the event that the blood sample 109 is very small, the digitization can also be carried out in one pass and only a single image can be output. At the output of microscope hardware 110, as shown schematically by arrow 122, is provided that a single image or the multiple individual images 120ι to 120 _{n is provided} for further processing.

The signal processing device or image processing device according to the invention, which is implemented, for example, in a computer, is described in more detail below with reference to FIG. The image processing is implemented in the computer 124, which receives the single image provided by the microscope hardware 110 or a plurality of the individual images 120 1 to 120 _{n in} succession at an input 126.

The image processing consists of substeps of color standardization, pixel block classification, cell grouping, cell separation, provision of the individual cells, feature calculation, cell classification and output of the differential blood count.

The color normalization 128 is provided in order to ensure that differently colored samples 109 are standardized to a “standard sample” with a defined color distribution and optionally defined illumination. The subsequent pixel block classification 130 serves to combine several pixels of the image into a block and the assignment of these blocks to one or more classes. In connection with the preferred exemplary embodiment relating to differential blood count analysis, preferred and possible classes are the background or white blood cells. The blocks are assigned as follows to the pixel block classification to certain cell groups 132, which is followed by a cell separation 134 in order to safely separate all cells, since it may well happen that some cells are arranged to overlap or adjoin one another. For every Individual cell 136, a feature calculation 138 is carried out in order to obtain corresponding features from the individual cells present. that are characteristic of individual cell types. This is followed by a cell classification 140 according to which it is decided on the basis of the features obtained to which cell type the individual cell belongs. All results of the cell classification 140 of the entire sample, that is to say all the processed individual images 120 1 to 120 _n, form the differential blood count 142 which is output at the end.

The individual sections of the approach according to the invention for the detection of cell types in a biological sample that have just been described with reference to FIG. IC are explained in more detail below.

The color standardization 128 ensures that differently colored samples 109 are standardized to a “standard sample” with a defined color distribution and optionally defined illumination.

For the subsequent pixel block classification 130, it is necessary that the images 120 1 to 120 _n received at the input 126 have a certain and always the same color distribution in order to ensure that the classification of the individual blocks can be carried out correctly. Conventional methods use methods, such as the color calibration of cameras, to ensure stable and constant recording of the images. The disadvantage of this approach is that it is static and is calculated only once. These methods are therefore inflexible, especially if something changes in the color of the sample material. In this case, the known color calibration no longer matches the situation and must be recalculated. The disadvantage is obvious, since a time expenditure and possibly an additional user interaction are required here, which is not practical for an automatic system as is aimed at according to the invention. The inventive method circumvents this known from the literature weakness by each image 120ι to 120 _n is individually treated, and a known and pre-defined color distribution is adjusted. This ensures that changes in the coloring and in the recording technique can be compensated for in a simple and precise manner.

According to a preferred exemplary embodiment of the present invention, a histogram adjustment based on the color values of the received individual images 120 1 to 120 _n is carried out for the normalization of the colored blood smears 109. A color histogram, which is based on the red channel, the green channel and the blue channel of the digital camera, has two characteristic maxima and a minimum enclosed by these two maxima in each color channel (RGB) for a typical section of a digitized blood smear. The locations where these extreme points appear are different for each channel. In order to achieve a normalization of a color image in this sense, the locations of the extremes must be calculated for each color channel of the image and made available to the method. As soon as these locations are known for all color channels, the histogram can be recalculated for each image and each color channel. For this purpose, the locations measured in the actual image for a color channel are shifted to the locations previously defined and defined for this channel, and values in the histogram between the three extremes are linearly interpolated. This is carried out for each channel, so that a standardized image with a known and defined color distribution results.

In the case of the pixel block classification 130, a plurality of pixels in the received digital image are combined to form a block and the corresponding block is assigned to a class. In the preferred embodiment, the possible classes are the background, red blood cells (erythrocyte), white blood cells (leukocyte) and the nucleus of white blood cells. In general, this method can also be used for other classes and other problems, in which case a corresponding adjustment of the color distribution in color normalization step 128 must then be carried out.

In accordance with a preferred exemplary embodiment, a digitized color image 120 1 to 120 _{n of} a blood section 109 is divided into blocks of sizes 8 × 8 pixels. The average color value per channel (R, G, B) is calculated for each block. The three mean values obtained in this way are fed to a classifier which, based on the three mean values, assigns the respective block to one of the four classes mentioned above. According to a preferred embodiment, a verification step is provided in order to avoid possible misclassifications of a block. Any misclassifications of a block are identified and corrected by a comparison with the surroundings of the block, so that the method according to the invention is more robust against lighting fluctuations.

2 and 3 again show the main steps of the color normalization 128 and the pixel block classification 130 just described. The individual steps of the pixel block classification are shown again in FIG. 2, the step 130a, as mentioned, here The digital color image is broken down into blocks of 8 x 8 pixels. In block 130b, the mean color value per channel is calculated for each block, and then at 130c, depending on the color value calculated for each channel, the block is classified as a background, erythrocyte, leukocyte or leukocyte nucleus. 3 shows the two main steps of the color normalization 128 again, with a color histogram being first generated for each channel of the generated image in accordance with 128a, and then the locations of the maxima and the minimum in the histogram at predetermined locations at 128b of a color anal. Following the pixel block classification 130, a cell group determination 132 is carried out, in which the classified blocks are combined into two classes according to the preferred exemplary embodiment. The first class is referred to as "background" and comprises the blocks which were classified as background or as red blood cells in step 130. The second class is the "white blood cells" class and comprises those in step 130 as white blood cells or nuclei of the white blood cells classified blocks. This information is in the form of a binary image which is made available to the subsequent cell separation. Depending on the type of cell types to be recorded, only some or all of the classified blocks can be sent for further processing. In the case of the evaluation of the sample with regard to the white blood cells, it is sufficient to use only the blocks classified as “white blood cells” and “core of a white blood cell”, these being assigned to a common cell group beforehand.

In the step of cell separation 134, all cells are separated, since it can happen that some cells collide or overlap. In order to ensure that touching cells, ie individual cells, are recognized, a cell separation must be carried out. For this purpose, the binary image from step 132 is treated with the aid of the distance transformation known in the prior art. Subsequent to this transformation, it is possible to use the known so-called watershed transformation, which can cut several touching cells apart at the contact line. After cell separation, a correlation analysis of the primary image is carried out in order to localize the individual cells. Such transformations are e.g. mentioned in the publications mentioned below.

"Applying watershed algorithms to the segmentation of clustered nuclei: Defining strategies for nuclei and back- ground ", Malpica N., Ortiz de Solόrzano C., Vaquero JJ, Santos A., Vallcorba I. Garcia-Sagredo JM, del Pozo F. * Cytometry * 28: pages 289-297 1997, ISSN 0196-4763.

"Watershed, hierarchical segmentation and waterfall algorithm" in Mathematical Morphology and its Applications to Image Processing, Beucher, S., J. Serra and P. Soille, Eds. Kluwer Acad. Publ. , Dordrecht, 1994, pages. 69-76.

"An Extension of the Watershed Transform for Color Image Segmentation" In Proc. 6th German Workshop on Color Image Processing ", A. Koschan and T. Harms, G. Stanke, M. Pochanke, Eds., Berlin, ISBN 3- 9807029-4-4, pages 5-12, October 2000.

The individual cells 136 present in this way are fed to the feature calculation 138 in order to acquire predetermined features which are characteristic of individual cell types, in order to then determine at 140 corresponding cell types for the single cell based on the so-called features.

All results of the Zeil classification of the entire sample form the differential blood count 142, the number of cells per cell type preferably being presented to the user in a medically easily understandable manner. According to the preferred exemplary embodiment, only cell types are counted here, that is to say a type of measuring system is implemented. The interpretation of these results is left to a specialist doctor.

Although preferred exemplary embodiments using a color image as a 3-channel image have been described above with reference to the figures, the present invention is not restricted to this. Instead of the RGB values for the Characterization of the color image can also be based on information relating to the luminance and chrominance, L * u * v * or information relating to the hue (H = Hue) of the saturation (S = Saturation) and the value (V = Value) of the image. The color image data can thus also be described in the HSV color frame or in the L * u * v * color space. For this, the RGB data obtained can be converted into the corresponding color spaces. The RGB data can also be transferred to other known color spaces.

Furthermore, the present invention is not limited to the recording of a 3-channel image, and the image can be generated by an n-channel, n ≥ 2, recording. In addition to the color data, the channels can also contain other multi-spectral data / information, such as Information based on the IR rays, UV rays and X-rays, etc.

Furthermore, with regard to the preferred exemplary embodiment described above, it should be pointed out that the steps described therein in connection with the color normalization and the pixel block classification can also be used individually in the detection of other cell types. Furthermore, the color standardization described above can also be used in isolation from the other steps in other classification methods in which it is necessary to provide images with a uniform color distribution. The same applies to the pixel block classification, which can also be used in other classification methods regardless of the method steps described above.

Although the preferred embodiment is described with the processing of a single image, it is obvious that, depending on the circumstances, several images are processed sequentially if the sample is represented by a plurality of images, so as to ensure an analysis of the entire sample.

Claims

claims 1. A method for the detection of different cell types of cells in a biological sample (109), with the following steps: (a) providing an image (120χ to 120 _n ) of the biological sample (109); (b) normalizing (128) the image of the biological sample (109) with respect to a distribution of image values in the image to obtain a normalized image; (c) decomposing (130a) the normalized image into a plurality of image sections; (d) assigning (130, 130b, 130c) each image section of the plurality of image sections to a predetermined class, depending on certain properties of the corresponding image section; (e) capturing (132, 134, 136) the individual cells or the cell group by combining image sections of the same class; (f) detecting (138) predetermined features from the single cells or the cell groups; and (g) assigning (140) the individual cells to different cell types based on the detected features. 2. The method of claim 1, wherein the image (120 _x to 120 _n ) in step (a) is generated by a multi-channel recording, the channels different color information (RGB, HSV, L * u * v *) or other multispectral Contain information, and if the channels contain color information, channels in- Formations with regard to the color RGB of the picture (120 120 to 120 _n ), with regard to the luminance and chrominance (L * u * v *) of the picture (120ι to 120 _n ) or with regard to the hue, saturation and value (HSV) of the Images (120χ to 120 _n ) are assigned, and the further multispectral information is based on recordings by the R rays, UV rays and X-rays. 3. The method of claim 2, wherein the channels contain different color information, and wherein step (d) comprises the following substeps for each image section: (d.l) acquiring color information values for each channel; (d.2) forming an average (130b) for each channel based on the color information values acquired in step (d.l), and (d.3) assigning the image section (130c) to a class based on the mean values determined for each channel. 4. The method according to any one of claims 1 to 3, wherein step (d) comprises verifying an assignment of an image section to a class based on one or more image sections that surround the affected image section. 5. The method according to any one of claims 1 to 4, in which a digital image is generated in step (a), and in which a predetermined number of pixels is selected for determining an image section in step (c) (130a). 6. The method according to any one of claims 1 to 5, wherein the property used in step (d) for the assignment to the classes comprises color values of the image. 7. The method according to any one of claims 1 to 6, wherein in step (b) the image is normalized based on a statistical distribution of the different image values in the image. 8. The method of claim 7, wherein the image values include color information for the image, and wherein the normalization is based on a histogram of the color information. 9. The method according to claim 8, in which each channel for assigned color information at predetermined locations comprises at least two maxima of the color information and a minimum of the color information enclosed by the same, and in which step (b) comprises the following substeps for each channel: (b.l) calculating (128a) the locations of the maxima and the minimum in the image, and (b.2) shifting (198b) the locations calculated in step (b.l) to the locations assigned to the channel under consideration. 10. The method of claim 9, wherein color information between the shifted locations was determined by interpolation between the maxima and the minimum. 11. The method according to any one of claims 1 to 10, in which, prior to step (e), the image sections are combined into specific classes in order to define corresponding image areas (132). 12. The method according to any one of claims 1 to 11, with the following step after step (e) Detection of single cells from the cell groups defined in step (e). 13. The method according to any one of claims 1 to 12, comprising the following steps: (h) determining (142) the number of individual cells per cell type; and (i) Output the number. 14. Device for detecting different cell types of cells in a biological sample (109), with an input (126) for receiving an image (120ι to 120 _n ) of the biological sample (109); to receive a signal processing means (124) which is adapted to the signal present in the input image (120χ to 120 _s) the biological sample (109) with respect to normalize the received image of a distribution of the image values (128), the normalized image decompose (130) a plurality of image sections, assign the image data to predetermined classes depending on predetermined properties, detect (132, 134, 136) individual cells in the image sections, determine (138) predetermined features of the individual cells, and the individual cells of different cell types assign (140) based on the determined features and the class of the assigned image section in which the single cell was contained; and an output for providing the cell types determined by the signal processing device (124). 15. The apparatus of claim 14, having a sample input (112) for receiving the biological sample (109); and a microscope (110) with an associated digital camera for producing a digital image (120ι to 120 _n), the biological sample (109) or a section thereof; the signal processing means (124) being adapted to receive the digital image.