TWI645176B - Threshold determining method, image processing method and image processing device - Google Patents

Abstract

An image processing device according to the present invention includes: an image acquisition unit 1 that acquires an original image obtained by imaging a stained cell; and an image processing member 2 that dyes the dyed region in the original image and The dyed non-stained areas are subjected to image processing different from each other; and the output member 3 outputs the processing result obtained by the image processing unit 2; and the image processing unit 2 obtains each pixel constituting the original image. The frequency distribution of the pixel values specifies the peaks and troughs of the frequency distribution map corresponding to the frequency distribution, and the depth of the trough is the largest among the troughs and the two peaks sandwiching the trough in the frequency distribution diagram. The pixel value corresponding to the position of one trough is used as a threshold, and the original image is divided into a dyed area and a non-stained area.

Description

Threshold determining method, image processing method and image processing device

The present invention relates to a technique for setting a threshold value of a pixel value for an original image obtained by photographing a stained cell, and distinguishing between a dyed region and an undyed region.

In the experiment and study using a sample obtained by two-dimensionally culturing a cell (also referred to as a monolayer culture) on a culture plate or a dish (culture dish), the sample is dyed in order to know the growth state of the cell. . The distribution of cells can be known by selective staining of cells present in the cells.

The pursuit of a technique for automatically distinguishing between a dyed region and an undyed region from an image of a sample dyed in this manner by image processing based on pixel values of pixels constituting the image is pursued. In the technique described in Japanese Laid-Open Patent Publication No. 2009-210409, for example, a method for determining the threshold value for distinguishing the pixel values of the dyed inter-substrate region from the undysed background region can be analyzed using Kittler's discriminant analysis. Discriminant analysis method of Law and Otsu. Further, as a technique for binarizing an image, a P-tile method (p-quantile method), an Mode method, and the like have been known from the past.

The growth state of the cells varies depending on the culture conditions, so that the ratio of the dyed regions in the image varies greatly from 0% to 100% depending on the sample. Again, depending on the cell The image density of the dyed area is also different depending on the color of the body and the state of the dyeing. For this reason, in the various prior art described above, there is a case where the dyed region and the non-stained region cannot be surely distinguished. In particular, when one of the dyed area or the non-stained area occupies most of the image, there is a distinction based on the set threshold that does not meet the judgment made by the expert's naked eye observation. Happening.

The present invention has been made in view of the above problems, and an object of the present invention is to provide a boundary between a dyed region and an undyed region while reducing the uncomfortable feeling to the user by appropriately setting the threshold value of the pixel value. technology.

One aspect of the present invention is a method for determining a threshold value, which is determined from the original image obtained by photographing the stained cells, and is used to distinguish the pixel values of the dyed stained region from the unstained non-stained region. a limit value; and in order to achieve the above object, comprising: a step of obtaining the original image; a step of determining a frequency distribution of pixel values of each pixel constituting the original image; and specifying a frequency distribution map corresponding to the frequency distribution a step of peaks and troughs: and setting a pixel value corresponding to a position of a trough in which the depth of the trough is the largest among the above-mentioned troughs with respect to the two peaks sandwiching the trough in the above-mentioned frequency map The steps for the above threshold.

In the image including the dyed area and the non-stained area, two main peaks corresponding to each of the dyed area and the non-stained area appear in the frequency layout prepared for the pixel values of the pixels constituting the image. The instant layout has bimodality. However, as described above, the image density is not necessarily the same in both the dyed region and the non-stained region, and for this reason, there are many cases in which three or more peaks appear in the frequency layout. Especially when the dyed area occupies most of the image, the unevenness of the pixel values becomes more pronounced, resulting in a plurality of peaks and troughs. The above prior art cannot cope with this situation.

It can be considered that when there are a plurality of troughs in the frequency layout, any one of them corresponds to the threshold value of the distinguishing dyed area and the non-stained area. To evaluate the significance of each trough, consider the method used to find the appropriate threshold. In the present invention, the evaluation is self-clamping between the troughs The depth of the trough observed by the two peaks, and the pixel value corresponding to the deepest trough are regarded as the threshold for distinguishing between the dyed region and the non-stained region. According to the inventor's opinion, when the original image is divided into the dyed area and the non-stained area based on the threshold determined in this way, it is possible to obtain a result that the expert does not seem to have any discomfort.

Furthermore, another aspect of the present invention is an image processing method comprising: a step of determining the threshold value by the threshold value determining method; and dividing the original image into the above based on the threshold value The step of dyeing the area with the above non-stained area.

Still another aspect of the present invention provides an image processing apparatus comprising: an image acquisition means for acquiring an original image obtained by imaging a stained cell; and an image processing means for the original image Image processing in which the dyed dyed area and the unstained non-stained area are different from each other; and an output member that outputs the processing result obtained by the image processing member; and the image processing member Forming a frequency distribution of pixel values of each pixel constituting the original image, and specifying peaks and troughs in the frequency distribution map corresponding to the frequency distribution, and sandwiching the above-mentioned troughs with respect to the frequency map In the two peaks of the trough, the pixel value corresponding to the position of the valley having the largest depth of the trough is used as a threshold value, and the original image is divided into the dyed region and the non-stained region.

In these inventions, the dyed area and the non-stained area in the original image are distinguished based on the threshold value appropriately set based on the result of evaluating the depth of each trough in the above manner. Therefore, the discrimination result close to the judgment of the expert expert can be automatically obtained.

According to the present invention, the threshold value of the pixel value is determined based on the evaluation of the respective troughs appearing in the frequency layout. Thereby, for the dyed area and the non-stained area in the original image, it is possible to achieve a distinction that does not give the user an uncomfortable feeling.

1‧‧‧Photography unit (image acquisition unit, imaging unit)

2‧‧‧Image Processing Unit (Image Processing Unit)

3‧‧‧UI (user interface) section (output component)

11‧‧‧Image Sensor

12‧‧‧A/D converter

21‧‧‧CPU

22‧‧‧Storage

23‧‧‧Interface (output member, receiving unit)

31‧‧‧ Input device

32‧‧‧Display (output member, display unit)

100‧‧‧Image processing device

A[i]‧‧‧ sequence variable

Cp‧‧‧scalar variables

Cv‧‧‧scalar variables

D3‧‧‧ distance

D8‧‧‧ distance

H[i]‧‧‧ sequence variable

H0‧‧ ‧ frequency value

H1‧‧‧ frequency value

H2‧‧‧ frequency value

H3‧‧‧ frequency value

H4‧‧‧ frequency value

H5‧‧‧ frequency value

H6‧‧‧ frequency value

H7‧‧‧ frequency value

H8‧‧‧ frequency value

H9‧‧‧ frequency value

H10‧‧‧ frequency value

Parameters of i‧‧‧ sequence variables

P2‧‧‧Crest

P6‧‧‧Crest

P9‧‧‧Crest

P[i]‧‧‧ sequence variable

Q3‧‧‧ intersection

Q8‧‧‧ intersection

S101‧‧‧Steps

S102‧‧‧Steps

S103‧‧‧Steps

S104‧‧‧Steps

S105‧‧‧Steps

S106‧‧‧Steps

S107‧‧‧Steps

S108‧‧‧Steps

S109‧‧‧Steps

S110‧‧‧Steps

S111‧‧‧Steps

S121‧‧‧Steps

S122‧‧‧Steps

S123‧‧‧Steps

S201‧‧‧ steps

S202‧‧‧Steps

S203‧‧‧Steps

S204‧‧‧Steps

S205‧‧‧Steps

S206‧‧‧ steps

S207‧‧‧Steps

S208‧‧‧Steps

Step S209‧‧‧

S210‧‧‧Steps

T‧‧‧scalar variables

Th‧‧‧ threshold

V3‧‧‧ trough

V8‧‧‧ trough

V[i]‧‧‧ sequence variable

Fig. 1 is a block diagram showing a schematic configuration of an embodiment of an image processing apparatus according to the present invention.

Fig. 2 is a flow chart showing an example of image processing.

3A and 3B are diagrams illustrating an original image and two-dimensional data obtained thereby.

Fig. 4 is a flow chart showing the detection processing of the peaks and troughs.

Fig. 5A is a view showing an example of a frequency layout diagram.

Fig. 5B is a view for explaining a method of evaluating a trough.

Fig. 6 is a view showing a state transition of variables in detection processing of peaks and troughs.

7A to 7C are diagrams showing an example of an original image and a binarized image obtained by binarizing the original image.

8A and 8B are views showing another example of the original image and the frequency distribution map corresponding thereto.

Fig. 1 is a block diagram showing a schematic configuration of an embodiment of an image processing apparatus according to the present invention. The image processing apparatus 100 has a function to perform the image processing method of the present invention. As a specific configuration for this function, the image processing apparatus 100 includes an imaging unit 1, an image processing unit 2, and a UI (User Interface) unit 3.

The imaging unit 1 has a function of photographing cells cultured in a medium carried in a sample container such as an orifice plate, a petri dish, or a disk. Specifically, the imaging unit 1 includes an image sensor 11 and an A/D (Analog to Digital) converter 12 that converts an electrical signal output from the image sensor 11 into a digital signal. A mechanism for holding the sample container may be provided in the imaging unit 1.

As the image sensor 11, for example, a CCD (Charge Coupled Device) sensor or a CMOS (Complementary Metal Oxide Semiconductor) sensor or the like can be used. The image sensor 11 can be any one of a regional image sensor in which a minute light receiving element is two-dimensionally arranged on a light receiving plane, and a linear image sensor in which the light receiving elements are arranged in a line. In the case of using a linear image sensor, a scanning moving mechanism for moving the image sensor relative to the object to be scanned relative to the object for obtaining a two-dimensional image is additionally provided. Moreover, the image sensor 11 can also be an example It is used in combination with a suitable imaging optical system such as a microscope optical system. The electric signal output by the image sensor 11 based on the amount of received light is converted into a digital signal by the A/D converter 12, and the imaging unit 1 outputs the digital image data generated in this manner to the image processing unit 2.

The image processing unit 2 includes a CPU (Central Processing Unit) 21 that executes a control program prepared in advance to implement specific image processing, and a memory 22 that stores a control program to be executed by the CPU 21, and The image data transmitted by the imaging unit 1, the intermediate data generated during the image processing, and the like; and the interface (I/F) 23, which is responsible for data exchange between the image processing unit 2 and the external device.

Further, the UI unit 3 includes an input device 31 such as a mouse, a keyboard, a touch panel, and an operation button, which receives an operation input such as a processing start instruction and a condition setting from the user, and a display 32 that displays the progress and results of the processing. Wait. Furthermore, the respective configurations of the image processing unit 2 and the UI unit 3 may be the same as those of a general personal computer. In other words, the configuration and functions of a general-purpose personal computer can be used as the image processing unit 2 and the UI unit 3.

The image processing apparatus 100 configured as described above is suitable for the purpose of observing the growth state of cells cultured in a sample container. For the purpose of this observation, the sample obtained by two-dimensionally culturing the cells in the sample container was dyed by a suitable dye. In the image obtained by photographing the inside of the sample container, the area occupied by the cells is dyed to form a dyed area, and on the other hand, the area where the cells are not present becomes a non-stained area where the color of the dye does not appear. Hereinafter, in the image processing executable by the image processing apparatus 100, the dyed area and the non-stained area are automatically recognized from the captured image, and one series of image binarization processing will be described based on the result.

Fig. 2 is a flow chart showing an example of image processing. This image processing is realized by the CPU 21 of the image processing unit 2 executing a control program stored in advance in the storage 22 to control each part of the apparatus. First, the imaging unit 1 captures a sample prepared in advance, that is, a cell obtained by culturing the cells in the sample container, thereby obtaining an original image of the sample (step S101).

3A and 3B are diagrams illustrating an original image and two-dimensional data obtained thereby. Figure 3A is A diagram showing an example of an original image. In the original image, in the inner bottom surface of the substantially circular sample container, there are two kinds of regions mixed: no cells exist, the medium is exposed, and the brightness is relatively high, that is, a region close to the lighter image density of white. (non-stained area); and a stained area in which the stained cells are distributed with a higher image density than the non-stained area (low brightness). Next, the two types of areas are distinguished in the following manner.

First, the frequency distribution of the pixel values of the respective pixels corresponding to the inside of the sample container in the pixels constituting the obtained original image is obtained, and a frequency distribution map corresponding thereto is created (step S102). Fig. 3B is a view showing an example of a frequency distribution diagram. Here, the original image is set as a monochrome image, and the pixel value of each pixel is set to indicate the brightness of the pixel in multiple colors. In the case where each pixel is represented by 8-bit data, the pixel value is represented by 256 gray scales from 0 to 255. The smaller the value, the lower the brightness of the pixel, that is, the darker, the closer to black; the larger the value, the higher the brightness of the pixel, that is, the brighter, the closer to white.

In the frequency layout diagram of the original image including the dyed area and the non-stained area, as shown in FIG. 3B, it is expected that a peak on the low-luminance side corresponding to the dyed area and a peak on the high-luminance side corresponding to the non-stained area appear. The position corresponding to the valley between the peaks has a threshold value of the pixel value separating the dyed region from the non-stained region. That is, it can be said that the pixel having the pixel value on the lower luminance side than the threshold value belongs to the dyed region, and the pixel having the pixel value higher than the threshold high luminance side is highly likely to belong to the non-stained region. Therefore, in this image processing, the threshold value for distinguishing the pixel values of the dyed region from the non-stained region is determined based on the frequency distribution obtained from the original image.

However, since the color of the dyed cells has individual differences and unevenness, two peaks do not necessarily appear clearly in the frequency layout, and as shown in FIG. 3B, a plurality of fine irregularities appear in addition to the main peaks. Therefore, in order to remove such small fluctuations, the frequency pattern is smoothed (step S103).

As a method of smoothing, for example, a moving average process, a median filter process, or the like can be considered. Here, in order to avoid the operation of the value on the frequency axis, the pixel value is entered in a certain period. The line sampling is performed by taking down the sampling of the pixel values, thereby obtaining a frequency layout diagram without fine concavities and convexities. This downsampling is equivalent to coarsening the resolution within the range of pixel values. According to the findings of the inventor of the present invention, it can be seen that when the pixel value is represented by an 8-bit 256 gray scale, by subtracting the pixel value to about (1/8), the meaningless can be eliminated without losing meaningful information. Bump. However, the degree of downsampling can also be changed depending on the purpose. Further, the smoothing method is not limited to this.

As shown by the black dot in FIG. 3B, the value of the frequency corresponding to the pixel value is extracted every 8 pixel values skipped, thereby obtaining a frequency distribution of the downsampling to (1/8). More generally, the pixel value corresponding to the value of the subtraction sample (8 in this example) as the tolerance difference column can be obtained as a new level, and will correspond to the pixel value. The frequency is set to the new frequency distribution of the new frequency, thereby obtaining the smoothed frequency distribution and the corresponding frequency layout.

In the example shown in FIG. 3B, the pixel value pair of each of the arithmetic sequence columns {0, 8, 16, 32, ...} is set to 0 with the first term of the arithmetic progression column and the tolerance is set to 8. The sampling is performed frequently, thereby achieving smoothing (subtraction sampling). However, the first item can also be set to any of 1 to 7. Further, the tolerance is not limited to 8 as described above, and can be appropriately changed. By smoothing in this manner, a new frequency distribution in which the meaningless unevenness is removed is obtained. Then, the positions of the peaks and troughs appearing in the frequency distribution map corresponding to the frequency distribution and the heights thereof are detected (step S104).

Fig. 4 is a flow chart showing the detection processing of the peaks and troughs. First, initialization of variables or the like for processing is performed (step S201). In this processing, a sequence variable H[i] storing the frequency of each level in the smoothed frequency pattern, a sequence variable A[i] indicating a change state of the frequency pattern, and a position for recording a peak are used. The sequence variable P[i], the sequence variable V[i] for recording the position of the trough, the scalar variable Cp for counting the number of occurrences of the peak, the scalar variable Cv for counting the number of occurrences of the trough, and the frequency distribution The scalar variable T of the tendency of the graph to change.

Furthermore, the parameter i of each sequence variable indicates the level of the smoothed frequency distribution and the level in the frequency distribution map, and corresponds to the pixel value of the original frequency pattern sampling. That is, the pixel values corresponding to the first item, the second item, and the third item of the arithmetic progression number at the time of sampling correspond to the stages 0, 1, 2, ..., respectively. In the case of the downsampling based on the first order 0 and the tolerance of the difference 8, the pixel values 0, 8, 16, ... correspond to the stages 0, 1, 2, ..., respectively. When the sampling is not performed, the pixel value directly becomes the number of stages. Hereinafter, the total number of smoothed levels is indicated by the symbol N. Subtract the raw data of 256 gray scales to the case of (1/8), N=32.

In the initial state, the value of the frequency corresponding to each of the series i (i = 0, 1, ..., N) of the frequency distribution after smoothing is stored in the sequence variable H[i]. Invalid data is set for another sequence variable A[i], P[i], V[i]. Further, the initial values "0" are set for the scalar variables Cp and Cv, respectively. Further, the initial value "up" is set to indicate that the frequency distribution map is a scalar variable T which is a tendency to rise with the level or a tendency to fall.

Then, based on the sequence variable H[i] indicating the frequency distribution after smoothing, the sequence variable A[i] indicating the state of change of the frequency distribution map is recorded (step S202). Specifically, by classifying the cases based on the comparison of the values H[i] and the values H[i-1] adjacent to each other, according to the following conditional expression:

When H[i]>H[i-1], "rise"

When H[i]=H[i-1], "unchanged"

"Hescent" when H[i]<H[i-1]

Set the value A[i]. As is clear from the above definition, when the value A[i] is "rise", the frequency is increased by one level at this level. Further, when the value A[i] is "unchanged", the frequency is the same as the lower level at this level. Further, when the value A[i] is "decline", the frequency is lowered by one level at this level. Thus, the sequence variable A[i] represents the state of increase or decrease in the frequency of each part of the frequency layout.

Then, the processing of steps S204 to S208 is repeatedly performed for each level i, thereby evaluating the frequency of each level, the position of the peaks and troughs in the specific frequency map, and the height thereof. Specifically In other words, the number i to be evaluated is set to the initial value 1 (step S203) until the number of stages becomes the last value N (step S209), and the number i is incremented one by one (step S210), and steps S204 to S209 are performed. Repeated processing.

The contents of each repeated process will be described. In step S204, it is determined whether or not the value A[i] corresponding to the current level i is "unchanged". If the value A[i] is "unchanged", then (YES in step S204), the processing of steps S205 to S208 is skipped. If the value A[i] is "rise" or "fall" (NO in step S204), then step S205 is performed.

In step S205, the value A[i] corresponding to the current number of stages i and the scalar variable T are evaluated. Specifically, when the value of the scalar variable T is "up" and the value A[i] is "down", step S206 is performed, and in other cases, step S206 is skipped.

In step S206, the rewriting of the variables is performed in the following manner. That is, the information indicating the presence of a peak at the position is recorded in the variable P[i-1] corresponding to the level (i-1) which is one level smaller than the current level i. It is arbitrary to record the peak position in any aspect. For example, the sequence variable may not be used, but only the memory or the register memory peak position. Further, 1 is added to the value of the variable Cp indicating the number of peaks, and the variable T is changed to "down".

In the next step S207, the value A[i] corresponding to the current number of stages i and the scalar variable T are evaluated. When the value of the scalar variable T is "down" and the value A[i] is "up", step S208 is performed. In the case other than this, step S208 is skipped.

In step S208, the rewriting of the variables is performed in the following manner. That is, the variable V[i-1] corresponding to the level (i-1) which is one level smaller than the current level i is recorded, and information indicating that there is a trough at the position is recorded. As with the record of the peak position, the information is arbitrary. Further, 1 is added to the value of the variable Cv indicating the number of valleys, and the variable T is changed to "up".

The procedure for detecting peaks and troughs by repeated processing as described above will be described by way of examples. For the convenience of the following description, the processing path obtained by the conditional branch in the above-mentioned repeated instruction is numbered. As shown in the table on the upper right of FIG. 4, step S204 is The processing result when the determination result is "YES" is to skip steps S205 to S208, and the path directly jumping from step S204 to step S209 is referred to as "path 1". Similarly, the processing path when the determination results of steps S204, S205, and S207 are "NO", "NO", and YES" is referred to as "path 2". Further, the processing path when the determination results of steps S204, S205, and S207 are "NO", "YES", and "NO" is referred to as "path 3". Further, the processing path when the determination results of steps S204, S205, and S207 are all "NO" is referred to as "path 4".

Further, as indicated by the addition of the number (5) in the table, the case in which the determination result is "YES" in steps S205 and S207 cannot be considered from the definition of the variable A[i]. That is, since the processing path advancement processing of all of steps S204 to S208 is not sequentially performed, it is not necessary to consider the processing path.

Fig. 5A is a view showing an example of a frequency layout diagram. Moreover, FIG. 6 is a view showing a state transition of the variables in the detection processing of the peaks and troughs. Hereinafter, the progress of the peak and valley detection processing will be described using the frequency layout diagram shown in FIG. 5A as an example. This frequency map is used to illustrate the hypothesis, and is not directly related to the frequency layout shown in FIG. 3B. Moreover, the frequency layout of FIG. 5A represents only one of the levels, specifically 0 to 10 of the 0 to N levels.

It is assumed that the smoothed frequency layout is as shown in Fig. 5A. The value H[i] of the frequency corresponding to each level i (i = 0, 1, 2, ..., N) is represented by the symbol Hi (i = 0, 1, 2, ..., N). That is, for example, H[0]=H0, H[1]=H1, . In the table of Fig. 6, the column "H[i]" at the left end is recorded in the value of the frequency H[i] initialized in step S201 and corresponding to the number of stages i. The symbols (=), (<), and (>) inserted between the values indicate the magnitude relationship between the frequencies in the adjacent levels, and correspond to the frequency layout of FIG. 5A. That is, H0=H1, H1<H2, H2>H3, .

In step S202, a sequence variable A[i] indicating a change state of the frequency distribution map is obtained based on the sequence variable H[i]. That is, the set value A[i] is any of "rise", "unchanged", and "declined" based on the change of the value H[i] with respect to the value H[i-1]. In the example shown in Figure 5A, H[0]=H[1], so the value of A[1] is "unchanged" as shown in FIG. 6. Also, since H[1]<H[2], the value of A[2] is "up". Further, since H[2]>H[3], the value of A[3] is "decreased". The subsequent values are also obtained in the same manner and provided to the repeated processing. As described above, the initial value of the variable T is "up".

In the first iterative process (i = 1), the value of A[i] = A[1] is "unchanged", so the processing along the processing path 1 is executed, and the values of the variables are not changed. In the second iteration (i=2), the value of A[2] is "up" and the value of T is "up", so the processing along the processing path 4 is executed. In this case, the values of the variables are also unchanged.

In the third iteration (i=3), the value of A[3] is "declined" and the value of T is "up". Therefore, the determination in step S205 is "YES", and the processing along the processing path 3 is executed. Therefore, it is recorded in P[i-1], that is, P[2] has information on the peak. In Fig. 6, the peak position is indicated by a black dot print. Further, the variable Cp for counting the number of peaks is changed from 0 to 1, and the variable T is changed from "up" to "down".

In the fourth iteration (i=4), the value of A[4] is "up" and the value of T is "down". Therefore, the determination in step S207 is "YES", and the processing along the processing path 2 is executed. Therefore, the information on the valley of V[3] is recorded. In Fig. 6, the trough position is indicated by a white dot print. Further, the variable Cv for counting the number of troughs is changed from 0 to 1, and the variable T is changed from "down" to "up".

In the fifth iteration (i=5), the value of A[5] is "unchanged", so the processing along the processing path 1 is executed, and the values of the variables are unchanged. In the sixth iteration (i=6), the value of A[6] is "up" and the value of T is "up", so the processing along the processing path 4 is executed. In this case, the values of the variables are also unchanged.

In the seventh iteration (i=7), the value of A[7] is "declined" and the value of T is "up". Therefore, the processing along the processing path 3 is executed in the same manner as the third time. Thereby, a peak is recorded in P[6], the value of the variable Cp for counting the number of peaks is increased by 1, and the value of T is changed to "down". In the 8th iteration (i=8), the value of A[8] is "declined" and the value of T is "down", therefore, the same as the sixth time, the processing along the processing path 4 is executed, and the values of the variables are unchanged.

In the ninth iteration (i=9), the value of A[9] is "up" and the value of T is "down", so the processing along the processing path 2 is executed. Therefore, information on the existence of troughs in V[8] is recorded. Further, the variable Cv for counting the number of troughs is increased from 1 to 2, and the variable T is changed from "down" to "up". In the tenth iteration (i=10), the value of A[10] is "declined" and the value of T is "up", so the processing along the processing path 3 is performed. Thereby, a peak is recorded in P(9), the value of the variable Cp for counting the number of peaks is increased by 1, and the value of T is changed to "down".

In this manner, the processing shown in the flowchart of FIG. 4 is performed, whereby the peak position and the trough position in the frequency map are specified and recorded. Referring to Fig. 6, peaks are recorded at i = 2, 6, and 9, and on the other hand, troughs are recorded at i = 3, 8. It can be seen that the result coincides with the peak position and the trough position of the frequency layout shown in FIG. 5A.

Returning to Fig. 2, the image processing of this embodiment will be described. If the peaks and troughs in the frequency layout are specified in the above manner, the evaluation of the trough and the determination of the threshold based on the evaluation are performed. The processing content is different depending on the number of detected troughs, that is, the value of the variable Cv.

When the number of detected troughs is 2 or more (YES in step S105), the pixel value corresponding to the position of the highest trough in each trough is regarded as a threshold (steps S106 and S107). The evaluation of the trough was carried out in the following manner.

Fig. 5B is a view for explaining a method of evaluating a trough. When the peaks and troughs are detected on the frequency pattern shown in FIG. 5A, as shown in FIG. 5B, the peak P2 having the value H2 is detected at the position of i=2, and the value is detected at the position of i=6. The peak P6 of H6 detects a peak P9 having a value of H9 at the position of i=9. Further, a valley V3 having a value H3 is detected at a position of i=3, and a valley V8 having a value H8 is detected at a position of i=8.

The threshold value that distinguishes the pixel values of the dyed area from the non-stained area is considered to be the frequency layout. The middle is between the peak corresponding to the dyed area and the peak corresponding to the non-stained area. Therefore, it is more effective to use a pixel value corresponding to the position of the deep trough existing between the two peaks as a threshold value. From this point of view, each trough is evaluated by the depth observed from the two peaks at the position sandwiching the trough in the frequency layout.

Specifically, as shown in FIG. 5B, the depth of the valley V3 is defined by a distance D3 which is a line segment (dashed line) connecting the two peaks P2 and P6 sandwiching the valley V3 and parallel to the valley V3. The intersection Q3 to the valley V3 of the line segment of the vertical axis (frequency axis) (corresponding to the straight line represented by the equation i=3). On the other hand, the depth of the valley V8 is defined by a distance D8 which connects the line segment of the two peaks P6 and P9 sandwiching the valley V8 with the intersection point Q8 of the line passing through the valley V8 and parallel to the vertical axis. Wave Valley V8.

If the formula is used to represent the depth of the trough, it is as follows. For example, regarding the depth of the valley V3, it can be expressed as follows. If the value corresponding to the frequency of the point Q3 is h3, it is known as the relationship between the peak P2 and the peak P6 of FIG. 5B, and can be expressed as: h3=H2+(H6-H2)‧(3-2)/( 6-2).

Here, the values 3, 2, and 6 are values of the level i corresponding to the positions of the peaks P3, P2, and P6, respectively. Therefore, the depth D3 with respect to the valley V3 can be expressed by the following formula: D3 = h3 - H3 = H2 + (H6 - H2) ‧ (3-2) / (6-2) - H3.

Thus, the depth of each trough can be calculated from the value of the position and frequency of the trough, and the value of the position and frequency of each of the two peaks sandwiching the trough.

It can be considered that among the troughs, the largest depth obtained by the above method is the most important trough of the two peaks. The pixel value corresponding to the level corresponding to the position of the trough is set as a threshold value that distinguishes the dyed area from the non-stained area (step S107). The threshold is determined in this way when a plurality of troughs are detected.

On the other hand, when there is one detected trough (NO in step S105 and YES in step S121), that is, when only two peaks sandwiching one trough are detected, of course It can be said that these two peaks correspond to the dyed area and the non-stained area, during which The trough is divided into dyed areas and non-stained areas. Therefore, the pixel value corresponding to the level corresponding to the trough position is set as the threshold value in the case (step S122).

Further, when the number of the troughs is 0, that is, when the troughs are not detected (NO in step S121), it is indicated that there is only one peak, which means that the entire sample container is occupied by the dyed area or the non-stained area. The possibility is higher. The whole is a state in which the cells in the non-stained region are not grown, and such a sample is less likely to be observed. More realistically, it is conceivable that the cells diffuse throughout the sample container, and the entire image becomes a stained area. Therefore, in this case, the pixel value corresponding to the end position on the high luminance side of the single peak appearing in the frequency layout is set as the threshold value (step S123).

When the threshold is determined in this manner, the dyed area and the non-stained area are distinguished based on the threshold (step S108). Specifically, a darker pixel having a pixel value having a lower luminance than a threshold value belongs to a dyed region, and a brighter pixel having a pixel value having a higher luminance than a threshold value is considered to belong to a non-stained region. A pixel having a pixel value equal to the threshold value is classified as a dyed region or a non-stained region. Thus, each pixel in the original image is divided into one of a dyed area or a non-stained area.

Then, a set of original images is calculated (step S109). The collection system indicates an index value of the ratio (area ratio) of the dyed region in the original image (strictly speaking, the region corresponding to the inside of the sample container). In the case where there is no cell and the entire original image is a non-stained region, the set is 0, and conversely, the set of the original image is a dyed region. The growth state of the cells in the sample container can be quantitatively represented by the value of the set. By appropriately setting the threshold value, the growth state of the cells can be quantitatively represented by the value of the set automatically calculated from the original image obtained by the photographing.

Further, by binarizing the pixel values different from each other in the dyed region and the non-stained region which are distinguished based on the threshold value, a binarized image is created (step S110). The created binarized image is displayed on the display 32 and presented to the user (step S111). Or, it is output to the external device via the interface 23.

7A to 7C are diagrams showing an example of an original image and a binarized image obtained by binarizing the original image. Specifically, FIG. 7A is an example of an original image, and FIG. 7B is a frequency diagram thereof. Further, Fig. 7C is a view showing an image of binarization. As shown in Fig. 7A, in the sample, cells were distributed and the stained stained region was complicatedly mixed with the non-stained region in which the cells were absent, and the unevenness in the stained region was also large. Therefore, as shown in FIG. 7B, a plurality of large irregularities also appear in the frequency layout.

The binarized image obtained by performing the above-described image processing on the original image having such a frequency map is the image shown in Fig. 7C. In Fig. 7C, the pixels which are divided into the dyed areas are differently painted in black, and the pixels which are distinguished as the non-stained areas are painted white. That is, the first pixel value of the relatively low luminance is assigned to the pixel divided into the dyed area, and the second pixel value of the relatively high luminance with respect to the first pixel value is assigned to the pixel divided into the non-stained area. If compared with the original image shown in FIG. 7A, it can be said that the boundary between the dyed area and the non-stained area grasped from the original image by the visual recognition and the binarized image automatically created by image processing The boundaries of these areas are very consistent. In Fig. 7B, the symbol Th indicates the threshold value set by the image processing of the present embodiment. It can be seen that the threshold value is set at the position of the valley having the largest depth observed from the two peaks.

8A and 8B are views showing another example of the original image and the frequency distribution map corresponding thereto. Specifically, FIG. 8A is an example of an image with a single peak, and FIG. 8B is an example of an image with a single trough. In the original image shown in Fig. 8A, the dyed area spreads throughout the sample container, and the unevenness of the concentration is also small. Such a sample can be considered to be a distribution of cells throughout the container. As indicated by the dotted line on the frequency layout diagram, the original image is divided into the dyed area as a whole by setting the threshold value at the high-luminance side end position of the peak.

Further, in the original image shown in Fig. 8B, for example, compared with the image shown in Fig. 7A, the difference between the brighter non-stained area and the darker stained area appears relatively clearly. In the frequency layout corresponding to the original image, the peak on the low-luminance side corresponding to the dyed region and the peak on the high-luminance side corresponding to the non-stained region are more clearly expressed. Therefore, such as frequency layout As indicated by the upper dotted line, a threshold value is set at a position sandwiched between the valleys of the two peaks, whereby the dyed area and the non-dyed area can be surely distinguished.

As described above, in this embodiment, the peaks and troughs in the frequency map of the pixel values of the respective pixels constituting the original image are detected. In the detected troughs, the threshold value for distinguishing the pixel values of the dyed region from the non-stained region is set from the position where the depth of the troughs of the two peaks of the trough is the largest. By such an operation, it is possible to automatically obtain a distinction between the result of the discrimination made by the user and the result of the discrimination by the image processing.

In evaluating the trough, the value of the frequency corresponding to the intersection of the line segment connecting the two peaks sandwiching the trough and the line segment parallel to the vertical axis (frequency axis), and the value of the frequency in the trough The difference is the depth of the trough. With this operation, the most obvious trough between the two peaks can be specified from the complex troughs.

In the frequency layout of the pixel values of the multi-gray scale, fine concavities and convexities due to uneven concentration of the sample are generated, and all of these are regarded as peaks and troughs, which degrades the determination accuracy. It is preferable to remove such small irregularities in advance by performing smoothing. In the above embodiment, as one of the smoothing modes, the peaks and troughs are detected in the frequency distribution pattern in which the pixel values are sampled and the frequency values corresponding thereto are used to determine the threshold. value. Specifically, the periodic sampling of the pixel values is performed so that the entire number of extracted pixel values becomes an arithmetic progression. By appropriately performing the downsampling of the pixel values, the threshold can be determined without being affected by the small unevenness.

In addition, when the detected trough is one, it is considered that the two peaks sandwiching the trough correspond to the dyed area and the non-stained area, respectively. Therefore, in this case, the dyed area and the non-stained area can be surely distinguished by setting the threshold value by a single trough position.

Further, since the trough is not detected, that is, when there is only a single peak in the frequency layout, the possibility that the original image as a whole is likely to be a dyed region is high, and the end position of the peak on the high luminance side is set. Limit. By this operation, all the pixels included in the peak are distinguished into the dyed regions, and the result consistent with the actual state can be obtained.

Further, based on the threshold determined in the above manner, the original image is divided into a dyed region and a non-stained region, whereby the dyed region and the non-stained region can be automatically recognized by image processing without depending on the naked eye of the user.

Further, by creating a binarized image in which pixel values corresponding to pixels corresponding to the dyed region and pixels corresponding to the non-stained region are different from each other, it is possible to provide the user with easy visibility of the dyed region and the non-stained region. An image of the distribution state.

As described above, in the image processing apparatus 100 of the embodiment, the imaging unit 1, the image processing unit 2, and the UI unit 3 are respectively the "image acquisition means" and the "image processing means" of the present invention. The "output component" functions. Further, the imaging unit 1 also functions as the "imaging unit" of the present invention, and the display 32 functions as the "display unit" of the present invention.

It is to be noted that the present invention is not limited to the above-described embodiments, and various modifications may be made without departing from the spirit and scope of the invention. For example, the image processing device 100 of the above-described embodiment includes the imaging unit 1 as the "image acquisition means" of the present invention. However, the present invention is also applicable to an image processing apparatus that does not have a function of capturing a sample. That is, the image processing apparatus of the present invention may be configured to receive image data of an original image captured by an external imaging device and perform the above-described processing on the original image.

In this case, the interface for receiving image data from the external device functions as the "receiving portion" of the "image acquisition means" of the present invention. In the above embodiment, the interface 23 of the image processing unit 2 assumes this function, and similarly to the original image captured by the imaging unit 1, the original image obtained by external imaging can be processed.

Furthermore, the embodiment of the present invention which does not include the camera function can also be realized by a combination of a hardware such as a general-purpose personal computer or a workstation and a software for realizing the processing algorithm of the present invention with the hardware as described above. . That is, the present invention can be implemented by a general hardware mounting control program based on the technical idea of the present invention.

Further, the image processing apparatus 100 of the above embodiment has a display 32 for displaying a binarized image obtained as a result of image processing. However, regarding the results of image processing, It is not only binarized and outputted, for example, image processing for giving visual information different from the region classified into the dyed region and the region classified as the non-stained region may be performed on the original image and output. Further, the function of displaying the image after the image processing can be performed by an external display device, and the image processing result is output to the display device. Moreover, the result can also be output to an external calculation device. In the above embodiment, the interface 23 of the image processing unit 2 can function as an "output member" of the present invention which is responsible for outputting data to the outside, thereby achieving such an object.

Further, in the above embodiment, the processing is performed based on the pixel value corresponding to the brightness of the pixel whose pixel is brighter. Accordingly, for example, a pixel value defined in such a manner that the image density is higher, that is, the darker pixel value is larger, can be similarly processed.

Further, in the present specification, the frequency layout is illustrated for easy understanding of the principle of the invention. However, in the actual processing, if the frequency distribution is obtained, the peaks and troughs on the frequency layout can be directly detected from the result. . Therefore, in the present invention, the production of the frequency layout itself is not an essential requirement.

The present invention is preferably applicable to the purpose of distinguishing between dyed regions and unstained regions from original images containing stained cells, such as experiments and observations particularly suitable for medical and biochemical fields.

Claims (11)

A threshold determining method for determining a threshold value for distinguishing pixel values of a dyed stained region from an unstained non-stained region by using an original image obtained by photographing the stained cells, and having: a step of obtaining the original image; a step of determining a frequency distribution of pixel values of each pixel constituting the original image; and a step of specifying a peak and a trough in a frequency distribution map corresponding to the frequency distribution: and according to the frequency The step of setting the threshold value is such that when the plurality of specific valleys are specified, the depths of the two peaks sandwiching the trough from the frequency map are evaluated for each of the troughs, and the depth is the largest. The pixel value corresponding to the position of one trough is set to the above threshold. A method for determining a threshold value according to claim 1, wherein an imaginary line segment connected to two of said peaks sandwiching one of said valleys in said frequency distribution map and an intersection of imaginary line segments parallel to said frequency axis and said frequency axis are equivalent The difference between the value of the frequency and the value of the frequency corresponding to the trough is the index value of the depth of the trough. The threshold value determining method according to claim 1, wherein the peak and the trough are specified based on a frequency distribution corresponding to a partial pixel value of one of the arithmetic progression values based on the pixel values obtained by the pixel. The threshold value determining method according to claim 1, wherein when the number of the valleys is one in the frequency distribution map, a pixel value corresponding to the valley is used as the threshold value. The threshold value determining method according to claim 1, wherein when the trough does not exist in the frequency distribution map, a pixel value of a pixel having the highest luminance in the image is used as the threshold value. An image processing method comprising: by any one of claims 1 to 5 The step of determining the threshold value by the limit value determining method; and the step of dividing the original image into the dyed region and the non-stained region based on the threshold value. The image processing method of claim 6, comprising the step of setting a pixel value of the pixel divided into the dyed region to a first value, and dividing the pixel of the pixel into the non-stained region on the other hand. The value is set to a second value different from the first value described above, and a binarized image obtained by performing binary expression on the original image is created. An image processing apparatus comprising: an image acquisition means for acquiring an original image obtained by imaging a stained cell; and an image processing means for dyeing the dyed area and the undyed image in the original image The non-stained areas perform different image processing; and the output means outputs the processing result obtained by the image processing means; and the image processing means obtains pixels of the pixels constituting the original image The frequency distribution of the values specifies peaks and troughs in the frequency distribution map corresponding to the frequency distribution, and sets the threshold value according to the frequency distribution map. When the specified plurality of valleys are plural, the evaluation of each of the valleys is performed. The pixel value corresponding to the position of one of the valleys having the largest depth is regarded as a threshold value from the depth seen by the two peaks of the valley in the above-mentioned frequency distribution diagram, and the original image is divided into the dyed area and the above Non-stained area. The image processing device of claim 8, wherein the image processing means creates a binarized image that performs binary representation on the original image based on the threshold value, and the output member has the display of the binarized image The display unit. The image processing device according to claim 8, wherein the image acquisition means has an imaging unit that images the cells and generates the original image. The image processing device of claim 8, wherein the image acquisition means has a receiving portion that receives image data of the original image from an external device.