JP6999118B2 - Image processing equipment - Google Patents

Description

The present invention relates to an image processing apparatus.

In biological science, medicine, etc., it is known that the state of living organisms such as health and disease is related to, for example, the state of cells and organelles in cells. Therefore, analyzing these relationships is one means of solving problems in various fields such as biological science and medicine. Further, analysis of the transmission route of information transmitted between cells or within cells can be useful for research on biosensors in industrial applications, pharmaceuticals for the purpose of disease prevention, and the like. As various analysis techniques for cells, tissue pieces, etc., for example, a technique using image processing is known (see, for example, Patent Document 1).

U.S. Pat. No. 0228069

However, when calculating intracellular or cell-cell interactions, if information obtained by image processing an image containing captured cells is used, the amount of information obtained is large, so in order to obtain a correlation, A huge amount of calculation is required, which may result in poor analysis.

The present invention has been made in view of the above problems, and an object of the present invention is to provide an image processing apparatus capable of suppressing analysis defects.

In order to solve the above problem, one aspect of the present invention comprises a cell image acquisition unit that acquires a cell image containing at least one cell, and a plurality of components existing inside the cell contained in the cell image. The plurality of features are calculated by using a sparse estimation based on a feature amount calculation unit that calculates a plurality of types of feature amounts corresponding to each component for each component and the likelihood of the plurality of the feature amounts . It is an image processing apparatus including a correlation extraction unit that extracts a specific correlation from a plurality of correlations between quantities .

According to the present invention, it is possible to suppress analysis defects due to image processing.

It is a schematic diagram which shows an example of the structure of the microscope observation system by embodiment of this invention. It is a block diagram which shows an example of the functional structure of each part provided in the image processing apparatus of this embodiment. It is a flow chart which shows an example of the calculation procedure of the calculation unit of this embodiment. It is a schematic diagram which shows an example of the calculation result of the feature amount by the feature amount calculation part of this embodiment. It is a figure which shows an example of the matrix of the feature amount for each cell of this embodiment. It is a schematic diagram which shows an example of the correlation between the feature quantity of this embodiment. It is a schematic diagram which shows an example of the specific correlation of this embodiment. It is a flow chart which shows an example of the procedure of cross-validation by the correlation extraction part of this embodiment. It is a schematic diagram which shows an example of the calculation procedure of the average likelihood by the correlation extraction part of this embodiment. It is a graph which shows an example of the relationship between the average likelihood and the regularization parameter of this embodiment. It is a schematic diagram which shows an example of the extraction result of the correlation of the feature amount by the correlation extraction part of this embodiment. It is a table which shows an example of the protein annotation database of this embodiment. It is a table which shows an example of the feature amount annotation database of this embodiment. It is a figure which shows an example of the comparative type of the feature amount network of this embodiment. It is a figure which shows an example of the determination procedure of the regularization parameter using the biological knowledge of this embodiment. It is a figure which shows the relationship between the knowledge base network and the prediction network about the connection existence or absence of the feature amount network of this embodiment.

[Embodiment]
Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic diagram showing an example of the configuration of the microscope observation system 1 according to the embodiment of the present invention.

The microscope observation system 1 performs image processing on an image acquired by imaging a cell or the like. In the following description, an image obtained by imaging a cell or the like is also simply referred to as a cell image. The microscope observation system 1 includes an image processing device 10, a microscope device 20, and a display unit 30.

The microscope device 20 is a biological microscope and includes an electric stage 21 and an imaging unit 22. The electric stage 21 can arbitrarily move the position of the object to be imaged in a predetermined direction (for example, a certain direction in a horizontal two-dimensional plane). The image pickup unit 22 includes an image pickup element such as a CCD (Charge-Coupled Device) or a CMOS (Complementary MOS), and images an image pickup object on the electric stage 21. The microscope device 20 may not be provided with the electric stage 21, and the stage may not operate in a predetermined direction.

More specifically, the microscope device 20 has functions such as a differential interference contrast microscope (DIC), a phase contrast microscope, a fluorescence microscope, a confocal microscope, and a super-resolution microscope. The microscope device 20 takes an image of the culture vessel placed on the electric stage 21. The culture vessel is, for example, a well plate WP. The microscope device 20 irradiates the cells cultured in a large number of wells W of the well plate WP with light, so that the transmitted light transmitted through the cells is imaged as an image of the cells. As a result, it is possible to acquire a transmission DIC image of cells, a phase contrast image, a dark field image, a bright field image, and the like. Further, by irradiating the cell with excitation light that excites the fluorescent substance, the fluorescence emitted from the biological substance is imaged as an image of the cell. Further, the microscope device 20 captures the fluorescence emitted from the color-developing substance itself taken into the biological substance and the fluorescence emitted when the substance having a chromophore binds to the biological substance as an image of the above-mentioned cells. You may. As a result, the microscope observation system 1 can acquire a fluorescence image, a confocal image, and a super-resolution image. The method of acquiring an image of a cell is not limited to an optical microscope. For example, an electron microscope may be used. Further, as an image containing cells described later, an image obtained by a different method may be used to obtain a correlation. That is, the type of the image including the cells may be appropriately selected. The cells in the present embodiment are, for example, primary cultured cells, established cultured cells, cells of tissue sections, and the like. In order to observe cells, the sample to be observed may be an aggregate of cells, a tissue sample, an organ, or a solid (animal or the like), and an image containing the cells may be obtained. The state of the cells is not particularly limited, and may be a living state or a fixed state, and may be either "in-vivo" or "in-vitro". Of course, a living state information and a fixed information may be combined.
The state of the cells may be appropriately selected according to the purpose. For example, fixed or unfixed may be selected depending on the type (eg, protein, organelle) to be discriminated in the intracellular structure. In addition, when acquiring the dynamic behavior of a fixed cell, a plurality of fixed cells having different conditions are created and the dynamic behavior is acquired. In the structure inside the cell, the type to be discriminated is not limited to the nucleus.
Further, in order to observe the cells, the cells may be observed after the cells have been treated in advance. Of course, in order to observe the cells, the cells may be observed without being treated. When observing the cells, the cells may be stained by immunostaining and observed. For example, the staining solution to be used may be selected in the type to be discriminated in the intracellular nuclear structure. As for the dyeing method, any dyeing method can be used. For example, there are various special stains mainly used for tissue staining, hybridization using base sequence binding, and the like.
Further, the cells may be treated with a photoprotein (for example, a photoprotein expressed from an introduced gene (such as a luciferase gene)) and observed. For example, the photoprotein to be used may be selected in the type to be discriminated in the intracellular nuclear structure. Further, the pretreatment for analyzing the correlation acquisition such as the means for observing these cells and / or the method for staining the cells may be appropriately selected for each purpose. For example, when obtaining the dynamic behavior of a cell, the dynamic information of the cell is acquired by the optimum method, and when obtaining the signal transduction in the cell, the information on the signal transduction in the cell is acquired by the optimum method. It doesn't matter. The preprocessing selected for each purpose may be different. Further, the types of preprocessing selected for each purpose may be reduced. For example, even if the optimum method for acquiring the dynamic behavior of a cell and the method for acquiring intracellular signal transduction are different, it is complicated to acquire each information by different methods. In addition, if it is sufficient to acquire each information, a common method may be used instead of the optimum method.

The well plate WP has a plurality of wells W. In this example, the well plate WP has 96 wells W of 12 × 8. The cells are cultured in well W under specific experimental conditions. Specific experimental conditions include temperature, humidity, culture period, elapsed time since the stimulus was applied, type and intensity of the stimulus applied, presence / absence of stimulus, induction of biological characteristics, and the like. The stimulus is, for example, a physical stimulus such as electricity, sound wave, magnetism, or light, or a chemical stimulus caused by administration of a substance or a drug. The biological characteristic is a characteristic indicating the stage of cell differentiation, morphology, cell number, and the like.

FIG. 2 is a block diagram showing an example of the functional configuration of each part included in the image processing apparatus 10 of the present embodiment. The image processing device 10 is a computer device that analyzes an image acquired by the microscope device 20. The image processing device 10 includes a calculation unit 100, a storage unit 200, and a result output unit 300. The image processed by the image processing device 10 is not limited to the image captured by the microscope device 20, for example, an image stored in advance in the storage unit 200 included in the image processing device 10 or an image (not shown). The image may be an image stored in advance in an external storage device.

The arithmetic unit 100 functions by the processor executing a program stored in the storage unit 200. Further, a part or all of each functional unit of these arithmetic units 110 may be configured by hardware such as LSI (Large Scale Integration) or ASIC (Application Specific Integrated Circuit). The calculation unit 100 includes a cell image acquisition unit 101, a feature amount calculation unit 102, a noise component removal unit 103, and a correlation extraction unit 104.

The cell image acquisition unit 101 acquires the cell image captured by the imaging unit 22, and supplies the acquired cell image to 102. Here, the cell image acquired by the cell image acquisition unit 101 includes a plurality of images in which the cell culture state is captured in time series, and a plurality of images in which the cells are cultured under various experimental conditions.

The feature amount calculation unit 102 calculates a plurality of types of feature amounts of the cell image supplied by the cell image acquisition unit 101. This feature amount includes the brightness, area, dispersion, etc. of the cell image.
That is, it is a feature derived from the information acquired from the captured cell image. For example, the brightness distribution in the acquired image is calculated. Using multiple images that differ due to changes in cell state such as time series or differentiation, changes in the calculated brightness distribution for a predetermined time, or changes due to changes in the cell state such as differentiation of the calculated brightness distribution, etc. Positional information indicating a change in luminance different from that of the above may be obtained, and the change in luminance may be used as a feature quantity. In this case, not only the change in time but also a plurality of images having different changes in the cell state such as differentiation may be used. Further, the information on the position indicating the change in different brightness may be used as the feature amount. For example, the behavior of the cell within a predetermined time or the behavior accompanying the change of the cell state such as the differentiation of the cell may be used, or the change of the shape of the cell within the predetermined time or the change of the cell state such as the differentiation of the shape of the cell. It does not matter if it changes with it. Further, if the captured cell image does not show a change within a predetermined time or a change due to a change in cell state such as differentiation, it may not change or may be used as a feature amount.
The noise component removing unit 103 removes a noise component (noise) from the feature amount calculated by the feature amount calculating unit 102.
The correlation extraction unit 104 describes a plurality of correlations between the feature amounts calculated by the feature amount calculation unit 102 based on the likelihood of the feature amount with respect to the feature amount after the noise component removal unit 103 removes the noise component. Extract a specific correlation from us. Here, the likelihood is a numerical value indicating the likelihood of inferring a predetermined condition from the result when the result is calculated according to the predetermined condition. The likelihood is a cost function that represents the likelihood of a parameter to be estimated when the data follows a probabilistic model.
The result output unit 300 outputs the calculation result of the calculation unit 100 to the display unit 30. The result output unit 300 may output the calculation result of the calculation unit 100 to an output device other than the display unit 30, a storage device, or the like.
The display unit 30 displays the calculation result output by the result output unit 300.
The specific calculation procedure of the above-mentioned calculation unit 100 will be described with reference to FIG.

FIG. 3 is a flow chart showing an example of the calculation procedure of the calculation unit 100 of the present embodiment. The calculation procedure shown here is an example, and the calculation procedure may be omitted or added.
The cell image acquisition unit 101 acquires a cell image (step S10). This cell image contains images of multiple types of biological tissues of different sizes, such as genes, proteins, and organelles. In addition, the cell image contains cell shape information. In addition, dynamic behavior is acquired from cell images having different time series or different cell states such as differentiation. Therefore, the time series is different from the cell image in a certain cell state such as a plurality of different scales from the fine structure in the cell to the cell shape, a certain predetermined time, or a certain cell state, or differentiation, etc. It is also possible to call it multi-scale analysis because the analysis is performed using scales having different dimensions in cell images having different cell states.

The feature amount calculation unit 102 extracts the image of the cell included in the cell image acquired in step S10 for each cell (step S20). The feature amount calculation unit 102 extracts an image of a cell by performing image processing on the cell image by a known method. In this example, the feature amount calculation unit 102 extracts an image of a cell by performing contour extraction, pattern matching, or the like of the image.

Next, the feature amount calculation unit 102 determines the cell type with respect to the image of the cell extracted in step S20 (step S30). Further, the feature amount calculation unit 102 determines the cell components included in the image of the cells extracted in step S20 based on the determination result in step S30 (step S40). Here, the cellular components include organelles such as cell nuclei, lysosomes, Golgi apparatus, and mitochondria, and proteins constituting organelles. Although the cell type is determined in step S30, it is not necessary to determine the cell type. In this case, if the type of cell to be introduced has been determined in advance, that information may be used. Of course, it is not necessary to specify the cell type.

Next, the feature amount calculation unit 102 calculates the feature amount of the image for each cell component determined in step S40 (step S50). This feature amount includes the brightness value of the pixel, the area of a certain area in the image, the dispersion value of the brightness of the pixel, and the like. In addition, there are a plurality of types of feature quantities depending on the constituent elements of the cell. As an example, the feature amount of the image of the cell nucleus includes the total luminance value in the nucleus, the area of the nucleus, and the like. The feature amount of the cytoplasmic image includes the total brightness value in the cytoplasm, the area of the cytoplasm, and the like. In addition, the feature amount of the image of the whole cell includes the total intracellular luminance value, the area of the cell, and the like. In addition, the feature amount of the mitochondrial image includes the fragmentation rate. The feature amount calculation unit 102 may calculate the feature amount by normalizing it to a value between 0 (zero) and 1, for example.

Further, the feature amount calculation unit 102 may calculate the feature amount based on the information of the experimental conditions for the cells associated with the cell image. For example, in the case of a cell image captured when an antibody is reacted with a cell, the feature amount calculation unit 102 may calculate a feature amount peculiar to the reaction of the antibody. Further, in the case of a cell image captured when the cells are stained or when the fluorescent protein is applied to the cells, the feature amount calculation unit 102 is used when the cells are stained or when the fluorescent protein is applied to the cells. You may calculate the feature amount peculiar to.
In these cases, the storage unit 200 may include the experimental condition storage unit 202. The experimental condition storage unit 202 stores information on experimental conditions for cells associated with the cell image for each cell image.

The feature amount calculation unit 102 supplies the feature amount calculated in step S50 to the noise component removing unit 103.

The noise component removing unit 103 removes the noise component from the feature amount calculated in step S50 (step S60). Specifically, the noise component removing unit 103 acquires information indicating a normal range or an abnormal range of the feature amount. The information indicating the normal range or the abnormal range of this feature amount is predetermined based on the characteristics of the cells imaged in the cell image. For example, among the feature quantities of the image of the cell nucleus, the normal range of the total luminance value in the nucleus is determined based on the characteristics of the image of the cell nucleus. When the calculated feature amount is not included in this normal range, the noise component removing unit 103 removes the feature amount as a noise component. Here, when the feature amount is removed as a noise component, the noise component removing unit 103 removes each cell. Specifically, a plurality of feature quantities may be calculated for a certain cell. For example, for a certain cell, the intracellular total luminance value, the intracellular total luminance value, and the area of the nucleus may be calculated as feature quantities, respectively. In this case, when the intracellular total luminance value of a certain cell is removed as a noise component, the noise component removing unit 103 also removes the intracellular total luminance value and the area of the nucleus of the cell. That is, when at least one feature amount among the plurality of feature amounts calculated for a certain cell is not included in the normal range, the noise component removing unit 103 also removes the other feature amounts of this cell.

That is, the noise component removing unit 103 captures the noise component in the cell image from the feature amounts supplied to the correlation extraction unit 104 based on the information indicating the characteristics of the cells captured in the cell image. Remove cell by cell. With this configuration, the noise component removing unit 103 can exclude the feature amount on a cell-by-cell basis when there is a feature amount having a relatively low reliability. That is, according to the noise component removing unit 103, the reliability of the feature amount can be improved.

Further, when the calculated feature amount is included in this normal range, the noise component removing unit 103 supplies the feature amount to the correlation extraction unit 104. The noise component removing unit 103 is not an essential component, and can be omitted depending on the state of the cell image and the state of calculating the feature amount.

The correlation extraction unit 104 uses sparse estimation to extract a specific correlation from a plurality of correlations between the features calculated by the feature calculation unit 102 based on the likelihood of the features ( Step S70). Hereinafter, the processing performed by the correlation extraction unit 104 will be described more specifically.

The correlation extraction unit 104 acquires the feature amount after the noise component is removed in step S60. This feature amount is calculated for each cell by the feature amount calculation unit 102 in step S50. The calculation result of the feature amount of a certain protein by the feature amount calculation unit 102 will be described with reference to FIG.

FIG. 4 is a schematic diagram showing an example of the calculation result of the feature amount by the feature amount calculation unit 102 of the present embodiment. The feature amount calculation unit 102 calculates a plurality of feature amounts for protein 1 for each cell and each time. In this example, the feature amount calculation unit 102 calculates the feature amount for N cells from cell 1 to cell N. Further, in this example, the feature amount calculation unit 102 calculates the feature amount for seven times from time 1 to time 7. Further, in this example, the feature amount calculation unit 102 calculates K types of feature amounts from the feature amount k1 to the feature amount kK. That is, in this example, the feature amount calculation unit 102 calculates the feature amount in the directions of the three axes. Here, the axis in the cell direction is described as the axis Nc, the axis in the time direction is described as the axis N, and the axis in the feature amount direction is described as the axis d1.

The K types of features from the feature amount k1 to the feature amount kK are a combination of the feature amounts for the protein 1. For proteins other than protein 1 or intracellular components other than protein 1, the types and combinations of feature amounts may differ.

FIG. 5 is a diagram showing an example of the matrix X of the feature amount for each cell of the present embodiment. The feature amount for a certain cell can also be shown by the matrix X shown in FIG. 5, which has an axis N in the row direction and an axis d in the column direction. In FIG. 5, each element of the matrix X is shown by the mean value of the cell population, but statistics such as the median and the mode can also be used. Of course, the matrix X of the feature amount for each cell may be used.

FIG. 6 is a schematic diagram showing an example of the correlation between the feature quantities of the present embodiment. The correlation extraction unit 104 extracts a specific correlation from a plurality of correlations between the feature quantities calculated by the feature quantity calculation unit 102.
FIG. 7 is a schematic diagram showing an example of a specific correlation of the present embodiment. Here, the specific correlation is a correlation selected by mathematical calculation from a plurality of correlations between feature quantities. The number of specific correlations is less than the number of plurality of correlations between the features before being extracted by the correlation extraction unit 104. That is, the correlation extraction unit 104 makes the number of correlations between the feature quantities sparse. The correlation extraction unit 104 makes the number of correlations between the features sparse by using sparse estimation for the correlations between the features. An example of sparse estimation by the correlation extraction unit 104 will be described.

The likelihood in the Gaussian model used by the correlation extraction unit 104 for sparse estimation is shown in Eq. (1).

In this equation (1), the term "φ (Λ)" is a regularization term. There are various forms of the functional form of this regularization term, depending on the attributes of the correlation to be extracted. In particular, when assuming a Gaussian model, it is known that an accuracy matrix having a sparse component can be obtained by adding an L1 regularization term as in Eq. (2). Λ in the equation (2) represents the strength of regularization, and the larger it is, the more likely it is that the components of the precision matrix become sparse.

Further, for example, it is possible to obtain a sparse submatrix by setting the regularization term as in the equation (3).

[Sparse estimation by Graphical Lasso]
Here, an example of sparse estimation will be described when the regularization term is a functional form by Graphical Lasso. Graphical Lasso is an efficient algorithm for estimating the accuracy matrix from a Gaussian model with L1 regularization. For example, Biostatistics (2008) by JEROME FRIEDMAN, TREVOR HASTIE and ROBERT TIBSHIRANI, 9, 3 432-441, "Space inverse covariance estimation" with the graphic.
First, the L1 regularized likelihood of the Gaussian model, which is the equation (1) when the equation (2) is adopted, is differentiated by the precision matrix.

Next, from the condition that the precision matrix is a positive-definite matrix, the diagonal components can be obtained as in Eq. (5).

Next, paying attention to the specific variable i, the variance-covariance matrix, the precision matrix, and the sample covariance matrix are divided into block matrices as in equation (6), so that equation (4) is as shown in equation (7). Can be transformed.

Equation (7) can be reduced to equation (8), which is a general Lasso regression optimization problem, and can be obtained by an existing solver or the like.

If the optimum solution β * (beta asterisk) of the equation (8) satisfying the condition of the equation (9) is obtained, the equation (6), the equation (7), and the equation (11) derived from the equation (10) can be obtained. By using the equation (12), each component of the block matrix of the equation (6) can be obtained. However, I in the equation (10) is an identity matrix.

The operations of the above equations (5) to (12) are repeated by exchanging the rows and columns.

As described above, in the image processing apparatus 10, the correlation extraction unit 104 performs sparse estimation to make the number of correlations between feature quantities sparse. Therefore, according to the image processing apparatus 10, the analysis can be performed in a state where the number of correlations between the feature quantities is reduced. That is, according to the image processing apparatus 10, it is possible to reduce the amount of calculation for analyzing the correlation between the feature quantities. Since the number of correlations between feature quantities can be reduced using the likelihood as an index, it is possible to suppress analysis defects by, for example, arbitrarily reducing the amount of calculation for the analysis of the correlations between feature quantities by a person. can. In addition, the feature amount was acquired in order to obtain the correlation of the feature amount from the image including the cells. In addition to the luminance information directly derived from the image, information other than the image information is also used as the feature amount. For example, the location of the luminance information in the cell (nucleus, nuclear envelope, cytoplasm) is estimated from the shape of the image. Further, for example, the location (nucleus, nuclear envelope, cytoplasm) of the stained site (site having luminance information) of the stained cell is estimated. As described above, in order to acquire the correlation of the feature amount by using the information estimated from the image information in addition to the information directly derived from the image information, the calculation amount becomes enormous. Therefore, by performing the sparse estimation by the correlation extraction unit 104, the number of correlations between the feature quantities can be sparse, and it is possible to suppress an enormous amount of calculation.
Further, for example, when the intracellular correlation is acquired, for example, when the intracellular correlation is acquired, for example, a plurality of cells may be acquired in an image, and in the plurality of cells, the intracellular correlation is obtained. It is possible to obtain the correlation. In this case, when the correlation within a plurality of cells is acquired, the correlation in a plurality of cells can be acquired as compared with the case where the correlation in a single cell is acquired. Therefore, signal transduction calculated, for example, as the acquisition of the correlation can be obtained. The accuracy of the route can be improved. When acquiring the correlation of a plurality of cells in order to improve the accuracy, the amount of calculation performed to acquire the correlation becomes enormous. In this case, the amount of calculation can be reduced by using the likelihood as an index, so that analysis failure can be suppressed. Further, for example, to explain the case of acquiring the correlation between cells as an example, when acquiring the correlation between cells, for example, a plurality of cells may be acquired in an image, and the cells of the plurality of cells may be acquired. It is possible to obtain the correlation. In this case, a predetermined cell may be correlated with a plurality of cells, and cells other than the predetermined cell may also be correlated with a plurality of cells. Further, in this case, by acquiring the cell correlation between the cells, it is possible to improve the accuracy of the signal transduction pathway between the cells, which is calculated, for example, as the acquisition of the correlation. When acquiring the correlation of a plurality of cells in order to improve the accuracy, the amount of calculation performed to acquire the correlation becomes enormous. In this case, the amount of calculation can be reduced by using the likelihood as an index, so that analysis failure can be suppressed.
Further, the feature amount calculated by the feature amount calculation unit 102 is involved in the intracellular signal transduction, for example, when the intracellular signal transduction after the cell receives the signal from the outside of the cell is obtained as a correlation. The type of protein to be used may be extracted as a feature amount. That is, for example, the type of substance involved in intracellular signal transduction may be used, or the resulting change in cell shape due to intracellular signal transduction may be used. Substances involved in intracellular signal transduction may be specified by NMR or the like, or may be a method of inferring the interacting partner from the staining solution to be used.

[Determining regularization parameters]
Next, the determination of the regularization parameter in the sparse estimation performed by the correlation extraction unit 104 will be described. Sparse estimation has the advantages of relatively high model flexibility and the ability to adjust the model for human or computer interpretation. On the other hand, sparse estimation has a problem that the model cannot be uniquely determined because of the regularization parameter dependence. The procedure for determining the regularization parameter for solving this problem is shown. Examples of the parameter determination procedure include a parameter determination procedure by cross-validation and a parameter determination procedure using biological knowledge. Of these, the procedure for determining parameters by cross-validation will be described.

[Λ estimation by cross validation]
FIG. 8 is a flow chart showing an example of the cross-validation procedure by the correlation extraction unit 104 of the present embodiment.
The correlation extraction unit 104 acquires the matrix X, the regularization parameter λ, and the number of divisions K (step S100). The correlation extraction unit 104 divides the matrix X based on the number of divisions K acquired in step S100 (step S110).
Next, the correlation extraction unit 104 processes a double loop of the loop of the regularization parameter λ (step S120) and the loop of the number of divisions K (step S130). In the loop of the number of divisions K, the correlation extraction unit 104 calculates the likelihood Lk using the matrix Xk and the matrix X / k (step S140). In the loop of the regularization parameter λ, the correlation extraction unit 104 calculates the average likelihood Lλ of the likelihood Lk in a certain regularization parameter λ (step S150). The details of the procedure for calculating the average likelihood Lλ by the correlation extraction unit 104 will be described with reference to FIG.

FIG. 9 is a schematic diagram showing an example of a procedure for calculating the average likelihood Lλ by the correlation extraction unit 104 of the present embodiment. The correlation extraction unit 104 divides the matrix X into K pieces in the cell direction Nc in the loop having the number of divisions K described above. The correlation extraction unit 104 calculates the likelihood Lk for the matrix X divided into K pieces.

Specifically, the correlation extraction unit 104 calculates the likelihood L1 for the elements included in the calculation frame W1 shown in FIG. 9 (k = 1). Further, the correlation extraction unit 104 calculates the likelihood L2 for the elements included in the calculation frame W2-1 and the calculation frame W2-2 shown in FIG. 9 (k = 2) for a certain regularization parameter λ. Further, the correlation extraction unit 104 calculates the likelihood LK for the elements included in the calculation frame WK shown in FIG. 9 (k = K). That is, the correlation extraction unit 104 calculates the likelihood Lk from k = 1 to k = K for a certain regularization parameter λ. The correlation extraction unit 104 calculates the average likelihood Lλ by the equation (13) for the likelihood Lk calculated for each regularization parameter λ.

Next, the correlation extraction unit 104 calculates the regularization parameter λ whose average likelihood Lλ indicates the maximum value Max (Lλ) as the regularization parameter λ to be calculated (step S160), and determines the regularization parameter λ. Finish the calculation. An example of the calculation procedure of the regularization parameter λ by the correlation extraction unit 104 will be described.

In this example, the correlation extraction unit 104 calculates the likelihood Lλ when the regularization parameter λ is changed from 0 to 1. Specifically, the correlation extraction unit 104 calculates the likelihood Lk shown in FIG. 9 by setting the regularization parameter λ to a certain value from 0 to 1. The correlation extraction unit 104 calculates the average of the likelihood Lk, that is, the average likelihood Lλ, and generates a correspondence between the regularization parameter λ and the average likelihood Lλ. FIG. 10 shows the correspondence between the regularization parameter λ generated by the correlation extraction unit 104 and the average likelihood Lλ.

FIG. 10 is a graph showing an example of the relationship between the average likelihood Lλ and the regularization parameter λ of the present embodiment. In the case of the example shown in FIG. 10, the average likelihood Lλ shows the maximum value in the regularization parameter λa. In this case, the correlation extraction unit 104 calculates the regularization parameter λa as the regularization parameter λ to be calculated.

In step S110 described above, the correlation extraction unit 104 divides the matrix X into K pieces by the number of divisions K in the cell direction Nc. In other words, the correlation extraction unit 104 extracts a specific correlation by obtaining the likelihood L of the feature amount based on the number of cells imaged in the cell image. Here, the correlation extraction unit 104 can also divide the matrix X in the time direction N. However, in order for the correlation extraction unit 104 to divide the matrix X in the time direction N, a plurality of cell images are required in the time direction N. That is, in order for the correlation extraction unit 104 to divide the matrix X in the time direction N, a plurality of cell images taken at different times are required. On the other hand, when the correlation extraction unit 104 divides the matrix X in the cell direction Nc, a plurality of cell images taken at different times from each other are unnecessary. That is, the correlation extraction unit 104 of the present embodiment can calculate the regularization parameter λ from, for example, one cell image even if there are no plurality of cell images captured at different times.

[Biological interpretation of sparse estimation results]
Next, the biological interpretation of the sparse estimation result by the correlation extraction unit 104 will be described with reference to FIGS. 11 to 13. Based on the biological information, a biological interpretation can be added to the sparse estimation result by the correlation extraction unit 104. In this example, the biological information will be described as being stored in the storage unit 200 in advance, but the present invention is not limited to this. The biological information may be supplied from the outside of the microscope observation system 1 via a network, a portable storage medium, or the like. Biological information includes intracellular component annotation database, feature amount annotation database, and the like. The intracellular component is a component that constitutes a cell. Examples of the elements constituting the cell include proteins, genes, compounds and the like. Hereinafter, a specific example of the procedure for biological interpretation of the sparse estimation result by the correlation extraction unit 104 will be described.

FIG. 11 is a schematic diagram showing an example of the extraction result of the correlation of the feature amount by the correlation extraction unit 104 of the present embodiment. In this example, the correlation extraction unit 104 extracts the correlation between the six types of proteins A to F and the mitochondria with respect to the feature amount.

In the case of this example, the cell image captures cells containing proteins A to F and mitochondria. The feature amount calculation unit 102 calculates the feature amounts of the types corresponding to these proteins A to F and mitochondria. That is, the feature amount calculation unit 102 calculates the type of feature amount corresponding to the type of the intracellular component contained in the cell captured in the cell image among the plurality of types of feature amounts.

Specifically, the correlation extraction unit 104 extracts three types of correlations, the correlation R1, the correlation R2, and the correlation R3, between the feature amount of protein A and the feature amount of protein D. .. Here, the correlation R1 is a correlation between the feature amount A3 of the protein A and the feature amount D2 of the protein D. The correlation R2 is a correlation between the feature amount A1 of the protein A and the feature amount D3 of the protein D. The correlation R3 is a correlation between the feature amount A4 of protein A and the feature amount D1 of protein D. For example, the feature amount A1 is a dispersion value of an image of protein A. The feature amount D3 is a luminance value of the image of the protein D.

Further, the correlation extraction unit 104 extracts the correlation R4 between the feature amount of protein A and the feature amount of mitochondria. Here, the correlation R4 is a correlation between the feature amount A2 of protein A and the feature amount M1 of mitochondria. The feature amount A2 is, for example, the ratio of the total intranuclear luminance value to the total intracellular luminance value of the image of protein A. For example, the feature amount M1 is the fragmentation rate of mitochondria shown by the image of mitochondria.

FIG. 12 is a table showing an example of the intracellular component annotation database of the present embodiment. This intracellular component annotation database associates the types of intracellular components with the functions of the intracellular components. In one example of this embodiment, the intracellular component annotation database is stored in the storage unit 200 in advance. Specifically, in the intracellular component annotation database, the type "protein A" is associated with the function "transcription factor (promoting)". Also, in the intracellular component annotation database, the type "protein B" is associated with the function "kinase".

FIG. 13 is a table showing an example of the feature amount annotation database of the present embodiment. This feature amount annotation database associates the NW element, the feature amount, the change direction of the feature amount, and the information indicating the biological meaning. In one example of this embodiment, the feature amount annotation database is stored in the storage unit 200 in advance. Specifically, in the feature amount annotation database, the NW element "transcription factor (promotion)", the feature amount "total intranuclear brightness value / total intracellular brightness value", the feature amount change "UP", and biology. The meaning "transcription promotion" is associated with each other. In addition, in the feature amount annotation database, the NW element "mitochondria", the feature amount "fragmentation rate", the feature amount change "UP", and the biological meaning "promotion of division, increase in oxidative stress, enhancement of mitophagy" Are associated with each other. The feature amount annotation database is a specific example of the type storage unit 201 included in the storage unit 200. That is, the type storage unit 201 stores the types of intracellular components contained in the cells imaged in the cell image and the types of the feature amount in association with each other.

When the correlation between the extracted correlation R4, that is, the ratio of the total intranuclear luminance value to the total intracellular luminance value of the image of protein A and the fragmentation rate of mitochondria is highly correlated, the following is performed. Can be used for biological interpretation.

That is, it is determined that the function of protein A is "transcription factor (promotion)" based on the intracellular component annotation database. In addition, if the feature amount "total intranuclear luminance value / total intracellular luminance value" associated with the "transcription factor (promotion)" indicates the feature amount change "UP" based on the feature amount annotation database, the organism. It can be determined that the scientific meaning is "promotion of transcription". In addition, if the feature amount "fragmentation rate" associated with "mitochondria" indicates a feature amount change "UP" based on the feature amount annotation database, its biological meaning is "promotion of division, increase in oxidative stress," It can be determined that "mittophagy is enhanced".

Based on these determination results, the following biological interpretation of the correlation R4 can be added. That is, (1) protein A promotes transcription of proteins involved in mitochondrial division. (2) Protein A promotes transcription of proteins that enhance mitophagy.

As described above, according to the image processing apparatus 10, it is possible to give a suggestion to the biological interpretation of the correlation based on the extraction result of the correlation between the cell features and the biological information. ..
From the cell features used to obtain the correlation, biological information on the features is created. That is, it creates biological information on the cell features used to obtain the correlation. This allows a biological interpretation of the extracted correlation.

[Extraction of features of feature network using sparse estimation results]
The elements of the feature amount network can be extracted by using the sparse estimation result by the correlation extraction unit 104. Here, the feature amount network refers to a network of correlations between feature amounts. The elements of the feature network include nodes, edges, and subgraphs (clusters ) . Features The features of the network include the presence or absence of hubs, the presence or absence of clusters, and the like. For example, whether or not a node has a hub can be determined based on the value of the partial correlation matrix. Here, the hub is a feature quantity having a relatively large number of correlations with other feature quantities.

When a hub exists in a certain node, it is considered that the feature quantity that is the hub or the node containing the hub has biologically important meaning. Therefore, the discovery of the presence of a hub may lead to the discovery of important proteins and important features. That is, by using the sparse estimation result by the correlation extraction unit 104, it is possible to contribute to the discovery of important proteins and important feature quantities.
Therefore, the correlation extraction unit 104 can specify one or more elements from the plurality of elements constituting the feature amount network from the calculated feature amount network by using the sparse estimation. As a result, it is possible to extract the elements to be considered preferentially when acquiring the correlation. By extracting, it can be taken into consideration when acquiring other correlations.
Of course, the correlation extraction unit 104 can specify one or more element groups from the plurality of element groups constituting the feature amount network from the calculated feature amount network by using the sparse estimation. .. As the element group, for example, a plurality of proteins having the same secondary structure of the protein may be used as one element group.

[Detection of changes between feature networks]
By using the sparse estimation result by the correlation extraction unit 104, it is possible to detect a change between a plurality of feature amount networks.
FIG. 14 is a diagram showing an example of a comparative type of the feature amount network of the present embodiment. Feature networks are comparable to each other at each particle size, from micro to macro. Specifically, it is possible to detect changes in edge particle size based on changes in relationships. This change detection of edge particle size has a score of d (d-1) / d. In addition, it is possible to detect changes in node particle size based on changes in variables. This change detection of node particle size has d scores. It is possible to detect changes in cluster particle size based on changes in EGONET. This change detection of cluster particle size has d scores. It is possible to detect changes in network particle size based on changes in the network. This change detection of network particle size has one score.

The change detection of the node particle size is performed by, for example, the stochastic neighbor method. The change detection of the node particle size by this stochastic neighborhood method is possible by calculating the neighborhood probability between the i-th element and the j-th element of the dissimilarity matrix and scoring the neighborhood probability. Further, by arranging the scores for a certain index of a certain protein for each protein and for each index, the change in the score can be graphed. For example, Tsuyoshi Ide, Spiros Papadimitriou, and Michael Vlachos are described in Proceedings of the Engineering IEEE International Conference on Data Mining (ICDM) -3

The change detection of the cluster particle size is performed by EGONET. Here, EGONET is a feature network composed of nodes and edges, focusing on individual nodes (Ego), and the relationship between the nodes of interest, the nodes related to the nodes of interest, and the edges stretched by these nodes. It is a figure which shows. According to EGONET, edges for the node of interest are extracted from all edges included in the feature network. Therefore, it is easier to compare EGONETs than to compare feature networks. It is useful for visualizing the correlation structure around the node of interest.

As a variant of change detection between feature networks, there is a quantitative comparison between two or more feature networks. For example, a feature network of cells taken from the tissue of a person suffering from a disease can be compared with a feature network of cells taken from the tissue of a healthy person. In this case, it is possible to discover the structure of the feature network peculiar to a certain disease by comparing the feature networks. Further, in the present embodiment, the correlation extraction unit 104 derives a sparse feature quantity network that is easy to interpret by using the sparse estimation, so that the comparison of the feature quantity networks is easier than in the case where the sparse estimation is not used. be.

In comparison between feature network, network learning by SVM, random forest, etc. may be used. Further, during this network learning, the regularization parameter may be adjusted in order to improve the classification performance.

[Identification of proteins with the same function by comparison between feature networks]
When one protein P1 and another protein P2 have the same function, there is a difference between the edge between the protein P1 and the other protein Pn and the edge between the protein P2 and the other protein Pn. May not be. That is, when the protein P1 and the protein P2 are the same protein, the structures of the feature amount networks are similar to each other. By utilizing this property, it is possible to identify a protein having the same function. For example, identification of the same functional protein by extracting only protein P1 from the feature network and excluding only protein P2 from the feature network during sparse estimation by the correlation extraction unit 104. It can be performed.

[Determination of regularization parameters using biological knowledge]
FIG. 15 is a diagram showing an example of a procedure for determining a regularization parameter using the biological knowledge of the present embodiment. In this example, biological knowledge-based binding between proteins is pre-defined. Specifically, based on biological knowledge, a bond that "should have a bond" and a bond that "should not have a bond" between a feature of one protein and a feature of another protein. Is defined in advance as a knowledge database. In the present embodiment, this predetermined knowledge database may be stored in the storage unit 200.

The regularization parameter λ is determined by calculating the distance between this knowledge database and the feature amount network indicated by the sparse estimation result by the correlation extraction unit 104, that is, Fscore. This Fscore is represented by the formula (16) by the precision (Precision) shown in the formula (14) and the reproducibility (Recall) shown in the formula (15). Fscore depends on the regularization parameter, takes the maximum value when the estimated network is closest to the knowledge database, and adopts the regularization parameter at this time. Also, as shown in FIG. 15, the range of regularization parameters can be narrowed even if the knowledge database is inadequate, i.e., even if the matrix components are not completely labeled.

The relationship between TP, FP, and FN represented by the formulas (14) and (15) is shown in FIG.
FIG. 16 is a diagram showing the relationship between the knowledge base network and the prediction network regarding the presence or absence of the combination of the feature amount networks of the present embodiment. The accuracy is the ratio of the elements of the feature network that are predicted to be edges to those that are actually edges. Further, the reproducibility (Recall) is a ratio of an element of the feature amount network, which is actually an edge, and which is predicted to be an edge.

As described above, the correlation extraction unit 104 may extract a specific correlation by obtaining the likelihood of the feature amount based on the information indicating the characteristics of the cells imaged in the cell image.
The correlation extracted by the above-described embodiment may be verified by using academic literature or a database as appropriate. When the database is divided into a plurality of parts, the information from the divided databases may be appropriately selected. In this case, the divided database may be divided according to the type of gene, protein, or the like. Further, the extracted correlation may be verified by combining the verification results of a plurality of databases.

A program for executing each process of the image processing apparatus 10 according to the embodiment of the present invention is recorded on a computer-readable recording medium, and the program recorded on the recording medium is read and executed by the computer system. Thereby, various processes described above may be performed.

The "computer system" here may include hardware such as an OS and peripheral devices. Further, the "computer system" includes the homepage providing environment (or display environment) if the WWW system is used. The "computer-readable recording medium" includes a flexible disk, a magneto-optical disk, a ROM, a writable non-volatile memory such as a flash memory, a portable medium such as a CD-ROM, a hard disk built in a computer system, and the like. Refers to the storage device of.

Further, the "computer-readable recording medium" is a volatile memory inside a computer system that serves as a server or client when a program is transmitted via a network such as the Internet or a communication line such as a telephone line (for example, DRAM (Dynamic)). It also includes those that hold the program for a certain period of time, such as Random Access Memory)). Further, the program may be transmitted from a computer system in which this program is stored in a storage device or the like to another computer system via a transmission medium or by a transmission wave in the transmission medium. Here, the "transmission medium" for transmitting a program refers to a medium having a function of transmitting information, such as a network (communication network) such as the Internet or a communication line (communication line) such as a telephone line. Further, the above program may be for realizing a part of the above-mentioned functions. Further, it may be a so-called difference file (difference program) that can realize the above-mentioned function in combination with a program already recorded in the computer system.

Although the embodiment of the present invention has been described in detail with reference to the drawings, the specific configuration is not limited to this embodiment, and the design and the like within a range not deviating from the gist of the present invention are also included.

1 ... Microscope observation system, 10 ... Image processing device, 20 ... Microscope device, 101 ... Cell image acquisition unit, 102 ... Feature amount calculation unit, 103 ... Noise component removal unit, 104 ... Correlation extraction unit

Claims (14)

A cell image acquisition unit that acquires a cell image containing at least one cell,
A feature amount calculation unit that calculates a plurality of types of feature amounts corresponding to each component of a plurality of components existing inside the cell included in the cell image for each component, and a feature amount calculation unit.
A correlation extraction unit that extracts a specific correlation from a plurality of correlations between the plurality of features using sparse estimation based on the likelihoods of the plurality of features.
An image processing device comprising. The image processing according to claim 1, wherein the feature amount calculation unit further calculates the overall feature amount of the cell and calculates the overall feature amount of the cell and a plurality of feature amounts corresponding to each of the components. Device. Further, a first storage unit in which the type of the component and the type of the feature amount are stored in association with each other is provided.
The feature amount calculation unit is
The image processing apparatus according to claim 1 or 2, wherein the feature amount of the component is calculated based on the information stored in the first storage unit. The first storage unit further stores the cell type and the feature amount type in association with each other.
The feature amount calculation unit is
The image processing apparatus according to claim 3, wherein the feature amount of the whole cell is calculated based on the information stored in the first storage unit. Further, a first output unit for outputting the specific correlation as a calculation result is provided.
The image processing apparatus according to any one of claims 1 to 4. The image processing apparatus according to claim 5, wherein the first output unit further outputs the plurality of correlations. The image processing apparatus according to any one of claims 1 to 6, further comprising a second output unit that extracts and outputs a hub included in the specific correlation network. The network is a network composed of a node which is a node associated with a feature amount corresponding to the component and an edge which is a side which shows the specific correlation by connecting the nodes. A plurality of the subgraphs are compared with respect to the subgraph which is a network obtained by extracting the node corresponding to the hub among the nodes constituting the network based on the hub output by the second output unit. The image processing apparatus according to claim 7 , further comprising a third output unit that outputs the amount of change between the networks . It is a network related to the specific correlation, and is composed of a node which is a node associated with a feature amount corresponding to the component and an edge which is a side connecting the nodes and shows the specific correlation. The image processing apparatus according to any one of claims 1 to 8 , further comprising a fourth output unit that quantitatively compares a plurality of the networks to be formed and outputs a change in the edge. By a node which is a network related to the specific correlation and which is a node associated with a feature amount corresponding to the component, and an edge which is a side which shows the specific correlation by connecting the nodes. The image processing according to any one of claims 1 to 8, further comprising a fifth output unit that quantitatively compares a plurality of the networks to be configured and outputs changes for each node. Device. If any of the plurality of feature quantities corresponding to each of the above components is not included in the normal range or is included in the abnormal range, a removal unit that excludes all the feature quantities of the corresponding components is removed. The image processing apparatus according to any one of claims 1 to 10 . Further, a second storage unit for storing information representing the biological meaning of the change in the feature amount in association with the feature amount is provided.
The correlation extraction unit extracts a biological interpretation of the specific correlation based on the specific correlation and the information stored in the second storage unit.
It further comprises a sixth output section that outputs the biological interpretation.
The image processing apparatus according to any one of claims 5 to 11 . Further provided with a third storage unit for storing data of biological knowledge about the components.
The correlation extraction unit is determined based on the result of comparing each correlation obtained by changing the regularization parameter with the data of the biological knowledge stored in the third storage unit. The image processing apparatus according to any one of claims 1 to 12 , which sets the regularization parameter of the above. The correlation extraction unit extracts one or more specific elements from the elements constituting the plurality of correlations.
The image processing apparatus according to any one of claims 1 to 13 .

Images