JP7679997B2 - Cell observation image capturing device and cell observation image acquiring method - Google Patents

Description

This disclosure relates to a cell observation image capturing device and a cell observation image acquisition method.

With the development of regenerative medicine technology, there are an increasing number of cases where cells are cultivated industrially. Unlike reactions of chemical substances, the properties of cultured cells vary widely and they are easily affected by factors such as the environment. For this reason, there is an increasing need to monitor the state of cell culture. In this regard, cells are currently observed using microscopes in the field of regenerative medicine.

Generally, microscopes have a very narrow field of view, so only a portion of the cell culture vessel can be observed at one time. Therefore, to obtain information about the entire surface of the vessel, a technique called tiling photography is used to increase the field of view by moving the stage on which the cell culture vessel is placed using an electric XY stage, or by moving the camera and lighting set used for photography, and then stitching together the images taken while positioning the vessel to generate an observation image that covers the entire surface of the vessel.

JP 2011-127972 A

However, tiling photography has issues such as differences in brightness between adjacent images, misalignment of the photography locations between adjacent images due to limitations in the positional accuracy of the motorized stage, and even inconsistencies between adjacent images caused by cells moving during photography, which results in discontinuities within the images. It is desirable to automate the monitoring of the culture status using a computer, but images with internal discontinuities are not compatible with automatic processing.

Another issue with tiling photography is that when you want to capture a wide area, the number of repetitions of stage movement and photography increases, which means the time required for photography is long. Unless special consideration is given, the environment for photography is often different from that suitable for cell culture. If cells are exposed to an environment that is not suitable for culture for a long period of time, it will have a negative effect on the culture itself. In addition, if the time required is long, the frequency of monitoring will decrease, making it difficult to respond promptly.
In view of such circumstances, the present disclosure proposes a technique for ensuring the continuity of observed images and shortening the time required for capturing images.

To solve the above problem, according to the present embodiment, a cell observation image capturing device is provided that captures an observation image of a cell by placing a culture vessel that supports cells to be cultured on the cell observation image capturing device, the cell observation image capturing device being provided with an illumination unit that irradiates illumination light onto the culture vessel and an imaging unit that captures an image of the culture vessel illuminated by the illumination light, a predetermined pattern having a difference in light transmittance is disposed between the illumination unit and the top cover of the culture vessel, and the focus of the imaging unit is adjusted to a position shifted a predetermined distance toward the illumination unit from the surface of the imaging unit where the culture vessel is placed.

Further features related to the present disclosure will be apparent from the description of the present specification and the accompanying drawings. Also, aspects of the present disclosure may be realized and realized by the elements and combinations of various elements and aspects set forth in the following detailed description and the appended claims.
It should be understood that the descriptions in this specification are exemplary and illustrative only and are not intended to limit the scope or application of the present disclosure in any way.

This disclosure makes it possible to ensure the continuity of observed images and shorten the time required for capturing images.

1 is a diagram showing an example of the configuration of a scanner device 100 for capturing a cell observation image according to an embodiment of the present invention. This figure compares an image when the cell 1102 is in focus (Figure 2A) with an image when the cell 1102 is out of focus (Figure 2B: when the focus is on the glass surface of the imaging unit, as with a normal scanner). 1A and 1B are diagrams showing an example of a pattern configuration of a pattern sheet 1011 used in the scanner device 100 for capturing cell observation images according to this embodiment. FIG. 13 is a diagram for explaining the principle by which cells can be photographed by adding a pattern sheet. 5A is a diagram showing an image observed by microscope tiling photography, FIG. 5B is an image observed by conventional scanner photography, and FIG. 5C is an image observed by scanner photography of the present method. 1A and 1B are diagrams showing examples of an image acquired by the cell observation image capturing scanner device 100 and an image paired with the image acquired by the cell observation image capturing scanner device 100. FIG. 1 is a diagram illustrating the concept of machine learning using paired images. FIG. 13 is a diagram showing the concept of inputting an actual observation image with a pattern sheet background and outputting an observation image without the background. 13A to 13C are diagrams for explaining the operation of the scanner device 200 for capturing cell observation images according to the second embodiment. 13A to 13C are diagrams for explaining the operation of the scanner device 300 for capturing cell observation images according to the third embodiment. 13A to 13C are diagrams for explaining the operation of the scanner device 400 for capturing cell observation images according to the fourth embodiment. 13A and 13B are diagrams illustrating an example of the configuration of a pattern sheet according to a fifth embodiment.

Hereinafter, an embodiment of the present disclosure will be described with reference to the attached drawings. In the attached drawings, functionally identical elements may be indicated by the same numbers. Note that the attached drawings show specific embodiments and implementation examples in accordance with the principles of the present disclosure, but these are intended to aid in understanding the present disclosure and are in no way intended to limit the interpretation of the present disclosure.

In this embodiment, the disclosure is described in sufficient detail for a person skilled in the art to implement the disclosure, but it should be understood that other implementations and forms are possible, and that changes to the configuration and structure and substitutions of various elements are possible without departing from the scope and spirit of the technical ideas of the disclosure. Therefore, the following description should not be interpreted as being limited to this.

This embodiment proposes a scanner suitable for acquiring observation images of cells in order to ensure the continuity of the observation images and shorten the time required for capturing images.

Since scanner photography has fundamentally big problems, there have been no attempts to use a scanner to photograph culture vessels and obtain observation images. This is because a normal scanner cannot photograph highly transparent objects such as cells. For example, if a culture vessel (dish) during cell culture is photographed with a scanner in the conventional way, the entire image will be almost completely white, and the cells will hardly be visible. This is the same for both reflective and transmissive scanners. In addition, in the case of microscopes, there are some innovations (phase contrast microscopes) that use the "wave properties of light" rather than "color (absorption at each wavelength)" for the purpose of observing cells, but scanners have not been introduced with such innovations because their original purpose is different from that of microscopes. In addition, in principle, the photographing surface of a microscope is a two-dimensional optical structure, while the photographing surface of a scanner is a one-dimensional optical structure. Furthermore, there is currently no consideration as to whether the same principle as a phase contrast microscope can be applied to a scanner.

Therefore, this embodiment proposes a method for acquiring cell observation images that replaces conventional tiling photography by enabling the previously impossible "cell photography with a scanner," ensuring the continuity of the observation images and shortening the time required for photography.

(1) First embodiment <Configuration example of a scanner for capturing cell observation images>
FIG. 1 is a diagram showing an example of the configuration of a scanner device 100 for capturing cell observation images according to this embodiment.

The scanner device 100 for photographing cell observation images is, for example, a transmission type scanner device, and is equipped with an illumination unit 101 and a photographing unit 102 that are equipped in a normal scanner, a computer 103 that controls the illumination unit 101 and the photographing unit (including the scanner head) 102, an input device (mouse, keyboard, etc.) 104 for, for example, a user to input commands and parameters, a storage device 105 for, for example, storing the photographed observation image, an output device (display device or printer) 106 for, for example, outputting (displaying, printing) the photographed observation image, and a drive unit (not shown) that is controlled by the computer 103 and moves the illumination unit 101 and the photographing unit 102 in parallel in a synchronized and linked manner.

The lighting unit 101 is attached to, for example, the cover of the scanner, in the same manner as in a normal scanner. In addition, a pattern sheet 1011 is attached to the surface (contact glass surface) of the lighting unit 101 facing the cell culture vessel 110 (alternatively, a pattern pattern may be printed (glass printing) on the glass surface, or the glass surface may be processed to have irregularities or the like to form a light and dark pattern). Alternatively, when taking a cell observation image, a pattern sheet 1011 independent of the scanner (separate: not attached to the glass surface of the lighting unit 101) may be placed on the top cover of the cell culture vessel 110. Alternatively, the pattern sheet 1011 may be fixed in some manner between the top cover of the cell culture vessel 110 and the lighting unit 101. In this way, a distance of a predetermined value or more can be provided between the cells to be photographed and the pattern sheet 1011, and the cells will not be blocked by the pattern. The pattern of the pattern sheet 1011 will be described later.

Unlike a normal scanner (where the scanner head is focused on the glass surface of the imaging unit on which the object to be imaged is placed), the imaging unit 102 focuses the scanner head at a predetermined distance (for example, at least the thickness of the bottom glass of the cell culture vessel 110) upward. In other words, the imaging unit such as the scanner head is configured to focus on the refractive surface of the cell to be imaged (cell surface: meaning including the case where the cell is attached to the bottom surface of the culture vessel 110 or slightly raised from the bottom surface). The position of the cell 1102 to be observed is the bottom surface of the cell culture vessel 110, but this position is about 1 mm above the glass surface of the imaging unit 102. FIG. 2 is a diagram for comparing an image when the cell 1102 is in focus (FIG. 2A) with an image when the cell 1102 is not in focus (FIG. 2B, where the glass surface of the imaging unit is in focus like a normal scanner). 2, when the cell 1102 is photographed by the scanner device 100 for cell observation and image capture according to this embodiment in which the cell 1102 is in focus, an observation image can be obtained in which each cell 1102 is very clearly shown. Note that the focus of the scanner head may be fixed, but the photographing unit 102 may be provided with a focus adjustment mechanism (function) so that the photographing unit 102 can be configured to adjust the focus, for example, in response to a command from the input device 104 or automatically.

In this embodiment, a scanner device is used as the photographing device, but an industrial camera (two-dimensional photographing device) and a micro lens may be used instead of the scanner device. In this case, since the camera has a wide field of view for each photographing operation, the driving unit does not translate the camera continuously, but moves the camera and the lighting unit so that the following operations are repeated: photographing → translate the camera and the lighting unit (for example, a few centimeters) → stop → photographing → translate → stop → ...

<Example of pattern configuration for pattern sheet>
3 is a diagram showing an example of the pattern configuration of the pattern sheet 1011 used in the cell observation image capturing scanner device 100 according to this embodiment. The patterns shown here are merely examples, and other patterns are also applicable.

The inventors thoroughly investigated a suitable pattern that can be applied to a scanner to obtain clear cell images, and found that if the boundaries between light and dark are too far apart in the pattern, the cells 1102 will not be captured in the middle, while if the boundaries between light and dark are too close, the clarity of the cells 1102 will decrease. As a result of further investigation based on this discovery, it was found that the pattern shown in Figure 3 (pattern: pattern formed by differences in light transmittance) was applicable.

That is, for example, as shown in FIG. 3, there are a block pattern (FIG. 3A), a checkered pattern (FIG. 3B), a dot pattern (FIG. 3C), a mesh pattern (FIG. 3D), a random pattern (FIG. 3E), a ripple pattern (FIG. 3F), a stripe pattern (FIG. 3G), a concentric circle pattern (FIG. 3H), a wave pattern (FIG. 3I), a hexagonal pattern with a middle fill (FIG. 3J), a dot pattern arranged in equilateral triangles (FIG. 3K), a dot pattern arranged in equilateral triangles with a middle fill (FIG. 3L), an equilateral triangle pattern (FIG. 3M), a hollow equilateral triangle pattern (FIG. 3N), and a hollow equilateral triangle pattern with a middle fill (FIG. 3O). Note that the pattern does not have to be a periodic pattern. For example, it may be a pattern such as the so-called Penrose style, or it may be a random pattern with no regularity.

For example, in the case of a block pattern (Figure 3A), the block appearance period can be 1780.9 μm (= 84 px x 25400 μm/1200 dpi), and one side of one block can be 890 μm (= 42 px x 25400 μm/1200 dpi). In addition, in the case of a stripe pattern (Figure G), the stripe appearance period can be 508.8 μm (= 24 px x 25400 μm/1200 dpi), and one stripe width can be 254.4 μm (= 12 px x 25400 μm/1200 dpi). Furthermore, in the case of a ripple pattern (Figure F), the ripple appearance period can be set to 508.8 μm (= 24 px x 25400 μm/1200 dpi), and the width of one ripple can be set to 254.4 μm (= 12 px x 25400 μm/1200 dpi). In addition, in the case of a wave pattern (Figure I), the wave period can be set to 636 μm (= 30 px x 25400 μm/1200 dpi), the amplitude to 424 μm (= 10 px x 2 x 25400 μm/1200 dpi), and the thickness to 318 μm (= 15 px x 25400 μm/1200 dpi).

For example, a mixed pattern can be a combination of mesh and blocks (a pattern in which a pattern like the kanji character "don" is repeated) or a combination of checkers, black blocks, and white blocks (checkers ± blocks) (a pattern in which a pattern like the kanji character "kai" is repeated).

By providing a pattern sheet 1011 having the above-mentioned pattern on top of the cell culture vessel 110, it becomes possible to obtain an observation image by a scanner that clearly identifies the presence and state of each individual cell 1102. In other words, if the transparency of the bottom surface of the cell culture vessel 110 decreases, it is clear that the culture is progressing, but even if this can be recognized to a certain extent, it is difficult to fully utilize this for regenerative medicine. According to this embodiment, it is possible to grasp the state of each individual cell, and this technology will contribute to the further development of regenerative medicine.

<Considerations on the principle of how adding a pattern sheet enables cells to be photographed>
FIG. 4 is a diagram for explaining the principle by which cells can be photographed by adding a pattern sheet.

Light does not pass through the black parts of the pattern sheet 1011. Since light emitted from each point of the lighting unit 101 located above the pattern sheet 1011 is emitted in all directions, the light passing through the transparent parts of the pattern sheet 1011 includes not only light 401 perpendicular to the pattern sheet 1011 but also light 402 obliquely.

As a result of vertical light 401 and diagonal light (diffused light) 402 being added throughout the lighting unit 101, the pattern of the pattern sheet 1011 appears blurred in the image captured by the photographing unit 102. In this case, even if the vertical light 401 enters the cell 1102, it passes straight down and the cell 1102 is not visible. On the other hand, when the diagonal light 402 enters the cell 1102, the light is refracted due to a subtle difference in refractive index. The diagonal light 402 reaches a different photographing element 404 due to refraction, instead of the photographing element 403 where it would normally reach if the cell 1102 was not present. As a result, the amount of light received by the photographing element 403 decreases slightly, and instead the amount of light received by the photographing element 404 increases slightly.

This change in the amount of light is not so large, but when the influence of the oblique light 402 is more dominant than the perpendicular light 401 at the point of the image capturing element 403 or 404, the cell 1102 will be imaged at a level where it can be seen. This property is satisfied in the vicinity directly below the portion of the pattern sheet 1011 where light and dark change, that is, the portion where the light and dark of the pattern are blurred to form a gradation on the captured image.

In addition, since the difference in refractive index between the cell 1102 and the culture solution is generally small, the refraction that occurs when the oblique light 402 enters the cell 1102 is small. If the distance from when the light enters the cell 1102 to when it reaches the image capture element 404 is short, the image capture element 403 and the image capture element 404 that should reach when the cell 1102 is not present (when there is no refraction) will be the same, and the presence of the cell cannot be confirmed on the image. That is, in order to effectively implement the present application, a sufficient distance is required between the surface on which the cell exists and the image capture element surface. Therefore, a sufficient effect cannot be obtained with a CIS (Contact Image Sensor) type scanner or a microscope, in which this distance is very short. On the other hand, it has been found that the effect is easily obtained with a configuration such as a CCD (Charge Coupled Device) type scanner, in which this distance is sufficiently large, or an industrial camera connected to a macro lens, etc.

<Comparison of observed images (experimental results)>
FIG. 5 shows an image observed by microscope tiling photography (FIG. 5A), an image observed by conventional scanner photography (FIG. 5B), and an image observed by scanner photography of the present method (FIG. 5C).

The cells used in this experiment were cartilage cells, with long sides (lengthwise direction) measuring 50 to 80 μm and short sides (widthwise direction) measuring 30 to 50 μm. The size of the pattern on the pattern sheet 1011 used to photograph Figure 5C was approximately 3 mm or less in terms of the size of the pattern in the direction in which the pattern is repeated and the size of one period of the pattern.

Comparing Figures 5A to 5C, in the observation image obtained using microscope tiling photography (Figure 5A), it was possible to capture cells, but as described above, the photography efficiency was poor and the image was not smooth because multiple microscope images were stitched together. In addition, in the case of conventional scanner photography (Figure 5B), the photography efficiency was good and the image obtained was smooth, but it was found that cells could not be captured unless they were stained. Since staining cells causes damage to the cells, conventional scanner photography is also not preferable. In contrast, in the case of photography according to this embodiment (Figure 5C), the photography efficiency is good and the image obtained is smooth.

The size of the pattern pattern may be related to the size of the cells and the height at which the pattern is placed (height of the container). When a tall (deep) cell culture container (dish) 110 is used, the size of the pattern pattern that was good for a shallow culture container (dish) 110 cannot capture the individual cells 1102 well. Specifically, when the boundary part of the pattern pattern on the photographing surface produces a simple gradation, the image can be captured well, but when the gradations originating from adjacent boundary parts intersect with each other and produce complex shadows, the visibility of the cells becomes poor. In a state where the visibility is poor, the distance between the pattern pattern and the photographing surface becomes far, and the main components of the oblique light 402 emitted from adjacent transparent regions in the pattern pattern intersect with each other. In this case, even if the effect of refraction by the cells occurs in one of the intersecting light rays 402, the effect of the cells does not occur in the other light ray 402'. As a result, the effect of highlighting the light and dark by refraction is masked, and the visibility of the cells decreases. Therefore, when the cell culture vessel 110 is deep, it is necessary to increase the pattern size (the distance between the transparent regions) to avoid intersection of the main components of the diagonal light rays. The main components are those components of the light rays 402 in each direction that have a non-negligible effect on the captured image.

<About condensation>
When condensation (temperature difference with the outside) occurs inside the cell culture vessel 110, the visibility is weakened and a good observation image cannot be obtained. In this regard, applying an anti-condensation agent to the inside of the top cover of the cell culture vessel 110 will have an adverse effect on the cells 1102.

For this reason, for example, the top lid (made of glass) of the cell culture vessel 110 may be heated with a heater. The best way to prevent condensation is to place the entire scanner device in a culture device (environment with a temperature of 37°C and humidity of 99%).

<Image output processing using machine learning>
When an observation image is acquired using the pattern sheet 1011 as in this embodiment, a part of the pattern (blurred state) is reflected in the background of the observation image, resulting in the appearance of light and dark undulations (see FIG. 5C). Therefore, the light and dark undulations due to the pattern can be removed by AI processing such as deep learning. For example, it is possible to convert the image into an image (without background) equivalent to an observation image obtained when the microscope tiling process is very successful. Here, an example of acquiring an observation image by such AI processing will be described. Note that various types of images can be used as pair images of the captured observation image, such as an input image "observation image with a pattern sheet background" and an output image corresponding to this "image as acquired by a phase contrast microscope,""an image like cytoplasmic staining (for example, a cell region is green and the background is black)," or "an image like nuclear staining (only the center of the cell is blue and the rest is black)."

(i) Step 1
A culture medium 1101 and cells 1102 are placed in a cell culture vessel 110 to prepare a target.

(ii) Step 2
The target prepared in step 1 is placed in the placement position (predetermined position of the photographing unit 102) of the scanner device 100 for photographing cell observation images, and then covered with a scanner cover (illumination unit 101) with the pattern sheet 1011 attached (or printed) (or the pattern sheet 1011 alone is placed on the top lid of the cell culture vessel 110), and a cell observation image is obtained. As described above, this cell observation image has the blurred pattern of the pattern sheet 1011 reflected in the background. It goes without saying that different images can be obtained depending on the type of pattern sheet (see FIG. 3).

(iii) Step 3
An image that pairs with a part or all of the cell observation image obtained in step 2 is prepared (obtained or created).

When acquiring paired images, the target from step 1 is photographed in a way that has the desired properties. For example, paired images can be acquired by photographing with a phase-contrast microscope and performing a tiling process, or nuclear staining can be performed, photographing with a fluorescence microscope, and performing a tiling process to acquire paired images. In addition, when creating paired images, images with the desired properties can be created by converting the acquired images manually or by performing separate image processing. Note that a sufficient number of paired images (such as multiple pairs or pairs with a sufficiently large image size) must be prepared.

Figure 6 shows an example of an image acquired by the scanner device 100 for capturing cell observation images and an example of an image paired with the image. The paired image in Figure 6 corresponds to "the case of creating a paired image," and is an example created manually by specifying the "desired property" as "an extracted cell region."

(iv) Step 4
Machine learning is performed to construct a machine learning model using the paired images prepared in step 3. Fig. 7 is a diagram illustrating the concept of machine learning using paired images.

For example, a "U-Net" (suitable for region extraction) is adopted as the network shape, and learning is performed with the input being "a partial (or full) image" of the cell observation image acquired by the cell observation image capture scanner device 100, and the output being "an image paired with the input" (Deep Learning).

As another method, for example, a technique called support vector regression (SVR) can be used to perform machine learning using the input as "vectorized versions of very small subregions in the paired input images" and the output as "values in the paired images that correspond to the center of the input subregion" (traditional machine learning compared to Deep Learning).

(v) Step 5
The steps up to step 4 are the preparatory steps. Step 5 corresponds to the process during actual operation. Fig. 8 is a diagram showing the concept of inputting an actual observation image with a pattern sheet background and outputting an observation image without the background.

Using the same pattern sheet 1011 as in step 1 (pattern sheet 1011 used for machine learning), a target other than the target used for learning (a target during actual operation) is captured by the scanner device 100 for capturing cell observation images, and the captured cell observation image X becomes the input for the machine learning model constructed in step 4. When such a cell observation image X is applied to the machine learning model, an image exhibiting the "desired properties" of the cell observation image X is obtained as the output image.

When the cell observation image X is different from the size Y assumed by the machine learning model, generally, regions of size Y are cut out one after another from within the cell observation image X, processed sequentially, and finally integrated. It is also possible to allow overlaps during cutting, and to perform processing such as averaging or maximum value selection during integration.

<Image analysis processing>
Various image analysis processes (e.g., edge extraction process, pixel counting process to calculate area, etc.) can be performed on the cell observation image obtained by photographing or the output image obtained by machine learning to obtain various analysis results. Such analysis processes can be performed, for example, by using the computer 103 in the cell observation image photographing scanner device 100. The image analysis results can then be used as specific management indexes.

Items to be analyzed include, for example, cell position, the area of each cell or the total area of all cells, cell morphology, cell differentiation state, cell movement, proximity, contact, division, and death, parent-child cell lineages and cell populations of the same lineage, and the proximity or contact between cell populations.

In addition, examples of management indices calculated based on the image analysis results include cell count (current value, progress, and future prediction), proliferation rate (current value, progress, and future prediction), and differentiation rate (current value, progress, and future prediction).

For example, by obtaining an output in which the cell area is white and the background is black, and then counting the number of white areas, the number of cells can be counted. In addition, by calculating the center of gravity of each white area, the position of each cell can be determined. Furthermore, by measuring the number of pixels in the white area, the area of each cell can be determined. Depending on the characteristics of the captured cells, it may be possible to distinguish the cell type and the differentiation state using the shape of the area. In particular, when taking time-series images by taking advantage of the characteristics of this technology that allows high frequency imaging, it is possible to obtain information such as cell movement, proximity, contact, division, and death by applying existing cell tracking technology. It is easy to calculate the proliferation rate from the change in cell number, and by organizing the information on division and death in a chronological order, it is possible to understand the parent-child lineage of cells. In addition, by organizing the information on proximity and parent-child lineage, it is possible to understand information such as proximity and contact about the cell population. The information obtained can be used, for example, to make a comprehensive judgment on the quality of the cell culture state, and can also be used to predict future cell numbers and cell culture states based on the changes in each index over time. This understanding of the situation and future predictions can be used to find a wide range of applications in this field, such as determining when to change the culture medium, scheduling transplant operations based on a prediction of when the required number of cells will be obtained, and reducing costs by detecting poor culture conditions early and ceasing cultivation.

(2) Second Embodiment FIG. 9 is a diagram for explaining the operation of a scanner device 200 for capturing cell observation images according to the second embodiment.

The scanner device 200 for cell observation image capture according to the second embodiment includes an illumination unit, a photographing unit, a computer, an input device, a display device, and a storage device, similar to the scanner device 100 for cell observation image capture according to the first embodiment. However, in the scanner device 200 for cell observation image capture according to the second embodiment, the illumination unit is not directly opposed to the photographing unit, but is disposed at an oblique position, and the illumination unit and the photographing unit are configured to move synchronously. In addition, the illumination unit is disposed so that the emitted parallel light traveling straight ahead is not exposed to (or is blocked from) the photographing element. For this reason, the illumination unit irradiates the cell 1102 with parallel light (collimated light) from above at an angle, and only the light refracted by the cell is incident on the photographing unit.

By adopting such a configuration, the scanner device 200 for capturing cell observation images according to the second embodiment does not need to place the pattern sheets shown in FIG. 3, and an image in which the background is black and the cell parts are gray to white is automatically obtained.

(3) Third Embodiment FIG. 10 is a diagram for explaining the operation of a scanner device 300 for capturing cell observation images according to the third embodiment.

The scanner device 300 for cell observation image capture according to the third embodiment has a configuration similar to that of the scanner device 200 for cell observation image capture according to the second embodiment, and is configured such that the illumination unit is not directly facing the capture unit but is disposed at an oblique position, and the illumination unit and the capture unit are moved synchronously. The illumination unit is disposed so that the emitted parallel light traveling straight is removed from (or blocked by) the capture element. The illumination unit irradiates the cell 1102 with parallel light (collimated light) from an obliquely upward direction, and only the light refracted by the cell is incident on the capture unit. However, the third embodiment differs from the second embodiment in that a virtual color is assigned to the capture unit (a capture element that outputs pixel data of any color is used). For this reason, as shown in FIG. 10, when the parallel light refracted by the cell 1102 spans multiple capture elements with different assigned colors, the light incident on each element is integrated, and an image of a color according to the amount of light incident on each element is obtained. For example, when light refracted by the cell 1102 is incident on a blue image sensor and a red image sensor, the colors are integrated to obtain a purple image (the image of the cell is colorized). In addition, when observing a cell 1102 that refracts more strongly, if light is incident across the red and green image sensors, a yellow image is obtained.

By adopting such a configuration, the scanner device 300 for capturing cell observation images according to the third embodiment does not need to place the pattern sheets shown in FIG. 3, as in the second embodiment, and automatically obtains an image in which the background is black and the cell portion is virtually colored (the color of the image varies depending on the refractive index of the cell).

(4) Fourth Embodiment FIG. 11 is a diagram for explaining the operation of a scanner device 400 for capturing cell observation images according to the fourth embodiment.

The scanner device 400 for cell observation image capture according to the fourth embodiment includes an illumination unit, an image capture unit, a computer, an input device, a display device, and a storage device, similar to the scanner device 100 for cell observation image capture according to the first embodiment. The illumination unit emits diffused light (non-collimated light) similar to the first embodiment. However, the fourth embodiment differs from the first embodiment in that multiple pattern sheets (see FIG. 11B) are arranged below the illumination unit, shifted vertically (at intervals vertically).

According to the above-mentioned photographing principle, it is known that the photographing effect of cells is high in areas where the light is mainly oblique. As shown in FIG. 11A, the pattern sheet 1 can remove most of the vertical light from the light source (lighting unit). Then, in order to remove the vertical and near-vertical light from the light that passes through the transparent part of the pattern sheet 1, the pattern sheet 2 is placed at a certain distance from the pattern sheet 1. The pattern sheet 2 is basically a pattern in which the black and white of the pattern sheet 1 is inverted, but the configuration of each sheet can be adjusted depending on how close to vertical light is to be removed. For example, if you want to leave only the oblique component with a stronger angle, you can increase the shielding part of the pattern sheet 2 or reduce the distance between the pattern sheet 1 and the pattern sheet 2. Alternatively, you can combine increasing the shielding part and shortening the distance between the pattern sheets 1 and 2. It is necessary to devise the arrangement and pattern of the pattern sheets 1 and 2 so that a sufficient amount of oblique light covers the entire cell culture surface as shown in FIG. 11A. It is also necessary to devise the arrangement and pattern of the pattern sheets 1 and 2 so that a sufficient amount of oblique light covers the entire cell culture surface. It is also possible to use a set of three or more sheets, although two sheet pairs are shown in FIG. 11B. In this case, three or more pattern sheets are arranged vertically at a predetermined distance apart, and the light transmitting portion of pattern sheet 1 is covered by the light blocking portions of a plurality of pattern sheets 2 to N.

(5) Fifth embodiment Fig. 12 is a diagram showing an example of the configuration of a pattern sheet according to the fifth embodiment. The pattern sheets shown in Figs. 12A and 12B can be used in the scanner device 100 for cell observation image capture. The pattern sheet can be configured, for example, to have a continuous change in transparency (Fig. 12A) or a stepwise change in transparency (Fig. 12B). By using these, it is possible to reduce the light irradiated to the culture vessel (cells) from the vertical direction and relatively increase the light from the oblique direction.

According to the above-mentioned photographing principle, it is known that the photographing effect of cells is high in areas where light mainly comes from an oblique direction. In the above embodiments, the pattern sheet is composed of black (completely opaque) parts and transparent parts, but as in this embodiment, it is also possible to form a pattern by distributing the transparency. For example, if it is desirable for the pattern not to be too strong in the background due to the convenience of subsequent analysis processing, the light and dark of the background can be suppressed and made closer to uniform by using a pattern sheet with a continuous or stepwise change in transparency.

<Summary>
(i) In the cell observation image capture device (scanner and camera) according to the above-mentioned embodiment, a pattern sheet is disposed between the illumination unit and the top lid of the culture vessel. The pattern sheet can be disposed by attaching it to the glass surface of the illumination unit, or it can be printed on the glass surface of the illumination unit, or it can be formed by processing the glass surface of the illumination unit into a concave-convex shape. Furthermore, the pattern sheet can be configured as a sheet member independent of the cell observation image capture device, and the sheet member can be placed on the top lid of the culture vessel during capture. By configuring the capture device in this way, a cell observation image in which each individual cell can be clearly recognized can be obtained. Examples of patterns that can produce this effect include block patterns, checker patterns, dot patterns, mesh patterns, random patterns, ripple patterns, stripe patterns, wave patterns, mixed patterns which are combinations of multiple patterns, and patterns consisting of any of a number of patterns including a concentric circle pattern, a hexagonal fill pattern, an equilateral triangular dot pattern, an equilateral triangular fill dot pattern, an equilateral triangle pattern, an equilateral triangle hollow pattern, an equilateral triangle hollow fill pattern, a non-periodic pattern, a pattern with continuous changes in transparency, and a pattern with gradual changes in transparency.
In addition, even if a two-dimensional imaging device (such as a camera) is used as the imaging device instead of a scanner device, the pattern is reflected as a background, so it is possible to align the position more accurately than before, and therefore the continuity of the observation image can be ensured. In addition, if a camera is used, a much wider field of view can be captured with one shot than with a microscope, so the number of shots can be reduced, and as a result, the required time can be shortened.

(ii) In the cell observation image capturing device according to the embodiment, unlike a normal capturing device (scanner device), the focus of the capturing unit is not set to the glass surface ( capture surface: surface on which the target is placed) of the capturing unit, but is set to a position shifted a predetermined distance toward the illumination unit from the surface on which the culture vessel is placed of the capturing unit (for example, above the thickness of the bottom surface of the cell culture vessel: in other words, the "predetermined distance" includes at least the distance of the thickness of the culture vessel). In other words, the focus of the capturing unit is configured to match the refractive surface of the object to be captured (cells). In this way, it is possible to provide a capturing device suitable for capturing cell observation images.

(iii) The storage device of the cell observation image capture device stores a machine learning model for outputting a transformed image exhibiting desired properties from the cell observation image. At this time, a cell observation image obtained by actually photographing a culture vessel may be applied as an input to the machine learning model to obtain a transformed image, which may then be displayed on the screen of an output device (display device). In this way, it becomes possible to remove light and dark undulations caused by light and dark patterns reflected in the cell observation image.

In addition, a specified image analysis process may be performed on the cell observation image or the converted image, and a specified management index may be calculated and displayed on the screen of the output device (display device).

The specified image analysis process is, for example, a process for analyzing (iii-1) cell position, (iii-2) the area of each cell or the total area of all cells, (iii-3) cell morphology, (iii-4) cell differentiation state, (iii-5) cell movement, proximity, contact, division, and death, (iii-6) parent-child lineage of cells and cell populations of the same lineage, or (iii-7) proximity or contact between cell populations. In addition, the number of cells, proliferation rate, or differentiation rate are calculated as the specified management index.

(iv) In the cell observation image capture device (scanner device) according to the second embodiment, the illumination unit is disposed so as to irradiate the capture surface of the capture unit with parallel light at an angle, and the parallel light traveling in a straight line does not enter the capture surface of the capture unit, and only the parallel light refracted by the cells enters the capture surface of the capture unit. By doing so, it is possible to obtain clear cell observation images without arranging a predetermined pattern between the illumination unit and the top cover of the culture vessel.

In the third embodiment, in addition to the configuration according to the second embodiment, the photographing unit is configured to have a plurality of photographing elements each assigned with a predetermined color. At this time, when parallel light refracted by a cell is incident on photographing elements assigned with different colors, the colors of the photographing elements on which the parallel light is incident are mixed and an image is output.

(v) In the fourth embodiment, a predetermined pattern having different light transmittances is also arranged between the lighting unit and the top lid of the culture vessel, and the predetermined pattern is composed of a set of multiple pattern sheets arranged at a predetermined distance apart. The light-transmitting portion of the pattern sheet arranged closest to the lighting unit is covered by the light-shielding portions of the other pattern sheets. At this time, the light transmitted through the multiple pattern sheets illuminates the entire culture surface of the culture vessel. In this way, by combining multiple pattern sheets, it is possible to reduce the light from the vertical direction and relatively increase the light from an oblique direction, making it possible to obtain clear cell observation images.

(vi) Some of the functions of this embodiment can also be realized by software program code. In this case, a storage medium on which the program code is recorded is provided to a system or device, and the computer (or CPU or MPU) of that system or device reads the program code stored in the storage medium. In this case, the program code itself read from the storage medium realizes the functions of the above-mentioned embodiment, and the program code itself and the storage medium on which it is stored constitute this disclosure. Examples of storage media for supplying such program code include flexible disks, CD-ROMs, DVD-ROMs, hard disks, optical disks, magneto-optical disks, CD-Rs, magnetic tapes, non-volatile memory cards, and ROMs.

Also, based on the instructions of the program code, an OS (operating system) running on a computer may perform some or all of the actual processing, and the functions of the above-mentioned embodiments may be realized by this processing. Furthermore, after the program code is read from a storage medium and written to memory on a computer, a CPU of the computer may perform some or all of the actual processing based on the instructions of the program code, and the functions of the above-mentioned embodiments may be realized by this processing.

Furthermore, the program code of the software that realizes the functions of the embodiments may be distributed over a network and stored in a storage means such as a hard disk or memory of a system or device, or in a storage medium such as a CD-RW or CD-R, so that when used, the computer (or CPU or MPU) of the system or device reads and executes the program code stored in the storage means or storage medium.

It should be understood that the processes and techniques described herein are not inherently related to any particular apparatus, but may be implemented by any suitable combination of components. Moreover, various types of general-purpose devices may be used in accordance with the disclosure described herein. It may be advantageous to construct a dedicated apparatus to perform the steps of the methods described herein, and various inventions may be formed by suitable combinations of multiple components disclosed in the embodiments. For example, some components may be omitted from all components shown in the embodiments, and different components may be combined as appropriate. Although the present disclosure has been described in relation to specific examples, these are provided for technical understanding rather than for the purpose of limiting the scope of the rights in all respects. It is also believed that those skilled in the art will readily recognize that there are numerous combinations of hardware, software, and firmware suitable for implementing the present disclosure.

Furthermore, in the above-mentioned embodiment, the control lines and information lines are those considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. All components may be interconnected.

100 Scanner device for capturing cell observation images 101 Lighting unit 102 Capture unit 103 Computer 104 Input device 105 Storage device 106 Output device 110 Cell culture vessel 1011 Pattern sheet 1101 Culture fluid 1102 Cells

Claims (24)

A cell observation image capturing device for capturing cell observation images by placing a culture vessel carrying cells to be cultured therein,
an illumination unit located above the culture vessel and configured to irradiate the culture vessel with illumination light;
A photographing unit is provided, the photographing unit being located under the culture vessel and photographing the culture vessel;
A predetermined pattern having a difference in light transmittance is arranged between the lighting unit and the upper cover of the culture vessel,
A cell observation image capturing device in which the focus of the photographing unit is adjusted to a position shifted a predetermined distance toward the illumination unit from the culture vessel mounting surface of the photographing unit, thereby aligning the focus with the cell surface, and no tiling photography is performed. In claim 1,
A cell observation and image capturing device, wherein the predetermined pattern is arranged by attaching a sheet having the predetermined pattern to the light transmitting surface of the lighting unit. In claim 1,
A cell observation and image capture device, wherein the predetermined pattern is printed on the light transmitting surface of the illumination unit, or formed by roughening the light transmitting surface of the illumination unit. In claim 1,
The predetermined pattern is formed of a sheet member independent of the cell observation image capturing device, and the sheet member is placed on the top lid of the culture vessel when capturing images of the cell observation image capturing device. In any one of claims 1 to 4,
A cell observation image capturing device, wherein the specified pattern is any one of a plurality of patterns including a block pattern, a checkered pattern, a dot pattern, a mesh pattern, a random pattern, a ripple pattern, a stripe pattern, a wave pattern, a concentric circle pattern, a filled hexagonal pattern, a dot pattern arranged in equilateral triangles, a filled dot pattern arranged in equilateral triangles, a regular triangle pattern, a hollowed out regular triangle pattern, a hollowed out regular triangle pattern, a pattern without periodicity, a pattern having a continuous change in transparency, and a pattern having a gradual change in transparency. In any one of claims 1 to 5,
The present invention further includes a computer that executes various calculations, a storage unit that stores the results of the calculations performed by the computer, and an output unit that outputs the results of the calculations,
The storage unit stores a machine learning model for outputting a converted image exhibiting desired properties from the cell observation image,
The computer applies the cell observation image obtained by actually photographing the culture vessel as an input to the machine learning model to obtain the transformed image, and outputs the transformed image to the output section, in a cell observation image capturing device. In claim 6,
The computer executes a predetermined image analysis process on the cell observation image, calculates a predetermined control index, and outputs it to the output unit. In claim 6,
The computer executes a predetermined image analysis process on the converted image, calculates a predetermined control index, and outputs it to the output unit. In claim 7 or 8,
The computer uses the specified image analysis processing to analyze (i) cell position, (ii) area of each cell or the total area of all cells, (iii) cell morphology, (iv) differentiation state of cells, (v) cell movement, proximity, contact, division, and death, (vi) parent-child cell lineage and cell populations of the same lineage, or (vii) proximity or contact between cell populations. In any one of claims 7 to 9,
The computer calculates the number of cells, the degree of proliferation, or the differentiation rate as the predetermined control index. A cell observation image capturing device for capturing cell observation images by placing a culture vessel carrying cells to be cultured therein,
an illumination unit located above the culture vessel and configured to irradiate the culture vessel with illumination light;
A photographing unit is provided, the photographing unit being located under the culture vessel and photographing the culture vessel;
A predetermined pattern having a difference in light transmittance is arranged between the lighting unit and the upper cover of the culture vessel,
The predetermined pattern is composed of a plurality of pattern sheet sets arranged at a predetermined distance apart, and a light-transmitting portion of the pattern sheet arranged closest to the lighting unit is covered by a light-shielding portion of another pattern sheet,
A cell observation image capturing device in which the focus of the photographing unit is adjusted to a position shifted a predetermined distance toward the illumination unit from the culture vessel mounting surface of the photographing unit, thereby aligning the focus with the cell surface, and no tiling photography is performed. In claim 11,
The set of multiple pattern sheets is arranged so that light transmitted through the multiple pattern sheets illuminates the entire culture surface of the culture vessel, in this cell observation screen imaging device. A cell observation image acquisition method for acquiring an observation image of a cell supported in a culture vessel using a cell observation image acquisition device, comprising:
Illuminating the culture vessel with illumination light by an illumination unit located above the culture vessel in the cell observation image capture device through a predetermined pattern disposed between the illumination unit and an upper cover of the culture vessel, the pattern having a difference in light transmittance;
and capturing an image of the culture vessel by an imaging unit located below the culture vessel in the cell observation image capturing device,
A cell observation image acquisition method in which the focus of the photographing unit is adjusted to a position shifted a predetermined distance toward the illumination unit from the culture vessel mounting surface of the photographing unit, thereby aligning the focus with the cell surface, and no tiling photography is performed. In claim 13,
A cell observation image acquisition method, wherein the predetermined pattern is arranged by attaching a sheet having the predetermined pattern to the light-transmitting surface of the lighting unit. In claim 13,
A cell observation image acquiring method, wherein the predetermined pattern is printed on the light transmitting surface of the lighting unit, or formed by roughening the light transmitting surface of the lighting unit. In claim 13,
A cell observation image acquiring method, in which the predetermined pattern is composed of a sheet member independent of the cell observation image capturing device, and the sheet member is placed on the top lid of the culture vessel during capturing. In any one of claims 13 to 16,
A cell observation image acquisition method, wherein the specified pattern is any one of a plurality of patterns including a block pattern, a checkered pattern, a dot pattern, a mesh pattern, a random pattern, a ripple pattern, a stripe pattern, a wave pattern, a concentric circle pattern, a filled hexagonal pattern, a dot pattern arranged in equilateral triangles, a filled dot pattern arranged in equilateral triangles, a regular triangle pattern, a hollowed out regular triangle pattern, a hollowed out regular triangle pattern, a pattern without periodicity, a pattern having a continuous change in transparency, and a pattern having a gradual change in transparency. In any one of claims 13 to 17, further comprising:
A computer reads in a machine learning model stored in a storage unit, the machine learning model being for outputting a converted image exhibiting a desired property from an observation image of the cell; the computer applies an observation image of the cell obtained by actually photographing the culture vessel as an input to the machine learning model to obtain the converted image, and outputs the converted image to an output unit;
A method for acquiring a cell observation image, comprising: In claim 18, further comprising:
The cell observation image acquisition method includes the computer executing a predetermined image analysis process on the cell observation image, calculating a predetermined management index, and outputting the calculated management index to the output unit. In claim 18, further comprising:
The cell observation image acquisition method includes the computer executing a predetermined image analysis process on the converted image, calculating a predetermined control index, and outputting the calculated control index to the output unit. In claim 19 or 20,
The computer uses the specified image analysis process to analyze (i) cell position, (ii) the area of each cell or the total area of all cells, (iii) cell morphology, (iv) cell differentiation state, (v) cell movement, proximity, contact, division, and death, (vi) parent-child cell lineage and cell populations of the same lineage, or (vii) proximity or contact between cell populations. In any one of claims 19 to 21,
The computer calculates the number of cells, the degree of proliferation, or the rate of differentiation as the predetermined control index. A cell observation image acquisition method for acquiring an observation image of a cell supported in a culture vessel using a cell observation image acquisition device, comprising:
Illuminating the culture vessel with illumination light by an illumination unit located above the culture vessel in the cell observation image capture device through a predetermined pattern disposed between the illumination unit and an upper cover of the culture vessel, the pattern having a difference in light transmittance;
and capturing an image of the culture vessel illuminated by the illumination light by an imaging unit located under the culture vessel in the cell observation image capturing device,
The predetermined pattern is composed of a plurality of pattern sheet sets arranged at a predetermined distance apart, and a light-transmitting portion of the pattern sheet arranged closest to the lighting unit is covered by a light-shielding portion of another pattern sheet,
A cell observation image acquisition method in which the focus of the photographing unit is adjusted to a position shifted a predetermined distance toward the illumination unit from the culture vessel mounting surface of the photographing unit, thereby aligning the focus with the cell surface, and no tiling photography is performed. In claim 23,
A cell observation screen acquiring method, wherein the plurality of pattern sheet sets are arranged so that light transmitted through the plurality of pattern sheets illuminates the entire culture surface of the culture vessel.