[go: up one dir, main page]

WO2025054302A1 - Filter-based multispectral fluorescence microscope optimized for virtual staining - Google Patents

Filter-based multispectral fluorescence microscope optimized for virtual staining Download PDF

Info

Publication number
WO2025054302A1
WO2025054302A1 PCT/US2024/045354 US2024045354W WO2025054302A1 WO 2025054302 A1 WO2025054302 A1 WO 2025054302A1 US 2024045354 W US2024045354 W US 2024045354W WO 2025054302 A1 WO2025054302 A1 WO 2025054302A1
Authority
WO
WIPO (PCT)
Prior art keywords
light
sample
excitation
image
emission
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/US2024/045354
Other languages
French (fr)
Inventor
Charles Santori
Michael GUTIERREZ
Dar Bahatt
Andrew Homyk
Tadayuki Yoshitake
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Verily Life Sciences LLC
Original Assignee
Verily Life Sciences LLC
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Verily Life Sciences LLC filed Critical Verily Life Sciences LLC
Publication of WO2025054302A1 publication Critical patent/WO2025054302A1/en
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N21/645Specially adapted constructive features of fluorimeters
    • G01N21/6456Spatial resolved fluorescence measurements; Imaging
    • G01N21/6458Fluorescence microscopy
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28Investigating the spectrum
    • G01J3/44Raman spectrometry; Scattering spectrometry ; Fluorescence spectrometry
    • G01J3/4406Fluorescence spectrometry
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/16Microscopes adapted for ultraviolet illumination ; Fluorescence microscopes
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/28Investigating the spectrum
    • G01J3/2823Imaging spectrometer
    • G01J2003/2826Multispectral imaging, e.g. filter imaging
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/02Details
    • G01J3/0205Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
    • G01J3/021Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows using plane or convex mirrors, parallel phase plates, or particular reflectors
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01JMEASUREMENT OF INTENSITY, VELOCITY, SPECTRAL CONTENT, POLARISATION, PHASE OR PULSE CHARACTERISTICS OF INFRARED, VISIBLE OR ULTRAVIOLET LIGHT; COLORIMETRY; RADIATION PYROMETRY
    • G01J3/00Spectrometry; Spectrophotometry; Monochromators; Measuring colours
    • G01J3/02Details
    • G01J3/0205Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows
    • G01J3/0216Optical elements not provided otherwise, e.g. optical manifolds, diffusers, windows using light concentrators or collectors or condensers
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N2021/6417Spectrofluorimetric devices
    • G01N2021/6419Excitation at two or more wavelengths
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N2021/6417Spectrofluorimetric devices
    • G01N2021/6421Measuring at two or more wavelengths
    • GPHYSICS
    • G01MEASURING; TESTING
    • G01NINVESTIGATING OR ANALYSING MATERIALS BY DETERMINING THEIR CHEMICAL OR PHYSICAL PROPERTIES
    • G01N21/00Investigating or analysing materials by the use of optical means, i.e. using sub-millimetre waves, infrared, visible or ultraviolet light
    • G01N21/62Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light
    • G01N21/63Systems in which the material investigated is excited whereby it emits light or causes a change in wavelength of the incident light optically excited
    • G01N21/64Fluorescence; Phosphorescence
    • G01N21/645Specially adapted constructive features of fluorimeters
    • G01N2021/6463Optics
    • G01N2021/6471Special filters, filter wheel
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B27/00Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
    • G02B27/10Beam splitting or combining systems
    • G02B27/14Beam splitting or combining systems operating by reflection only
    • G02B27/141Beam splitting or combining systems operating by reflection only using dichroic mirrors

Definitions

  • tissue slides may be stained for use in white light brightfield microscopes or for fluorescent microscopy. Each stain may require a separate slide and each separate slide consumes available tissue volume. In cases where the tissue sample is small (e.g., needle biopsy), the number of available stained slides is accordingly limited. The requirement for multiple slides also adds to the time, money and labor required for a complete set of images.
  • a fluorescence microscope can be used to image fluorescence from dye-stained tissue samples or autofluorescence from unstained tissue samples. In the latter case, the natural fluorescence from tissue can be imaged, without the need to apply stains to any tissue samples. However, photobleaching of the tissue sample can occur following repeated exposure to excitation light, which can diminish the fluorescence signal.
  • An example imaging system includes an excitation module, an emission module, and a sample stage configured to support a sample.
  • the excitation module is configured to transmit a light beam to the sample supported by the sample stage to cause the sample to emit fluorescence light that is detectable by an emission module.
  • the excitation module includes one or more partial reflectors; two or more light sources arranged in a tree configuration based on positions of the one or more partial reflectors , in which each light source emits light of a particular color; two or more collimating lenses corresponding to the two or more light sources; a tube lens; and an excitation objective configured to direct light to the sample, in which a first partial reflector of the one or more partial reflectors is configured to allow light from a first light source of the two or more light sources and light from a second light source of the two or more light sources to traverse the same path to the excitation objective.
  • the emission module is configured to capture emitted fluorescence light from the sample when the sample is supported by the sample stage, including an emission objective; one or more partial reflectors configured to generate a first transmitted beam and a first reflected beam ; and two or more image sensors, in which each image sensor is configured to receive fluorescence light of at least one color emitted from the sample responsive to a light beam output from an excitation module, wherein the two or more image sensors are arranged in a tree configuration based on positions of the one or more partial reflectors.
  • a system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardw are, or a combination of them installed on the system that in operation causes or cause the system to perform the actions.
  • One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.
  • One general aspect includes a computer-implemented method. The computer-implemented method includes outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light.
  • each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module.
  • the computer-implemented method also includes receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor.
  • the computer-implemented method also includes generating a first raw image from the first imaging signal and generating a second raw image from the second imaging signal.
  • the computer-implemented method also includes generating a composite multispectral image comprising the first raw image and the second raw' image and outputting the composite multispectral image.
  • Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.
  • FIG. 1 illustrates an example imaging system in which techniques for implementing a filter-based fluorescent microscope for virtual staining are illustrated, according to at least one example.
  • FIG. 2 illustrates an example slide that may be imaged using the imaging system of FIG. 1, according to at least one example.
  • FIG. 3 shows a simplified schematic diagram of a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 4 shows a simplified schematic diagram of an excitation module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 5 shows a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 6 shows a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 7A depicts a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 7B depicts a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 8 A depicts a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 8B depicts a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 9A shows a relationship between certain components that may be used in one example emission module for a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 9B shows another relationship between certain components that may be used in one example emission module for a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 10 shows a simplified schematic of an example imaging system that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 11 illustrates an example flowchart illustrating a process for operating a filterbased fluorescent microscope for virtual staining, according to at least one example.
  • FIG. 12 illustrates an example system for implementing techniques for filter-based fluorescent microscope for virtual staining, according to at least one example.
  • Some examples may thus use virtual staining techniques to increase the information that can be obtained from a single, unstained tissue section.
  • virtual staining a multispectral fluorescence image is first obtained from an unstained section via autofluorescence, or the natural emission of light by the tissue sample following the absorption of excitation light without the application of a specific fluorescent stain.
  • a trained machine learning (ML) model is used to output a set of predicted “virtual” stains that may cause an output that mirrors that which would be produced by actual staining of tissue samples.
  • the tissue section is left unstained and largely unaltered, leaving it available for subsequent staining or other types of analysis.
  • the microscope should be configured to scan the entire tissue sample in a relatively short amount of time without compromising the amount of useful spectral information that is obtained.
  • the microscope should be sufficiently stable to generate consistent data quality without requiring excessive maintenance.
  • high spatial resolution and image sharpness directly correspond to virtual staining accuracy.
  • the microscope should not acquire an excessively large number of spectral channels, compared with the density 7 of spectral features found in the autofluorescence data, for the data storage and processing to be practical.
  • the techniques described herein may be used to implement a widefield. filter-based fluorescent microscope for virtual staining that solves the problems mentioned above.
  • a fluorescence microscope can be constructed that has the requisite speed.
  • the choice of optical components and associated control software lend themselves to widefield scanning with numerous excitation-emission combinations (i.e., spectral channels) in periods of time significantly shorter than is possible using existing techniques.
  • moving parts subject to mechanical breakdown may include a translation stage holding the sample, a slide loader for moving slides from storage onto the translation stage, a focusing mechanism for the emission objective, and a focusing stage for the excitation objective.
  • a microscope constructed according to the systems described herein does not involve the movement of filter wheels for each measurement, due to the selection and positioning of optical components. Thus, vast numbers of mechanical filter wheel motions are resulting in a longer lifetime.
  • a widefield, filter-based fluorescent microscope for virtual staining includes an excitation optics system and an emission optics system.
  • the excitation optics system includes a sample stage configured to support a sample.
  • the sample may be a tissue section placed upon a glass slide and optionally covered with a glass cover slip.
  • the excitation optics system also includes an excitation module that is configured to transmit a light beam to the sample to cause the sample to emit fluorescence light that is detectable by an emission module.
  • the excitation light sources can be pulsed or activated sequentially, such that a single light source is used to illuminate the sample at a time.
  • the arrangement of partial reflectors can be used to cause the individual light beams to share paths to the sample (i.e., overlap in space), but not to overlap in time.
  • multiple light sources can be used to illuminate the sample at a time.
  • a combined light beam can be used to excite the sample, which may include one or more combined or uncombined, collimated light-emitting diode (LED) narrowband light beams, with narrowband wavelength ranges selected to cause autofluorescence of the sample.
  • LED collimated light-emitting diode
  • the excitation module includes one or more partial reflectors such as dichroic mirrors.
  • a dichroic mirror is an optical component that transmits light at certain wavelengths and reflects light at other wavelengths.
  • the dichroic mirror can act as either a light beam splitter or be used to combine two light beams.
  • a first dichroic mirror of the one or more dichroic mirrors is configured to generate a first combined beam comprising light from at least a first light source and light from a second light source, the two light sources generating light of different wavelengths.
  • the dichroic mirror can similarly be used to direct light such that light from different sources share the same optical path but traverse the path at different times.
  • dichroic mirrors Although the present disclosure describes examples of light beam combining and splitting using dichroic mirrors, one of ordinary skill in the art will recognize that other components may be used to accomplish similar design objectives. For instance, in some examples, 50/50 beam splitters may be used for beam splitting in lieu of dichroic mirrors. Likewise, fluorescence filters can be used to split an incoming light beam into two light beams of approximately equal intensity. Other components may also be used.
  • the excitation module includes two or more light sources.
  • LED light sources with wavelength ranges chosen for autofluorescence may be used.
  • the light sources - each light source emitting light of a particular color (i.e., a narrow range centered on a particular wavelength) - are arranged in a tree configuration based on positions of the one or more dichroic mirrors.
  • wavelength used in reference to a light source refers to a characteristic wavelength. All light sources produce a distribution of wavelengths, for example, a narrow band w ith a central peak intensity that falls off rapidly.
  • one tree configuration may include two “branches.”
  • the first branch may itself include tw o branches, or sub-branches.
  • Each sub-branch of the first branch may include a light source of a particular color.
  • a dichroic mirror can be used to establish overlapping beam paths, for a single light source in operation, and to combine or overlap multiple light beams for multiple, simultaneous light source operation.
  • the second branch may also include a light source of yet another color.
  • a dichroic mirror can combine the light beams of the first and second branches such that the transmitted beam includes three overlapping light beams (two from the first branch and one from the second branch).
  • the light source from each sub-branch or branch may be pulsed to illuminate the sample sequentially.
  • Branches may also include turning mirrors to allow for branches to be oriented in orthogonal planes.
  • the tree configuration may thus be used to reduce the footprint of the optical setup, which would otherwise occupy space in a single 2D plane.
  • the tree configuration can include an arbitrary number of branches, although in practical implementations, the number of branches will be constrained by the available space.
  • the excitation module also includes two or more collimating lenses corresponding to the two or more light sources.
  • LED light sources may be immediately followed by collimating lenses to cause the divergent light from the LED sources to arrive uniformly and parallel at the sample.
  • the light beam then passes through a tube lens prior to an excitation objective lens.
  • the excitation objective is configured to direct the first combined beam to the sample.
  • a combined beam including light from multiple branches passes through the excitation objective.
  • the emission optics system includes an emission module configured to capture emitted fluorescence light from the sample when the sample is supported by the sample stage.
  • the emission module includes a collection objective for receiving the autofluorescence light emitted from the sample.
  • the emission module includes one or more partial reflectors such as dichroic mirrors in analogy 7 to the excitation module, in which a first dichroic mirror of the one or more dichroic mirrors is configured to generate a transmitted beam and a reflected beam.
  • the transmitted beam includes at least autofluorescence light of a first color emitted from the sample and the reflected beam includes at least autofluorescence light of a second color emitted from the sample.
  • the emission module includes two or more image sensors. Each image sensor is configured to receive fluorescence light of a particular color emitted from the sample responsive to a light beam output from the excitation module. Like the excitation module, the two or more image sensors are arranged in a tree configuration based on positions of the one or more dichroic mirrors as well as turning mirrors.
  • one tree configuration of the emission module may include two “branches” corresponding to the two branches of the excitation module.
  • the first branch may itself include two sub-branches.
  • Each sub-branch of the first branch may include an image sensor for a particular color corresponding to the color of light emitted from the sample.
  • the second branch may also include an image sensor corresponding to a particular color corresponding to another color emitted from the sample.
  • two dichroic mirrors can be used to split the combined light beams from the branches of the excitation module (e g., two from a first branch and one from a second branch).
  • the dichroic mirrors can be used to direct the light beam to the appropriate image sensor.
  • Branches or sub-branches may also include turning mirrors to place branches in orthogonal planes.
  • the tree configuration may thus be used to reduce the footprint of the optical setup, which would otherwise occupy space in a single 2D plane.
  • the tree configuration can again include an arbitrary number of branches.
  • a computer-implemented method for operating a widefield, filter-based fluorescent microscope for virtual staining includes a computing device outputting a command to cause one or more light sources arranged in a tree configuration, as described above, to emit light.
  • Each light source emits light of a particular color.
  • one light source may be used to illuminate the sample at a time, while in other examples, multiple light sources may be used simultaneously to illuminate the sample.
  • the light sources can be activated sequentially or in a pulsed fashion or they can all be activated simultaneously or simultaneously in subsets.
  • fluorescence is spectrally continuous and broadband
  • sets of colors that are eligible for imaging may be identified.
  • Some images are not meaningful fluorescence images because (1) the wavelength of fluorescence light is generally longer than the wavelength of excitation light or (2) excitation light may be too bright and hide the resulting fluorescence. The latter may occur when the wavelength of the fluorescence overlaps with the excitation wavelength. In some cases, the excitation light may be considerably brighter than the fluorescence, so the fluorescence signal cannot be detected. Thus, excitation wavelengths and fluorescence wavelengths may be selected such that the fluorescence wavelength is longer than excitation wavelength for optimized imaging outcomes.
  • blue, green, yellow, or red fluorescence images may be obtained.
  • yellow or red fluorescence may be obtained.
  • a blue image may not contain any detectable fluorescence signal as the detection wavelength is shorter than excitation light.
  • a green image may be difficult to obtain since excitation light may make the faint fluorescent signal invisible.
  • the computing device receives, from two or more image sensors arranged in a tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor.
  • the image sensors may be selected or configured to receive the red and green emitted fluorescence light, either simultaneously or in sequence.
  • Raw images are generated using the first imaging signal and second imaging signal. From these raw images, the computing device generates a composite multispectral image and outputs the composite multispectral image.
  • FIG. 1 illustrates an example imaging system 100 in which techniques for a filter-based fluorescent microscope for virtual staining system may be implemented, according to at least one example.
  • the imaging system 100 includes a microscope 102 and a computer system 104.
  • the microscope 102 includes a camera 106, a filter 108, a tube lens 110, an objective 112, a lateral stage 114, and an illumination system 116 including an illumination source 118 and one or more lenses 120.
  • the imaging system 100 may be configured to image a sample 122. which in the illustrated example, may be held on a slide, such as a glass slide.
  • filters 108 may also be included in the imaging system 100 to achieve design objectives. Filters 108 may be used both before and after the sample 122 is illuminated.
  • the imaging system 100 images the exit face of a light guide to a plane close to the sample, but other configurations are possible.
  • imaging system 100 can be configured to be a fluorescent microscope in which illumination source 118 is used to excite fluorescent stains applied to the sample 122.
  • masks at a plane conjugate to the sample surface may be used to limit the illumination area and reduce photobleaching.
  • Imaging system 100 is thus an example of a simple fluorescent microscope that includes an excitation module with a single light source and an emission module with a single image sensor, shown here to illustrate certain components and concepts.
  • FIGs. 3-8B and 10 depict excitation modules and emission modules with multiple light sources and image sensors, respectively, in accordance with some aspects of the present disclosure.
  • the camera 106 may be any suitable device that includes at least one image sensor. In some examples, the camera 106 may have any suitable range of resolution and be capable of imaging any suitable wavelength of light. In some examples, the camera 106 may be suitable for imaging fluorescent wavelengths, though it may also image in other wavelengths.
  • the filter 108 may be any suitable filter capable of altering the characteristics of the light that is seen by the sensor(s) of the camera 106 in accordance with the design objectives of the imaging system 100. Thus, in some examples, the filter 108 may be selected to enable the camera 106 to capture images of different wavelengths of light.
  • the objective 112 may be an infinity-corrected objective together with the tube lens 110. In some examples, the objective 112 may be corrected for a fixed tube length (e.g., 160 mm) without a separate tube lens 1 10. Various arrangements of these and other components are possible to accomplish various design and operational objectives. For instance, in some examples, the objective 112 may be moved along the optical Z axis for focusing.
  • the lateral stage 114 may be configured to retain the sample 122.
  • the lateral stage 114 may also be configured for movement in multiple axes with respect to the camera 106.
  • the lateral stage 1 14 may be moved in an X direction and a Y direction (e.g., front to back and side to side along a plane parallel to the surface of the lateral stage 114 on which the sample 122 is held).
  • the lateral stage 114 may also be moveable in a Z direction (e.g., vertically with respect to the objective 112). Movement of the lateral stage 114 may be manual or automated. In a manual example, a set of knobs and gears may be used to move the lateral stage 114.
  • Sensors and/or readouts may be connected to the lateral stage 114 in the manual example to provide a user with positional feedback of the relative position of the lateral stage 114.
  • a set of servo motor actuators (or other automated mechanisms) may be coupled with the lateral stage 114 and electrically coupled with a controller.
  • the controller may provide signals to the servo motor actuators to control movement of the servo motor actuators.
  • the controller may be included, or otherwise be, the computer system 104.
  • the computer system 104 may provide electric signals to the servo motor actuators to cause the servo motor actuators to move the lateral stage 114 to certain positions.
  • sensors and/or readouts may be connected to the lateral stage 114 to provide positional information.
  • the positional information either collected from the sensors or derived specifically from the servo motor actuators, may be used to perform the techniques described herein.
  • the positional information may be used in place of image registration to determine the actual shifts of the motorized lateral stage 114 between a set of images.
  • the illumination source 118 and the one or more lenses 120 of the illumination system 116 may be configured to provide light for imaging the sample 122.
  • properties of the illumination source 118 and the one or more lenses 120 may be adjusted to achieve the particular imaging objectives of the system.
  • the illumination source 118 and the one or more lenses 120 may be selected to provide fluorescent illumination of the fluorescent stains applied to the sample 122.
  • the excited stains may emit light at a longer wavelength than illumination source 118.
  • the emitted light can be separated by a spectral emission filter, and then imaged, for example using camera 106, to create high-contrast pictures of the sample.
  • imaging system 100 can image specific structures or molecules within the sample, which have been marked with the fluorescent stains.
  • imaging system 100 can be configured to utilize the autofluorescence properties of the sample 122 for virtual staining.
  • virtual staining a multispectral fluorescence image is first obtained from an unstained sample 122 via autofluorescence, or the natural emission of light by the tissue sample following the absorption of light without the application of a specific fluorescent stain.
  • a trained ML model is used to predict a desired set of stains from the autofluorescence data. The tissue section is left unstained and largely unaltered, leaving it available for subsequent staining or other types of analysis.
  • the computer system 104 may be any suitable computing device including a desktop computer, a server computer, a tablet, a laptop computer, a microprocessor and coupled memory, and any other suitable combination of the foregoing.
  • the computer system 104 may be integrally formed with the imaging system 100 and may include one or more ports for input/output components, such as a display, keyboard, keypad, mouse, and the like.
  • the computer system 104 may be configured to control the operation of the microscope 102 and/or perform techniques described herein relating to computing detection profiles.
  • the computer system 104 outputs information relating to the imaging system 100 (e.g., image data, state data for light the illumination system 116, positional information of the lateral stage 114, and any other suitable information), which is then processed by a different computer system for performing the techniques described herein.
  • information relating to the imaging system 100 e.g., image data, state data for light the illumination system 116, positional information of the lateral stage 114, and any other suitable information
  • FIG. 12 An example of components that may be included in the computer system 104 is shown in FIG. 12.
  • FIG. 2 illustrates an example slide 200 that may be imaged using the imaging system 100 of FIG. 1, according to at least one example.
  • the slide 200 may be formed from any suitable material including, for example, glass, plastic, quartz, and any other suitable material.
  • the slide 200 may have any suitable shape including rectangular, square, oval, round, and the like.
  • the slide 200 may have any suitable dimension, which may be standard (e.g., 75 mm by 25 mm) or non-standard.
  • the slide 200 may also be used with a cover slip or cover glass (not shown) to help retain a sample 202 on a slide surface 204 of the slide 200.
  • the sample 202 which is an example of the sample 122, may be any suitable object to be imaged by the imaging system 100.
  • the sample 202 may be a tissue sample that has been obtained from a subject (e.g., a human).
  • the sample 202 may include one or more distinct features 208 and 210.
  • the feature 208 e.g., a first feature
  • the feature 210 e.g., a second feature
  • the feature 208 and the feature 210 may represent aspects of the sample 202 that are distinct or otherwise more highly visible under certain conditions. In some examples, these conditions may be within the visible wavelength or may be within a non- visible wavelength. In some examples, these aspects of features 208 and 210 may be surfaced, identified, highlighted, or otherwise imaged using the techniques described herein.
  • FIG. 2 includes an X axis 212 and a Y axis 214 extending parallel to the slide surface 204 of the slide 200.
  • the X axis 212 and the Y axis 214 are included for illustrative purposes to explain how the position of the slide 200 may be moved (e.g., on the lateral stage 1 14) to capture images of the slide 200 at different positions.
  • the slide 200 is illustrated at a centered position depicted by a center marker 216 (e.g., centered at an origin of the X axis 212 and the Y axis 214).
  • a center marker 216 e.g., centered at an origin of the X axis 212 and the Y axis 214.
  • positional markers 218 are also illustrated in FIG. 2. A few of these positional markers 218 are also labeled.
  • the positional markers 218 are illustrative of other positions to which the slide can be moved relative to the camera.
  • the slide 200 can be moved laterally along the Y axis 214 to positional marker 218(1), e.g., (0, +y) and 218(3), e.g., (0, -y).
  • the slide 200 can also be moved laterally along the X axis 212 to positional markers 218(2), e.g., (+x, 0) and 218(4), e.g., (-x, 0).
  • the slide 200 can also be moved in both the X and Y directions, e.g., as illustrated by positional markers 218(5) and 218(6), while remaining on the slide surface 204 (e.g.. 218(5) and off of the slide surface 204 (e.g.. 218(6)). While a few positional markers are illustrated, it should be understood that the slide 200 may be moved to any suitable position relative to the camera.
  • the positional markers 218 are included for illustrative purposes, these markers are not to scale.
  • the image shifts between positions may be much smaller than those illustrated by the positional markers 218.
  • the camera field of view may be much smaller than the sample, such that movements of the field of view remain on the sample 202.
  • the image shift values e.g., distances between positions
  • the image shift points would be confined to some small region within the sample 202.
  • another set of points outside of the sample may be obtained, but still within the slide (e.g., such as illustrated by the positional marker 218(5).
  • tissue texture may be needed to perform image registration, but beyond that the calibration results may be most accurate if the tissue brightness is relatively uniform across the camera field of view.
  • one or more images may be captured in each position.
  • the positions may be predetermined, randomly assigned within some bounding, and/or any suitable combination of the foregoing.
  • the positions may include larger displacements and smaller displacements.
  • the smaller displacements may be suitable for capturing information on structure of the sample 202 and the larger displacements may be suitable for generating information useful for determining large scale variation of the imaging system.
  • a set of positions, in pixel units (x, y) may include ⁇ (0, 0), (0, 118), (0, -118), (118, 0), (-118, 0), (0, 1000), (0. -1000). (1000, 0), (-1000, 0) ⁇ .
  • FIG. 3 shows a simplified schematic diagram of a filter-based fluorescent microscope 300 for virtual staining, according to some aspects of the present disclosure.
  • FIG. 3 illustrates an example filter-based fluorescent microscope 300 for virtual staining in a typical configuration.
  • FIGs. 4-8B and FIG. 10 include depictions of filter-based fluorescent microscopes for virtual staining in various other configurations in accordance with differing design parameters and considerations.
  • microscope 300 includes 4 LED light sources in a tree configuration. For clarity, only LED 305 is described and labeled, but each “branch” of the “tree” is substantially the same except for spectral parameters determined by the wavelength of the light associated with each branch. Similarly, microscope 300 includes 8 image sensors (e.g., cameras), each on a sub-sub-branch of two main branches, but only image sensor 350 is labeled.
  • image sensors e.g., cameras
  • Example microscope 300 includes excitation module 301 and emission module 302.
  • Excitation module 301 includes two or more light sources, like LED 305.
  • Excitation module 301 can use partial reflectors such as dichroic mirrors 325. 330 to direct the light from multiple LEDs 305 to a common optical path.
  • the LEDs 305 may be operated simultaneously to generate a combined light beam. The wavelength of each LED can be chosen to excite the sample at a different wavelength.
  • Example microscope 300 shows 4 LEDs 305, but an arbitrary' number of LEDs 305 may be used, subject to physical space constraints. In some typical examples, between 4 and 7 LEDs 305 may be used.
  • Dichroic mirrors 325, 330 may be used to configure excitation module 301 in a tree configuration.
  • Dichroic mirror 325 or 330 is a mirror with significantly different reflection or transmission properties at two different wavelengths. When light from two LEDs 305 is received by dichroic mirror 325 or 330, it will reflect light of one wavelength while allowing light of the other wavelength to pass through.
  • light from both LEDs 305 can thus be combined into a single beam without significantly disturbing the individual characteristics of the light from each LED 305.
  • single dichroic mirror 330 is used to combine the light from two branches, each branch including one LED 305.
  • Each additional dichroic mirror 325 splits a branch (or subbranch) to create two sub-branches. Sub-branches can be further split using additional dichroic mirrors 325, and so on.
  • the LEDs 305 may be operated sequentially or pulsed. For instance, in an example, the shorter wavelength LEDs 305 (e.g., blue) are pulsed followed by a period for data collection using all or most image sensors 350. In the next step, a longer wavelength LED 305 (e.g., yellow) can be pulsed followed by data collection by a smaller subset of image sensors 350. Finally, the longest wavelength LED 305 (e.g., red) may be pulsed followed by data collection using only a single image sensor 350. This sequence reflects that emitted light is typically at a longer wavelength compared with the excitation light wavelength.
  • the shorter wavelength LEDs 305 e.g., blue
  • a longer wavelength LED 305 e.g., yellow
  • the longest wavelength LED 305 e.g., red
  • This sequence reflects that emitted light is typically at a longer wavelength compared with the excitation light wavelength.
  • turning mirrors may be used to lift a branch or subbranch out of the starting plane of excitation module 301 or to introduce other non-functional turns.
  • the combination of dichroic mirrors 325, 330 and turning mirrors may be used to configure the tree configuration of excitation module 301 in a compact three-dimensional shape, thus minimizing the footprint of microscope 300 or otherwise adapting the physical dimensions of microscope 300 to meet physical space requirements.
  • Excitation module 301 includes collimating lens 310. Collimating lens 310, tube lens 320, and excitation objective 335 map LED 305 structure into the angular distribution of light at sample 340. This ty pe of configuration is sometimes referred to as Kohler illumination. In Kohler illumination, the divergent light from LED 305 source is collimated and uniformly distributed at the sample and is not affected by the physical structure of LEDs 305 itself. This ensures that the final image will not include the structure or arrangement of LED 305 sources. [0069] Excitation module 301 includes photodiode 315. The photodiode 315 may be positioned at the unused port of dichroic mirror 330.
  • Dichroic mirror 330 is the one used to join the two main branches of the tree configuration of excitation module 301 .
  • two light beams themselves each possibly a combination of two or more light beams from two or more light sources 305, are combined at dichroic mirror 330, producing light beam 322 for exciting sample 340.
  • only a single, monochromatic light beam 322 is directed to the sample 340.
  • the unused port of dichroic mirror 330 refers to the location orthogonal to light beam 322 that receives neither reflected nor transmitted light. Because dichroic mirror 330 does not perfectly transmit or reflect all incident light, a leakage signal proportional to LED 305 powers can be received by photodiode 315. The leakage signal received by photodiode 315 can be used for continuous calibration of the excitation LED 305 sources or to monitor the stability of excitation module 301.
  • Excitation module 301 includes tube lens 320.
  • the light beam 322 reflected/transmitted from dichroic mirror 330 next passes through tube lens 320.
  • Tube lens 320 may form an image of LEDs 305 at the back focal plane of or onto excitation objective 335.
  • Excitation objective 335 in turn focuses or collimates the light onto sample 340 for excitation.
  • Excitation refers to the process whereby light energy is absorbed by an atom or molecule of sample 340, raising it from its ground state to a higher energy, or excited, state.
  • the absorbed energy corresponds to incident light of a specific wavelength.
  • the excited atom or molecule will '‘relax” back to a ground state after a very short time (e.g., nanoseconds).
  • relaxation energy is released in the form of light. This emitted light may be at a longer wavelength than the incident light. This cycle of excitation and relaxation is usually referred to as fluorescence.
  • Emission objective 345 may be an infinity-corrected lens that collects the fluorescence from the sample and outputs a collimated, light beam 347.
  • Collimated, light beam 347 is received by dichroic mirror 365, in this case acting as a beam splitter. Collimated in this context refers to a collection of parallel light beams, although not necessarily parallel to the optical axis.
  • Emission module 302 is depicted as having 2 main branches with 2 sub-branches each. Each sub-branch also has two sub-sub-branches, for a total of 8 sub-sub-branches and 8 corresponding image sensors 350.
  • the number of image sensors 350 does not necessarily match the number of LED 305 sources because some wavelengths of excitation light may cause more than one emitted wavelength of light.
  • second 370 and third dichroic mirrors 360 are used to further split or direct light beam 347.
  • Each terminal branch (e.g., sub-sub-branches ending with an image sensor 350) of emission module 302 includes tube lens 355.
  • tube lens 355 can form an image of sample 340 fluorescence in the plane of image sensor 350 at the end of the branch.
  • Image sensor 350 may be any type of image sensor or camera suitable for the gathering of emitted light at the target wavelengths and desired resolution.
  • image sensor 350 may be a charge-coupled device (CCD), a complementary metal-oxide-semiconductor (CMOS) sensor, or a photomultiplier tube (PMT). among others.
  • CCD charge-coupled device
  • CMOS complementary metal-oxide-semiconductor
  • PMT photomultiplier tube
  • image sensor 350 is a scientific-grade CMOS sensor.
  • a scientific-grade CMOS sensor may provide significantly improved performance characteristics that make them suitable for high-precision scientific applications, like fluorescence microscopy.
  • FIG. 4 shows a simplified schematic diagram of excitation module 400 that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 4 illustrates a detailed view' of excitation module 301 introduced in FIG. 3, including additional components such as mask 412 that will be discussed below.
  • Excitation module 400 may be used in conjunction with an emission module similar to examples 500, 600, 700. and 800 from FIGs. 5-8B as discussed below.
  • excitation module 400 several components appear multiple times but are only labeled and described once for clarity.
  • excitation module 400 includes light sensor 405 that may be an LED light source. Light from one or multiple light sources 405 maybe directed or combined using a tree configuration through a combination of partial reflectors, such as dichroic mirrors, and turning mirrors.
  • two main branches are shown. with tw o sub-branches each.
  • Excitation module 400 is depicted in a 2D plane, but a branch or sub-branch may include a turning mirror that effectively rotates the branch (and included sub-branches) into an orthogonal plane.
  • dichroic mirrors can be used as needed to split branches and turning mirrors can be used to rotate branches (or sub-branches) through rotations.
  • branches or sub-branches are rotated 90 degrees but configurations using other angles are possible.
  • Excitation module 400 including rotated branches can be compactly organized with a minimum of wasted space between the optical components, thus significantly reducing the footprint of the assembled microscope.
  • excitation module 400 emitted light from light source 405 may be followed by collimating lens 410, similar to collimating lens 310 of FIG. 3. Collimating lens 410 can map the light source structure into the angular distribution of light beam 470 arriving at sample 440, effectively producing a ty pe of Kohler illumination.
  • collimating lens 410 is followed by mask 412.
  • Mask 412 can define the illumination area on sample 440.
  • mask 412 is a rectangular aperture, but other mask 412 shapes are possible.
  • the exit aperture of collimating lens 410 effectively acts as the mask for the light emitted from collimating lens 410.
  • widefield illumination i.e., uniform illumination of sample 440
  • mask 412 is used to limit the illumination area of sample 340 to a region not much larger than the imaging field of view, as defined by the emission optics. Limiting the illumination area in this way can maximize contrast and avoid excessive photobleaching of sample 440.
  • bandpass filter 414 may be used to limit the wavelength range of the excitation light, to avoid any overlap between the excitation and emission spectra.
  • bandpass filter 414 can be tilted, so that sample 440 fluorescence inadvertently collected by the excitation optics that reflects off of the filters is not able to return back to sample 340 and on to the image sensors of emission module 302. The tilt of bandpass filter 414 can be chosen to maintain the requisite transmission efficiency of bandpass filter 414 and other spectral characteristics.
  • Excitation module 400 includes photodiode 415 and tube lens 420 that are substantially similar to the complementary' components depicted in FIG. 3.
  • dichroic mirror 425 is used to allow monochromatic light beams from light sources 405 on a sub-branch to traverse the same optical path.
  • Dichroic mirror 430 similarly allows light beams 450 and 460 to traverse the same optical path.
  • Light beam 470 illuminates the sample 440 via excitation objective 435, similar to the description of FIG. 3.
  • the light sources 405 can be activated sequentially' or in a pulsed fashion or they can all be activated simultaneously or simultaneously in subsets.
  • excitation objective 435 may require some working distance between excitation objective 435 and sample 440 to avoid mechanical collisions with sample 440. Therefore, an aspheric lens may be used for excitation objective 435 that can correct for spherical aberrations while taking up less space. However, a single-element lens may have a large chromatic focal shift. Requiring excitation objective 435 to be moved to a different position for each excitation wavelength, may slow scanning unacceptably.
  • the upstream optics of excitation module 400 can be configured to keep all of the excitation profiles (i.e., different wavelengths) simultaneously in focus.
  • the chromatic focal shift caused by excitation objective 435 e.g., a single-element aspheric lens
  • excitation objective 435 can be compensated for by selecting an appropriate distance between fixed tube lens 420 and the variable mask 412 position.
  • a set of mask-tube lens distances providing simultaneous focusing of the 2 LED masks is 132 and 165 mm, respectively.
  • FIG. 4 correspondingly depicts significantly different mask-tube lens distances for light beams 450 and 460, where light beam 450 has a longer wavelength than light beam 460.
  • the mask-tube distance for light beam 450 is longer, which can result in simultaneous focusing of the light beams 450 and 460 by excitation objective 435 at sample 440.
  • the light beams from the various light sources 405 can become superimposed after excitation objective 435 as a result of the excitation objective’s 435 chromatic aberration. This superposition may occur when the distance between tube lens 420 and excitation objective 435 is equal to the focal length of tube lens 420.
  • the size and shape of mask 412 for each respective light source 405 can be identical.
  • the excitation module 400 may include turning mirror 445 for configurations in which the excitation optics he in a horizontal plane, while the light beam is directed vertically towards sample 440.
  • turning mirror 445 for configurations in which the excitation optics he in a horizontal plane, while the light beam is directed vertically towards sample 440.
  • a combination of dichroic mirrors and turning mirrors can be used to construct a tree configuration in 3D space, resulting in significant flexibility in the orientation of all of the optical components.
  • the excitation optics could be arranged in a vertical plane or a nonplanar configuration.
  • excitation objective 435 may be mounted on a vertical actuator for focusing using manual manipulation, software, or electronics.
  • the vertical actuator (not shown) may be a piezo motor, a conventional motor, or other suitable component for affecting vertical or horizontal motion of excitation objective 435.
  • Excitation objective 435 may also be mounted on a 2D lateral translator (not shown). The lateral translator can be adjusted using manual manipulation, software, or electronics to enable alignment of mask 412 image relative to the detection field of view in the detection optics.
  • FIG. 5 shows a simplified schematic diagram of emission module 500 that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 5 shows a simplified schematic diagram of emission module 500 that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • Emission module 500 illustrates an example emission module 500 corresponding to a detailed view of emission module 302 introduced in FIG. 3, including additional components such as turning mirror 550 that will be discussed below.
  • Emission module 500 may be used in conjunction with an excitation module similar to examples 300, 400 from FIGs. 3 and 4. In emission module 500, several components appear multiple times but are only labeled and described once for clarity.
  • Emission module 500 includes emission objective 505.
  • Emission objective 505 may be an infinity -corrected objective configured to collect the fluorescence from sample 340 and output a collimated beam.
  • infinity -corrected refers to a configuration in which the light leaving the objective is collimated. Infinity-correction may allow for maximum flexibility in placement of downstream optical components.
  • Emission objective 505 may also include a correction for the presence or absence of coverglasses on sample 340.
  • emission objective 505 may include a rotatable ring, collar mechanism, or other component that can adjust the spacing or position of the internal optical elements within emission objective 505, thus allowing for the imaging of samples 340 in either case.
  • emission objective 505 may include a kinematic adjustment to compensate for the tilt of the microscope or of sample 340.
  • Emission module 500 includes turning mirror 510 for changing the plane of emission module 500 optics with respect to the incident light beam.
  • a combination of dichroic mirrors and turning mirrors can be used to construct a tree configuration for emission module 500 in 3D space, resulting in significant flexibility in the orientation of all of the optical components.
  • emission module 500 optics could be arranged in either a horizontal or vertical plane, or even anonplanar configuration.
  • turning mirror 510 may include a kinematic adjustment.
  • emission module 500 includes one or more partial reflectors such as dichroic mirrors 515, 525, 530.
  • Dichroic mirrors 515, 525, 530 may include kinematic adjustments for fine-tuning optical alignment and alignment of image sensor 555 field of view.
  • a combination of turning mirrors 510 and dichroic mirrors 515, 525, 530 can be used to arrange the optical components of emission module 500 into a compact tree configuration.
  • the dichroic mirrors 515, 525, 530 may be used to direct or split the incident light beam 520 according to wavelength.
  • incident light beam 520 received from the excited sample 340 includes 8 wavelengths of light
  • 7 dichroic mirrors can be used to separate combined light beam 520 into 8 separate paths, assuming a symmetrical arrangement.
  • some image sensors 555 may not detect a faint fluorescence signal, an effect that is stronger for longer excitation wavelengths.
  • each path terminates with image sensor 555 as will be described in more detail below.
  • the tree depth is 3 (i.e. , a branch, a sub-branch, and a sub-sub- branch), in which each path includes 3 dichroic mirrors 515, 525, 530.
  • emission module 500 could be configured using a tree configuration with 10 paths by adding two additional dichroic mirrors to split beams 545 and 547.
  • the existing paths would include 3 dichroic mirrors and the two paths corresponding to the added dichroic mirrors would include 4 dichroic mirrors.
  • dichroic mirrors 515, 525, and 530 reflect shorter wavelengths and transmit longer wavelengths. However, dichroic mirrors that reflect longer wavelengths and transmit shorter wavelengths can also be used. However, selection of dichroic mirrors with the same reflectance/transmission profile may result in a more consistent assembly. For example, if dichroic mirrors 515, 525, and 530 reflect shorter wavelengths and transmit longer wavelengths, then image sensor 555 corresponding to the shortest-wavelength incident light receives light reflected by all 3 dichroic mirrors in its path. In contrast, image sensor 555 corresponding to the longest-wavelength incident light receives light transmitted by all 3 dichroic mirrors in its path.
  • Some examples may be configured to minimize the number of dichroic mirrors that transmit longer-wavelength fluorescence light (e.g., red) since the fluorescence signal may decrease with wavelength and the efficiency of transmission of dichroic filters is imperfect (e.g., typically ⁇ 100% transmission). For example, transmission through five 95% efficient dichroic mirrors may be 77%, a significant loss of signal.
  • dichroic mirrors that transmit longer-wavelength fluorescence light (e.g., red) since the fluorescence signal may decrease with wavelength and the efficiency of transmission of dichroic filters is imperfect (e.g., typically ⁇ 100% transmission). For example, transmission through five 95% efficient dichroic mirrors may be 77%, a significant loss of signal.
  • Emission module 500 includes bandpass filter 535 on each path following one or more dichroic mirrors 515, 525, 530. Bandpass filter 535 further defines the detection spectrum for each image sensor 555. Bandpass filter 535 also provides additional attenuation outside of the detection band, which may be required because of the large intensity difference between the excitation light and the fluorescence light emitted from sample 340.
  • bandpass filter 535 is followed by tube lens 540.
  • Tube lens 540 can form an image of sample 340 fluorescence in the plane of image sensor 555 at the end of the path. Tube lens 540 thus focuses the light from sample 340 onto image sensor 555.
  • tube lens 540 is followed by turning mirror 550, to reduce the footprint of the optical setup as described above.
  • turning mirrors 550 are configured for a tree configuration that is a spiral pattern.
  • Turning mirrors 550 may include kinematic adjustments that can be used to align the camera fields of view.
  • Each path of emission module 500 terminates with image sensor 555.
  • image sensor 555 Typical configurations for widefield imaging of a rectangular field of view may utilize two-dimensional sensors, but other configurations are possible.
  • a hne sensor could be used in a line scanning mode. In the line scanning mode, data may be captured one line at a time.
  • Image sensor 555 may be any type of image sensor or camera suitable for the gathering of emitted light at the target wavelengths and desired resolution.
  • image sensor 555 may be CCD, a CMOS sensor, or a PMT, among others.
  • image sensor 555 is a scientific-grade CMOS sensor.
  • the scientific-grade CMOS sensor may include features such as fast readout, low readout noise, and a global shutter.
  • image sensor 555 size is chosen to match the field of view of the imaging system where the imaging performance is close to diffraction-limited.
  • Image sensor 555 may be controlled by electronics allowing for, for example, parallel exposure, triggering, and readout.
  • bandpass filters 535 could instead be positioned after tube lenses 540.
  • FIG. 6 shows a simplified schematic diagram of emission module 600 that may be used in a filter-based fluorescent microscope for virtual staining in which bandpass filters 535 are positioned after tube lenses 540, according to some aspects of the present disclosure.
  • Emission module 600 may be used in imaging systems in conjunction with an excitation module similar to examples 300, 400 from FIGs. 3 and 4. Note that in emission module 600, several components appear multiple times but are only labeled and described once for clarity.
  • bandpass filter 535 follows tube lens 540 in each path. Bypositioning bandpass filters 535 after tube lenses 540 (e.g., close to image sensors 555) several potential issues can be mitigated.
  • Certain optical components on each path may have angular dependencies that result in a spectral shift that may affect the final image imaged by image sensor 555. For instance, some dichroic mirrors used at 45 degree angles of incidence, as shown, may result in a first-order spectral shift that can significantly affect the image. Because a large field of view of sample 340 may be imaged, a range of angles of collimated light beams may be emitted from emission objective 505. For a point offset by /lx from the center of the field of view, the angle of the Ax collimated light beam with respect to the optical axis is given by approximately — , where fi is A the effective focal length of emission objective 505.
  • the range of angles making up the collimated light beams is proportional to Ax.
  • the range of angles of the emitted non-collimated light is given bv approximately 2(W/I) ' r ' where NA is the numerical aperture of emission objective 505, and f2 is the effective focal length of tube lens 540.
  • the total range of ray angles in collimated space may exceed the range of angles in non-collimated space, depending on the size of the field of view relative to emission objective 505 characteristics.
  • this may result in undesirable or uncorrectable non-uniform responses (e.g., spectral shifts) or aberrations in the final image due to the spectral shift from optical components with an angular dependency. It may thus be preferable to position bandpass filter 535 after tube lenses 540 in such cases.
  • emission module 600 can reduce spatial- spectral coupling.
  • An additional advantage is that positioning bandpass filters 535 closer to image sensors 555 can allow for smaller bandpass filters 535 to be used without vignetting, thereby reducing cost. Vignetting refers to the darkening of the comers of an image compared to its center.
  • FIGs. 7A and 7B show simplified schematic diagrams of emission module 700 that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIGs. 7A and 7B illustrate example emission module 700 with some components repositioned and/or in different quantities in accordance with certain design parameters.
  • Emission module 700 may be used in imaging systems in conjunction with an excitation module similar to examples 300, 400 from FIGs. 3 and 4. In emission module 700, several components appear multiple times but are only labeled and described once for clarity.
  • FIG. 7A depicts example emission module 700 that includes only single tube lens 710, instead of tube lens 540 for each branch, as shown in FIGs. 5 and 6.
  • dichroic mirrors 720 are located in non-collimated space (between tube lens 710 and image sensor 715). Similar to the configuration of FIG. 6, in which bandpass filter 725 is positioned after tube lenses 540, it may also be preferable to position dichroic mirrors 720 after tube lenses 710. As discussed above, each point in the imaging field of view transmits the same range of ray angles through the dichroic mirrors. As a result, the transmission spectrum received at image sensor 715 may be uniform across the field of view, despite any angular dependence of the transmission spectra of dichroic mirrors 720.
  • an astigmatism can be introduced by the converging (i.e., emitted from tube lens 710) beams passing through dichroic mirrors 720.
  • An astigmatism may occur when different orthogonal orientations of light have different focal lengths, leading to aberrations in the focused image.
  • This problem may manifest when dichroic mirrors 720 are constructed as dielectric coatings on plane-parallel substrates. In that case, the amount of astigmatism introduced upon transmission through dichroic mirrors 720 may be proportional to the thickness of the substrate. Typical dichroic mirror substrates may be 3 millimeters or more to ensure sufficient flatness for diffraction-limited imaging performance.
  • compensation plates 730 can be introduced along each optical path terminating with image sensor 715.
  • Compensation plates 730 may include glass plates with plane-parallel surfaces. Furthermore, compensation plates 730 may incorporate antireflection coatings on both surfaces. In some examples, compensation plates 730 can be angled with respect to the light beam axis.
  • the thickness of compensation plates 730 is equal to the total thickness of the dichroic mirror substrates through which a given path has been transmitted. For example, for a branch including one sub-branch (i.e., 2 dichroic mirrors) the required compensation plate 730 thickness is 2 times the dichroic mirror thickness. In some cases, for branches in which the light arrives at image sensor 715 following only reflections by dichroic mirrors, and no transmission, no compensation plate 730 may be required.
  • the amount of required compensation can be reduced by sending some of the paths out of the plane of the optical system.
  • FIG. 7B depicts example emission module 700 in which some dichroic mirrors 750, 755, 760 are oriented to reflect or transmit light beams out-of-plane.
  • out-of-plane dichroic mirrors 755, 760 are represented by rectangles with crosses connecting the comers and out-of-plane paths are represented by dashed lines.
  • required compensation plate 765 thickness may be significantly reduced. For example, for the labeled path in FIG. 7B, only a compensation plate thickness equal to the width of dichroic mirror 760 may be needed.
  • Such a configuration can also reduce uncompensated higher-order aberrations introduced by orthogonally tilted plates.
  • FIGs. 8A and 8B show a simplified schematic diagrams of emission module 800 that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIGs. 8A and 8B illustrate example emission module 800 with some components repositioned and/or in different quantities in accordance with certain design parameters.
  • Emission module 800 may be used in imaging systems in conjunction with an excitation module similar to examples 300, 400 from FIGs. 3 and 4. In emission module 800, several components appear multiple times but are only labeled and described once for clarity.
  • FIG. 8 A depicts example emission module 800 in which dichroic mirrors 805 and 810 are positioned in both collimated space and non-collimated space, respectively.
  • This configuration may be advantageous in applications that are space-constrained. Emitted light from emission objective 802 is reflected or transmitted by dichroic mirror 805 in collimated space. The light beam then passes through tube lens 815, followed by dichroic mirrors 810 in noncollimated space. As in FIGs. 7A and 7B, compensation plate 820 can be added to correct for undesirable astigmatism effects.
  • FIG. 8A all of the optical components of emission module 800 are in a single horizontal plane.
  • the thickness of required compensation plates 820 is accordingly reduced to match the thickness of dichroic mirrors 810 in collimated space that are transmitted light (i.e., light passes through dichroic mirrors 810).
  • FIG. 8B depicts emission module 800 with some components lying out-of-plane.
  • dichroic mirror 805 is in collimated space and dichroic mirrors 825. 830 are in non- collimated space.
  • dichroic mirror 830 is oriented to reflect the incident light beam out of plane.
  • the astigmatism effect may cancel.
  • the width of the compensation plate 835 may be accordingly reduced further.
  • the width of compensation plate 835 need only match the width of single transmitting dichroic mirror 830. Judicious selection of the number and position of tube lenses 815 and dichroic mirrors 805, 825, 830 can thus minimize the physical footprint of the emission module 800.
  • FIGs. 9A and 9B illustrate example configurations of certain components that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
  • FIG. 9A shows relationship 900 between characteristics of bandpass filters and the transmission spectra of dichroic mirrors in one example emission module.
  • the schematic of an example emission module to the right of graph 905 shown serves as a key 910 for graph 905.
  • Graph 905 depicts a relationship between the transmission spectra of the bandpass filters 920 depicted in key 910 and the transmission spectra of dichroic mirrors 915 depicted in key 910.
  • the transmission spectra of dichroic mirrors 915 and bandpass filters 920 may be selected to obtain a suitable overall transmission across the desired detection range, while avoiding unwanted effects near the transmission edges of the components.
  • relationship 900 may be used to select the bandpass filters and dichroic mirrors as shown in key 910.
  • dichroic minor transmission spectrum 925 corresponding to D3 in key 910, transmits 90% to 95% of incident light for typical commercially available dichroic mirrors.
  • bandpass filter transmission profile 930 is chosen such that bandpass filter 930 (BP3/C3) transmits only light that is fully transmitted by dichroic mirror 925 (D3).
  • bandpass filter 930 BP3/C3
  • D3 dichroic mirror 925
  • FIG. 9B depicts another example relationship 950 between transmission spectra of the bandpass filters 970 and transmission spectra of the dichroic mirrors 975.
  • dichroic mirrors 975 primarily define the detection bands, however bandpass filters 970 are still needed to provide additional attenuation of the excitation light.
  • bandpass filter transmission bands 970 are expanded to include the entire transmission bands provided by the combination of dichroic mirrors 975.
  • bandpass filter transmission profile 960 is chosen such that bandpass filter 960 (BP3/C3) transmission profile edges coincide with transmission spectra edge of dichroic mirror 965 (D3).
  • This configuration may significantly increase throughput.
  • this relationship 950 may result in spatial-spectral coupling effects due to the angle-dependent transmission spectra of the dichroic minors.
  • Relationship 950 may be optimized configurations in which the at least one dichroic mirror is positioned in non-collimated space.
  • FIG. 10 show s a simplified schematic of example imaging system 1000 that may be used in a filter-based fluorescent microscope for virtual staining.
  • FIG. 10 illustrates example imaging system 1000 in an epi-fluorescence configuration. In imaging system 1000, several components appear multiple times but are only labeled and described once for clarity.
  • Example imaging system 1000 depicts a fluorescent microscope in an epi-fluorescence configuration.
  • both the excitation light and the emitted fluorescence pass through same objective 1005.
  • Same objective 1005 is thus used for both illumination of the sample with the excitation light and collection of the emitted fluorescence light.
  • the epi-fluorescence configuration may be associated with a smaller physical footprint.
  • excitation and emission module components may be configured in a tree configuration side-by-side.
  • a colored light beam from light source 1010 passes through collimator 1012, mask 1014, bandpass filter 1016, and is then reflected a first time by dichroic mirror 1018.
  • the colored light beam 1008 then passes through tube lens 1020, is reflected by dichroic mirror 1022 and then again by dichroic mirror 1024 towards objective 1005.
  • the light beam excites sample 340 on the sample stage (not shown) and the fluorescent emitted light is received by the same objective 1005.
  • the emitted light beam is reflected by dichroic mirror 1024, transmitted by dichroic mirror 1022 such that only light beam of a particular color 1029 is transmitted.
  • Light beam of a particular color 1029 passes through tube lens 1028, is reflected by dichroic mirror 1030, passes through bandpass filter 1032, and is finally received by image sensor 1034.
  • the epi-fluorescence configuration of FIG. 10 focusing is no longer affected by variations in the thickness of sample 340 (e.g., the microscope slide), and only same objective 1005 is required to be moved for focusing of the image.
  • the epi-fluorescence configuration may simplify mechanical clearance around sample 340 (e.g., the translation stage that holds the sample slides). Additionally, the intensity of the back-reflected illumination sent towards image sensor 1034 may be smaller than the direct illumination sent towards the image sensor in the conventional configurations of FIG. 1 , thereby reducing the performance requirements for the filters.
  • FIG. 11 illustrates an example flowchart showing process 1100. according to at least a few examples.
  • This process 1100, and any other processes described herein, are illustrated as logical flow diagrams, each operation of which represents a sequence of operations that can be implemented in hardware, computer instructions, or a combination thereof (e.g., computer system 1200 as described in FIG. 12 and the accompanying description).
  • the operations may represent computer-executable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, perform the recited operations.
  • computer-executable instructions include routines, programs, objects, components, data structures and the like that perform particular functions or implement particular data types.
  • the order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes.
  • any, or all of the processes described herein may be performed under the control of one or more computer systems configured with specific executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executing collectively on one or more processors, by hardware, or combinations thereof.
  • code e.g., executable instructions, one or more computer programs, or one or more applications
  • the code may be stored on a non-transitory computer readable storage medium, for example, in the form of a computer program including a plurality of instructions executable by one or more processors.
  • Process 1100 relates to operating a filter-based fluorescent microscope for virtual staining, according to at least one example.
  • the process 1 100 in particular is directed to a process for generating a composite image using a fluorescent microscope as described above and using virtual staining techniques based on autofluorescence from sample 340.
  • Some description may be given with respect to the example of FIG. 3 for illustrative purposes, but many other configurations are possible.
  • the process 1100 begins at block 1102 at which a computing device outputs a first command to cause one or more light sources 305 arranged in an excitation tree configuration included in excitation module 301 to emit light.
  • light sources 305 may be LED light sources 305.
  • LED light sources 305 may have narrow, well-defined excitation wavelength range for precision autofluorescence. Some LED light sources 305 may have rapid switching capabilities to minimize photobleaching or allow for in situ switching between excitation wavelengths.
  • each light source 305 emits light of a particular color.
  • a single excitation light source 305 is turned on at a time, while in other examples two or more light sources 305 are used.
  • LED light sources 305 are well-suited to fluorescing applications, often having a narrow spectral profile.
  • a typical LED light source may have a spectral width (full width at half maximum or FWHM) as narrow as 20-40 nm.
  • a 488 nm LED might have a bandwidth of approximately 24 nm. emitting light primarily between 476 nm and 500 nm.
  • additional optical components like bandpass filters may be used to narrow the spectral profile further both prior to excitation and subsequent to emission.
  • optical components such as partial reflectors (e.g., dichroic mirrors) and turning mirrors can be used to combine the light from the one or more light sources 305 to generate a combined light beam. While the light beams are closely aligned, light sources 305 are ty pically incoherent and lack a well-defined phase relationship. Thus, the combined light beam does not experience constructive or destructive interferences, while the light beam's intensity at any point is roughly the sum of the intensities from the two individual beams.
  • partial reflectors e.g., dichroic mirrors
  • turning mirrors can be used to combine the light from the one or more light sources 305 to generate a combined light beam. While the light beams are closely aligned, light sources 305 are ty pically incoherent and lack a well-defined phase relationship. Thus, the combined light beam does not experience constructive or destructive interferences, while the light beam's intensity at any point is roughly the sum of the intensities from the two individual beams.
  • the light beam is directed to sample 340 supported by a sample stage. Illuminated sample 340 emits fluorescence light responsive to the light beam received from the excitation module. Since sample 340 is not stained or dyed, this fluorescence is referred to as autofluorescence, or the emission of light by sample 340 that has not been externally stained or labeled.
  • the computing device receives, from one or more image sensors 350 arranged in an emission tree configuration included in emission module 302, a first imaging signal from first image sensor 350 and a second imaging signal from second image sensor 350.
  • optical components such as dichroic mirrors and turning mirrors can be used to direct the light beam from emission objective 345 to image sensors 350 in a densely compact tree configuration to economize on space or cost.
  • first image sensor 350 may receive light of a first emitted wavelength and second image sensor 350 may receive light of a second emitted wavelength.
  • images may be collected from each image sensor 350.
  • the emitted wavelengths are longer than the excitation wavelengths because the energy of the emitted photons in the emitted light is lower than that of the absorbed photons in the excitation light.
  • a molecule absorbs a photon, it transitions to a higher energy excited state. Upon returning to its ground state, some energy is lost, usually as non-radiative processes, and thus the emitted photon has lower energy.
  • the computing device generates a first raw image from the first imaging signal.
  • the computing device generates a second raw image from the second imaging signal.
  • the first imaging signal may correspond to fluorescence of a particular autofluorescent molecule having a first wavelength range
  • the second imaging signal corresponds to fluorescent emissions from a different autofluorescent molecule with another wavelength range.
  • the dichroic mirrors are used to ensure that image sensors 350 receive these distinct imaging signals separately.
  • the raw images correspond to the intensity of fluorescence emission from sample 340 at each pixel location.
  • the value at each pixel represents the amount of light detected, which may correlate to the concentration and distribution of the autofluorescent molecules in sample 340 at that specific point.
  • Digital processing techniques can be applied to these raw images to correct for, for example, optical aberrations, noise, or artifacts. For example, some corrections may include dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction.
  • the computing device generates a composite multispectral image comprising the first raw image and the second raw image.
  • an image registration component can be used to align the raw images.
  • the image registration component may identify common features or patterns among the raw images and then apply suitable transformations to align the features to produce a composite image.
  • the computing device outputs the composite multispectral image.
  • the composite multispectral image may be shown on a suitable display device, like a computer or smartphone screen.
  • a notification or alert may be sent including information about the composite multispectral image, time of completion, links for access, and so forth.
  • the composite multispectral image may be saved or persisted using a suitable memory device.
  • FIG. 12 illustrates an example system 1200 for implementing techniques for filterbased fluorescent microscope for virtual staining, according to at least one example.
  • FIG. 12 in particular illustrates examples of components of the computer system 1200, according to at least one example.
  • the computer system 1200 may be a single computer, such as a user computing device and/or can represent a distributed computing system, such as one or more server computing devices.
  • the computer system 1200 is an example of the computer system 104.
  • the computer system 1200 may include at least a processor 1202, a memory 1204, a storage device 1206, input/output peripherals (I/O) 1208, communication peripherals 1210, and an interface bus 1212.
  • the interface bus 1212 is configured to communicate, transmit, and transfer data, controls, and commands among the various components of the computer system 1200.
  • the memory 1204 and the storage device 1206 include computer-readable storage media, such as random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), hard drives, CD-ROMs, optical storage devices, magnetic storage devices, electronic non-volatile computer storage, for example, Flash® memory, and other tangible storage media.
  • the memory 1204 and the storage device 1206 also include computer-readable signal media.
  • a computer-readable signal medium includes a propagated data signal with computer-readable program code embodied therein. Such a propagated signal takes any of a variety of forms including, but not limited to. electromagnetic, optical, or any combination thereof.
  • a computer- readable signal medium includes any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transport a program for use in connection with the computer system 1200.
  • the memory 1204 includes an operating system, programs, and applications.
  • the processor 1202 is configured to execute the stored instructions and includes, for example, a logical processing unit, a microprocessor, a digital signal processor, and other processors.
  • the memory 1204 and/or the processor 1202 can be virtualized and can be hosted within another computing system of, for example, a cloud network or a data center.
  • the I/O peripherals 1208 include user interfaces, such as a keyboard, screen (e.g.. a touch screen), microphone, speaker, other input/output devices, and computing components, such as graphical processing units, serial ports, parallel ports, universal serial buses, and other input/output peripherals.
  • the I/O peripherals 1208 are connected to the processor 1202 through any of the ports coupled to the interface bus 1212.
  • the communication peripherals 1210 are configured to facilitate communication between the computer system 1200 and other computing devices over a communications network and include, for example, a network interface controller, modem, wireless and wired interface cards, antenna, and other communication peripherals.
  • Example 1 is an excitation optics system , comprising: a sample stage configured to support a sample; and an excitation module configured to transmit a light beam to the sample supported by the sample stage to cause the sample to emit fluorescence light that is detectable by an emission module, the excitation module comprising: one or more partial reflectors; two or more light sources arranged in a tree configuration based on positions of the one or more partial reflectors , wherein each light source emits light of a particular color; two or more collimating lenses corresponding to the two or more light sources; a tube lens; and an excitation objective configured to direct light to the sample, wherein a first partial reflector of the one or more partial reflectors is configured to allow light from a first light source of the two or more light sources and light from a second light source of the two or more light sources to traverse the same path to the excitation objective.
  • Example 2 is the excitation optics system of example(s) 1. wherein the two or more light sources are activated sequentially.
  • Example 3 is the excitation optics system of example(s) 1. wherein: wherein the two or more light sources are activated simultaneously; the first partial reflector of the one or more partial reflectors is configured to generate a first combined beam comprising light from a first light source of the two or more light sources and light from a second light source of the two or more light sources; and the excitation objective is configured to direct the first combined beam to the sample.
  • Example 4 is the excitation optics system of example(s) 3, wherein: a second partial reflector of the one or more partial reflectors is configured to generate a second combined light beam comprising the light from a third light source of the two or more light sources and the light from a fourth light source of the two or more light sources; a third partial reflector of the one or more partial reflectors is configured to generate a third combined light beam comprising the first combined beam and the second combined light beam; and the third combined light beam is transmitted to the sample by the excitation objective.
  • Example 5 is the excitation optics system of example(s) 1. wherein the excitation module further comprises a first mask, wherein: the light from the first light source traverses a first collimating lens of the two or more collimating lenses; and the light exiting the first collimating lens passes through the first mask, the first mask configured to define an illumination area on the sample.
  • Example 6 is the excitation optics system of example(s) 5. wherein the excitation module further comprises a bandpass filter configured to narrow a wavelength range of the light from the first light source, wherein the light passing through the first mask passes through the bandpass filter.
  • the excitation module further comprises a bandpass filter configured to narrow a wavelength range of the light from the first light source, wherein the light passing through the first mask passes through the bandpass filter.
  • Example 7 is the excitation optics system of example(s) 6, wherein the bandpass filter is oriented with an angle with respect to a plane perpendicular to a centerline of the first collimating lens, wherein the angled bandpass filter is configured to reflect fluorescence light emitted from the sample away from the centerline of the first collimating lens.
  • Example 8 is the excitation optics system of example(s) 5, wherein the excitation module further comprises a second mask, wherein: the light from the second light source traverses a second collimating lens of the two or more collimating lenses; the light exiting the second collimating lens passes through the second mask, the second mask configured to define the illumination area on the sample; the light from the first light source has a longer wavelength than the light from the second light source; and the first mask and the tube lens define a first mask-tube lens distance and the second mask and the tube lens define a second mask-tube lens distance, wherein the first mask-tube lens distance and the second mask-tube lens distance are configured to keep light from the first light source exiting the tube lens and light from the second light source exiting the tube lens in parallel.
  • the excitation module further comprises a second mask, wherein: the light from the second light source traverses a second collimating lens of the two or more collimating lenses; the light exiting the second collimating lens passes through the second mask, the second mask configured to
  • Example 9 is the excitation optics system of example(s) 1. wherein the excitation module further comprises a vertical turning mirror, wherein the vertical turning mirror is configured to maintain the plane of the sample stage perpendicular to a plane perpendicular to the centerline of the tube lens.
  • Example 10 is the excitation optics system of example(s) 1, wherein: the excitation objective is mounted on a vertical actuator; and the excitation objective is mounted on a lateral translator.
  • Example 11 is the excitation optics system of example(s) 1, wherein the one or more partial reflectors are dichroic mirrors.
  • Example 12 is the excitation optics system of example(s) 1, wherein the excitation module further comprises a photodiode coupled to an unused port of the first partial reflector, wherein the photodiode is configured to receive a leakage signal for calibration of the two or more light sources.
  • Example 13 is an emission optics system, comprising: a sample stage configured to support a sample; and an emission module configured to capture emitted fluorescence light from the sample when the sample is supported by the sample stage, comprising: an emission objective; one or more partial reflectors configured to generate a first transmitted beam and a first reflected beam ; and two or more image sensors, wherein each image sensor is configured to receive fluorescence light of at least one color emitted from the sample responsive to a light beam output from an excitation module, wherein the two or more image sensors are arranged in a tree configuration based on positions of the one or more partial reflectors.
  • Example 14 is the emission optics system of example(s) 13, wherein: the light beam output from the excitation module comprises fluorescence light of a first color emitted from the sample and fluorescence light of a second color emitted from the sample; the first transmitted beam includes the light of the first color; and the first reflected beam includes the light of the second color.
  • Example 15 is the emission optics system of example(s) 13, wherein the one or more partial reflectors are dichroic mirrors.
  • Example 16 is the emission optics system of example(s) 13, wherein at least one of the two or more image sensors further comprises a bandpass filter configured to narrow a wavelength of the fluorescence light emitted from sample prior to the at least one of the two or more image sensors.
  • Example 17 is the emission optics system of example(s) 16, wherein a range of the bandpass filter is configured to be within a transmission threshold of at least one of the one or more partial reflectors.
  • Example 18 is the emission optics system of example(s) 16. wherein a range of the bandpass filter is configured to include a transmission band resulting from a combination of two or more partial reflectors.
  • Example 19 is the emission optics system of example(s) 1 , wherein the at least one of the two or more image sensors further comprises a tube lens configured to receive light passing through the bandpass filter.
  • Example 20 is the emission optics system of example(s) 16, wherein the at least one of the two or more image sensors further comprises a tube lens configured to receive the fluorescence light emitted from the sample prior to the bandpass filter.
  • Example 21 is the emission optics system of example(s) 13, wherein the emission module further comprises one or more horizontal turning mirrors configured to reduce the footprint of the emission module by angling a path of the fluorescence light prior to reception by at least one image sensor.
  • Example 22 is the emission optics system of example(s) 13, wherein at least one of the two or more image sensors is a two-dimensional sensor.
  • Example 23 is the emission optics system of example(s) 13, wherein at least one of the two or more image sensors is a line sensor.
  • Example 24 is the emission optics system of example(s) 13, wherein the emission module further comprises a tube lens and one or more compensation plates configured to receive the fluorescence light emitted from sample, wherein a thickness of each compensation plate is equal to a thickness of the one or more partial reflectors through which the fluorescence light emitted by the sample has passed.
  • Example 25 is the emission optics system of example(s) 24, wherein the emission module further comprises at least two partial reflectors, wherein the at least two partial reflectors are configured with orthogonal tilt axes.
  • Example 26 is the emission optics system of example(s) 13, wherein the emission module further comprises a vertical turning mirror, wherein the vertical turning mirror is configured to maintain the plane of the sample stage perpendicular to the plane of the center of the first partial reflector.
  • Example 27 is the emission optics system of example(s) 13. wherein: the first transmitted beam comprises light of a first color and light of a third color; the first reflected beam comprises light of a second color and light of a fourth color; a second partial reflector of the one or more partial reflectors is configured to generate a second transmitted beam and a second reflected beam from the first transmitted beam, wherein the second transmitted beam comprises the light of the third color and the second reflected beam comprises the fluorescence light of the third color emitted from the sample; and a third partial reflector of the one or more partial reflectors is configured to generate a third transmitted beam and a third reflected beam, wherein the third transmitted beam comprises the light of the second color and the third reflected beam comprises the light of the fourth color.
  • Example 28 is a computer-implemented method, comprising: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.
  • Example 29 is the method of example(s) 28, wherein receiving the first imaging signal from the first image sensor occurs in parallel to receiving the second imaging signal from the second image sensor.
  • Example 30 is the method of example(s) 28, further comprising: outputting a second command to cause a capture of a low-resolution image of the sample; and outputting a third command to cause a focusing procedure of a portion of the low-resolution image.
  • Example 31 is the method of example(s) 28, further comprising: applying one or more image corrections to the composite multispectral image including at least one of dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction.
  • Example 32 is a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw 7 image; and outputting the composite multispectral image.
  • Example 33 is the non-transitory computer-readable medium of example(s) 32, wherein receiving the first imaging signal from the first image sensor occurs in parallel to receiving the second imaging signal from the second image sensor.
  • Example 34 is the non-transitory computer-readable medium of example(s) 32, further comprising: applying one or more image corrections to the composite multispectral image including at least one of dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction.
  • Example 35 is a system comprising: one or more processors; and one or more computer-readable storage media storing instructions which, when executed by the one or more processors, cause the one or more processors to perform operations including: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.
  • a computing device can include any suitable arrangement of components that provide a result conditioned on one or more inputs.
  • Suitable computing devices include multipurpose microprocessor-based computing systems accessing stored software that programs or configures the computing system from a general purpose computing apparatus to a specialized computing apparatus implementing one or more embodiments of the present subject matter. Any suitable programming, scripting, or other type of language or combinations of languages may be used to implement the teachings contained herein in software to be used in programming or configuring a computing device.
  • Embodiments of the methods disclosed herein may be performed in the operation of such computing devices.
  • the order of the blocks presented in the examples above can be varied — for example, blocks can be re-ordered, combined, and/or broken into sub-blocks. Certain blocks or processes can be performed in parallel.
  • Disjunctive language such as the phrase “at least one of X, Y, or Z,” unless specifically stated otherwise, is otherwise understood within the context as used in general to present that an item, term, etc., may be either X. Y, or Z, or any combination thereof (e.g., X, Y. and/or Z). Thus, such disjunctive language is not generally intended to, and should not, imply that certain examples require at least one of X, at least one of Y, or at least one of Z to each be present.
  • a or B or C includes any or all of the following alternative combinations as appropriate for a particular usage: A alone; B alone; C alone; A and B only; A and C only; B and C only; and all three of A and B and C.
  • based on is meant to be open and inclusive, in that a process, step, calculation, or other action “based on” one or more recited conditions or values may, in practice, be based on additional conditions or values beyond those recited.
  • use of “based at least in part on” is meant to be open and inclusive, in that a process, step, calculation, or other action “based at least in part on” one or more recited conditions or values may, in practice, be based on additional conditions or values beyond those recited. Headings, lists, and numbering included herein are for ease of explanation only and are not meant to be limiting.
  • any reference to a series of examples is to be understood as a reference to each of those examples disjunctively (e.g., "Examples 1-4" is to be understood as “Examples 1, 2, 3, or 4").
  • Example 1 is an excitation optics system, comprising: a sample stage configured to support a sample; and an excitation module configured to transmit a light beam to the sample supported by the sample stage to cause the sample to emit fluorescence light that is detectable by an emission module, the excitation module comprising: one or more partial reflectors; two or more light sources arranged in a tree configuration based on positions of the one or more partial reflectors, wherein each light source emits light of a particular color; two or more collimating lenses corresponding to the two or more light sources; a tube lens; and an excitation objective configured to direct light to the sample, wherein a first partial reflector of the one or more partial reflectors is configured to allow light from a first light source of the two or more light sources and light from a second light source of the two or more light sources to traverse the same path to the excitation objective.
  • Example 2 is the excitation optics system of the preceding or any of the subsequent example(s), wherein the two or more light sources are activated sequentially.
  • Example 3 is the excitation optics system any of the preceding or subsequent exampl e(s). wherein: wherein the two or more light sources are activated simultaneously; the first partial reflector of the one or more partial reflectors is configured to generate a first combined beam comprising light from a first light source of the two or more light sources and light from a second light source of the two or more light sources; and the excitation objective is configured to direct the first combined beam to the sample. 4.
  • a second partial reflector of the one or more partial reflectors is configured to generate a second combined light beam comprising the light from a third light source of the two or more light sources and the light from a fourth light source of the two or more light sources;
  • a third partial reflector of the one or more partial reflectors is configured to generate a third combined light beam comprising the first combined beam and the second combined light beam; and the third combined light beam is transmitted to the sample by the excitation objective.
  • Example 5 is the excitation optics system of any of the preceding or subsequent example(s), wherein the excitation module further comprises a first mask, wherein: the light from the first light source traverses a first collimating lens of the two or more collimating lenses; and the light exiting the first collimating lens passes through the first mask, the first mask configured to define an illumination area on the sample.
  • Example 6 is the excitation optics system of any of the preceding or subsequent example(s), w herein the excitation module further comprises a bandpass filter configured to narrow a wavelength range of the light from the first light source, wherein the light passing through the first mask passes through the bandpass filter.
  • the excitation module further comprises a bandpass filter configured to narrow a wavelength range of the light from the first light source, wherein the light passing through the first mask passes through the bandpass filter.
  • Example 7 is the excitation optics system of any of the preceding or subsequent example(s), wherein the bandpass filter is oriented with an angle with respect to a plane perpendicular to a centerline of the first collimating lens, wherein the angled bandpass filter is configured to reflect fluorescence light emitted from the sample away from the centerline of the first collimating lens.
  • Example 8 is the excitation optics system of any of the preceding or subsequent example(s), wherein the excitation module further comprises a second mask, wherein: the light from the second light source traverses a second collimating lens of the two or more collimating lenses; the light exiting the second collimating lens passes through the second mask, the second mask configured to define the illumination area on the sample; the light from the first light source has a longer wavelength than the light from the second light source; and the first mask and the tube lens define a first mask-tube lens distance and the second mask and the tube lens define a second mask-tube lens distance, wherein the first mask-tube lens distance and the second mask-tube lens distance are configured to keep light from the first light source exiting the tube lens and light from the second light source exiting the tube lens in parallel.
  • the excitation module further comprises a second mask, wherein: the light from the second light source traverses a second collimating lens of the two or more collimating lenses; the light exiting the second collimating lens passes through the second mask
  • Example 9 is the excitation optics system of any of the preceding or subsequent example(s), wherein the excitation module further comprises a vertical turning mirror, wherein the vertical turning mirror is configured to maintain the plane of the sample stage perpendicular to a plane perpendicular to the centerline of the tube lens.
  • Example 10 is the excitation optics system of any of the preceding or subsequent example(s), wherein: the excitation objective is mounted on a vertical actuator; and the excitation objective is mounted on a lateral translator.
  • Example 11 is the excitation optics system of any of the preceding or subsequent example(s), wherein the one or more partial reflectors are dichroic mirrors.
  • Example 12 is the excitation optics system of any of the preceding or subsequent example(s). wherein the excitation module further comprises a photodiode coupled to an unused port of the first partial reflector, wherein the photodiode is configured to receive a leakage signal for calibration of the two or more light sources.
  • Example 13 is an emission optics system, comprising: a sample stage configured to support a sample; and an emission module configured to capture emitted fluorescence light from the sample when the sample is supported by the sample stage, comprising: an emission objective; one or more partial reflectors configured to generate a first transmitted beam and a first reflected beam; and two or more image sensors, wherein each image sensor is configured to receive fluorescence light of at least one color emitted from the sample responsive to a light beam output from an excitation module, wherein the two or more image sensors are arranged in a tree configuration based on positions of the one or more partial reflectors.
  • Example 14 is the emission optics system of the preceding or any of the subsequent example(s). wherein: the light beam output from the excitation module comprises fluorescence light of a first color emitted from the sample and fluorescence light of a second color emitted from the sample; the first transmitted beam includes the light of the first color; and the first reflected beam includes the light of the second color.
  • Example 15 is the emission optics system of any of the preceding or subsequent example(s). wherein the one or more partial reflectors are dichroic mirrors.
  • Example 16 is the emission optics system of any of the preceding or subsequent example(s), wherein at least one of the two or more image sensors further comprises a bandpass filter configured to narrow a wavelength of the fluorescence light emitted from sample prior to the at least one of the two or more image sensors.
  • Example 17 is the emission optics system of any of the preceding or subsequent example(s), wherein a range of the bandpass filter is configured to be within a transmission threshold of at least one of the one or more partial reflectors.
  • Example 18 is the emission optics system of any of the preceding or subsequent example(s), wherein a range of the bandpass filter is configured to include a transmission band resulting from a combination of two or more partial reflectors.
  • Example 19 is the emission optics system of any of the preceding or subsequent example(s), wherein the at least one of the two or more image sensors further comprises a tube lens configured to receive light passing through the bandpass filter.
  • Example 20 is the emission optics system of any of the preceding or subsequent exampl e(s), wherein the at least one of the two or more image sensors further comprises a tube lens configured to receive the fluorescence light emitted from the sample prior to the bandpass filter.
  • Example 21 is the emission optics system of any of the preceding or subsequent example(s), wherein the emission module further comprises one or more horizontal turning mirrors configured to reduce the footprint of the emission module by angling a path of the fluorescence light prior to reception by at least one image sensor.
  • Example 22 is the emission optics system of any of the preceding or subsequent example(s), wherein at least one of the two or more image sensors is a two-dimensional sensor.
  • Example 23 is the emission optics system of any of the preceding or subsequent example(s). wherein at least one of the two or more image sensors is a line sensor.
  • Example 24 is the emission optics system of any of the preceding or subsequent example(s), wherein the emission module further comprises a tube lens and one or more compensation plates configured to receive the fluorescence light emitted from sample, wherein a thickness of each compensation plate is equal to a thickness of the one or more partial reflectors through which the fluorescence light emitted by the sample has passed.
  • Example 25 is the emission optics system of any of the preceding or subsequent example(s), wherein the emission module further comprises at least two partial reflectors, wherein the at least two partial reflectors are configured with orthogonal tilt axes.
  • Example 26 is the emission optics system of any of the preceding or subsequent example(s), wherein the emission module further comprises a vertical turning mirror, wherein the vertical turning mirror is configured to maintain the plane of the sample stage perpendicular to the plane of the center of the first partial reflector.
  • Example 27 is the emission optics system of any of the preceding or subsequent example(s), wherein: the first transmitted beam comprises light of a first color and light of a third color; the first reflected beam comprises light of a second color and light of a fourth color; a second partial reflector of the one or more partial reflectors is configured to generate a second transmitted beam and a second reflected beam from the first transmitted beam, wherein the second transmitted beam comprises the light of the third color and the second reflected beam comprises the fluorescence light of the third color emitted from the sample; and a third partial reflector of the one or more partial reflectors is configured to generate a third transmitted beam and a third reflected beam, wherein the third transmitted beam comprises the light of the second color and the third reflected beam comprises the light of the fourth color.
  • Example 28 is a computer-implemented method, comprising: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.
  • Example 29 is the method of the preceding or any of the subsequent example(s), wherein receiving the first imaging signal from the first image sensor occurs in parallel to receiving the second imaging signal from the second image sensor.
  • Example 30 is the method of any of the preceding or subsequent example(s), further comprising: outputting a second command to cause a capture of a low-resolution image of the sample; and outputting a third command to cause a focusing procedure of a portion of the low- resolution image.
  • Example 31 is the method of any of the preceding or subsequent example(s), further comprising: applying one or more image corrections to the composite multispectral image including at least one of dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction.
  • Example 32 is a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.
  • Example 33 is the non-transitory computer-readable medium of the preceding or any of the subsequent example(s). wherein receiving the first imaging signal from the first image sensor occurs in parallel to receiving the second imaging signal from the second image sensor.
  • Example 34 is the non-transitory computer-readable medium of any of the preceding or subsequent example(s), further comprising: applying one or more image corrections to the composite multispectral image including at least one of dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction.
  • Example 35 is a system comprising: one or more processors; and one or more computer-readable storage media storing instructions which, when executed by the one or more processors, cause the one or more processors to perform operations including: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.

Landscapes

  • Physics & Mathematics (AREA)
  • Spectroscopy & Molecular Physics (AREA)
  • General Physics & Mathematics (AREA)
  • Health & Medical Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Chemical & Material Sciences (AREA)
  • Optics & Photonics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Biochemistry (AREA)
  • General Health & Medical Sciences (AREA)
  • Immunology (AREA)
  • Pathology (AREA)
  • Investigating, Analyzing Materials By Fluorescence Or Luminescence (AREA)
  • Microscoopes, Condenser (AREA)

Abstract

Systems and methods for filter-based multispectral fluorescence microscope optimized for virtual staining techniques are provided. An example imaging system includes an excitation module, an emission module, and a sample stage configured to support a sample. The excitation module is configured to transmit light to the sample to cause fluorescence that is detectable by the emission module. The excitation module includes dichroic mirrors and light sources arranged in a tree configuration based on the positions of the dichroic mirrors. The emission module is configured to capture the emitted fluorescence from the sample. The emission module includes dichroic mirrors and image sensors, in which each image sensor is configured to receive fluorescence of a particular color emitted from the sample responsive to excitation light of a particular color and are arranged in a tree configuration based on the positions of the dichroic mirrors.

Description

FILTER-BASED MULTISPECTRAL FLUORESCENCE MICROSCOPE
OPTIMIZED FOR VIRTUAL STAINING
CROSS-REFERENCE TO RELATED APPLICATION
[0001] This international application claims priority to U.S. Patent Application No.
63/537.270, filed on September 8, 2023 and entitled FILTER-BASED MULTISPECTRAL FLUORESCENCE MICROSCOPE OPTIMIZED FOR VIRTUAL STAINING,” the disclosure of which is herein incorporated by reference in its entirety' for all purposes.
BACKGROUND
[0002] In a ty pical pathology workflow, tissue slides may be stained for use in white light brightfield microscopes or for fluorescent microscopy. Each stain may require a separate slide and each separate slide consumes available tissue volume. In cases where the tissue sample is small (e.g., needle biopsy), the number of available stained slides is accordingly limited. The requirement for multiple slides also adds to the time, money and labor required for a complete set of images.
[0003] A fluorescence microscope can be used to image fluorescence from dye-stained tissue samples or autofluorescence from unstained tissue samples. In the latter case, the natural fluorescence from tissue can be imaged, without the need to apply stains to any tissue samples. However, photobleaching of the tissue sample can occur following repeated exposure to excitation light, which can diminish the fluorescence signal.
BRIEF SUMMARY
[0004] Systems and methods for a filter-based multispectral fluorescence microscope optimized for virtual staining techniques are provided. An example imaging system includes an excitation module, an emission module, and a sample stage configured to support a sample. The excitation module is configured to transmit a light beam to the sample supported by the sample stage to cause the sample to emit fluorescence light that is detectable by an emission module.
The excitation module includes one or more partial reflectors; two or more light sources arranged in a tree configuration based on positions of the one or more partial reflectors , in which each light source emits light of a particular color; two or more collimating lenses corresponding to the two or more light sources; a tube lens; and an excitation objective configured to direct light to the sample, in which a first partial reflector of the one or more partial reflectors is configured to allow light from a first light source of the two or more light sources and light from a second light source of the two or more light sources to traverse the same path to the excitation objective.
[0005] The emission module is configured to capture emitted fluorescence light from the sample when the sample is supported by the sample stage, including an emission objective; one or more partial reflectors configured to generate a first transmitted beam and a first reflected beam ; and two or more image sensors, in which each image sensor is configured to receive fluorescence light of at least one color emitted from the sample responsive to a light beam output from an excitation module, wherein the two or more image sensors are arranged in a tree configuration based on positions of the one or more partial reflectors.
[0006] A system of one or more computers can be configured to perform particular operations or actions by virtue of having software, firmware, hardw are, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions. One general aspect includes a computer-implemented method. The computer-implemented method includes outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light. In this example method, each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module. The computer-implemented method also includes receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor. The computer-implemented method also includes generating a first raw image from the first imaging signal and generating a second raw image from the second imaging signal. The computer-implemented method also includes generating a composite multispectral image comprising the first raw image and the second raw' image and outputting the composite multispectral image. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods. BRIEF DESCRIPTION OF THE DRAWINGS
[0007] The accompanying drawings, which are incorporated into and constitute a part of this specification, illustrate one or more certain examples and, together with the description of the example, serve to explain the principles and implementations of the certain examples.
[0008] FIG. 1 illustrates an example imaging system in which techniques for implementing a filter-based fluorescent microscope for virtual staining are illustrated, according to at least one example.
[0009] FIG. 2 illustrates an example slide that may be imaged using the imaging system of FIG. 1, according to at least one example.
[0010] FIG. 3 shows a simplified schematic diagram of a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
[0011] FIG. 4 shows a simplified schematic diagram of an excitation module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
[0012] FIG. 5 shows a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
[0013] FIG. 6 shows a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
[0014] FIG. 7A depicts a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
[0015] FIG. 7B depicts a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
[0016] FIG. 8 A depicts a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure. [0017] FIG. 8B depicts a simplified schematic diagram of an emission module that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
[0018] FIG. 9A shows a relationship between certain components that may be used in one example emission module for a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
[0019] FIG. 9B shows another relationship between certain components that may be used in one example emission module for a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
[0020] FIG. 10 shows a simplified schematic of an example imaging system that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure.
[0021] FIG. 11 illustrates an example flowchart illustrating a process for operating a filterbased fluorescent microscope for virtual staining, according to at least one example.
[0022] FIG. 12 illustrates an example system for implementing techniques for filter-based fluorescent microscope for virtual staining, according to at least one example.
DETAILED DESCRIPTION
[0023] Examples are described herein in the context of a filter-based multispectral fluorescence microscope optimized for virtual staining techniques. Those of ordinary skill in the art will realize that the following description is illustrative only and is not intended to be in any way limiting. For example, the components of the optical systems described herein can be configured in various ways to achieve the desired imaging capabilities. Reference will now be made in detail to implementations of examples as illustrated in the accompanying drawings. The same reference indicators will be used throughout the drawings and the following description to refer to the same or like items.
[0024] In the interest of clarity, not all of the routine features of the examples described herein are shown and described. It will, of course, be appreciated that in the development of any such actual implementation, numerous implementation-specific decisions must be made in order to achieve the developer’s specific goals, such as compliance with application- and business-related constraints, and that these specific goals will vary from one implementation to another and from one developer to another. [0025] The techniques described herein may be used for, among other things, manufacture, construction, or operation of filter-based multispectral fluorescence microscopes that are optimized for virtual staining techniques. For example, a conventional pathology workflow typically involves applying a number of stains to a number of different tissue sections. The stained sections can be examined using white light brightfield microscopes or fluorescent microscopes. In these cases, (1) for some applications there may be insufficient tissue available for all of the desired stains, and (2) the cost of the tissue samples, reagents, and labor associated with staining may be prohibitively high. With respect to fluorescent microscopy in particular, the stained tissue sections samples can degrade from repeated measurements (photobleaching). An approach to fluorescence microscopes with a reduced reliance on permanently staining tissue samples can address these issues.
[0026] Some examples may thus use virtual staining techniques to increase the information that can be obtained from a single, unstained tissue section. In virtual staining, a multispectral fluorescence image is first obtained from an unstained section via autofluorescence, or the natural emission of light by the tissue sample following the absorption of excitation light without the application of a specific fluorescent stain. A trained machine learning (ML) model is used to output a set of predicted “virtual” stains that may cause an output that mirrors that which would be produced by actual staining of tissue samples. The tissue section is left unstained and largely unaltered, leaving it available for subsequent staining or other types of analysis. However, using virtual staining in concert with a fluorescence microscope is a highly constrained operation that involves at least several challenges. First, the microscope should be configured to scan the entire tissue sample in a relatively short amount of time without compromising the amount of useful spectral information that is obtained. Second, the microscope should be sufficiently stable to generate consistent data quality without requiring excessive maintenance. Third, high spatial resolution and image sharpness directly correspond to virtual staining accuracy. Fourth, the microscope should not acquire an excessively large number of spectral channels, compared with the density7 of spectral features found in the autofluorescence data, for the data storage and processing to be practical.
[0027] The techniques described herein may be used to implement a widefield. filter-based fluorescent microscope for virtual staining that solves the problems mentioned above. For example, using the techniques described herein, a fluorescence microscope can be constructed that has the requisite speed. The choice of optical components and associated control software lend themselves to widefield scanning with numerous excitation-emission combinations (i.e., spectral channels) in periods of time significantly shorter than is possible using existing techniques.
[0028] In addition, using the techniques of the present disclosure for a given excitation wavelength, all of the emission wavelengths are measured in parallel, dramatically reducing the potential for photobleaching. In some existing systems the emission wavelengths must be measured serially resulting in many more repeated exposures of the sample, which can cause significant photodegradation.
[0029] The techniques of the present disclosure are significantly more reliable as compared with some existing systems. In some existing systems, moving parts subject to mechanical breakdown may include a translation stage holding the sample, a slide loader for moving slides from storage onto the translation stage, a focusing mechanism for the emission objective, and a focusing stage for the excitation objective. Compared with existing systems, a microscope constructed according to the systems described herein, does not involve the movement of filter wheels for each measurement, due to the selection and positioning of optical components. Thus, vast numbers of mechanical filter wheel motions are resulting in a longer lifetime.
[0030] In two illustrative examples, a widefield, filter-based fluorescent microscope for virtual staining includes an excitation optics system and an emission optics system. The excitation optics system includes a sample stage configured to support a sample. For example, the sample may be a tissue section placed upon a glass slide and optionally covered with a glass cover slip. The excitation optics system also includes an excitation module that is configured to transmit a light beam to the sample to cause the sample to emit fluorescence light that is detectable by an emission module.
[0031] In some examples, the excitation light sources can be pulsed or activated sequentially, such that a single light source is used to illuminate the sample at a time. In this example, the arrangement of partial reflectors can be used to cause the individual light beams to share paths to the sample (i.e., overlap in space), but not to overlap in time. In other examples, multiple light sources can be used to illuminate the sample at a time. In that case, a combined light beam can be used to excite the sample, which may include one or more combined or uncombined, collimated light-emitting diode (LED) narrowband light beams, with narrowband wavelength ranges selected to cause autofluorescence of the sample.
[0032] The excitation module includes one or more partial reflectors such as dichroic mirrors. A dichroic mirror is an optical component that transmits light at certain wavelengths and reflects light at other wavelengths. Thus, the dichroic mirror can act as either a light beam splitter or be used to combine two light beams. In this example, a first dichroic mirror of the one or more dichroic mirrors is configured to generate a first combined beam comprising light from at least a first light source and light from a second light source, the two light sources generating light of different wavelengths. The dichroic mirror can similarly be used to direct light such that light from different sources share the same optical path but traverse the path at different times.
[0033] Although the present disclosure describes examples of light beam combining and splitting using dichroic mirrors, one of ordinary skill in the art will recognize that other components may be used to accomplish similar design objectives. For instance, in some examples, 50/50 beam splitters may be used for beam splitting in lieu of dichroic mirrors. Likewise, fluorescence filters can be used to split an incoming light beam into two light beams of approximately equal intensity. Other components may also be used.
[0034] The excitation module includes two or more light sources. In some examples, LED light sources with wavelength ranges chosen for autofluorescence may be used. The light sources - each light source emitting light of a particular color (i.e., a narrow range centered on a particular wavelength) - are arranged in a tree configuration based on positions of the one or more dichroic mirrors. One of ordinary skill in the art will appreciate that the word “wavelength” used in reference to a light source refers to a characteristic wavelength. All light sources produce a distribution of wavelengths, for example, a narrow band w ith a central peak intensity that falls off rapidly.
[0035] For example, one tree configuration may include two “branches.” The first branch may itself include tw o branches, or sub-branches. Each sub-branch of the first branch may include a light source of a particular color. A dichroic mirror can be used to establish overlapping beam paths, for a single light source in operation, and to combine or overlap multiple light beams for multiple, simultaneous light source operation. For example, the second branch may also include a light source of yet another color. A dichroic mirror can combine the light beams of the first and second branches such that the transmitted beam includes three overlapping light beams (two from the first branch and one from the second branch). In another example, the light source from each sub-branch or branch may be pulsed to illuminate the sample sequentially.
[0036] Branches may also include turning mirrors to allow for branches to be oriented in orthogonal planes. The tree configuration may thus be used to reduce the footprint of the optical setup, which would otherwise occupy space in a single 2D plane. The tree configuration can include an arbitrary number of branches, although in practical implementations, the number of branches will be constrained by the available space.
[0037] The excitation module also includes two or more collimating lenses corresponding to the two or more light sources. For example, LED light sources may be immediately followed by collimating lenses to cause the divergent light from the LED sources to arrive uniformly and parallel at the sample. The light beam then passes through a tube lens prior to an excitation objective lens. The excitation objective is configured to direct the first combined beam to the sample. In examples in which two or more light sources are operated simultaneously, a combined beam including light from multiple branches passes through the excitation objective.
[0038] Turning now to the emission optics system, the emission optics system includes an emission module configured to capture emitted fluorescence light from the sample when the sample is supported by the sample stage. The emission module includes a collection objective for receiving the autofluorescence light emitted from the sample. The emission module includes one or more partial reflectors such as dichroic mirrors in analogy7 to the excitation module, in which a first dichroic mirror of the one or more dichroic mirrors is configured to generate a transmitted beam and a reflected beam. The transmitted beam includes at least autofluorescence light of a first color emitted from the sample and the reflected beam includes at least autofluorescence light of a second color emitted from the sample.
[0039] The emission module includes two or more image sensors. Each image sensor is configured to receive fluorescence light of a particular color emitted from the sample responsive to a light beam output from the excitation module. Like the excitation module, the two or more image sensors are arranged in a tree configuration based on positions of the one or more dichroic mirrors as well as turning mirrors.
[0040] For example, one tree configuration of the emission module may include two “branches” corresponding to the two branches of the excitation module. The first branch may itself include two sub-branches. Each sub-branch of the first branch may include an image sensor for a particular color corresponding to the color of light emitted from the sample. Likewise, the second branch may also include an image sensor corresponding to a particular color corresponding to another color emitted from the sample. In this example, two dichroic mirrors can be used to split the combined light beams from the branches of the excitation module (e g., two from a first branch and one from a second branch). In other examples in which a single light source is operated at a time, the dichroic mirrors can be used to direct the light beam to the appropriate image sensor. Branches or sub-branches may also include turning mirrors to place branches in orthogonal planes. The tree configuration may thus be used to reduce the footprint of the optical setup, which would otherwise occupy space in a single 2D plane. The tree configuration can again include an arbitrary number of branches.
[0041] In another illustrative example, a computer-implemented method for operating a widefield, filter-based fluorescent microscope for virtual staining includes a computing device outputting a command to cause one or more light sources arranged in a tree configuration, as described above, to emit light. Each light source emits light of a particular color. In some examples, one light source may be used to illuminate the sample at a time, while in other examples, multiple light sources may be used simultaneously to illuminate the sample. For instance, the light sources can be activated sequentially or in a pulsed fashion or they can all be activated simultaneously or simultaneously in subsets.
[0042] Because fluorescence is spectrally continuous and broadband, sets of colors that are eligible for imaging may be identified. Some images are not meaningful fluorescence images because (1) the wavelength of fluorescence light is generally longer than the wavelength of excitation light or (2) excitation light may be too bright and hide the resulting fluorescence. The latter may occur when the wavelength of the fluorescence overlaps with the excitation wavelength. In some cases, the excitation light may be considerably brighter than the fluorescence, so the fluorescence signal cannot be detected. Thus, excitation wavelengths and fluorescence wavelengths may be selected such that the fluorescence wavelength is longer than excitation wavelength for optimized imaging outcomes.
[0043] For instance, for violet excitation light, blue, green, yellow, or red fluorescence images may be obtained. In contrast, for green excitation light, yellow or red fluorescence may be obtained. A blue image may not contain any detectable fluorescence signal as the detection wavelength is shorter than excitation light. A green image may be difficult to obtain since excitation light may make the faint fluorescent signal invisible.
[0044] The computing device receives, from two or more image sensors arranged in a tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor. For instance, the image sensors may be selected or configured to receive the red and green emitted fluorescence light, either simultaneously or in sequence. Raw images are generated using the first imaging signal and second imaging signal. From these raw images, the computing device generates a composite multispectral image and outputs the composite multispectral image. [0045] These illustrative examples are given to introduce the reader to the general subject matter discussed herein and the disclosure is not limited to this example. The following sections describe various additional non-limiting examples relating to a filter-based multispectral fluorescence microscope optimized for virtual staining techniques.
[0046] Turning now to the figures, FIG. 1 illustrates an example imaging system 100 in which techniques for a filter-based fluorescent microscope for virtual staining system may be implemented, according to at least one example. The imaging system 100 includes a microscope 102 and a computer system 104. The microscope 102 includes a camera 106, a filter 108, a tube lens 110, an objective 112, a lateral stage 114, and an illumination system 116 including an illumination source 118 and one or more lenses 120. The imaging system 100 may be configured to image a sample 122. which in the illustrated example, may be held on a slide, such as a glass slide. In some examples, filters 108 may also be included in the imaging system 100 to achieve design objectives. Filters 108 may be used both before and after the sample 122 is illuminated. The imaging system 100 images the exit face of a light guide to a plane close to the sample, but other configurations are possible.
[0047] For instance, imaging system 100 can be configured to be a fluorescent microscope in which illumination source 118 is used to excite fluorescent stains applied to the sample 122. In some examples, masks (at a plane conjugate to the sample surface) may be used to limit the illumination area and reduce photobleaching. Imaging system 100 is thus an example of a simple fluorescent microscope that includes an excitation module with a single light source and an emission module with a single image sensor, shown here to illustrate certain components and concepts. FIGs. 3-8B and 10 depict excitation modules and emission modules with multiple light sources and image sensors, respectively, in accordance with some aspects of the present disclosure.
[0048] The camera 106 may be any suitable device that includes at least one image sensor. In some examples, the camera 106 may have any suitable range of resolution and be capable of imaging any suitable wavelength of light. In some examples, the camera 106 may be suitable for imaging fluorescent wavelengths, though it may also image in other wavelengths.
[0049] The filter 108 may be any suitable filter capable of altering the characteristics of the light that is seen by the sensor(s) of the camera 106 in accordance with the design objectives of the imaging system 100. Thus, in some examples, the filter 108 may be selected to enable the camera 106 to capture images of different wavelengths of light. [0050] In some examples, the objective 112 may be an infinity-corrected objective together with the tube lens 110. In some examples, the objective 112 may be corrected for a fixed tube length (e.g., 160 mm) without a separate tube lens 1 10. Various arrangements of these and other components are possible to accomplish various design and operational objectives. For instance, in some examples, the objective 112 may be moved along the optical Z axis for focusing.
[0051] The lateral stage 114 may be configured to retain the sample 122. The lateral stage 114 may also be configured for movement in multiple axes with respect to the camera 106. For example, the lateral stage 1 14 may be moved in an X direction and a Y direction (e.g., front to back and side to side along a plane parallel to the surface of the lateral stage 114 on which the sample 122 is held). In some examples, the lateral stage 114 may also be moveable in a Z direction (e.g., vertically with respect to the objective 112). Movement of the lateral stage 114 may be manual or automated. In a manual example, a set of knobs and gears may be used to move the lateral stage 114. Sensors and/or readouts, may be connected to the lateral stage 114 in the manual example to provide a user with positional feedback of the relative position of the lateral stage 114. In an automated example, a set of servo motor actuators (or other automated mechanisms) may be coupled with the lateral stage 114 and electrically coupled with a controller. The controller may provide signals to the servo motor actuators to control movement of the servo motor actuators. In some examples, the controller may be included, or otherwise be, the computer system 104. For example, the computer system 104 may provide electric signals to the servo motor actuators to cause the servo motor actuators to move the lateral stage 114 to certain positions. In some examples, sensors and/or readouts may be connected to the lateral stage 114 to provide positional information. In some examples, the positional information, either collected from the sensors or derived specifically from the servo motor actuators, may be used to perform the techniques described herein. For example, the positional information may be used in place of image registration to determine the actual shifts of the motorized lateral stage 114 between a set of images.
[0052] The illumination source 118 and the one or more lenses 120 of the illumination system 116 may be configured to provide light for imaging the sample 122. In some examples, properties of the illumination source 118 and the one or more lenses 120 may be adjusted to achieve the particular imaging objectives of the system. For example, the illumination source 118 and the one or more lenses 120 may be selected to provide fluorescent illumination of the fluorescent stains applied to the sample 122. The excited stains may emit light at a longer wavelength than illumination source 118. The emitted light can be separated by a spectral emission filter, and then imaged, for example using camera 106, to create high-contrast pictures of the sample. In this configuration, imaging system 100 can image specific structures or molecules within the sample, which have been marked with the fluorescent stains.
[0053] In some examples, imaging system 100 can be configured to utilize the autofluorescence properties of the sample 122 for virtual staining. In virtual staining, a multispectral fluorescence image is first obtained from an unstained sample 122 via autofluorescence, or the natural emission of light by the tissue sample following the absorption of light without the application of a specific fluorescent stain. A trained ML model is used to predict a desired set of stains from the autofluorescence data. The tissue section is left unstained and largely unaltered, leaving it available for subsequent staining or other types of analysis.
[0054] The computer system 104 may be any suitable computing device including a desktop computer, a server computer, a tablet, a laptop computer, a microprocessor and coupled memory, and any other suitable combination of the foregoing. In some examples, the computer system 104 may be integrally formed with the imaging system 100 and may include one or more ports for input/output components, such as a display, keyboard, keypad, mouse, and the like. In some examples, the computer system 104 may be configured to control the operation of the microscope 102 and/or perform techniques described herein relating to computing detection profiles. In some examples, the computer system 104 outputs information relating to the imaging system 100 (e.g., image data, state data for light the illumination system 116, positional information of the lateral stage 114, and any other suitable information), which is then processed by a different computer system for performing the techniques described herein. An example of components that may be included in the computer system 104 is shown in FIG. 12.
[0055] FIG. 2 illustrates an example slide 200 that may be imaged using the imaging system 100 of FIG. 1, according to at least one example. The slide 200 may be formed from any suitable material including, for example, glass, plastic, quartz, and any other suitable material. The slide 200 may have any suitable shape including rectangular, square, oval, round, and the like. The slide 200 may have any suitable dimension, which may be standard (e.g., 75 mm by 25 mm) or non-standard. The slide 200 may also be used with a cover slip or cover glass (not shown) to help retain a sample 202 on a slide surface 204 of the slide 200.
[0056] On the slide surface 204 is included a sample 202. The sample 202, which is an example of the sample 122, may be any suitable object to be imaged by the imaging system 100. In some examples, the sample 202 may be a tissue sample that has been obtained from a subject (e.g., a human). The sample 202 may include one or more distinct features 208 and 210. The feature 208 (e.g., a first feature) and the feature 210 (e.g., a second feature) may represent aspects of the sample 202 that are distinct or otherwise more highly visible under certain conditions. In some examples, these conditions may be within the visible wavelength or may be within a non- visible wavelength. In some examples, these aspects of features 208 and 210 may be surfaced, identified, highlighted, or otherwise imaged using the techniques described herein.
[0057] FIG. 2 includes an X axis 212 and a Y axis 214 extending parallel to the slide surface 204 of the slide 200. The X axis 212 and the Y axis 214 are included for illustrative purposes to explain how the position of the slide 200 may be moved (e.g., on the lateral stage 1 14) to capture images of the slide 200 at different positions. The slide 200 is illustrated at a centered position depicted by a center marker 216 (e.g., centered at an origin of the X axis 212 and the Y axis 214). Thus, at the centered position (0. 0), an image captured of the slide 200 would be focused and centered on the center marker 216. While the stage and the slide 200 are moved, orientation and position of the camera remain fixed on the center marker 216.
[0058] A number of other positional markers 218 are also illustrated in FIG. 2. A few of these positional markers 218 are also labeled. The positional markers 218 are illustrative of other positions to which the slide can be moved relative to the camera. For example, the slide 200 can be moved laterally along the Y axis 214 to positional marker 218(1), e.g., (0, +y) and 218(3), e.g., (0, -y). The slide 200 can also be moved laterally along the X axis 212 to positional markers 218(2), e.g., (+x, 0) and 218(4), e.g., (-x, 0). In some examples, the slide 200 can also be moved in both the X and Y directions, e.g., as illustrated by positional markers 218(5) and 218(6), while remaining on the slide surface 204 (e.g.. 218(5) and off of the slide surface 204 (e.g.. 218(6)). While a few positional markers are illustrated, it should be understood that the slide 200 may be moved to any suitable position relative to the camera.
[0059] As the positional markers 218 are included for illustrative purposes, these markers are not to scale. In practice, the image shifts between positions may be much smaller than those illustrated by the positional markers 218. For example, the camera field of view may be much smaller than the sample, such that movements of the field of view remain on the sample 202. In this example, if estimating just a single detection profile, the image shift values (e.g., distances between positions) may be smaller than the camera field of view. Thus, the image shift points would be confined to some small region within the sample 202. When estimating a background profile, another set of points outside of the sample may be obtained, but still within the slide (e.g., such as illustrated by the positional marker 218(5). While the features 208 and 210 are illustrated as bright or dark features, such is not required to perform the techniques described here. For example, some tissue texture may be needed to perform image registration, but beyond that the calibration results may be most accurate if the tissue brightness is relatively uniform across the camera field of view.
[0060] In some examples, one or more images may be captured in each position. The positions may be predetermined, randomly assigned within some bounding, and/or any suitable combination of the foregoing. In some examples, the positions may include larger displacements and smaller displacements. The smaller displacements may be suitable for capturing information on structure of the sample 202 and the larger displacements may be suitable for generating information useful for determining large scale variation of the imaging system. In a particular example, a set of positions, in pixel units (x, y) may include {(0, 0), (0, 118), (0, -118), (118, 0), (-118, 0), (0, 1000), (0. -1000). (1000, 0), (-1000, 0)}.
[0061] Turning next to FIG. 3. FIG. 3 shows a simplified schematic diagram of a filter-based fluorescent microscope 300 for virtual staining, according to some aspects of the present disclosure. FIG. 3 illustrates an example filter-based fluorescent microscope 300 for virtual staining in a typical configuration. FIGs. 4-8B and FIG. 10 include depictions of filter-based fluorescent microscopes for virtual staining in various other configurations in accordance with differing design parameters and considerations.
[0062] In microscope 300, several components appear multiple times but are only labeled and described once for clarity. For example, microscope 300 includes 4 LED light sources in a tree configuration. For clarity, only LED 305 is described and labeled, but each “branch” of the “tree” is substantially the same except for spectral parameters determined by the wavelength of the light associated with each branch. Similarly, microscope 300 includes 8 image sensors (e.g., cameras), each on a sub-sub-branch of two main branches, but only image sensor 350 is labeled.
[0063] Example microscope 300 includes excitation module 301 and emission module 302. Excitation module 301 includes two or more light sources, like LED 305. Excitation module 301 can use partial reflectors such as dichroic mirrors 325. 330 to direct the light from multiple LEDs 305 to a common optical path. In some examples, the LEDs 305 may be operated simultaneously to generate a combined light beam. The wavelength of each LED can be chosen to excite the sample at a different wavelength. Example microscope 300 shows 4 LEDs 305, but an arbitrary' number of LEDs 305 may be used, subject to physical space constraints. In some typical examples, between 4 and 7 LEDs 305 may be used. [0064] Dichroic mirrors 325, 330 may be used to configure excitation module 301 in a tree configuration. Dichroic mirror 325 or 330 is a mirror with significantly different reflection or transmission properties at two different wavelengths. When light from two LEDs 305 is received by dichroic mirror 325 or 330, it will reflect light of one wavelength while allowing light of the other wavelength to pass through.
[0065] In some examples, light from both LEDs 305 can thus be combined into a single beam without significantly disturbing the individual characteristics of the light from each LED 305. In the simplest case, single dichroic mirror 330 is used to combine the light from two branches, each branch including one LED 305. Each additional dichroic mirror 325 splits a branch (or subbranch) to create two sub-branches. Sub-branches can be further split using additional dichroic mirrors 325, and so on.
[0066] However, during simultaneous operation, some LEDs 305 may interfere with the sensitivity or resolution of the image sensors 350s in the emission module 302. Thus, in a typical configuration, the LEDs 305 may be operated sequentially or pulsed. For instance, in an example, the shorter wavelength LEDs 305 (e.g., blue) are pulsed followed by a period for data collection using all or most image sensors 350. In the next step, a longer wavelength LED 305 (e.g., yellow) can be pulsed followed by data collection by a smaller subset of image sensors 350. Finally, the longest wavelength LED 305 (e.g., red) may be pulsed followed by data collection using only a single image sensor 350. This sequence reflects that emitted light is typically at a longer wavelength compared with the excitation light wavelength.
[0067] In some examples, turning mirrors (not shown) may be used to lift a branch or subbranch out of the starting plane of excitation module 301 or to introduce other non-functional turns. The combination of dichroic mirrors 325, 330 and turning mirrors may be used to configure the tree configuration of excitation module 301 in a compact three-dimensional shape, thus minimizing the footprint of microscope 300 or otherwise adapting the physical dimensions of microscope 300 to meet physical space requirements.
[0068] Excitation module 301 includes collimating lens 310. Collimating lens 310, tube lens 320, and excitation objective 335 map LED 305 structure into the angular distribution of light at sample 340. This ty pe of configuration is sometimes referred to as Kohler illumination. In Kohler illumination, the divergent light from LED 305 source is collimated and uniformly distributed at the sample and is not affected by the physical structure of LEDs 305 itself. This ensures that the final image will not include the structure or arrangement of LED 305 sources. [0069] Excitation module 301 includes photodiode 315. The photodiode 315 may be positioned at the unused port of dichroic mirror 330. Dichroic mirror 330 is the one used to join the two main branches of the tree configuration of excitation module 301 . In some examples, two light beams, themselves each possibly a combination of two or more light beams from two or more light sources 305, are combined at dichroic mirror 330, producing light beam 322 for exciting sample 340. In some examples, during sequential or pulsed operation of the light sources 305, only a single, monochromatic light beam 322 is directed to the sample 340.
[0070] The unused port of dichroic mirror 330 refers to the location orthogonal to light beam 322 that receives neither reflected nor transmitted light. Because dichroic mirror 330 does not perfectly transmit or reflect all incident light, a leakage signal proportional to LED 305 powers can be received by photodiode 315. The leakage signal received by photodiode 315 can be used for continuous calibration of the excitation LED 305 sources or to monitor the stability of excitation module 301.
[0071] Excitation module 301 includes tube lens 320. The light beam 322 reflected/transmitted from dichroic mirror 330 next passes through tube lens 320. Tube lens 320 may form an image of LEDs 305 at the back focal plane of or onto excitation objective 335. Excitation objective 335 in turn focuses or collimates the light onto sample 340 for excitation.
[0072] Light arriving at the sample "excites" sample 340. Excitation refers to the process whereby light energy is absorbed by an atom or molecule of sample 340, raising it from its ground state to a higher energy, or excited, state. For the particular atoms and molecules making up the biological tissue of sample 340. the absorbed energy corresponds to incident light of a specific wavelength. Typically, the excited atom or molecule will '‘relax” back to a ground state after a very short time (e.g., nanoseconds). During relaxation, energy is released in the form of light. This emitted light may be at a longer wavelength than the incident light. This cycle of excitation and relaxation is usually referred to as fluorescence.
[0073] The emitted light is received by emission objective 345 of emission module 302. Emission objective 345 may be an infinity-corrected lens that collects the fluorescence from the sample and outputs a collimated, light beam 347. Collimated, light beam 347 is received by dichroic mirror 365, in this case acting as a beam splitter. Collimated in this context refers to a collection of parallel light beams, although not necessarily parallel to the optical axis.
[0074] Emission module 302 is depicted as having 2 main branches with 2 sub-branches each. Each sub-branch also has two sub-sub-branches, for a total of 8 sub-sub-branches and 8 corresponding image sensors 350. The number of image sensors 350 does not necessarily match the number of LED 305 sources because some wavelengths of excitation light may cause more than one emitted wavelength of light. In one branch, second 370 and third dichroic mirrors 360 are used to further split or direct light beam 347.
[0075] Each terminal branch (e.g., sub-sub-branches ending with an image sensor 350) of emission module 302 includes tube lens 355. For an infinity corrected emission objective 345, tube lens 355 can form an image of sample 340 fluorescence in the plane of image sensor 350 at the end of the branch.
[0076] Image sensor 350 may be any type of image sensor or camera suitable for the gathering of emitted light at the target wavelengths and desired resolution. For example, image sensor 350 may be a charge-coupled device (CCD), a complementary metal-oxide-semiconductor (CMOS) sensor, or a photomultiplier tube (PMT). among others. In a typical implementation, image sensor 350 is a scientific-grade CMOS sensor. A scientific-grade CMOS sensor may provide significantly improved performance characteristics that make them suitable for high-precision scientific applications, like fluorescence microscopy.
[0077] Turning next to FIG. 4, FIG. 4 shows a simplified schematic diagram of excitation module 400 that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure. FIG. 4 illustrates a detailed view' of excitation module 301 introduced in FIG. 3, including additional components such as mask 412 that will be discussed below. Excitation module 400 may be used in conjunction with an emission module similar to examples 500, 600, 700. and 800 from FIGs. 5-8B as discussed below. In excitation module 400, several components appear multiple times but are only labeled and described once for clarity.
[0078] As with example microscope 300 of FIG. 3, excitation module 400 includes light sensor 405 that may be an LED light source. Light from one or multiple light sources 405 maybe directed or combined using a tree configuration through a combination of partial reflectors, such as dichroic mirrors, and turning mirrors. In excitation module 400, two main branches are shown. with tw o sub-branches each. Excitation module 400 is depicted in a 2D plane, but a branch or sub-branch may include a turning mirror that effectively rotates the branch (and included sub-branches) into an orthogonal plane. Thus, dichroic mirrors can be used as needed to split branches and turning mirrors can be used to rotate branches (or sub-branches) through rotations. Typically, branches or sub-branches are rotated 90 degrees but configurations using other angles are possible. Excitation module 400 including rotated branches can be compactly organized with a minimum of wasted space between the optical components, thus significantly reducing the footprint of the assembled microscope.
[0079] In example excitation module 400. emitted light from light source 405 may be followed by collimating lens 410, similar to collimating lens 310 of FIG. 3. Collimating lens 410 can map the light source structure into the angular distribution of light beam 470 arriving at sample 440, effectively producing a ty pe of Kohler illumination.
[0080] In some examples, collimating lens 410 is followed by mask 412. Mask 412 can define the illumination area on sample 440. In a typical configuration mask 412 is a rectangular aperture, but other mask 412 shapes are possible. In the absence of mask 412, the exit aperture of collimating lens 410 effectively acts as the mask for the light emitted from collimating lens 410. While widefield illumination (i.e., uniform illumination of sample 440) does not require diffraction-limited imaging performance, mask 412 is used to limit the illumination area of sample 340 to a region not much larger than the imaging field of view, as defined by the emission optics. Limiting the illumination area in this way can maximize contrast and avoid excessive photobleaching of sample 440.
[0081] In some examples, collimating lens 410 or mask 412 is followed by bandpass filter 414. Bandpass filter 414 may be used to limit the wavelength range of the excitation light, to avoid any overlap between the excitation and emission spectra. In some examples, bandpass filter 414 can be tilted, so that sample 440 fluorescence inadvertently collected by the excitation optics that reflects off of the filters is not able to return back to sample 340 and on to the image sensors of emission module 302. The tilt of bandpass filter 414 can be chosen to maintain the requisite transmission efficiency of bandpass filter 414 and other spectral characteristics.
[0082] Excitation module 400 includes photodiode 415 and tube lens 420 that are substantially similar to the complementary' components depicted in FIG. 3. Likewise, dichroic mirror 425 is used to allow monochromatic light beams from light sources 405 on a sub-branch to traverse the same optical path. Dichroic mirror 430 similarly allows light beams 450 and 460 to traverse the same optical path. Light beam 470 illuminates the sample 440 via excitation objective 435, similar to the description of FIG. 3. The light sources 405 can be activated sequentially' or in a pulsed fashion or they can all be activated simultaneously or simultaneously in subsets.
[0083] In some examples, excitation objective 435 may require some working distance between excitation objective 435 and sample 440 to avoid mechanical collisions with sample 440. Therefore, an aspheric lens may be used for excitation objective 435 that can correct for spherical aberrations while taking up less space. However, a single-element lens may have a large chromatic focal shift. Requiring excitation objective 435 to be moved to a different position for each excitation wavelength, may slow scanning unacceptably.
[0084] Therefore, the upstream optics of excitation module 400 can be configured to keep all of the excitation profiles (i.e., different wavelengths) simultaneously in focus. The chromatic focal shift caused by excitation objective 435 (e.g., a single-element aspheric lens) can be compensated for by selecting an appropriate distance between fixed tube lens 420 and the variable mask 412 position. For example, for a simple example excitation module 400 including 2 LEDs 405 with wavelengths 365 mm and 450 nm, tube lens 420 focal length of 200 mm, and excitation objective 435 with an effective focal length of 16 mm, a set of mask-tube lens distances providing simultaneous focusing of the 2 LED masks is 132 and 165 mm, respectively.
[0085] FIG. 4 correspondingly depicts significantly different mask-tube lens distances for light beams 450 and 460, where light beam 450 has a longer wavelength than light beam 460. The mask-tube distance for light beam 450 is longer, which can result in simultaneous focusing of the light beams 450 and 460 by excitation objective 435 at sample 440. In some examples, the light beams from the various light sources 405 can become superimposed after excitation objective 435 as a result of the excitation objective’s 435 chromatic aberration. This superposition may occur when the distance between tube lens 420 and excitation objective 435 is equal to the focal length of tube lens 420. In this configuration, the size and shape of mask 412 for each respective light source 405 can be identical.
[0086] In some examples, the excitation module 400 may include turning mirror 445 for configurations in which the excitation optics he in a horizontal plane, while the light beam is directed vertically towards sample 440. However, as discussed above, a combination of dichroic mirrors and turning mirrors can be used to construct a tree configuration in 3D space, resulting in significant flexibility in the orientation of all of the optical components. For example, the excitation optics could be arranged in a vertical plane or a nonplanar configuration.
[0087] In some examples, excitation objective 435 may be mounted on a vertical actuator for focusing using manual manipulation, software, or electronics. The vertical actuator (not shown) may be a piezo motor, a conventional motor, or other suitable component for affecting vertical or horizontal motion of excitation objective 435. Excitation objective 435 may also be mounted on a 2D lateral translator (not shown). The lateral translator can be adjusted using manual manipulation, software, or electronics to enable alignment of mask 412 image relative to the detection field of view in the detection optics. [0088] Turning next to FIG. 5, FIG. 5 shows a simplified schematic diagram of emission module 500 that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure. FIG. 5 illustrates an example emission module 500 corresponding to a detailed view of emission module 302 introduced in FIG. 3, including additional components such as turning mirror 550 that will be discussed below. Emission module 500 may be used in conjunction with an excitation module similar to examples 300, 400 from FIGs. 3 and 4. In emission module 500, several components appear multiple times but are only labeled and described once for clarity.
[0089] Emission module 500 includes emission objective 505. Emission objective 505 may be an infinity -corrected objective configured to collect the fluorescence from sample 340 and output a collimated beam. In this context, infinity -corrected refers to a configuration in which the light leaving the objective is collimated. Infinity-correction may allow for maximum flexibility in placement of downstream optical components.
[0090] Emission objective 505 may also include a correction for the presence or absence of coverglasses on sample 340. For example, emission objective 505 may include a rotatable ring, collar mechanism, or other component that can adjust the spacing or position of the internal optical elements within emission objective 505, thus allowing for the imaging of samples 340 in either case. In some examples, emission objective 505 may include a kinematic adjustment to compensate for the tilt of the microscope or of sample 340.
[0091] Emission module 500 includes turning mirror 510 for changing the plane of emission module 500 optics with respect to the incident light beam. A combination of dichroic mirrors and turning mirrors can be used to construct a tree configuration for emission module 500 in 3D space, resulting in significant flexibility in the orientation of all of the optical components. For example, emission module 500 optics could be arranged in either a horizontal or vertical plane, or even anonplanar configuration. In some examples, turning mirror 510 may include a kinematic adjustment.
[0092] As mentioned, emission module 500 includes one or more partial reflectors such as dichroic mirrors 515, 525, 530. Dichroic mirrors 515, 525, 530 may include kinematic adjustments for fine-tuning optical alignment and alignment of image sensor 555 field of view. A combination of turning mirrors 510 and dichroic mirrors 515, 525, 530 can be used to arrange the optical components of emission module 500 into a compact tree configuration. [0093] Since the light sources of the excitation module 301 may activated sequentially or in a pulsed fashion or they can all be activated simultaneously or simultaneously in subsets, the dichroic mirrors 515, 525, 530 may be used to direct or split the incident light beam 520 according to wavelength. In one example, if incident light beam 520 received from the excited sample 340 includes 8 wavelengths of light, 7 dichroic mirrors can be used to separate combined light beam 520 into 8 separate paths, assuming a symmetrical arrangement. In some cases, for certain excitation colors, some image sensors 555 may not detect a faint fluorescence signal, an effect that is stronger for longer excitation wavelengths.
[0094] Each path terminates with image sensor 555 as will be described in more detail below. In general, for a symmetrical arrangement of paths, the number of paths is a power of 2 (e.g.. 2° = I. 21 = 2, 22 = 4, etc.). In FIG. 5, the tree depth is 3 (i.e. , a branch, a sub-branch, and a sub-sub- branch), in which each path includes 3 dichroic mirrors 515, 525, 530.
[0095] Non-symmetric tree configurations arrangements are also possible. For example, emission module 500 could be configured using a tree configuration with 10 paths by adding two additional dichroic mirrors to split beams 545 and 547. In such a configuration, the existing paths would include 3 dichroic mirrors and the two paths corresponding to the added dichroic mirrors would include 4 dichroic mirrors.
[0096] In some examples, dichroic mirrors 515, 525, and 530 reflect shorter wavelengths and transmit longer wavelengths. However, dichroic mirrors that reflect longer wavelengths and transmit shorter wavelengths can also be used. However, selection of dichroic mirrors with the same reflectance/transmission profile may result in a more consistent assembly. For example, if dichroic mirrors 515, 525, and 530 reflect shorter wavelengths and transmit longer wavelengths, then image sensor 555 corresponding to the shortest-wavelength incident light receives light reflected by all 3 dichroic mirrors in its path. In contrast, image sensor 555 corresponding to the longest-wavelength incident light receives light transmitted by all 3 dichroic mirrors in its path. Some examples may be configured to minimize the number of dichroic mirrors that transmit longer-wavelength fluorescence light (e.g., red) since the fluorescence signal may decrease with wavelength and the efficiency of transmission of dichroic filters is imperfect (e.g., typically <100% transmission). For example, transmission through five 95% efficient dichroic mirrors may be 77%, a significant loss of signal.
[0097] Emission module 500 includes bandpass filter 535 on each path following one or more dichroic mirrors 515, 525, 530. Bandpass filter 535 further defines the detection spectrum for each image sensor 555. Bandpass filter 535 also provides additional attenuation outside of the detection band, which may be required because of the large intensity difference between the excitation light and the fluorescence light emitted from sample 340.
[0098] In some examples, bandpass filter 535 is followed by tube lens 540. Tube lens 540 can form an image of sample 340 fluorescence in the plane of image sensor 555 at the end of the path. Tube lens 540 thus focuses the light from sample 340 onto image sensor 555.
[0099] In some examples, tube lens 540 is followed by turning mirror 550, to reduce the footprint of the optical setup as described above. For instance, in example emission module 500, turning mirrors 550 are configured for a tree configuration that is a spiral pattern. Turning mirrors 550 may include kinematic adjustments that can be used to align the camera fields of view.
[0100] Each path of emission module 500 terminates with image sensor 555. Typical configurations for widefield imaging of a rectangular field of view may utilize two-dimensional sensors, but other configurations are possible. For example, a hne sensor could be used in a line scanning mode. In the line scanning mode, data may be captured one line at a time.
[0101] Image sensor 555 may be any type of image sensor or camera suitable for the gathering of emitted light at the target wavelengths and desired resolution. For example, image sensor 555 may be CCD, a CMOS sensor, or a PMT, among others. In a typical implementation, image sensor 555 is a scientific-grade CMOS sensor. The scientific-grade CMOS sensor may include features such as fast readout, low readout noise, and a global shutter. In some examples, image sensor 555 size is chosen to match the field of view of the imaging system where the imaging performance is close to diffraction-limited. Image sensor 555 may be controlled by electronics allowing for, for example, parallel exposure, triggering, and readout.
[0102] In some examples, bandpass filters 535 could instead be positioned after tube lenses 540. For example, FIG. 6 shows a simplified schematic diagram of emission module 600 that may be used in a filter-based fluorescent microscope for virtual staining in which bandpass filters 535 are positioned after tube lenses 540, according to some aspects of the present disclosure. Emission module 600 may be used in imaging systems in conjunction with an excitation module similar to examples 300, 400 from FIGs. 3 and 4. Note that in emission module 600, several components appear multiple times but are only labeled and described once for clarity. In emission module 600, bandpass filter 535 follows tube lens 540 in each path. Bypositioning bandpass filters 535 after tube lenses 540 (e.g., close to image sensors 555) several potential issues can be mitigated. [0103] Certain optical components on each path may have angular dependencies that result in a spectral shift that may affect the final image imaged by image sensor 555. For instance, some dichroic mirrors used at 45 degree angles of incidence, as shown, may result in a first-order spectral shift that can significantly affect the image. Because a large field of view of sample 340 may be imaged, a range of angles of collimated light beams may be emitted from emission objective 505. For a point offset by /lx from the center of the field of view, the angle of the Ax collimated light beam with respect to the optical axis is given by approximately — , where fi is A the effective focal length of emission objective 505. Thus, the range of angles making up the collimated light beams is proportional to Ax. Following downstream emission from tube lens 540, the range of angles of the emitted non-collimated light is given bv approximately 2(W/I)'r' where NA is the numerical aperture of emission objective 505, and f2 is the effective focal length of tube lens 540. The total range of ray angles in collimated space (between emission objective 505 and tube lens 540) may exceed the range of angles in non-collimated space, depending on the size of the field of view relative to emission objective 505 characteristics. In some configurations, this may result in undesirable or uncorrectable non-uniform responses (e.g., spectral shifts) or aberrations in the final image due to the spectral shift from optical components with an angular dependency. It may thus be preferable to position bandpass filter 535 after tube lenses 540 in such cases.
[0104] Moreover, the example configuration of emission module 600 can reduce spatial- spectral coupling. An additional advantage is that positioning bandpass filters 535 closer to image sensors 555 can allow for smaller bandpass filters 535 to be used without vignetting, thereby reducing cost. Vignetting refers to the darkening of the comers of an image compared to its center.
[0105] Turning next to FIGs. 7A and 7B, FIGs. 7A and 7B show simplified schematic diagrams of emission module 700 that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure. FIGs. 7A and 7B illustrate example emission module 700 with some components repositioned and/or in different quantities in accordance with certain design parameters. Emission module 700 may be used in imaging systems in conjunction with an excitation module similar to examples 300, 400 from FIGs. 3 and 4. In emission module 700, several components appear multiple times but are only labeled and described once for clarity. [0106] FIG. 7A depicts example emission module 700 that includes only single tube lens 710, instead of tube lens 540 for each branch, as shown in FIGs. 5 and 6. In this configuration, all of the fluorescence emitted by sample 340 may be collected by emission objective 705 and focused towards image sensors 715 by single tube lens 710. In this configuration, dichroic mirrors 720 are located in non-collimated space (between tube lens 710 and image sensor 715). Similar to the configuration of FIG. 6, in which bandpass filter 725 is positioned after tube lenses 540, it may also be preferable to position dichroic mirrors 720 after tube lenses 710. As discussed above, each point in the imaging field of view transmits the same range of ray angles through the dichroic mirrors. As a result, the transmission spectrum received at image sensor 715 may be uniform across the field of view, despite any angular dependence of the transmission spectra of dichroic mirrors 720.
[0107] In this configuration, however, an astigmatism can be introduced by the converging (i.e., emitted from tube lens 710) beams passing through dichroic mirrors 720. An astigmatism may occur when different orthogonal orientations of light have different focal lengths, leading to aberrations in the focused image. This problem may manifest when dichroic mirrors 720 are constructed as dielectric coatings on plane-parallel substrates. In that case, the amount of astigmatism introduced upon transmission through dichroic mirrors 720 may be proportional to the thickness of the substrate. Typical dichroic mirror substrates may be 3 millimeters or more to ensure sufficient flatness for diffraction-limited imaging performance.
[0108] To compensate for astigmatism introduced by the placement of dichroic mirrors 720 in non-collimated space, blank substrates or compensation plates 730 can be introduced along each optical path terminating with image sensor 715. Compensation plates 730 may include glass plates with plane-parallel surfaces. Furthermore, compensation plates 730 may incorporate antireflection coatings on both surfaces. In some examples, compensation plates 730 can be angled with respect to the light beam axis.
[0109] In some examples, the thickness of compensation plates 730 is equal to the total thickness of the dichroic mirror substrates through which a given path has been transmitted. For example, for a branch including one sub-branch (i.e., 2 dichroic mirrors) the required compensation plate 730 thickness is 2 times the dichroic mirror thickness. In some cases, for branches in which the light arrives at image sensor 715 following only reflections by dichroic mirrors, and no transmission, no compensation plate 730 may be required.
[0110] In some examples, the amount of required compensation can be reduced by sending some of the paths out of the plane of the optical system. FIG. 7B depicts example emission module 700 in which some dichroic mirrors 750, 755, 760 are oriented to reflect or transmit light beams out-of-plane. In FIG. 7B, out-of-plane dichroic mirrors 755, 760 are represented by rectangles with crosses connecting the comers and out-of-plane paths are represented by dashed lines. In this configuration, whenever a path is transmitted through two dichroic mirrors with orthogonal tilt axes (e.g., the axis that is normal to the plane of the dichroic mirror), the astigmatism effect may cancel. Thus, required compensation plate 765 thickness may be significantly reduced. For example, for the labeled path in FIG. 7B, only a compensation plate thickness equal to the width of dichroic mirror 760 may be needed. Such a configuration can also reduce uncompensated higher-order aberrations introduced by orthogonally tilted plates.
[0111] Turning next to FIGs. 8A and 8B, FIGs. 8A and 8B show a simplified schematic diagrams of emission module 800 that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure. FIGs. 8A and 8B illustrate example emission module 800 with some components repositioned and/or in different quantities in accordance with certain design parameters. Emission module 800 may be used in imaging systems in conjunction with an excitation module similar to examples 300, 400 from FIGs. 3 and 4. In emission module 800, several components appear multiple times but are only labeled and described once for clarity.
[0112] FIG. 8 A depicts example emission module 800 in which dichroic mirrors 805 and 810 are positioned in both collimated space and non-collimated space, respectively. This configuration may be advantageous in applications that are space-constrained. Emitted light from emission objective 802 is reflected or transmitted by dichroic mirror 805 in collimated space. The light beam then passes through tube lens 815, followed by dichroic mirrors 810 in noncollimated space. As in FIGs. 7A and 7B, compensation plate 820 can be added to correct for undesirable astigmatism effects. In FIG. 8A, all of the optical components of emission module 800 are in a single horizontal plane. In the planar configuration, with dichroic mirrors 805, 810 split between collimated space and non-collimated space, the thickness of required compensation plates 820 is accordingly reduced to match the thickness of dichroic mirrors 810 in collimated space that are transmitted light (i.e., light passes through dichroic mirrors 810).
[0113] FIG. 8B depicts emission module 800 with some components lying out-of-plane. As in FIG. 8A, dichroic mirror 805 is in collimated space and dichroic mirrors 825. 830 are in non- collimated space. However, dichroic mirror 830 is oriented to reflect the incident light beam out of plane. In this configuration, as above, whenever a path is transmitted through two dichroic mirrors with orthogonal tilt axes (e.g., the axis that is normal to the plane of the dichroic mirror), the astigmatism effect may cancel. Thus, the width of the compensation plate 835 may be accordingly reduced further. In example emission module 800 of FIG. 8B, the width of compensation plate 835 need only match the width of single transmitting dichroic mirror 830. Judicious selection of the number and position of tube lenses 815 and dichroic mirrors 805, 825, 830 can thus minimize the physical footprint of the emission module 800.
[0114] Turning next to FIGs. 9A and 9B, FIGs. 9A and 9B illustrate example configurations of certain components that may be used in a filter-based fluorescent microscope for virtual staining, according to some aspects of the present disclosure. FIG. 9A shows relationship 900 between characteristics of bandpass filters and the transmission spectra of dichroic mirrors in one example emission module. The schematic of an example emission module to the right of graph 905 shown serves as a key 910 for graph 905.
[0115] Graph 905 depicts a relationship between the transmission spectra of the bandpass filters 920 depicted in key 910 and the transmission spectra of dichroic mirrors 915 depicted in key 910. In general, the transmission spectra of dichroic mirrors 915 and bandpass filters 920 may be selected to obtain a suitable overall transmission across the desired detection range, while avoiding unwanted effects near the transmission edges of the components. In some examples, to minimize spatial-spectral coupling associated with angle-dependent transmission spectra of the dichroic mirrors, relationship 900 may be used to select the bandpass filters and dichroic mirrors as shown in key 910.
[0116] In graph 905, the peak transmission values of dichroic mirrors 915 are shifted vertically for visibility. The transmission spectra of dichroic mirrors 915 show near-ideal transmission above some wavelength. For example, dichroic minor transmission spectrum 925, corresponding to D3 in key 910, transmits 90% to 95% of incident light for typical commercially available dichroic mirrors.
[0117] For the tree configuration depicted in key 910, the transmission spectra associated with bandpass filters 920 are labeled BP0 through BP7 in graph 905 and are associated with the cameras labeled CO through C7 in key 910. In graph 905, transmission edge wavelengths of bandpass filters 920 are positioned inside of relevant dichroic mirror transmission edges 915. For example, bandpass filter transmission profile 930 is chosen such that bandpass filter 930 (BP3/C3) transmits only light that is fully transmitted by dichroic mirror 925 (D3). This configuration may cause the overall detection spectra of the cameras to be mostly unaffected if dichroic mirror transmission edges 915 shift slightly in wavelength because of angular dependencies. Thus, bandpass filters 920 primarily restrict the detection spectra of the cameras in this configuration.
[0118] FIG. 9B depicts another example relationship 950 between transmission spectra of the bandpass filters 970 and transmission spectra of the dichroic mirrors 975. The schematic of an example emission module to the right of graph 955 show n again serves as key 910 for graph 955. This relationship may be selected to maximize optical throughput, to reduce required exposure times, or to reduce photobleaching of sample 340. In relationship 950, dichroic mirrors 975 primarily define the detection bands, however bandpass filters 970 are still needed to provide additional attenuation of the excitation light.
[0119] In this example relationship 950, bandpass filter transmission bands 970 are expanded to include the entire transmission bands provided by the combination of dichroic mirrors 975. For example, bandpass filter transmission profile 960 is chosen such that bandpass filter 960 (BP3/C3) transmission profile edges coincide with transmission spectra edge of dichroic mirror 965 (D3). This configuration may significantly increase throughput. However, this relationship 950 may result in spatial-spectral coupling effects due to the angle-dependent transmission spectra of the dichroic minors. Relationship 950 may be optimized configurations in which the at least one dichroic mirror is positioned in non-collimated space.
[0120] Turning next to FIG. 10, FIG. 10 show s a simplified schematic of example imaging system 1000 that may be used in a filter-based fluorescent microscope for virtual staining. FIG. 10 illustrates example imaging system 1000 in an epi-fluorescence configuration. In imaging system 1000, several components appear multiple times but are only labeled and described once for clarity.
[0121] Example imaging system 1000 depicts a fluorescent microscope in an epi-fluorescence configuration. In an epi-fluorescence configuration, both the excitation light and the emitted fluorescence pass through same objective 1005. Same objective 1005 is thus used for both illumination of the sample with the excitation light and collection of the emitted fluorescence light. In some examples, the epi-fluorescence configuration may be associated with a smaller physical footprint. For instance, in the epi-fluorescence configuration, excitation and emission module components may be configured in a tree configuration side-by-side.
[0122] In example imaging system 1000, a colored light beam from light source 1010 passes through collimator 1012, mask 1014, bandpass filter 1016, and is then reflected a first time by dichroic mirror 1018. The colored light beam 1008 then passes through tube lens 1020, is reflected by dichroic mirror 1022 and then again by dichroic mirror 1024 towards objective 1005. The light beam excites sample 340 on the sample stage (not shown) and the fluorescent emitted light is received by the same objective 1005. In one possible return path shown for illustrative purposes, the emitted light beam is reflected by dichroic mirror 1024, transmitted by dichroic mirror 1022 such that only light beam of a particular color 1029 is transmitted. Light beam of a particular color 1029 passes through tube lens 1028, is reflected by dichroic mirror 1030, passes through bandpass filter 1032, and is finally received by image sensor 1034.
[0123] In the epi-fluorescence configuration of FIG. 10, focusing is no longer affected by variations in the thickness of sample 340 (e.g., the microscope slide), and only same objective 1005 is required to be moved for focusing of the image. The epi-fluorescence configuration may simplify mechanical clearance around sample 340 (e.g., the translation stage that holds the sample slides). Additionally, the intensity of the back-reflected illumination sent towards image sensor 1034 may be smaller than the direct illumination sent towards the image sensor in the conventional configurations of FIG. 1 , thereby reducing the performance requirements for the filters.
[0124] FIG. 11 illustrates an example flowchart showing process 1100. according to at least a few examples. This process 1100, and any other processes described herein, are illustrated as logical flow diagrams, each operation of which represents a sequence of operations that can be implemented in hardware, computer instructions, or a combination thereof (e.g., computer system 1200 as described in FIG. 12 and the accompanying description). In the context of computer instructions, the operations may represent computer-executable instructions stored on one or more non-transitory computer-readable storage media that, when executed by one or more processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, components, data structures and the like that perform particular functions or implement particular data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described operations can be combined in any order and/or in parallel to implement the processes.
[0044] Additionally, some, any, or all of the processes described herein may be performed under the control of one or more computer systems configured with specific executable instructions and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) executing collectively on one or more processors, by hardware, or combinations thereof. As noted above, the code may be stored on a non-transitory computer readable storage medium, for example, in the form of a computer program including a plurality of instructions executable by one or more processors.
[0125] Process 1100 relates to operating a filter-based fluorescent microscope for virtual staining, according to at least one example. The process 1 100 in particular is directed to a process for generating a composite image using a fluorescent microscope as described above and using virtual staining techniques based on autofluorescence from sample 340. Some description may be given with respect to the example of FIG. 3 for illustrative purposes, but many other configurations are possible.
[0126] The process 1100 begins at block 1102 at which a computing device outputs a first command to cause one or more light sources 305 arranged in an excitation tree configuration included in excitation module 301 to emit light. For example, light sources 305 may be LED light sources 305. LED light sources 305 may have narrow, well-defined excitation wavelength range for precision autofluorescence. Some LED light sources 305 may have rapid switching capabilities to minimize photobleaching or allow for in situ switching between excitation wavelengths.
[0127] In block 1102, each light source 305 emits light of a particular color. In some examples, only a single excitation light source 305 is turned on at a time, while in other examples two or more light sources 305 are used. LED light sources 305 are well-suited to fluorescing applications, often having a narrow spectral profile. For example, a typical LED light source may have a spectral width (full width at half maximum or FWHM) as narrow as 20-40 nm. Thus, a 488 nm LED might have a bandwidth of approximately 24 nm. emitting light primarily between 476 nm and 500 nm. As discussed above, additional optical components like bandpass filters may be used to narrow the spectral profile further both prior to excitation and subsequent to emission.
[0128] In some examples, optical components such as partial reflectors (e.g., dichroic mirrors) and turning mirrors can be used to combine the light from the one or more light sources 305 to generate a combined light beam. While the light beams are closely aligned, light sources 305 are ty pically incoherent and lack a well-defined phase relationship. Thus, the combined light beam does not experience constructive or destructive interferences, while the light beam's intensity at any point is roughly the sum of the intensities from the two individual beams.
[0129] The light beam is directed to sample 340 supported by a sample stage. Illuminated sample 340 emits fluorescence light responsive to the light beam received from the excitation module. Since sample 340 is not stained or dyed, this fluorescence is referred to as autofluorescence, or the emission of light by sample 340 that has not been externally stained or labeled.
[0130] At block 1104, the computing device receives, from one or more image sensors 350 arranged in an emission tree configuration included in emission module 302, a first imaging signal from first image sensor 350 and a second imaging signal from second image sensor 350. As with excitation module 301, optical components such as dichroic mirrors and turning mirrors can be used to direct the light beam from emission objective 345 to image sensors 350 in a densely compact tree configuration to economize on space or cost.
[0131] For example, first image sensor 350 may receive light of a first emitted wavelength and second image sensor 350 may receive light of a second emitted wavelength. In some examples, for each light source 305 used in block 1102, images may be collected from each image sensor 350. The emitted wavelengths are longer than the excitation wavelengths because the energy of the emitted photons in the emitted light is lower than that of the absorbed photons in the excitation light. Generally, when a molecule absorbs a photon, it transitions to a higher energy excited state. Upon returning to its ground state, some energy is lost, usually as non-radiative processes, and thus the emitted photon has lower energy.
[0132] At block 1106, the computing device generates a first raw image from the first imaging signal. Likewise, at block 1108, the computing device generates a second raw image from the second imaging signal. For example, the first imaging signal may correspond to fluorescence of a particular autofluorescent molecule having a first wavelength range, while the second imaging signal corresponds to fluorescent emissions from a different autofluorescent molecule with another wavelength range. The dichroic mirrors are used to ensure that image sensors 350 receive these distinct imaging signals separately.
[0133] The raw images correspond to the intensity of fluorescence emission from sample 340 at each pixel location. The value at each pixel represents the amount of light detected, which may correlate to the concentration and distribution of the autofluorescent molecules in sample 340 at that specific point. Digital processing techniques can be applied to these raw images to correct for, for example, optical aberrations, noise, or artifacts. For example, some corrections may include dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction. [0134] At block 1110, the computing device generates a composite multispectral image comprising the first raw image and the second raw image. For example, an image registration component can be used to align the raw images. The image registration component may identify common features or patterns among the raw images and then apply suitable transformations to align the features to produce a composite image. At block 1112. the computing device outputs the composite multispectral image. The composite multispectral image may be shown on a suitable display device, like a computer or smartphone screen. In some examples, upon completion of imaging process 1100 a notification or alert may be sent including information about the composite multispectral image, time of completion, links for access, and so forth. The composite multispectral image may be saved or persisted using a suitable memory device.
[0135] FIG. 12 illustrates an example system 1200 for implementing techniques for filterbased fluorescent microscope for virtual staining, according to at least one example. FIG. 12 in particular illustrates examples of components of the computer system 1200, according to at least one example. The computer system 1200 may be a single computer, such as a user computing device and/or can represent a distributed computing system, such as one or more server computing devices. The computer system 1200 is an example of the computer system 104.
[0136] The computer system 1200 may include at least a processor 1202, a memory 1204, a storage device 1206, input/output peripherals (I/O) 1208, communication peripherals 1210, and an interface bus 1212. The interface bus 1212 is configured to communicate, transmit, and transfer data, controls, and commands among the various components of the computer system 1200. The memory 1204 and the storage device 1206 include computer-readable storage media, such as random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), hard drives, CD-ROMs, optical storage devices, magnetic storage devices, electronic non-volatile computer storage, for example, Flash® memory, and other tangible storage media. Any of such computer-readable storage media can be configured to store instructions or program codes embodying aspects of the disclosure. The memory 1204 and the storage device 1206 also include computer-readable signal media. A computer-readable signal medium includes a propagated data signal with computer-readable program code embodied therein. Such a propagated signal takes any of a variety of forms including, but not limited to. electromagnetic, optical, or any combination thereof. A computer- readable signal medium includes any computer-readable medium that is not a computer-readable storage medium and that can communicate, propagate, or transport a program for use in connection with the computer system 1200. [0137] Further, the memory 1204 includes an operating system, programs, and applications. The processor 1202 is configured to execute the stored instructions and includes, for example, a logical processing unit, a microprocessor, a digital signal processor, and other processors. The memory 1204 and/or the processor 1202 can be virtualized and can be hosted within another computing system of, for example, a cloud network or a data center. The I/O peripherals 1208 include user interfaces, such as a keyboard, screen (e.g.. a touch screen), microphone, speaker, other input/output devices, and computing components, such as graphical processing units, serial ports, parallel ports, universal serial buses, and other input/output peripherals. The I/O peripherals 1208 are connected to the processor 1202 through any of the ports coupled to the interface bus 1212. The communication peripherals 1210 are configured to facilitate communication between the computer system 1200 and other computing devices over a communications network and include, for example, a network interface controller, modem, wireless and wired interface cards, antenna, and other communication peripherals.
[0138] In the following, further examples are described to facilitate the understanding of the present disclosure. As used below, any reference to a series of examples is to be understood as a reference to each of those examples disjunctively (e.g., "Examples 1-4" is to be understood as "Examples 1, 2, 3, or 4").
[0139] Example 1 is an excitation optics system , comprising: a sample stage configured to support a sample; and an excitation module configured to transmit a light beam to the sample supported by the sample stage to cause the sample to emit fluorescence light that is detectable by an emission module, the excitation module comprising: one or more partial reflectors; two or more light sources arranged in a tree configuration based on positions of the one or more partial reflectors , wherein each light source emits light of a particular color; two or more collimating lenses corresponding to the two or more light sources; a tube lens; and an excitation objective configured to direct light to the sample, wherein a first partial reflector of the one or more partial reflectors is configured to allow light from a first light source of the two or more light sources and light from a second light source of the two or more light sources to traverse the same path to the excitation objective.
[0140] Example 2 is the excitation optics system of example(s) 1. wherein the two or more light sources are activated sequentially.
[0141] Example 3 is the excitation optics system of example(s) 1. wherein: wherein the two or more light sources are activated simultaneously; the first partial reflector of the one or more partial reflectors is configured to generate a first combined beam comprising light from a first light source of the two or more light sources and light from a second light source of the two or more light sources; and the excitation objective is configured to direct the first combined beam to the sample.
[0142] Example 4 is the excitation optics system of example(s) 3, wherein: a second partial reflector of the one or more partial reflectors is configured to generate a second combined light beam comprising the light from a third light source of the two or more light sources and the light from a fourth light source of the two or more light sources; a third partial reflector of the one or more partial reflectors is configured to generate a third combined light beam comprising the first combined beam and the second combined light beam; and the third combined light beam is transmitted to the sample by the excitation objective.
[0143] Example 5 is the excitation optics system of example(s) 1. wherein the excitation module further comprises a first mask, wherein: the light from the first light source traverses a first collimating lens of the two or more collimating lenses; and the light exiting the first collimating lens passes through the first mask, the first mask configured to define an illumination area on the sample.
[0144] Example 6 is the excitation optics system of example(s) 5. wherein the excitation module further comprises a bandpass filter configured to narrow a wavelength range of the light from the first light source, wherein the light passing through the first mask passes through the bandpass filter.
[0145] Example 7 is the excitation optics system of example(s) 6, wherein the bandpass filter is oriented with an angle with respect to a plane perpendicular to a centerline of the first collimating lens, wherein the angled bandpass filter is configured to reflect fluorescence light emitted from the sample away from the centerline of the first collimating lens.
[0146] Example 8 is the excitation optics system of example(s) 5, wherein the excitation module further comprises a second mask, wherein: the light from the second light source traverses a second collimating lens of the two or more collimating lenses; the light exiting the second collimating lens passes through the second mask, the second mask configured to define the illumination area on the sample; the light from the first light source has a longer wavelength than the light from the second light source; and the first mask and the tube lens define a first mask-tube lens distance and the second mask and the tube lens define a second mask-tube lens distance, wherein the first mask-tube lens distance and the second mask-tube lens distance are configured to keep light from the first light source exiting the tube lens and light from the second light source exiting the tube lens in parallel.
[0147] Example 9 is the excitation optics system of example(s) 1. wherein the excitation module further comprises a vertical turning mirror, wherein the vertical turning mirror is configured to maintain the plane of the sample stage perpendicular to a plane perpendicular to the centerline of the tube lens.
[0148] Example 10 is the excitation optics system of example(s) 1, wherein: the excitation objective is mounted on a vertical actuator; and the excitation objective is mounted on a lateral translator.
[0149] Example 11 is the excitation optics system of example(s) 1, wherein the one or more partial reflectors are dichroic mirrors.
[0150] Example 12 is the excitation optics system of example(s) 1, wherein the excitation module further comprises a photodiode coupled to an unused port of the first partial reflector, wherein the photodiode is configured to receive a leakage signal for calibration of the two or more light sources.
[0151] Example 13 is an emission optics system, comprising: a sample stage configured to support a sample; and an emission module configured to capture emitted fluorescence light from the sample when the sample is supported by the sample stage, comprising: an emission objective; one or more partial reflectors configured to generate a first transmitted beam and a first reflected beam ; and two or more image sensors, wherein each image sensor is configured to receive fluorescence light of at least one color emitted from the sample responsive to a light beam output from an excitation module, wherein the two or more image sensors are arranged in a tree configuration based on positions of the one or more partial reflectors.
[0152] Example 14 is the emission optics system of example(s) 13, wherein: the light beam output from the excitation module comprises fluorescence light of a first color emitted from the sample and fluorescence light of a second color emitted from the sample; the first transmitted beam includes the light of the first color; and the first reflected beam includes the light of the second color.
[0153] Example 15 is the emission optics system of example(s) 13, wherein the one or more partial reflectors are dichroic mirrors. [0154] Example 16 is the emission optics system of example(s) 13, wherein at least one of the two or more image sensors further comprises a bandpass filter configured to narrow a wavelength of the fluorescence light emitted from sample prior to the at least one of the two or more image sensors.
[0155] Example 17 is the emission optics system of example(s) 16, wherein a range of the bandpass filter is configured to be within a transmission threshold of at least one of the one or more partial reflectors.
[0156] Example 18 is the emission optics system of example(s) 16. wherein a range of the bandpass filter is configured to include a transmission band resulting from a combination of two or more partial reflectors.
[0157] Example 19 is the emission optics system of example(s) 1 , wherein the at least one of the two or more image sensors further comprises a tube lens configured to receive light passing through the bandpass filter.
[0158] Example 20 is the emission optics system of example(s) 16, wherein the at least one of the two or more image sensors further comprises a tube lens configured to receive the fluorescence light emitted from the sample prior to the bandpass filter.
[0159] Example 21 is the emission optics system of example(s) 13, wherein the emission module further comprises one or more horizontal turning mirrors configured to reduce the footprint of the emission module by angling a path of the fluorescence light prior to reception by at least one image sensor.
[0160] Example 22 is the emission optics system of example(s) 13, wherein at least one of the two or more image sensors is a two-dimensional sensor.
[0161] Example 23 is the emission optics system of example(s) 13, wherein at least one of the two or more image sensors is a line sensor.
[0162] Example 24 is the emission optics system of example(s) 13, wherein the emission module further comprises a tube lens and one or more compensation plates configured to receive the fluorescence light emitted from sample, wherein a thickness of each compensation plate is equal to a thickness of the one or more partial reflectors through which the fluorescence light emitted by the sample has passed. [0163] Example 25 is the emission optics system of example(s) 24, wherein the emission module further comprises at least two partial reflectors, wherein the at least two partial reflectors are configured with orthogonal tilt axes.
[0164] Example 26 is the emission optics system of example(s) 13, wherein the emission module further comprises a vertical turning mirror, wherein the vertical turning mirror is configured to maintain the plane of the sample stage perpendicular to the plane of the center of the first partial reflector.
[0165] Example 27 is the emission optics system of example(s) 13. wherein: the first transmitted beam comprises light of a first color and light of a third color; the first reflected beam comprises light of a second color and light of a fourth color; a second partial reflector of the one or more partial reflectors is configured to generate a second transmitted beam and a second reflected beam from the first transmitted beam, wherein the second transmitted beam comprises the light of the third color and the second reflected beam comprises the fluorescence light of the third color emitted from the sample; and a third partial reflector of the one or more partial reflectors is configured to generate a third transmitted beam and a third reflected beam, wherein the third transmitted beam comprises the light of the second color and the third reflected beam comprises the light of the fourth color.
[0166] Example 28 is a computer-implemented method, comprising: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.
[0167] Example 29 is the method of example(s) 28, wherein receiving the first imaging signal from the first image sensor occurs in parallel to receiving the second imaging signal from the second image sensor. [0168] Example 30 is the method of example(s) 28, further comprising: outputting a second command to cause a capture of a low-resolution image of the sample; and outputting a third command to cause a focusing procedure of a portion of the low-resolution image.
[0169] Example 31 is the method of example(s) 28, further comprising: applying one or more image corrections to the composite multispectral image including at least one of dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction.
[0170] Example 32 is a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw7 image; and outputting the composite multispectral image.
[0171] Example 33 is the non-transitory computer-readable medium of example(s) 32, wherein receiving the first imaging signal from the first image sensor occurs in parallel to receiving the second imaging signal from the second image sensor.
[0172] Example 34 is the non-transitory computer-readable medium of example(s) 32, further comprising: applying one or more image corrections to the composite multispectral image including at least one of dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction.
[0173] Example 35 is a system comprising: one or more processors; and one or more computer-readable storage media storing instructions which, when executed by the one or more processors, cause the one or more processors to perform operations including: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.
[0174] While the present subject matter has been described in detail with respect to specific embodiments thereof, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing may readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, it should be understood that the present disclosure has been presented for purposes of example rather than limitation, and does not preclude inclusion of such modifications, variations, and/or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art. Indeed, the methods and systems described herein may be embodied in a variety’ of other forms; furthermore, various omissions, substitutions and changes in the form of the methods and systems described herein may be made without departing from the spirit of the present disclosure. The accompanying claims and their equivalents are intended to cover such forms or modifications as would fall within the scope and spirit of the present disclosure.
[0175] Unless specifically stated otherwise, it is appreciated that throughout this specification discussions utilizing terms, such as “processing;’ “computing,” “calculating,” “determining,” and “identifying” or the like refer to actions or processes of a computing device, such as one or more computers or a similar electronic computing device or devices, that manipulate or transform data represented as physical electronic or magnetic quantities within memories, registers, or other information storage devices, transmission devices, or display devices of the computing platform.
[0176] The system or systems discussed herein are not limited to any particular hardware architecture or configuration. A computing device can include any suitable arrangement of components that provide a result conditioned on one or more inputs. Suitable computing devices include multipurpose microprocessor-based computing systems accessing stored software that programs or configures the computing system from a general purpose computing apparatus to a specialized computing apparatus implementing one or more embodiments of the present subject matter. Any suitable programming, scripting, or other type of language or combinations of languages may be used to implement the teachings contained herein in software to be used in programming or configuring a computing device.
[0177] Embodiments of the methods disclosed herein may be performed in the operation of such computing devices. The order of the blocks presented in the examples above can be varied — for example, blocks can be re-ordered, combined, and/or broken into sub-blocks. Certain blocks or processes can be performed in parallel.
[0178] Conditional language used herein, such as. among others, “can,” “could,” “might.” “may,” “e.g..” and the like, unless specifically stated otherwise, or otherwise understood within the context as used, is generally intended to convey that certain examples include, while other examples do not include, certain features, elements, and/or steps. Thus, such conditional language is not generally intended to imply that features, elements and/or steps are in any way required for one or more examples or that one or more examples necessarily include logic for deciding, with or without author input or prompting, whether these features, elements and/or steps are included or are to be performed in any particular example.
[0179] Disjunctive language, such as the phrase “at least one of X, Y, or Z,” unless specifically stated otherwise, is otherwise understood within the context as used in general to present that an item, term, etc., may be either X. Y, or Z, or any combination thereof (e.g., X, Y. and/or Z). Thus, such disjunctive language is not generally intended to, and should not, imply that certain examples require at least one of X, at least one of Y, or at least one of Z to each be present.
[0180] Use herein of the word “or” is intended to cover inclusive and exclusive OR conditions. In other words, A or B or C includes any or all of the following alternative combinations as appropriate for a particular usage: A alone; B alone; C alone; A and B only; A and C only; B and C only; and all three of A and B and C.
[0181] The use of the terms “a” and “an” and “the” and similar referents in the context of describing the disclosed examples (especially in the context of the following claims) are to be construed to cover both the singular and the plural, unless otherwise indicated herein or clearly contradicted by context. The terms “comprising.” “including,” “having.” and the like are synonymous and are used inclusively, in an open-ended fashion, and do not exclude additional elements, features, acts, operations, and so forth. Also, the term “or” is used in its inclusive sense (and not in its exclusive sense) so that when used, for example, to connect a list of elements, the term “or” means one, some, or all of the elements in the list. The use of “adapted to” or “configured to” herein is meant as open and inclusive language that does not foreclose devices adapted to or configured to perform additional tasks or steps. The term ‘‘connected'’ is to be construed as partly or wholly contained within, attached to, or joined together, even if there is something intervening. Recitation of ranges of values herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, unless otherwise indicated herein, and each separate value is incorporated into the specification as if it were individually recited herein. Additionally, the use of “based on” is meant to be open and inclusive, in that a process, step, calculation, or other action “based on” one or more recited conditions or values may, in practice, be based on additional conditions or values beyond those recited. Similarly, the use of “based at least in part on” is meant to be open and inclusive, in that a process, step, calculation, or other action “based at least in part on” one or more recited conditions or values may, in practice, be based on additional conditions or values beyond those recited. Headings, lists, and numbering included herein are for ease of explanation only and are not meant to be limiting.
[0182] The various features and processes described above may be used independently of one another or may be combined in various ways. All possible combinations and sub-combinations are intended to fall within the scope of the present disclosure. In addition, certain method or process blocks may be omitted in some implementations. The methods and processes described herein are also not limited to any particular sequence, and the blocks or states relating thereto can be performed in other sequences that are appropriate. For example, described blocks or states may be performed in an order other than that specifically disclosed, or multiple blocks or states may be combined in a single block or state. The example blocks or states may be performed in serial, in parallel, or in some other manner. Blocks or states may be added to or removed from the disclosed examples. Similarly, the example systems and components described herein maybe configured differently than described. For example, elements may be added to, removed from, or rearranged compared to the disclosed examples.
[0183] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.
EXAMPLES
[0184] As used below, any reference to a series of examples is to be understood as a reference to each of those examples disjunctively (e.g., "Examples 1-4" is to be understood as "Examples 1, 2, 3, or 4"). [0185] Example 1 is an excitation optics system, comprising: a sample stage configured to support a sample; and an excitation module configured to transmit a light beam to the sample supported by the sample stage to cause the sample to emit fluorescence light that is detectable by an emission module, the excitation module comprising: one or more partial reflectors; two or more light sources arranged in a tree configuration based on positions of the one or more partial reflectors, wherein each light source emits light of a particular color; two or more collimating lenses corresponding to the two or more light sources; a tube lens; and an excitation objective configured to direct light to the sample, wherein a first partial reflector of the one or more partial reflectors is configured to allow light from a first light source of the two or more light sources and light from a second light source of the two or more light sources to traverse the same path to the excitation objective.
[0186] Example 2 is the excitation optics system of the preceding or any of the subsequent example(s), wherein the two or more light sources are activated sequentially.
[0187] Example 3 is the excitation optics system any of the preceding or subsequent exampl e(s). wherein: wherein the two or more light sources are activated simultaneously; the first partial reflector of the one or more partial reflectors is configured to generate a first combined beam comprising light from a first light source of the two or more light sources and light from a second light source of the two or more light sources; and the excitation objective is configured to direct the first combined beam to the sample. 4. The excitation optics system of example(s) 3. wherein: a second partial reflector of the one or more partial reflectors is configured to generate a second combined light beam comprising the light from a third light source of the two or more light sources and the light from a fourth light source of the two or more light sources; a third partial reflector of the one or more partial reflectors is configured to generate a third combined light beam comprising the first combined beam and the second combined light beam; and the third combined light beam is transmitted to the sample by the excitation objective.
[0188] Example 5 is the excitation optics system of any of the preceding or subsequent example(s), wherein the excitation module further comprises a first mask, wherein: the light from the first light source traverses a first collimating lens of the two or more collimating lenses; and the light exiting the first collimating lens passes through the first mask, the first mask configured to define an illumination area on the sample.
[0189] Example 6 is the excitation optics system of any of the preceding or subsequent example(s), w herein the excitation module further comprises a bandpass filter configured to narrow a wavelength range of the light from the first light source, wherein the light passing through the first mask passes through the bandpass filter.
[0190] Example 7 is the excitation optics system of any of the preceding or subsequent example(s), wherein the bandpass filter is oriented with an angle with respect to a plane perpendicular to a centerline of the first collimating lens, wherein the angled bandpass filter is configured to reflect fluorescence light emitted from the sample away from the centerline of the first collimating lens.
[0191] Example 8 is the excitation optics system of any of the preceding or subsequent example(s), wherein the excitation module further comprises a second mask, wherein: the light from the second light source traverses a second collimating lens of the two or more collimating lenses; the light exiting the second collimating lens passes through the second mask, the second mask configured to define the illumination area on the sample; the light from the first light source has a longer wavelength than the light from the second light source; and the first mask and the tube lens define a first mask-tube lens distance and the second mask and the tube lens define a second mask-tube lens distance, wherein the first mask-tube lens distance and the second mask-tube lens distance are configured to keep light from the first light source exiting the tube lens and light from the second light source exiting the tube lens in parallel.
[0192] Example 9 is the excitation optics system of any of the preceding or subsequent example(s), wherein the excitation module further comprises a vertical turning mirror, wherein the vertical turning mirror is configured to maintain the plane of the sample stage perpendicular to a plane perpendicular to the centerline of the tube lens.
[0193] Example 10 is the excitation optics system of any of the preceding or subsequent example(s), wherein: the excitation objective is mounted on a vertical actuator; and the excitation objective is mounted on a lateral translator.
[0194] Example 11 is the excitation optics system of any of the preceding or subsequent example(s), wherein the one or more partial reflectors are dichroic mirrors.
[0195] Example 12 is the excitation optics system of any of the preceding or subsequent example(s). wherein the excitation module further comprises a photodiode coupled to an unused port of the first partial reflector, wherein the photodiode is configured to receive a leakage signal for calibration of the two or more light sources. [0196] Example 13 is an emission optics system, comprising: a sample stage configured to support a sample; and an emission module configured to capture emitted fluorescence light from the sample when the sample is supported by the sample stage, comprising: an emission objective; one or more partial reflectors configured to generate a first transmitted beam and a first reflected beam; and two or more image sensors, wherein each image sensor is configured to receive fluorescence light of at least one color emitted from the sample responsive to a light beam output from an excitation module, wherein the two or more image sensors are arranged in a tree configuration based on positions of the one or more partial reflectors.
[0197] Example 14 is the emission optics system of the preceding or any of the subsequent example(s). wherein: the light beam output from the excitation module comprises fluorescence light of a first color emitted from the sample and fluorescence light of a second color emitted from the sample; the first transmitted beam includes the light of the first color; and the first reflected beam includes the light of the second color.
[0198] Example 15 is the emission optics system of any of the preceding or subsequent example(s). wherein the one or more partial reflectors are dichroic mirrors.
[0199] Example 16 is the emission optics system of any of the preceding or subsequent example(s), wherein at least one of the two or more image sensors further comprises a bandpass filter configured to narrow a wavelength of the fluorescence light emitted from sample prior to the at least one of the two or more image sensors.
[0200] Example 17 is the emission optics system of any of the preceding or subsequent example(s), wherein a range of the bandpass filter is configured to be within a transmission threshold of at least one of the one or more partial reflectors.
[0201] Example 18 is the emission optics system of any of the preceding or subsequent example(s), wherein a range of the bandpass filter is configured to include a transmission band resulting from a combination of two or more partial reflectors.
[0202] Example 19 is the emission optics system of any of the preceding or subsequent example(s), wherein the at least one of the two or more image sensors further comprises a tube lens configured to receive light passing through the bandpass filter.
[0203] Example 20 is the emission optics system of any of the preceding or subsequent exampl e(s), wherein the at least one of the two or more image sensors further comprises a tube lens configured to receive the fluorescence light emitted from the sample prior to the bandpass filter.
[0204] Example 21 is the emission optics system of any of the preceding or subsequent example(s), wherein the emission module further comprises one or more horizontal turning mirrors configured to reduce the footprint of the emission module by angling a path of the fluorescence light prior to reception by at least one image sensor.
[0205] Example 22 is the emission optics system of any of the preceding or subsequent example(s), wherein at least one of the two or more image sensors is a two-dimensional sensor.
[0206] Example 23 is the emission optics system of any of the preceding or subsequent example(s). wherein at least one of the two or more image sensors is a line sensor.
[0207] Example 24 is the emission optics system of any of the preceding or subsequent example(s), wherein the emission module further comprises a tube lens and one or more compensation plates configured to receive the fluorescence light emitted from sample, wherein a thickness of each compensation plate is equal to a thickness of the one or more partial reflectors through which the fluorescence light emitted by the sample has passed.
[0208] Example 25 is the emission optics system of any of the preceding or subsequent example(s), wherein the emission module further comprises at least two partial reflectors, wherein the at least two partial reflectors are configured with orthogonal tilt axes.
[0209] Example 26 is the emission optics system of any of the preceding or subsequent example(s), wherein the emission module further comprises a vertical turning mirror, wherein the vertical turning mirror is configured to maintain the plane of the sample stage perpendicular to the plane of the center of the first partial reflector.
[0210] Example 27 is the emission optics system of any of the preceding or subsequent example(s), wherein: the first transmitted beam comprises light of a first color and light of a third color; the first reflected beam comprises light of a second color and light of a fourth color; a second partial reflector of the one or more partial reflectors is configured to generate a second transmitted beam and a second reflected beam from the first transmitted beam, wherein the second transmitted beam comprises the light of the third color and the second reflected beam comprises the fluorescence light of the third color emitted from the sample; and a third partial reflector of the one or more partial reflectors is configured to generate a third transmitted beam and a third reflected beam, wherein the third transmitted beam comprises the light of the second color and the third reflected beam comprises the light of the fourth color.
[0211] Example 28 is a computer-implemented method, comprising: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.
[0212] Example 29 is the method of the preceding or any of the subsequent example(s), wherein receiving the first imaging signal from the first image sensor occurs in parallel to receiving the second imaging signal from the second image sensor.
[0213] Example 30 is the method of any of the preceding or subsequent example(s), further comprising: outputting a second command to cause a capture of a low-resolution image of the sample; and outputting a third command to cause a focusing procedure of a portion of the low- resolution image.
[0214] Example 31 is the method of any of the preceding or subsequent example(s), further comprising: applying one or more image corrections to the composite multispectral image including at least one of dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction.
[0215] Example 32 is a non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.
[0216] Example 33 is the non-transitory computer-readable medium of the preceding or any of the subsequent example(s). wherein receiving the first imaging signal from the first image sensor occurs in parallel to receiving the second imaging signal from the second image sensor.
[0217] Example 34 is the non-transitory computer-readable medium of any of the preceding or subsequent example(s), further comprising: applying one or more image corrections to the composite multispectral image including at least one of dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction.
[0218] Example 35 is a system comprising: one or more processors; and one or more computer-readable storage media storing instructions which, when executed by the one or more processors, cause the one or more processors to perform operations including: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.

Claims

WHAT IS CLAIMED IS:
1. An excitation optics system, comprising: a sample stage configured to support a sample; and an excitation module configured to transmit a light beam to the sample supported by the sample stage to cause the sample to emit fluorescence light that is detectable by an emission module, the excitation module comprising: one or more partial reflectors; two or more light sources arranged in a tree configuration based on positions of the one or more partial reflectors, wherein each light source emits light of a particular color; two or more collimating lenses corresponding to the two or more light sources; a tube lens; and an excitation objective configured to direct light to the sample, wherein a first partial reflector of the one or more partial reflectors is configured to allow light from a first light source of the two or more light sources and light from a second light source of the two or more light sources to traverse the same path to the excitation objective.
2. The excitation optics system of claim 1, wherein the two or more light sources are activated sequentially.
3. The excitation optics system of claim 1, wherein: wherein the two or more light sources are activated simultaneously; the first partial reflector of the one or more partial reflectors is configured to generate a first combined beam comprising light from a first light source of the two or more light sources and light from a second light source of the two or more light sources; and the excitation objective is configured to direct the first combined beam to the sample.
4. The excitation optics system of claim 3, wherein:
47
KI a second partial reflector of the one or more partial reflectors is configured to generate a second combined light beam comprising the light from a third light source of the two or more light sources and the light from a fourth light source of the two or more light sources; a third partial reflector of the one or more partial reflectors is configured to generate a third combined light beam comprising the first combined beam and the second combined light beam; and the third combined light beam is transmitted to the sample by the excitation objective.
5. The excitation optics system of claim 1, wherein the excitation module further comprises a first mask, wherein: the light from the first light source traverses a first collimating lens of the two or more collimating lenses; and the light exiting the first collimating lens passes through the first mask, the first mask configured to define an illumination area on the sample.
6. The excitation optics system of claim 5, wherein the excitation module further comprises a bandpass filter configured to narrow a wavelength range of the light from the first light source, wherein the light passing through the first mask passes through the bandpass filter.
7. The excitation optics system of claim 6, wherein the bandpass filter is oriented with an angle with respect to a plane perpendicular to a centerline of the first collimating lens, wherein the angled bandpass filter is configured to reflect fluorescence light emitted from the sample away from the centerline of the first collimating lens.
8. The excitation optics system of claim 5, wherein the excitation module further comprises a second mask, wherein: the light from the second light source traverses a second collimating lens of the two or more collimating lenses; the light exiting the second collimating lens passes through the second mask, the second mask configured to define the illumination area on the sample;
48
KI the light from the first light source has a longer wavelength than the light from the second light source; and the first mask and the tube lens define a first mask-tube lens distance and the second mask and the tube lens define a second mask-tube lens distance, wherein the first mask-tube lens distance and the second mask-tube lens distance are configured to keep light from the first light source exiting the tube lens and light from the second light source exiting the tube lens in parallel.
9. The excitation optics system of claim 1, wherein the excitation module further comprises a vertical turning mirror, wherein the vertical turning mirror is configured to maintain the plane of the sample stage perpendicular to a plane perpendicular to the centerline of the tube lens.
10. The excitation optics system of claim 1, wherein: the excitation objective is mounted on a vertical actuator; and the excitation objective is mounted on a lateral translator.
11. The excitation optics system of claim 1, wherein the one or more partial reflectors are dichroic mirrors.
12. The excitation optics system of claim 1, wherein the excitation module further comprises a photodiode coupled to an unused port of the first partial reflector, wherein the photodiode is configured to receive a leakage signal for calibration of the two or more light sources.
13. An emission optics system, comprising: a sample stage configured to support a sample; and an emission module configured to capture emitted fluorescence light from the sample when the sample is supported by the sample stage, comprising: an emission objective; one or more partial reflectors configured to generate a first transmitted beam and a first reflected beam; and
49
KI two or more image sensors, wherein each image sensor is configured to receive fluorescence light of at least one color emitted from the sample responsive to a light beam output from an excitation module, wherein the two or more image sensors are arranged in a tree configuration based on positions of the one or more partial reflectors.
14. The emission optics system of claim 13, wherein: the light beam output from the excitation module comprises fluorescence light of a first color emitted from the sample and fluorescence light of a second color emitted from the sample; the first transmitted beam includes the light of the first color; and the first reflected beam includes the light of the second color.
15. The emission optics system of claim 13, wherein the one or more partial reflectors are dichroic mirrors.
16. The emission optics system of claim 13, wherein at least one of the two or more image sensors further comprises a bandpass filter configured to narrow a wavelength of the fluorescence light emitted from sample prior to the at least one of the two or more image sensors.
17. The emission optics system of claim 16, wherein a range of the bandpass filter is configured to be within a transmission threshold of at least one of the one or more partial reflectors.
18. The emission optics system of claim 16, wherein a range of the bandpass filter is configured to include a transmission band resulting from a combination of two or more partial reflectors.
19. The emission optics system of claim 16, wherein the at least one of the two or more image sensors further comprises a tube lens configured to receive light passing through the bandpass filter.
50
KI
20. The emission optics system of claim 16, wherein the at least one of the two or more image sensors further comprises a tube lens configured to receive the fluorescence light emitted from the sample prior to the bandpass filter.
21. The emission optics system of claim 13, wherein the emission module further comprises one or more horizontal turning mirrors configured to reduce the footprint of the emission module by angling a path of the fluorescence light prior to reception by at least one image sensor.
22. The emission optics system of claim 13, wherein at least one of the two or more image sensors is a two-dimensional sensor.
23. The emission optics system of claim 13, wherein at least one of the two or more image sensors is a line sensor.
24. The emission optics system of claim 13, wherein the emission module further comprises a tube lens and one or more compensation plates configured to receive the fluorescence light emitted from sample, wherein a thickness of each compensation plate is equal to a thickness of the one or more partial reflectors through which the fluorescence light emitted by the sample has passed.
25. The emission optics system of claim 24, wherein the emission module further comprises at least two partial reflectors, wherein the at least two partial reflectors are configured with orthogonal tilt axes.
26. The emission optics system of claim 13, wherein the emission module further comprises a vertical turning mirror, wherein the vertical turning mirror is configured to maintain the plane of the sample stage perpendicular to the plane of the center of the first partial reflector.
27. The emission optics system of claim 13, wherein: the first transmitted beam comprises light of a first color and light of a third color; the first reflected beam comprises light of a second color and light of a fourth color;
51
KI a second partial reflector of the one or more partial reflectors is configured to generate a second transmitted beam and a second reflected beam from the first transmitted beam, wherein the second transmitted beam comprises the light of the third color and the second reflected beam comprises the fluorescence light of the third color emitted from the sample; and a third partial reflector of the one or more partial reflectors is configured to generate a third transmitted beam and a third reflected beam, wherein the third transmitted beam comprises the light of the second color and the third reflected beam comprises the light of the fourth color.
28. A computer-implemented method, comprising: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.
29. The method of claim 28, wherein receiving the first imaging signal from the first image sensor occurs in parallel to receiving the second imaging signal from the second image sensor.
30. The method of claim 28, further comprising:
52
KI outputting a second command to cause a capture of a low-resolution image of the sample; and outputting a third command to cause a focusing procedure of a portion of the low- resolution image.
31. The method of claim 28, further comprising: applying one or more image corrections to the composite multispectral image including at least one of dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction.
32. A non-transitory computer-readable medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations including: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.
33. The non-transitory computer-readable medium of claim 32, wherein receiving the first imaging signal from the first image sensor occurs in parallel to receiving the second imaging signal from the second image sensor.
53
KI
34. The non-transitory computer-readable medium of claim 32, further comprising: applying one or more image corrections to the composite multi spectral image including at least one of dark-frame subtraction, glass fluorescence background subtraction, ghost image subtraction, or flat-field correction.
35. A system comprising: one or more processors; and one or more computer-readable storage media storing instructions which, when executed by the one or more processors, cause the one or more processors to perform operations including: outputting a first command to cause one or more light sources arranged in an excitation tree configuration included in an excitation module to emit light, wherein: each light source emits light of a particular color; the emitted light from the one or more light sources is directed to a sample supported by a sample stage; and the sample emits fluorescence light responsive to the emitted light received from the excitation module; receiving, from one or more image sensors arranged in an emission tree configuration included in an emission module, a first imaging signal from a first image sensor and a second imaging signal from a second image sensor; generating a first raw image from the first imaging signal; generating a second raw image from the second imaging signal; generating a composite multispectral image comprising the first raw image and the second raw image; and outputting the composite multispectral image.
54
KI
PCT/US2024/045354 2023-09-08 2024-09-05 Filter-based multispectral fluorescence microscope optimized for virtual staining Pending WO2025054302A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363537270P 2023-09-08 2023-09-08
US63/537,270 2023-09-08

Publications (1)

Publication Number Publication Date
WO2025054302A1 true WO2025054302A1 (en) 2025-03-13

Family

ID=94924537

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2024/045354 Pending WO2025054302A1 (en) 2023-09-08 2024-09-05 Filter-based multispectral fluorescence microscope optimized for virtual staining

Country Status (1)

Country Link
WO (1) WO2025054302A1 (en)

Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130242595A1 (en) * 2012-03-16 2013-09-19 Lumencor, Inc. Solid state light source with hybrid optical and electrical intensity control
US20160331230A1 (en) * 2014-01-13 2016-11-17 Nanyang Technological University Device for determining a condition of an organ and method of operating the same
US20170048435A1 (en) * 2014-05-01 2017-02-16 Sony Corporation Illuminating device, control method for illuminating device, and image acquisition system
US20180011021A1 (en) * 2015-01-27 2018-01-11 Hitachi High-Technologies Corporation Multicolor fluorescence analysis device
US20180129863A1 (en) * 2013-07-23 2018-05-10 Northwestern University High throughput partial wave spectroscopic microscopy and associated systems and methods
US20190250385A1 (en) * 2016-10-26 2019-08-15 University Of Science And Technology Of China Microscope having three-dimensional imaging capability and three-dimensional microscopic imaging method
US20200218046A1 (en) * 2017-06-23 2020-07-09 Leica Microsystems Cms Gmbh Microscope system with light sheet microscopy functional unit
US20200225456A1 (en) * 2017-10-02 2020-07-16 Sony Corporation Fluorescence microscope apparatus and fluorescence microscope system
US20210325309A1 (en) * 2020-04-21 2021-10-21 Essen Instruments, Inc. D/B/A Essen Bioscience, Inc. Optical Module with Three or More Color Fluorescent Light Sources and Methods for Use Thereof
US20230103346A1 (en) * 2017-01-11 2023-04-06 Avedro, Inc. Systems and methods for determining cross-linking distribution in a cornea and/or structural characteristics of a cornea
US20230221251A1 (en) * 2022-01-11 2023-07-13 Modulight Corporation Apparatus and method for fluorescence excitation and detection

Patent Citations (11)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20130242595A1 (en) * 2012-03-16 2013-09-19 Lumencor, Inc. Solid state light source with hybrid optical and electrical intensity control
US20180129863A1 (en) * 2013-07-23 2018-05-10 Northwestern University High throughput partial wave spectroscopic microscopy and associated systems and methods
US20160331230A1 (en) * 2014-01-13 2016-11-17 Nanyang Technological University Device for determining a condition of an organ and method of operating the same
US20170048435A1 (en) * 2014-05-01 2017-02-16 Sony Corporation Illuminating device, control method for illuminating device, and image acquisition system
US20180011021A1 (en) * 2015-01-27 2018-01-11 Hitachi High-Technologies Corporation Multicolor fluorescence analysis device
US20190250385A1 (en) * 2016-10-26 2019-08-15 University Of Science And Technology Of China Microscope having three-dimensional imaging capability and three-dimensional microscopic imaging method
US20230103346A1 (en) * 2017-01-11 2023-04-06 Avedro, Inc. Systems and methods for determining cross-linking distribution in a cornea and/or structural characteristics of a cornea
US20200218046A1 (en) * 2017-06-23 2020-07-09 Leica Microsystems Cms Gmbh Microscope system with light sheet microscopy functional unit
US20200225456A1 (en) * 2017-10-02 2020-07-16 Sony Corporation Fluorescence microscope apparatus and fluorescence microscope system
US20210325309A1 (en) * 2020-04-21 2021-10-21 Essen Instruments, Inc. D/B/A Essen Bioscience, Inc. Optical Module with Three or More Color Fluorescent Light Sources and Methods for Use Thereof
US20230221251A1 (en) * 2022-01-11 2023-07-13 Modulight Corporation Apparatus and method for fluorescence excitation and detection

Similar Documents

Publication Publication Date Title
US11131840B2 (en) Microscope system and method for microscopic imaging
JP7424286B2 (en) Fluorescence observation device and fluorescence observation method
US10838190B2 (en) Hyperspectral imaging methods and apparatuses
US10580128B2 (en) Whole slide multispectral imaging systems and methods
JP6272617B2 (en) Method for generating a composite image of an object composed of multiple sub-images
JP4998618B2 (en) Biological imaging device
US12468167B2 (en) Systems and methods for illuminating and imaging objects
ES2959361T3 (en) Real-time autofocus scanning
JP7003241B2 (en) Paste slide judgment system
US10656401B2 (en) Optical apparatus, measuring apparatus, measuring method, screening apparatus, and screening method
US10409049B2 (en) Low resolution slide imaging and slide label imaging and high resolution slide imaging using dual optical paths and a single imaging sensor
JP6845971B2 (en) High throughput hyperspectral imaging system
US20160048014A1 (en) SPIM Arrangement
KR20130123190A (en) Spectral fluorescence lifetime imaging microscope with the use of tunable bandpass filters
US7645971B2 (en) Image scanning apparatus and method
WO2017049226A1 (en) Automated stain finding in pathology bright-field images
CN114730067B (en) Microscopes, beam splitters and microscope systems
WO2025054302A1 (en) Filter-based multispectral fluorescence microscope optimized for virtual staining
US10627638B2 (en) Devices, systems, and methods for illuminating objects
JP2016224368A (en) Imaging apparatus and imaging system
WO2018016010A1 (en) Spectroscopic unit and spectroscopic device
WO2022234484A1 (en) Devices and methods for rapid channel-by-channel fluorescence microscopy

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24863588

Country of ref document: EP

Kind code of ref document: A1