[go: up one dir, main page]

WO2025109168A1 - Slide imaging apparatus for hematological digital morphology - Google Patents

Slide imaging apparatus for hematological digital morphology Download PDF

Info

Publication number
WO2025109168A1
WO2025109168A1 PCT/EP2024/083288 EP2024083288W WO2025109168A1 WO 2025109168 A1 WO2025109168 A1 WO 2025109168A1 EP 2024083288 W EP2024083288 W EP 2024083288W WO 2025109168 A1 WO2025109168 A1 WO 2025109168A1
Authority
WO
WIPO (PCT)
Prior art keywords
slide
image
imaging apparatus
pattern
autofocus
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/EP2024/083288
Other languages
French (fr)
Inventor
Wenzel Salzmann
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Roche Diagnostics International AG
Original Assignee
Roche Diagnostics International AG
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Roche Diagnostics International AG filed Critical Roche Diagnostics International AG
Publication of WO2025109168A1 publication Critical patent/WO2025109168A1/en
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/24Base structure
    • G02B21/241Devices for focusing
    • G02B21/244Devices for focusing using image analysis techniques
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/24Base structure
    • G02B21/241Devices for focusing
    • G02B21/245Devices for focusing using auxiliary sources, detectors
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B21/00Microscopes
    • G02B21/36Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
    • G02B21/365Control or image processing arrangements for digital or video microscopes
    • GPHYSICS
    • G02OPTICS
    • G02BOPTICAL ELEMENTS, SYSTEMS OR APPARATUS
    • G02B7/00Mountings, adjusting means, or light-tight connections, for optical elements
    • G02B7/28Systems for automatic generation of focusing signals
    • G02B7/36Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals
    • G02B7/38Systems for automatic generation of focusing signals using image sharpness techniques, e.g. image processing techniques for generating autofocus signals measured at different points on the optical axis, e.g. focussing on two or more planes and comparing image data

Definitions

  • the present disclosure relates to a slide imaging apparatus for hematological digital morphology, a method for focus control using a slide imaging apparatus, a method for focus finding using a slide imaging apparatus, a computer program and computer-readable storage medium.
  • the devices and methods can be used in the field of tissue diagnostics and hematology, in particular for digital pathology. However, further uses are feasible.
  • Hematological digital morphology requires high image quality of a digital microscopy unit.
  • Most instruments use image analysis methods (e.g. software autofocus) to find a best focus positions. This becomes very challenging as soon as other sample types need to be imaged that also frequently occur in hematological laboratories, such as cerebrospinal fluids (CSF) or pleural fluids. These samples typically have very low cell counts and consequently very low contrast for a software autofocus to optimize to. So far, all tested commercially available systems can have problems in this respect.
  • CSF cerebrospinal fluids
  • pleural fluids pleural fluids.
  • EP 3175 302 B1 describes a method for lens-free imaging of a sample or objects within the sample which uses multi-height iterative phase retrieval and rotational field transformations to perform wide FOV imaging of pathology samples.
  • US6677565B1 describes a method and apparatus for high-speed autofocus and tilt of an inspection surface in a microscope system.
  • An array of spots, lines, circles, grids or other shapes is projected on the surface to be adjusted.
  • the superposition of the array on the surface is imaged by a CCD camera and captured for subsequent analysis. Analysis of the captured image determines both the distance and angle through which the surface must be adjusted to bring it into the focal plane of the optical system.
  • Focus and tilt error is estimated by comparing image dilation and distortion with calibrated data.
  • US 2010/033811 Al describes a microscope including an objective which images a sample along a microscope beam path, and an autofocus device, which is coupled into the microscope beam path via a beam splitter at a location behind the objective direction.
  • a light modulator for generating a two-dimensional, intensity- modulated modulation object is located in the autofocus beam path in a plane conjugated to the focal plane of the objective or intersects the latter and is imaged into the focal plane of the objective.
  • a camera records a two-dimensional image onto which the modulation object's image is imaged. The image plane of the camera intersects a plane that is conjugated to the modulation object or is located in the plane and the camera detecting the contrast of the modulation object's image located in the sample.
  • US 2013/062501 Al describes an autofocus device for an imaging device which has an imaging lens system with a first focal plane, an object stage for holding an object, and a first movement module for the relative movement of object stage and imaging lens system.
  • the autofocus device comprises an image-recording module with a second focal plane, a second movement module for the relative movement of object stage and image-recording module, and a control module which controls the image-recording module for focusing the imaging device.
  • the control module controls the first movement module such that evaluated change in distance between the object stage and the imaging lens system is carried out, and controls the second movement module such that, during the first exposure time for recording the first two-dimensional image, the object stage is moved relative to the image-recording module in a plane parallel to the second focal plane.
  • US 9 229 209 B2 describes an autofocus device for an imaging device, which has an imaging optic having a first focal plane and an object table for moving an object to be imaged relative to the first focal plane, wherein said autofocus device comprises a) an image recording module having a second focal plane, the location thereof relative to the first focal plane being known, b) a lighting module (BM) for imaging a focusing image along a lighting beam path in a focusing image plane such that, of the object is positioned in a target position at a predetermined distance to the second focal plane, the lighting beam path is folded because of reflection on the object and the focusing image, which lies in the focusing image plane, intersects the second focal plane or lies therein, and c) a control module, which activates the object table to focus the imaging device so that the object is positioned in the target position, from a signal of the image recording module, which the image recording module generates on the basis of the recording thereof of the focusing image when the object is positioned in the target position, derives the deviation of the object
  • US 9 389 405 B2 describes an autofocus method for a microscope with an objective which images a sample lying in an object plane, including the steps: projecting a longitudinally extended grating slit which lies in a grating slit plane onto the sample, and imaging the projection of the grating slit onto an autofocus camera; determining an intensity distribution of the grating slit image and from this, deducing a preset for a relative adjustment of sample and object plane; projecting a likewise longitudinally extended comparison slit onto the sample, and imaging the projection of the comparison slit onto the autofocus camera; evaluating the width of the comparison slit image at right angles to the longitudinal extension at at least two sites which are spaced apart along the longitudinal extension, and determining a width variation of the comparison slit image, a gradient of the width variation and a direction of the relative adjustment.
  • the terms “have”, “comprise” or “include” or any arbitrary grammatical variations thereof are used in a non-exclusive way. Thus, these terms may both refer to a situation in which, besides the feature introduced by these terms, no further features are present in the entity described in this context and to a situation in which one or more further features are present.
  • the expressions “A has B”, “A comprises B” and “A includes B” may both refer to a situation in which, besides B, no other element is present in A (i.e. a situation in which A solely and exclusively consists of B) and to a situation in which, besides B, one or more further elements are present in entity A, such as element C, elements C and D or even further elements.
  • the terms “at least one”, “one or more” or similar expressions indicating that a feature or element may be present once or more than once typically will be used only once when introducing the respective feature or element.
  • the expressions “at least one” or “one or more” will not be repeated, non-withstanding the fact that the respective feature or element may be present once or more than once.
  • a slide imaging apparatus for hematological digital morphology is proposed.
  • the term “slide”, also denoted as “microscopic slide” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term may, specifically, refer, without limitation, to a substrate which is designated for a biological specimen to be mounted on a surface of the substrate.
  • the substrate may be mechanically stable and can, therefore comprise any material which provides sufficient mechanical stability.
  • the substrate may preferably exhibit a surface which is configured to be compatible with biological material.
  • the slide is a glass slide. Glass is known, on one hand, to provide sufficient mechanical stability and, on the other hand, to have a high compatibility with biological material.
  • the slide may be a plate having a 2D extension and a thickness.
  • the 2D extension of the plate may exhibit a rectangular or circular form.
  • the thickness of the plate may be small compared to a size of the extension, preferably 20 %, more preferred 10 %, most preferably 5 %, or less than a measure for a linear extent of the 2D extension of the plate.
  • the slide may have a shape which may enable imaging of the biological specimen mounted on the slide.
  • biological specimen as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term may, specifically, refer, without limitation, to any biological sample material such as a fluid or a tissue sample.
  • the biological specimen may be or may comprise any sample material in which blood cells might occur.
  • the biological specimen may be or may comprise a biological material selected from the group consisting of a blood smear sample, a cerebrospinal fluid, or a pleural fluid, urine, synovial fluid, peritoneal fluid, ascites fluid, bone marrow aspirate.
  • other kinds of specimen may also be feasible.
  • the biological specimen is mountable on the microscopic slide.
  • the term “mountable on the slide” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term may, specifically, refer, without limitation, to suitability of the slide to provide the biological specimen.
  • the biological specimen can be applied or inserted onto the slide.
  • the biological specimen may be pretreated during at least one sample preparation step before being mounted on the slide. One or more sample preparation steps may be performed when the biological specimen is mounted on the slide.
  • apparatus as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to a device or system comprising an arbitrary set of interacting or interdependent components. The components may interact with each other in order to fulfill at least one common function. The components may be handled independently or may be coupled or connectable.
  • imaging apparatus as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an apparatus configured for performing at least one imaging function.
  • sample imaging apparatus as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term may, specifically, refer, without limitation, to an apparatus configured for imaging a biological specimen mounted on a slide.
  • imaging or “generating an image” as used herein are broad terms and are to be given their ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning.
  • the terms may, specifically, refer, without limitation, to generating and/or providing a two-dimensional representation of at least one property of the specimen, also denoted by the term “image”.
  • the image can typically, be processed and displayed on a screen for being regarded by eyes of a viewer, e.g. without any further aids, apart from eyeglasses of the viewer.
  • the imaging may comprise generating and/or providing a digital image.
  • digital image as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to a discrete and discontinuous representation of the image.
  • digital image may refer to a two-dimensional function, f(x,y), wherein intensity and/or color values are given for any x, y-position in the digital image, wherein the position may be discretized corresponding to recording pixels of the digital image.
  • imaging system as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning.
  • the term may, specifically, refer, without limitation, to a system configured for generating and/or providing at least one image.
  • the imaging system may comprise at least one imaging sensor and at least one transfer system, e.g. at least one objective.
  • hematological digital morphology is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to morphological analysis of a biological sample, e.g. a blood smear, using digital imaging.
  • morphological analysis may comprise assessing of red blood cell (RBC) findings such as shape, size, color, inclusions, and arrangement.
  • morphological analysis may comprise assessing of white blood cell (WBC) findings such as shape, size, color, size and shape of (intra)cellular structures, inclusions, and arrangement.
  • RBC red blood cell
  • WBC white blood cell
  • the slide imaging apparatus comprises at least one z-stage configured for mounting a microscopic slide at a sample plane, wherein a biological specimen is mountable on the microscopic slide, wherein the z- stage is configured for setting a relative distance between the microscopic slide and an imaging system of the slide imaging apparatus along an optical axis of the slide imaging apparatus; at least one sample beam path comprising at least one sample illumination source configured for illuminating the sample plane and at least one imaging sensor configured for generating at least one digital sample image of at least one part of the microscopic slide upon illumination by the sample illumination source, wherein the sample beam path comprises at least one first transfer system configured for allowing light from the sample plane to propagate to the imaging sensor; at least one autofocus beam path comprising at least one pattern illumination source comprising at least one light emitter and at least one optical grating, wherein the optical grating is tilted with respect to the optical axis of the slide imaging apparatus, wherein the autofocus beam path comprises at least one second transfer system and at least one autofocus camera, wherein
  • stage as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning.
  • the term may, specifically, refer, without limitation, to an element configured for adapting position of the slide relative to further elements of the slide imaging apparatus, e.g. relative to the imaging system.
  • z-stage as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning.
  • the term may, specifically, refer, without limitation, to a stage configured for setting a relative distance between the microscope slide and the imaging system along the optical axis of the slide imaging apparatus.
  • the z-stage is configured for controlling axial movement of the microscopic slide along an optical axis of the slide imaging apparatus.
  • the imaging apparatus comprises a motorized focusing stage.
  • the motorized focusing stage may be coupled to a microscope objective.
  • the motorized focusing stage may be configured for controlling axial movement of the microscope objective along the optical axis.
  • the slide imaging apparatus may comprise an XY-moving stage coupled to the slide.
  • the XY-moving stage may be configured for controlling movement of the slide transversal to the optical axis.
  • the present invention proposes using two beam paths, i.e. the sample beam path and the autofocus beam path.
  • sample beam path is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning.
  • the term may, specifically, refer, without limitation, to a beam path used for magnified imaging the biological specimen on the microscopic slide, e.g. for subsequent assessing of hematological morphology.
  • the sample beam path may be configured for generating a magnified image of the biological specimen on the imaging sensor.
  • autofocus beam path is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning.
  • the term may, specifically, refer, without limitation, to a beam path used for focus finding, e.g. for controlling an optimal position, e.g. within the focal plane, of the microscopic slide along the optical axis of the slide imaging apparatus.
  • the sample beam path comprises at least one sample illumination source configured for illuminating the sample plane and at least one imaging sensor configured for generating at least one digital sample image of at least one part of the microscopic slide upon illumination by the sample illumination source.
  • sample plane as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to a plane perpendicular to the optical axis at which the microscopic slide is positioned.
  • illumination source is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to an arbitrary device configured for emitting light, such as one or more of light in the visible spectral range, the light in the infrared spectral range or light in the ultraviolet spectral range.
  • sample illumination source as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to an illumination source configured and/or arranged such that it can illuminate the sample plane.
  • the sample illumination source may comprise at least one light source.
  • the sample beam path may comprise a bright field microscope.
  • the sample illumination source may be a bright field illumination source and the imaging sensor may be a bright field camera.
  • the sample illumination source may be configured for illuminating the complete specimen on the slide or for at least partially illuminating the specimen.
  • the term “at least partially illuminating” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to embodiments in which the whole slide is illuminated and to embodiments in which sub-portions of the specimen are illuminated.
  • the slide imaging apparatus may be configured for determining an imaging area comprising a region of the slide that comprises at least a portion of the specimen. The imaging area may be further divided into a plurality sub-regions.
  • the slide imaging apparatus may be configured for illuminating the sub-regions subsequently, e.g. by moving the slide in x,y.
  • the illuminating of the specimen may comprise transillumination of the slide.
  • the slide may be arranged between the sample illumination source and the objective.
  • the specimen disposed on the slide may generate in response to the illumination at least one light beam such as by transmission.
  • image sensor as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to at least one sensor device having at least one imaging element configured for recording or capturing spatially resolved one-dimensional, two-dimensional or even three-dimensional optical data or information.
  • the imaging sensor may comprise at least one camera chip, such as at least one CCD chip and/or at least one CMOS chip.
  • the image sensor may comprise at least one camera, wherein the camera is a charge-coupled device (CCD) and/or a complimentary metal-oxide semiconductor (CMOS) image sensor.
  • CCD charge-coupled device
  • CMOS complimentary metal-oxide semiconductor
  • the sample beam path comprises at least one first transfer system configured for allowing light from the sample plane to propagate to the imaging sensor.
  • the term “transfer system” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to one or more optical elements, e.g. having a focal length in response to an impinging light beam and/or at least one further optical property for influencing impinging light.
  • the first transfer system comprises one or more of at least one objective, at least one spectral filter, at least one tube lens, and at least one beam splitter.
  • object also denoted as microscope objective, as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to at least one optical element configured for receiving light generated by the specimen in response to illumination and for focusing the impinging light rays to produce an image.
  • the objective may comprise at least one lens.
  • the objective may comprise a plurality of lenses such as a lens system.
  • the objective may be configured for receiving at least one incident light beam generated by the specimen in response to the illumination.
  • the objective may be configured for generating in combination with a respective imaging lens an image of the specimen on the imaging sensor.
  • the objective may have a defined magnification.
  • the magnification may range from 4x to lOOx.
  • the sample beam path may be configured for generating a magnified image of the specimen on the image sensor.
  • the slide imaging apparatus may constitute a coordinate system, wherein “z” is a coordinate along an optical axis of the microscope objective, also denoted as z-axis or z-direction. A coordinate along the z-axis may be considered a longitudinal coordinate z. The directions transversal to the z-axis may be considered as x- and y-directions.
  • the first transfer system may be configured such that the imaging sensor is parfocal.
  • parfocal as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to a property of the transfer system that the imaging sensor, within tolerances, stays in focus when magnification and/or focal length is changed.
  • the autofocus beam path comprises at least one pattern illumination source.
  • the pattern illumination source comprises at least one light emitter and at least one optical grating.
  • the light emitter may comprise at least one light emitting diode (LED).
  • the light emitter may be configured for generating at least one light beam and the optical grating may be configured for generating the pattern from the at least one impinging light beam.
  • the pattern illumination source may comprise an array of light emitters each configured for generating a light beam.
  • pattern as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one arbitrary pattern comprising a plurality of light features.
  • the pattern comprises at least one pattern selected from the group consisting of: a streak pattern having a plurality of equally spaced strips; a checker board, e.g. a checkered pattern, a pattern having different line density in a field of view of the autofocus camera.
  • the light beams of the pattern may have the same or different wavelengths. Wavelength of the complete optical spectral range may be possible.
  • the pattern is an infrared pattern.
  • the projected light beams of the pattern have a wavelength in the near ultraviolet or near infrared spectral region such as from 400 to 300 nm or a spectral region from 760 nm to 1000 pm, more preferably of 780 nm.
  • the wavelengths of the light beams forming the pattern may be significantly different from the wavelengths used for the sample beam path.
  • the term “ray” generally refers to a line that is perpendicular to wavefronts of light which points in a direction of energy flow.
  • the term “beam” generally refers to a collection of rays. In the following, the terms “ray” and “beam” will be used as synonyms.
  • the term “light beam” generally refers to an amount of light, specifically an amount of light traveling essentially in the same direction, including the possibility of the light beam having a spreading angle or widening angle.
  • the optical grating is tilted with respect to the optical axis of the slide imaging apparatus.
  • the optical grating is tilted by 10° to 45° with respect to the optical axis.
  • the autofocus beam path comprises at least one second transfer system and at least one autofocus camera.
  • the second transfer system is configured for allowing a pattern generated by the pattern illumination source to be projected into the sample plane and for allowing the pattern reflected from the sample plane to propagate to the autofocus camera.
  • the second transfer system may comprise at least one beam splitter configured for reflecting at least a part of a tilted pattern generated by the pattern illumination source into the sample plane and for transmitting the pattern reflected from the sample plane to propagate to the autofocus camera.
  • the second transfer system may comprise at least achromat comprising a plurality of lenses configured for minimizing an imaging error between the pattern and the autofocus camera.
  • the second transfer system may be configured such that the autofocus camera is parfocal.
  • the autofocus camera is configured for generating at least one digital autofocus image of the pattern reflected from the sample plane.
  • the term “autofocus camera” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a camera used for focus finding.
  • the autofocus camera may comprise at least one camera.
  • the camera may a charge-coupled device (CCD) and/or a complimentary metal-oxide semiconductor (CMOS) image sensor.
  • CMOS complimentary metal-oxide semiconductor
  • the camera may have a sufficiently large resolution. The resolution may be sufficient to resolve the streak pattern, e.g. about 1/3 of a streak width.
  • an image of a pattern mask of e.g. equally spaced, parallel strips may be projected into the sample plane.
  • the back reflection off the slide surface is imaged onto a dedicated camera sensor of the autofocus camera.
  • the pattern as well as a dense cell layer is visible (in case of whole blood). Both change contrast as the sample is moved in and out of the focus plane.
  • the slide imaging apparatus comprises at least one processing unit configured for analyzing the digital autofocus image.
  • processing unit as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to an arbitrary logic circuitry configured for performing basic operations of a computer or system, and/or, generally, to a device which is configured for performing calculations or logic operations.
  • the processing unit may be configured for processing basic instructions that drive the computer or system.
  • the processing unit may comprise at least one arithmetic logic unit (ALU), at least one floatingpoint unit (FPU), such as a math coprocessor or a numeric coprocessor, a plurality of registers, specifically registers configured for supplying operands to the ALU and storing results of operations, and a memory, such as an LI and L2 cache memory.
  • ALU arithmetic logic unit
  • FPU floatingpoint unit
  • the processing unit may be a multicore processor.
  • the processing unit may be or may comprise a central processing unit (CPU).
  • the processing unit may be or may comprise a microprocessor, thus specifically the processing unit’s elements may be contained in one single integrated circuitry (IC) chip.
  • the processing unit may be or may comprise one or more application specific-integrated circuits (ASICs) and/or one or more field-programmable gate arrays (FPGAs) or the like.
  • the processing unit may provide one or more hardware elements for performing one or more of the named operations and/or may provide one or more processors with software running thereon for performing one or more of the named operations.
  • the processing unit may comprise one or more programmable devices such as one or more computers, application-specific integrated circuits (ASICs), Digital Signal Processors (DSPs), or Field Programmable Gate Arrays (FPGAs) which are configured to perform the named operations. Additionally or alternatively, however, the processing unit may also fully or partially be embodied by hardware.
  • the analyzing of the digital autofocus image comprises spatially analyzing of contrast and determining an image location of maximum contrast.
  • the analyzing comprises using at least one digital image filtering method based on convolution of the digital autofocus image with one or more sine and cosine functions. This can allow to extract a measure for the pattern contrast only. This way the detected contrast is largely insensitive to presence of cells in the digital autofocus image.
  • the imaged pattern is only in focus within a limited region in the digital autofocus image.
  • the location of highest contrast allows deducing a bi-directional signal that can be used as an input to a focus-finder- procedure and/or algorithm.
  • the focused pattern fraction may move to sides of the autofocus camera.
  • a direction of shift can indicate the direction of defocus (above/below).
  • the slide imaging apparatus is configured for performing at least one focus- finder-procedure comprising the following steps:
  • Step III) of the focus-finder-procedure may comprise the following steps: a) grouping pixels of each of the digital autofocus images into a plurality of image strips; b) cross correlating, per image strip, with a dedicated sine and/or cosine function of the same frequency; c) for each autofocus image, integrating over all envelopes integrals for all image strips and determining the maximum; d) determining a distribution of the maxima as a function of the position of the microscopic slide along the optical axis and applying a linear regression for identifying the focus plane.
  • the focus-finder-procedure may comprise in step b) applying a linear fit per image strip, thereby subtracting background.
  • the focus-finder-procedure may comprise determining a focus error by calculating a deviation for each point of the distribution of the maxima as a function of the position of the microscopic slide.
  • the focus-finder-procedure may be performed repeatedly.
  • the focus-finder-procedure may be performed with and/or without the biological specimen on the slide.
  • the slide imaging apparatus comprises the control unit configured for controlling the z-stage for setting a position of the microscopic slide along the optical axis of the slide imaging apparatus considering the determined image location.
  • control unit generally refers to an arbitrary device configured for performing the named operations, preferably by using at least one processing unit and, more preferably, by using at least one processor and/or at least one application-specific integrated circuit.
  • the control unit may comprise at least one processing unit.
  • the control unit may in particular be programmatically arranged, for example for controlling positioning of the slide.
  • the control unit may be a single-part or multi-part device, which is arranged to control and/or regulate, in whole or in part, an operation of the slide imaging apparatus.
  • the control unit may be arranged to change, in particular to control and/or regulate, one or more operating parameters such as controlling the z-stage or the motorized focusing stage and/or the motorized XY-moving stage and/or imaging.
  • the control unit may be configured for setting the microscopic slide into the focus plane. In case the determined image location is within a central region of the digital autofocus image, the control unit may maintain the position of the microscopic slide. Otherwise, in case the determined image location is shifted from the central region of the digital autofocus image, the control unit may be configured for adapting the position of the microscopic slide. The control unit is configured for adapting the position of the microscopic slide depending on a direction of shift.
  • the slide imaging apparatus may comprise a storage device which is loadable with a plurality of slides and configured for storing the slides.
  • storage device as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term may, specifically, refer, without limitation, to a slide repository which is designated for receiving an individual slide holder or, alternatively, more than one slide holder simultaneously, wherein each slide holder is configured to hold more than one slide.
  • the storage device may be selected from a slide tray or a slide rack, however, further types of storage devices may also be feasible.
  • the storage device may be loadable with the at least one slide, preferably, in a manual fashion, wherein, however, an automatic loading of the storage device may also be conceivable.
  • the slide imaging apparatus may comprise a supply device which is configured to supply the at least one slide from the storage device to the z-stage.
  • supply device as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term may, specifically, refer, without limitation, to a device which is configured to transfer the slide from the storage device to the z-stage.
  • the supply device may comprise a robotic arm.
  • a method for focus control comprises using a slide imaging apparatus according to the present invention such as disclosed in an embodiment of the first aspect or as described in more detail below. With respect to definitions and embodiments for the method reference is made to the description of the slide imaging apparatus.
  • the method comprises the following method steps which, specifically, may be performed in the given order. Still, a different order is also possible. It is further possible to perform two or more of the method steps fully or partially simultaneously. Further, one or more or even all of the method steps may be performed once or may be performed repeatedly, such as repeated once or several times. Further, the method may comprise additional method steps which are not listed.
  • the method comprises the following steps: i. generating at least one pattern by using the pattern illumination source and projecting the pattern into the sample plane; ii. generating at least one digital autofocus image of the pattern reflected from the sample plane by using the autofocus camera; iii. analyzing the digital autofocus image by spatially analyzing of contrast and determining an image location of maximum contrast; iv. controlling the z-stage for setting a position of the microscopic slide along the optical axis of the slide imaging apparatus considering the determined image location by using the control unit.
  • the method may be computer-implemented.
  • computer-implemented as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning.
  • the term specifically may refer, without limitation, to a process which is fully or partially implemented by using a data processing means, such as data processing means comprising at least one processing unit.
  • the term “computer”, thus, may generally refer to a device or to a combination or network of devices having at least one data processing means such as at least one processing unit.
  • the computer additionally, may comprise one or more further components, such as at least one of a data storage device, an electronic interface or a human-machine interface.
  • a method for focus finding comprises using a slide imaging apparatus according to the present invention such as disclosed in an embodiment of the first aspect or as described in more detail below. With respect to definitions and embodiments for the method reference is made to the description of the slide imaging apparatus.
  • the method comprises the following method steps which, specifically, may be performed in the given order. Still, a different order is also possible. It is further possible to perform two or more of the method steps fully or partially simultaneously. Further, one or more or even all of the method steps may be performed once or may be performed repeatedly, such as repeated once or several times. Further, the method may comprise additional method steps which are not listed.
  • the method comprising the following steps:
  • the method may be computer-implemented.
  • a computer program including computer-executable instructions for performing one or both of the methods according to the present invention in one or more of the embodiments enclosed herein when the instructions are executed on a computer or computer network.
  • the computer program may be stored on a computer-readable data carrier and/or on a computer-readable storage medium.
  • computer-readable data carrier and “computer-readable storage medium” specifically may refer to non-transitory data storage means, such as a hardware storage medium having stored thereon computer-executable instructions.
  • the computer- readable data carrier or storage medium specifically may be or may comprise a storage medium such as a random-access memory (RAM) and/or a read-only memory (ROM).
  • RAM random-access memory
  • ROM read-only memory
  • one, more than one or even all of method steps of one or both of the methods as indicated above may be performed by using a computer or a computer network, preferably by using a computer program.
  • program code means may be stored on a computer-readable data carrier and/or on a computer-readable storage medium.
  • a data carrier having a data structure stored thereon, which, after loading into a computer or computer network, such as into a working memory or main memory of the computer or computer network, may execute one or both of the methods according to one or more of the embodiments disclosed herein.
  • non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform one or both of the methods according to one or more of the embodiments disclosed herein.
  • a computer program product with program code means stored on a machine-readable carrier, in order to perform one or both of the methods according to one or more of the embodiments disclosed herein, when the program is executed on a computer or computer network.
  • a computer program product refers to the program as a tradable product.
  • the product may generally exist in an arbitrary format, such as in a paper format, or on a computer-readable data carrier and/or on a computer- readable storage medium.
  • the computer program product may be distributed over a data network.
  • a modulated data signal which contains instructions readable by a computer system or computer network, for performing one or both of the methods according to one or more of the embodiments disclosed herein.
  • one or more of the method steps or even all of the method steps of one or both of the methods according to one or more of the embodiments disclosed herein may be performed by using a computer or computer network.
  • any of the method steps including provision and/or manipulation of data may be performed by using a computer or computer network.
  • these method steps may include any of the method steps, typically except for method steps requiring manual work, such as providing the samples and/or certain aspects of performing the actual measurements.
  • a computer or computer network comprising at least one processor, wherein the processor is adapted to perform one or both of the methods according to one of the embodiments described in this description,
  • a data structure is stored on the storage medium and wherein the data structure is adapted to perform one or both of the methods according to one of the embodiments described in this description after having been loaded into a main and/or working storage of a computer or of a computer network, and
  • program code means can be stored or are stored on a storage medium, for performing one or both of the methods according to one of the embodiments described in this description, if the program code means are executed on a computer or on a computer network.
  • Embodiment 1 A slide imaging apparatus for hematological digital morphology, comprising at least one z-stage configured for mounting a microscopic slide at a sample plane, wherein a biological specimen is mountable on the microscopic slide, wherein the z-stage is configured for setting a relative distance between the microscopic slide and an imaging system of the slide imaging apparatus along an optical axis of the slide imaging apparatus; at least one sample beam path comprising at least one sample illumination source configured for illuminating the sample plane and at least one imaging sensor configured for generating at least one digital sample image of at least one part of the microscopic slide upon illumination by the sample illumination source, wherein the sample beam path comprises at least one first transfer system configured for allowing light from the sample plane to propagate to the imaging sensor; at least one autofocus beam path comprising at least one pattern illumination source comprising at least one light emitter and at least one optical grating, wherein the optical grating is tilted with respect
  • Embodiment 2 The slide imaging apparatus according to the preceding embodiment, wherein the slide imaging apparatus is configured for performing at least one focus- finder-procedure comprising the following steps:
  • Embodiment 3 The slide imaging apparatus according to any one of the preceding embodiments, wherein the pattern comprises at least one pattern selected from the group consisting of a streak pattern having a plurality of equally spaced strips; a checker board, a pattern having different line density in a field of view of the autofocus camera.
  • Embodiment 4 The slide imaging apparatus according to any one of the two preceding embodiments, wherein step III) of the focus-finder-procedure comprises the following steps: a) grouping pixels of each of the digital autofocus images into a plurality of image strips; b) cross correlating, per image strip, with a dedicated sine and/or cosine function of the same frequency; c) for each autofocus image, integrating over all envelopes integrals for all image strips and determining the maximum; d) determining a distribution of the maxima as a function of the position of the microscopic slide along the optical axis and applying a linear regression for identifying the focus plane.
  • Embodiment 5 The slide imaging apparatus according to the preceding embodiment, wherein the focus-finder-procedure comprises in step b) applying a linear fit per image strip, thereby subtracting background.
  • Embodiment 6 The slide imaging apparatus according to any one of the two preceding embodiments, wherein the focus-finder-procedure comprises determining a focus error by calculating a deviation for each point of the distribution of the maxima as a function of the position of the microscopic slide.
  • Embodiment 7 The slide imaging apparatus according to any one of the five preceding embodiments, wherein the control unit is configured for setting the microscopic slide into the focus plane.
  • Embodiment 8 The slide imaging apparatus according to any one of the six preceding embodiments, wherein the focus-finder-procedure is performed repeatedly, wherein the focus-finder-procedure is performed with and/or without the biological specimen on the slide.
  • Embodiment 9 The slide imaging apparatus according to the preceding embodiments, wherein, in case the determined image location is within a central region of the digital autofocus image, the control unit maintains the position of the microscopic slide, wherein otherwise, in case the determined image location is shifted from the central region of the digital autofocus image, the control unit is configured for adapting the position of the microscopic slide, wherein the control unit is configured for adapting the position of the microscopic slide depending on a direction of shift.
  • Embodiment 10 The slide imaging apparatus according to any one of the preceding embodiments, wherein the emitter comprises at least one light emitting diode.
  • Embodiment 11 The slide imaging apparatus according to any one of the preceding embodiments, wherein the pattern is an infrared pattern, wherein projected light beams of the pattern have a wavelength in the near ultraviolet or near infrared spectral region such as from 400 to 300 nm or a spectral region from 760 nm to 1000 pm, more preferably of 780 nm.
  • Embodiment 12 The slide imaging apparatus according to any one of the preceding embodiments, wherein the optical grating is tilted by 10° to 45° with respect to the optical axis.
  • Embodiment 13 The slide imaging apparatus according to any one of the preceding embodiments, wherein the first transfer system comprises one or more of at least one objective, at least one spectral filter, at least one tube lens, and at least one beam splitter.
  • the second transfer system comprises at least one beam splitter configured for reflecting at least a part of a tilted pattern generated by the pattern illumination source into the sample plane and for transmitting the pattern reflected from the sample plane to propagate to the autofocus camera.
  • Embodiment 15 The slide imaging apparatus according to any one of the preceding embodiments, wherein the sample beam path is configured for generating a magnified image of the biological specimen on the imaging sensor.
  • Embodiment 16 The slide imaging apparatus according to any one of the preceding embodiments, wherein the sample beam path comprises a bright field microscope, wherein the sample illumination source is a bright field illumination source and the imaging sensor is a bright field camera.
  • Embodiment 17 The slide imaging apparatus according to any one of the preceding embodiments, wherein the autofocus camera comprises at least one camera, wherein the camera is a charge-coupled device (CCD) and/or a complimentary metal-oxide semiconductor (CMOS) image sensor.
  • CCD charge-coupled device
  • CMOS complimentary metal-oxide semiconductor
  • Embodiment 18 The slide imaging apparatus according to any one of the preceding embodiments, wherein the biological specimen is a blood smear sample, a cerebrospinal fluid, or a pleural fluid, urine, synovial fluid, peritoneal fluid, ascites fluid, bone marrow aspirate.
  • the biological specimen is a blood smear sample, a cerebrospinal fluid, or a pleural fluid, urine, synovial fluid, peritoneal fluid, ascites fluid, bone marrow aspirate.
  • Embodiment 19 A method for focus control using a slide imaging apparatus according to any one of the preceding embodiments, the method comprising the following steps: i. generating at least one pattern by using the pattern illumination source and projecting the pattern into the sample plane; ii. generating at least one digital autofocus image of the pattern reflected from the sample plane by using the autofocus camera; iii. analyzing the digital autofocus image by spatially analyzing of contrast and determining an image location of maximum contrast; iv. controlling the z-stage for setting a position of the microscopic slide along the optical axis of the slide imaging apparatus considering the determined image location by using the control unit.
  • Embodiment 20 The method according to the preceding embodiment, wherein the method is computer-implemented.
  • Embodiment 21 A method for focus finding using a slide imaging apparatus according to any one of the preceding embodiments, the method comprising the following steps:
  • Embodiment 22 The method according to the preceding embodiment, wherein the method is computer-implemented.
  • Embodiment 23 A computer program comprising instructions which, when the program is executed by the slide imaging apparatus according to any one of the preceding embodiments referring to a slide imaging apparatus, cause the slide imaging apparatus to perform one or both of the methods according to any one of the preceding embodiments referring to a method.
  • Embodiment 24 A computer-readable storage medium comprising instructions which, when the instructions are executed by the slide imaging apparatus according to any one of the preceding embodiments referring to a slide imaging apparatus, cause the slide imaging apparatus to perform one or both of the methods according to any one of the preceding embodiments referring to a method.
  • Embodiment 25 A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform one or both of the methods according to any one of the preceding embodiments referring to a method.
  • Figure 1 shows an exemplary embodiment slide imaging apparatus
  • Figures 2A and 2B show exemplary pattern
  • Figures 3 and 4 show exemplary digital autofocus images
  • Figures 5 shows an exemplary analysis of a digital autofocus image
  • Figure 1 shows an exemplary embodiment slide imaging apparatus 110 for hematological digital morphology.
  • the slide imaging apparatus 110 comprises at least one z-stage 112 configured for mounting a microscopic slide 114 at a sample plane.
  • a biological specimen is mountable on the microscopic slide 114.
  • the z-stage 112 is configured for setting a relative distance between the microscopic slide 114 and an imaging system 116 of the slide imaging apparatus 110 along an optical axis 118 of the slide imaging apparatus 110.
  • the slide imaging apparatus 110 comprises two beam paths, i.e. at least one sample beam path and at least one autofocus beam path.
  • the sample beam path may be a beam path used for magnified imaging of the biological specimen on the microscopic slide 114, e.g. for subsequent assessing of hematological morphology.
  • the sample beam path comprises at least one sample illumination source 120 configured for illuminating the sample plane and at least one imaging sensor 122 configured for generating at least one digital sample image of at least one part of the microscopic slide upon illumination by the sample illumination source 120.
  • the sample beam path may comprise a bright field microscope.
  • the sample illumination source 120 may be a bright field illumination source and the imaging sensor 122 may be a bright field camera.
  • the sample beam path comprises at least one first transfer system 124 configured for allowing light from the sample plane to propagate to the imaging sensor 122.
  • the illuminating of the specimen may comprise transillumination of the slide 114.
  • the slide 114 may be arranged between the sample illumination source and an objective 126.
  • the specimen disposed on the slide 114 may generate in response to the illumination at least one light beam such as by transmission.
  • the first transfer system 124 in the embodiment of Figure 1, further comprises at least one spectral filter 128 and at least one tube lens 130.
  • the first transfer system 124 may be configured such that the imaging sensor 122 is parfocal.
  • the autofocus beam path comprises at least one pattern illumination source 132 comprising at least one light emitter 134 and at least one optical grating 136.
  • the light emitter 134 may comprise at least one light emitting diode (LED).
  • the light emitter 134 may be configured for generating at least one light beam and the optical grating 136 may be configured for generating the pattern from the at least one impinging light beam.
  • the pattern illumination source 132 may comprise an array of light emitters 134.
  • the pattern comprises at least one pattern selected from the group consisting of a streak pattern having a plurality of equally spaced strips; a checker board, e.g. a checkered pattern, a pattern having different line density in a field of view of the autofocus camera.
  • Figures 2A and 2B show exemplary pattern.
  • Figure 2A shows a line pattern with constant line density
  • Figure 2B shows a line pattern with different line density in a field of view of the autofocus camera 138.
  • the light beams of the pattern may have the same or different wavelengths, wherein wavelength of the complete optical spectral range are possible.
  • the pattern is an infrared pattern.
  • the projected light beams of the pattern have a wavelength in the near ultraviolet or near infrared spectral region such as from 400 to 300 nm or a spectral region from 760 nm to 1000 pm, more preferably of 780 nm.
  • the wavelengths of the light beams forming the pattern may be significantly different from the wavelengths used for the sample beam path.
  • the optical grating 136 is tilted with respect to the optical axis 118.
  • the optical grating 136 is tilted by 10° to 45° with respect to the optical axis 118.
  • the autofocus beam path comprises at least one second transfer system 140 and the at least one autofocus camera 138.
  • the second transfer system 140 may be configured such that the autofocus camera 138 is parfocal.
  • the second transfer system 140 is configured for allowing a pattern generated by the pattern illumination source 132 to be projected into the sample plane and for allowing the pattern reflected from the sample plane to propagate to the autofocus camera 138.
  • the second transfer system 140 may comprise at least one beam splitter 142 configured for reflecting at least a part of a tilted pattern generated by the pattern illumination source 132 into the sample plane and for transmitting the pattern reflected from the sample plane to propagate to the autofocus camera 138.
  • the second transfer system 140 may comprise two beam splitters 142, e.g.
  • the second transfer system 140 may comprise in the embodiment of Figure 1, two achromats 144, e.g. comprising a plurality of lenses configured for minimizing an imaging error between the pattern and the autofocus camera 138.
  • the second transfer system 140 may comprise further elements such as at least one aspheric condenser lens 146.
  • the autofocus camera 138 is configured for generating at least one digital autofocus image of the pattern reflected from the sample plane.
  • the autofocus camera 138 may comprise at least one camera.
  • the camera may a charge-coupled device (CCD) and/or a complimentary metal-oxide semiconductor (CMOS) image sensor.
  • CMOS complimentary metal-oxide semiconductor
  • the camera may have a sufficiently large resolution. The resolution may be sufficient to resolve the streak pattern, e.g. about 1/3 of a streak width.
  • the slide imaging apparatus 110 comprises at least one processing unit 148 configured for analyzing the digital autofocus image.
  • the analyzing comprises spatially analyzing of contrast and determining an image location of maximum contrast.
  • the analyzing comprises using at least one digital image filtering method based on convolution of the digital autofocus image with one or more sine and cosine functions. This can allow to extract a measure for the pattern contrast only. This way the detected contrast is largely insensitive to presence of cells in the digital autofocus image.
  • the slide imaging apparatus 110 comprises at least one control unit 150 configured for controlling the z-stage 112 for setting a position of the microscopic slide 114 along the optical axis 118 considering the determined image location.
  • the imaged pattern is only in focus within a limited region in the digital autofocus image.
  • the location of highest contrast allows to deduce a bi-directional signal that can be used as an input to a focus- finder-procedure and/or algorithm.
  • the focused pattern fraction may move to sides of the autofocus camera.
  • a direction of shift can indicate the direction of defocus (above/below).
  • an image of a pattern mask of e.g. equally spaced, parallel strips may be projected into the sample plane.
  • the back reflection off the slide surface is imaged onto a dedicated camera sensor of the autofocus camera 138.
  • the pattern as well as a dense cell layer is visible (in case of whole blood). Both change contrast as the sample is moved in and out of the focus plane.
  • Figures 3 and 4 show experimental results of digital autofocus images.
  • the top image shows a digital autofocus image when the slide 114 is in focus.
  • the central region of the digital autofocus image is the image location of maximum contrast.
  • a left image region has the maximum contrast.
  • This image relates to a position of the slide 114 above the focus.
  • a right image region has the maximum contrast. This image relates to a position of the slide 114 below the focus.
  • Figure 4 shows further experimental digital autofocus images using such a line pattern.
  • experimental results of a clean slide i.e. without a sample
  • experimental results of a whole blood printed slide are shown.
  • respective slide 114 is in focus and the central region of the digital autofocus images is the image location of maximum contrast.
  • a left image region has the maximum contrast.
  • This image relates to a position of the slide 114 above the focus.
  • a right image region has the maximum contrast.
  • This image relates to a position of the slide 114 below the focus.
  • the images of the middle and lower row may be obtained at ⁇ 2pm from the focus position.
  • Figures 5 shows an exemplary analysis of a digital autofocus image, e.g. of the middle row of Figure 4 for the clean slide.
  • the analysis may comprise calculating a pixel mean of the original image within 10 vertical strips. The contrast may be analyzed on these strips only, e.g. 400x2000pixel are combined to 400x10.
  • Per strip a linear fit may be applied to remove 0 th and 1 st order background, see top right figure of Figure 5.
  • Per strip a cross correlation with sine and cosine functions of the same frequency may be applied, see middle row left figure of Figure 5. This step can effectively remove impact from printed cells as only the specific spatial frequency of the strip pattern is measured.
  • Figures 6A to 6D show further experimental results, in particular a comparison between a clean slide (Fig. 6A and B) and printed slide (with sample) (Fig. 6C and D).
  • Fig. 6A and 6C the sensor position vs the z position is depicted.
  • Figures 6B and 6D the focus error vs the z position is depicted.
  • Figures 7 A to 7L show further experimental results of the focus-finder-procedure and/or algorithm for different slide conditions.
  • Figures 7A, D, G and J the respective slide condition is shown which relates to the plots of the same row, e.g. Figure 7A shows a clean slide and Figures 7B and 7C relate to results using the clean slide.
  • Figure 7D shows a slide with whole blood
  • Figure 7G a slide with a high cell content
  • Figure 7J a slide with low cell content.
  • Figures 7B, 7C, 7E, 7F, 7H, 71, 7K and 7L show repeated focusing movements in which the autofocus determined the focus position and the z stage is moved in accordingly.
  • the x-axis of these plots shows the position measurement of the z-axis
  • the y-axis of these plots shows the autofocus signal with which the autofocus is moved.
  • Figures 7B, E, H, K show the autofocus signal as a function of the z-position in pm when starting from below the focus position for a number of runs (7 runs for 7B, 3 runs for 7E, 9 runs for 7H and 11 runs for 7K).
  • Figures 7C, F, I, L show the autofocus signal as a function of the z-position in pm when starting from above the focus position for a number of runs (8 runs for 7C, 6 runs for 7F, 10 runs for 71 and 12 runs for 7L).
  • the experimental results show that the focus precision is consistent within ⁇ ⁇ 0.1 pm, e.g. a factor of about 5 less than the objective depth of field.

Landscapes

  • Physics & Mathematics (AREA)
  • Engineering & Computer Science (AREA)
  • General Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Computer Vision & Pattern Recognition (AREA)
  • Chemical & Material Sciences (AREA)
  • Analytical Chemistry (AREA)
  • Multimedia (AREA)
  • Microscoopes, Condenser (AREA)

Abstract

A slide imaging apparatus (110) for hematological digital morphology is disclosed, comprising at least one z-stage (112) configured for mounting a microscopic slide (114) at a sample plane, wherein a biological specimen is mountable on the microscopic slide (114), wherein the z-stage (112) is configured for setting a relative distance between the microscopic slide (114) and an imaging system (116) of the slide imaging apparatus (110) along an optical axis (118) of the slide imaging apparatus (110). The slide imaging apparatus (110) further comprises at least one sample beam path and at least one autofocus beam path comprising at least one pattern illumination source (132) comprising at least one light emitter (134) and at least one optical grating (136), wherein the optical grating (136) is tilted with respect to the optical axis (118) of the slide imaging apparatus (110), wherein the autofocus beam path comprises at least one second transfer system (140) and at least one autofocus camera (138), wherein the second transfer system (140) is configured for allowing a pattern generated by the pattern illumination source (132) to be projected into the sample plane and for allowing the pattern reflected from the sample plane to propagate to the autofocus camera (138), wherein the autofocus camera (138) is configured for generating at least one digital autofocus image of the pattern reflected from the sample plane. The slide imaging apparatus (110) further comprises at least one processing unit (148) configured for analyzing the digital autofocus image, wherein the analyzing comprises spatially analyzing of contrast and determining an image location of maximum contrast. The slide imaging apparatus (110) further comprises at least one control unit (150) configured for controlling the z-stage (112) for setting a position of the microscopic slide (114) along the optical axis (118) of the slide imaging apparatus (110) considering the determined image location.

Description

Slide imaging apparatus for hematological digital morphology
Technical Field
The present disclosure relates to a slide imaging apparatus for hematological digital morphology, a method for focus control using a slide imaging apparatus, a method for focus finding using a slide imaging apparatus, a computer program and computer-readable storage medium. Herein, the devices and methods can be used in the field of tissue diagnostics and hematology, in particular for digital pathology. However, further uses are feasible.
Background art
Hematological digital morphology requires high image quality of a digital microscopy unit. Most instruments use image analysis methods (e.g. software autofocus) to find a best focus positions. This becomes very challenging as soon as other sample types need to be imaged that also frequently occur in hematological laboratories, such as cerebrospinal fluids (CSF) or pleural fluids. These samples typically have very low cell counts and consequently very low contrast for a software autofocus to optimize to. So far, all tested commercially available systems can have problems in this respect. On the other hand, among the available hardware autofocus methods, none succeeded to focus whole blood smears due to the high level of signal disturbance. Thus, there is a need for a method that is capable to give robust focusing performance for all sample types, in particular for a future fully automated hematology solution. EP 3175 302 B1 describes a method for lens-free imaging of a sample or objects within the sample which uses multi-height iterative phase retrieval and rotational field transformations to perform wide FOV imaging of pathology samples.
D. Clark et al., “A Rapid Image Acquisition Method for Focus Stacking in Microscopy”, Microscopy Today , Volume 23 , Issue 4 , July 2015 , pp. 18 - 25, DOI: https://doi.org/10.1017/S1551929515000577 a system for rapidly obtaining a set of images at video rates.
US6677565B1 describes a method and apparatus for high-speed autofocus and tilt of an inspection surface in a microscope system. An array of spots, lines, circles, grids or other shapes is projected on the surface to be adjusted. The superposition of the array on the surface is imaged by a CCD camera and captured for subsequent analysis. Analysis of the captured image determines both the distance and angle through which the surface must be adjusted to bring it into the focal plane of the optical system. Focus and tilt error is estimated by comparing image dilation and distortion with calibrated data.
US 2010/033811 Al describes a microscope including an objective which images a sample along a microscope beam path, and an autofocus device, which is coupled into the microscope beam path via a beam splitter at a location behind the objective direction. A light modulator for generating a two-dimensional, intensity- modulated modulation object, is located in the autofocus beam path in a plane conjugated to the focal plane of the objective or intersects the latter and is imaged into the focal plane of the objective. A camera records a two-dimensional image onto which the modulation object's image is imaged. The image plane of the camera intersects a plane that is conjugated to the modulation object or is located in the plane and the camera detecting the contrast of the modulation object's image located in the sample.
US 2013/062501 Al describes an autofocus device for an imaging device which has an imaging lens system with a first focal plane, an object stage for holding an object, and a first movement module for the relative movement of object stage and imaging lens system. The autofocus device comprises an image-recording module with a second focal plane, a second movement module for the relative movement of object stage and image-recording module, and a control module which controls the image-recording module for focusing the imaging device. The control module controls the first movement module such that evaluated change in distance between the object stage and the imaging lens system is carried out, and controls the second movement module such that, during the first exposure time for recording the first two-dimensional image, the object stage is moved relative to the image-recording module in a plane parallel to the second focal plane.
US 9 229 209 B2 describes an autofocus device for an imaging device, which has an imaging optic having a first focal plane and an object table for moving an object to be imaged relative to the first focal plane, wherein said autofocus device comprises a) an image recording module having a second focal plane, the location thereof relative to the first focal plane being known, b) a lighting module (BM) for imaging a focusing image along a lighting beam path in a focusing image plane such that, of the object is positioned in a target position at a predetermined distance to the second focal plane, the lighting beam path is folded because of reflection on the object and the focusing image, which lies in the focusing image plane, intersects the second focal plane or lies therein, and c) a control module, which activates the object table to focus the imaging device so that the object is positioned in the target position, from a signal of the image recording module, which the image recording module generates on the basis of the recording thereof of the focusing image when the object is positioned in the target position, derives the deviation of the object position from the target position, and, based on the derived deviation, the predetermined distance, and the relative location of the first and second focal planes, activates the object table so that the object is positioned in the first focal plane.
US 9 389 405 B2 describes an autofocus method for a microscope with an objective which images a sample lying in an object plane, including the steps: projecting a longitudinally extended grating slit which lies in a grating slit plane onto the sample, and imaging the projection of the grating slit onto an autofocus camera; determining an intensity distribution of the grating slit image and from this, deducing a preset for a relative adjustment of sample and object plane; projecting a likewise longitudinally extended comparison slit onto the sample, and imaging the projection of the comparison slit onto the autofocus camera; evaluating the width of the comparison slit image at right angles to the longitudinal extension at at least two sites which are spaced apart along the longitudinal extension, and determining a width variation of the comparison slit image, a gradient of the width variation and a direction of the relative adjustment.
Problem to be solved
It is therefore an objective of the present invention to provide a slide imaging apparatus for hematological digital morphology, a method for focus control using a slide imaging apparatus, a method for focus finding using a slide imaging apparatus, a computer program and computer-readable storage medium, which at least partially avoid the shortcomings of known devices and methods of this kind and which at least partially address the above-mentioned challenges. Specifically, robust focusing performance for all sample types is desired.
Summary
This problem is addressed by a slide imaging apparatus for hematological digital morphology, a method for focus control using a slide imaging apparatus, a method for focus finding using a slide imaging apparatus, a computer program and computer-readable storage medium with the features of the independent claims. Advantageous embodiments which might be realized in an isolated fashion or in any arbitrary combinations are listed in the dependent claims as well as throughout the specification.
As used in the following, the terms “have”, “comprise” or “include” or any arbitrary grammatical variations thereof are used in a non-exclusive way. Thus, these terms may both refer to a situation in which, besides the feature introduced by these terms, no further features are present in the entity described in this context and to a situation in which one or more further features are present. As an example, the expressions “A has B”, “A comprises B” and “A includes B” may both refer to a situation in which, besides B, no other element is present in A (i.e. a situation in which A solely and exclusively consists of B) and to a situation in which, besides B, one or more further elements are present in entity A, such as element C, elements C and D or even further elements.
Further, it shall be noted that the terms “at least one”, “one or more” or similar expressions indicating that a feature or element may be present once or more than once typically will be used only once when introducing the respective feature or element. In the following, in most cases, when referring to the respective feature or element, the expressions “at least one” or “one or more” will not be repeated, non-withstanding the fact that the respective feature or element may be present once or more than once.
Further, as used in the following, the terms "preferably", "more preferably", "particularly", "more particularly", "specifically", "more specifically" or similar terms are used in conjunction with optional features, without restricting alternative possibilities. Thus, features introduced by these terms are optional features and are not intended to restrict the scope of the claims in any way. The invention may, as the skilled person will recognize, be performed by using alternative features. Similarly, features introduced by "in an embodiment of the invention" or similar expressions are intended to be optional features, without any restriction regarding alternative embodiments of the invention, without any restrictions regarding the scope of the invention and without any restriction regarding the possibility of combining the features introduced in such way with other optional or non-optional features of the invention.
In a first aspect, a slide imaging apparatus for hematological digital morphology is proposed.
The term “slide”, also denoted as “microscopic slide” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term may, specifically, refer, without limitation, to a substrate which is designated for a biological specimen to be mounted on a surface of the substrate. The substrate may be mechanically stable and can, therefore comprise any material which provides sufficient mechanical stability. The substrate may preferably exhibit a surface which is configured to be compatible with biological material. By way of example, the slide is a glass slide. Glass is known, on one hand, to provide sufficient mechanical stability and, on the other hand, to have a high compatibility with biological material. However, further kinds of materials for the slides may also be feasible. The slide may be a plate having a 2D extension and a thickness. The 2D extension of the plate may exhibit a rectangular or circular form. The thickness of the plate may be small compared to a size of the extension, preferably 20 %, more preferred 10 %, most preferably 5 %, or less than a measure for a linear extent of the 2D extension of the plate. Further, the slide may have a shape which may enable imaging of the biological specimen mounted on the slide.
The term “biological specimen” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term may, specifically, refer, without limitation, to any biological sample material such as a fluid or a tissue sample. The biological specimen may be or may comprise any sample material in which blood cells might occur. The biological specimen may be or may comprise a biological material selected from the group consisting of a blood smear sample, a cerebrospinal fluid, or a pleural fluid, urine, synovial fluid, peritoneal fluid, ascites fluid, bone marrow aspirate. However, other kinds of specimen may also be feasible.
The biological specimen is mountable on the microscopic slide. The term “mountable on the slide” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term may, specifically, refer, without limitation, to suitability of the slide to provide the biological specimen. For example, the biological specimen can be applied or inserted onto the slide. The biological specimen may be pretreated during at least one sample preparation step before being mounted on the slide. One or more sample preparation steps may be performed when the biological specimen is mounted on the slide.
The term "apparatus" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a device or system comprising an arbitrary set of interacting or interdependent components. The components may interact with each other in order to fulfill at least one common function. The components may be handled independently or may be coupled or connectable. The term “imaging apparatus” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an apparatus configured for performing at least one imaging function. The term “slide imaging apparatus” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term may, specifically, refer, without limitation, to an apparatus configured for imaging a biological specimen mounted on a slide.
The term “imaging” or “generating an image” as used herein are broad terms and are to be given their ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning. The terms may, specifically, refer, without limitation, to generating and/or providing a two-dimensional representation of at least one property of the specimen, also denoted by the term “image”. The image can typically, be processed and displayed on a screen for being regarded by eyes of a viewer, e.g. without any further aids, apart from eyeglasses of the viewer. The imaging may comprise generating and/or providing a digital image. The term “digital image” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a discrete and discontinuous representation of the image. Thus, the term “digital image” may refer to a two-dimensional function, f(x,y), wherein intensity and/or color values are given for any x, y-position in the digital image, wherein the position may be discretized corresponding to recording pixels of the digital image. The term “imaging system” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning. The term may, specifically, refer, without limitation, to a system configured for generating and/or providing at least one image. The imaging system may comprise at least one imaging sensor and at least one transfer system, e.g. at least one objective.
The term "hematological digital morphology" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to morphological analysis of a biological sample, e.g. a blood smear, using digital imaging. For example, morphological analysis may comprise assessing of red blood cell (RBC) findings such as shape, size, color, inclusions, and arrangement. In another example, morphological analysis may comprise assessing of white blood cell (WBC) findings such as shape, size, color, size and shape of (intra)cellular structures, inclusions, and arrangement.
The slide imaging apparatus comprises at least one z-stage configured for mounting a microscopic slide at a sample plane, wherein a biological specimen is mountable on the microscopic slide, wherein the z- stage is configured for setting a relative distance between the microscopic slide and an imaging system of the slide imaging apparatus along an optical axis of the slide imaging apparatus; at least one sample beam path comprising at least one sample illumination source configured for illuminating the sample plane and at least one imaging sensor configured for generating at least one digital sample image of at least one part of the microscopic slide upon illumination by the sample illumination source, wherein the sample beam path comprises at least one first transfer system configured for allowing light from the sample plane to propagate to the imaging sensor; at least one autofocus beam path comprising at least one pattern illumination source comprising at least one light emitter and at least one optical grating, wherein the optical grating is tilted with respect to the optical axis of the slide imaging apparatus, wherein the autofocus beam path comprises at least one second transfer system and at least one autofocus camera, wherein the second transfer system is configured for allowing a pattern generated by the pattern illumination source to be projected into the sample plane and for allowing the pattern reflected from the sample plane to propagate to the autofocus camera, wherein the autofocus camera is configured for generating at least one digital autofocus image of the pattern reflected from the sample plane; at least one processing unit configured for analyzing the digital autofocus image, wherein the analyzing comprises spatially analyzing of contrast and determining an image location of maximum contrast; at least one control unit configured for controlling the z-stage for setting a position of the microscopic slide along the optical axis of the slide imaging apparatus considering the determined image location.
The term “stage” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning. The term may, specifically, refer, without limitation, to an element configured for adapting position of the slide relative to further elements of the slide imaging apparatus, e.g. relative to the imaging system. The term “z-stage” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning. The term may, specifically, refer, without limitation, to a stage configured for setting a relative distance between the microscope slide and the imaging system along the optical axis of the slide imaging apparatus. For example, the z-stage is configured for controlling axial movement of the microscopic slide along an optical axis of the slide imaging apparatus. Additionally or alternatively, the imaging apparatus comprises a motorized focusing stage. The motorized focusing stage may be coupled to a microscope objective. The motorized focusing stage may be configured for controlling axial movement of the microscope objective along the optical axis. Further, the slide imaging apparatus may comprise an XY-moving stage coupled to the slide. The XY-moving stage may be configured for controlling movement of the slide transversal to the optical axis.
The present invention proposes using two beam paths, i.e. the sample beam path and the autofocus beam path.
The term “sample beam path” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning. The term may, specifically, refer, without limitation, to a beam path used for magnified imaging the biological specimen on the microscopic slide, e.g. for subsequent assessing of hematological morphology. The sample beam path may be configured for generating a magnified image of the biological specimen on the imaging sensor.
The term “autofocus beam path” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and are not to be limited to a special or customized meaning. The term may, specifically, refer, without limitation, to a beam path used for focus finding, e.g. for controlling an optimal position, e.g. within the focal plane, of the microscopic slide along the optical axis of the slide imaging apparatus. The sample beam path comprises at least one sample illumination source configured for illuminating the sample plane and at least one imaging sensor configured for generating at least one digital sample image of at least one part of the microscopic slide upon illumination by the sample illumination source.
The term "sample plane" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a plane perpendicular to the optical axis at which the microscopic slide is positioned.
The term "illumination source" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary device configured for emitting light, such as one or more of light in the visible spectral range, the light in the infrared spectral range or light in the ultraviolet spectral range. The term "sample illumination source" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an illumination source configured and/or arranged such that it can illuminate the sample plane. The sample illumination source may comprise at least one light source. For example, the sample beam path may comprise a bright field microscope. The sample illumination source may be a bright field illumination source and the imaging sensor may be a bright field camera.
The sample illumination source may be configured for illuminating the complete specimen on the slide or for at least partially illuminating the specimen. The term “at least partially illuminating” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to embodiments in which the whole slide is illuminated and to embodiments in which sub-portions of the specimen are illuminated. The slide imaging apparatus may be configured for determining an imaging area comprising a region of the slide that comprises at least a portion of the specimen. The imaging area may be further divided into a plurality sub-regions. The slide imaging apparatus may be configured for illuminating the sub-regions subsequently, e.g. by moving the slide in x,y. The illuminating of the specimen may comprise transillumination of the slide. Thus, the slide may be arranged between the sample illumination source and the objective. The specimen disposed on the slide may generate in response to the illumination at least one light beam such as by transmission.
The term "image sensor" as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one sensor device having at least one imaging element configured for recording or capturing spatially resolved one-dimensional, two-dimensional or even three-dimensional optical data or information. As an example, the imaging sensor may comprise at least one camera chip, such as at least one CCD chip and/or at least one CMOS chip. The image sensor may comprise at least one camera, wherein the camera is a charge-coupled device (CCD) and/or a complimentary metal-oxide semiconductor (CMOS) image sensor.
The sample beam path comprises at least one first transfer system configured for allowing light from the sample plane to propagate to the imaging sensor. The term “transfer system” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to one or more optical elements, e.g. having a focal length in response to an impinging light beam and/or at least one further optical property for influencing impinging light. For example, the first transfer system comprises one or more of at least one objective, at least one spectral filter, at least one tube lens, and at least one beam splitter. The term "objective", also denoted as microscope objective, as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one optical element configured for receiving light generated by the specimen in response to illumination and for focusing the impinging light rays to produce an image. The objective may comprise at least one lens. The objective may comprise a plurality of lenses such as a lens system. The objective may be configured for receiving at least one incident light beam generated by the specimen in response to the illumination. The objective may be configured for generating in combination with a respective imaging lens an image of the specimen on the imaging sensor. The objective may have a defined magnification. The magnification may range from 4x to lOOx. The sample beam path may be configured for generating a magnified image of the specimen on the image sensor. The slide imaging apparatus may constitute a coordinate system, wherein “z” is a coordinate along an optical axis of the microscope objective, also denoted as z-axis or z-direction. A coordinate along the z-axis may be considered a longitudinal coordinate z. The directions transversal to the z-axis may be considered as x- and y-directions.
The first transfer system may be configured such that the imaging sensor is parfocal. The term “parfocal” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a property of the transfer system that the imaging sensor, within tolerances, stays in focus when magnification and/or focal length is changed.
The autofocus beam path comprises at least one pattern illumination source. The pattern illumination source comprises at least one light emitter and at least one optical grating. The light emitter may comprise at least one light emitting diode (LED). The light emitter may be configured for generating at least one light beam and the optical grating may be configured for generating the pattern from the at least one impinging light beam. The pattern illumination source may comprise an array of light emitters each configured for generating a light beam. The term “pattern” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to at least one arbitrary pattern comprising a plurality of light features. For example, the pattern comprises at least one pattern selected from the group consisting of: a streak pattern having a plurality of equally spaced strips; a checker board, e.g. a checkered pattern, a pattern having different line density in a field of view of the autofocus camera.
The light beams of the pattern may have the same or different wavelengths. Wavelength of the complete optical spectral range may be possible. For example, the pattern is an infrared pattern. The projected light beams of the pattern have a wavelength in the near ultraviolet or near infrared spectral region such as from 400 to 300 nm or a spectral region from 760 nm to 1000 pm, more preferably of 780 nm. The wavelengths of the light beams forming the pattern may be significantly different from the wavelengths used for the sample beam path.
As used herein, the term “ray” generally refers to a line that is perpendicular to wavefronts of light which points in a direction of energy flow. As used herein, the term “beam” generally refers to a collection of rays. In the following, the terms “ray” and “beam” will be used as synonyms. As further used herein, the term “light beam” generally refers to an amount of light, specifically an amount of light traveling essentially in the same direction, including the possibility of the light beam having a spreading angle or widening angle.
The optical grating is tilted with respect to the optical axis of the slide imaging apparatus. For example, the optical grating is tilted by 10° to 45° with respect to the optical axis.
The autofocus beam path comprises at least one second transfer system and at least one autofocus camera. The second transfer system is configured for allowing a pattern generated by the pattern illumination source to be projected into the sample plane and for allowing the pattern reflected from the sample plane to propagate to the autofocus camera. The second transfer system may comprise at least one beam splitter configured for reflecting at least a part of a tilted pattern generated by the pattern illumination source into the sample plane and for transmitting the pattern reflected from the sample plane to propagate to the autofocus camera. The second transfer system may comprise at least achromat comprising a plurality of lenses configured for minimizing an imaging error between the pattern and the autofocus camera. The second transfer system may be configured such that the autofocus camera is parfocal.
The autofocus camera is configured for generating at least one digital autofocus image of the pattern reflected from the sample plane. The term “autofocus camera” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a camera used for focus finding. The autofocus camera may comprise at least one camera. The camera may a charge-coupled device (CCD) and/or a complimentary metal-oxide semiconductor (CMOS) image sensor. The camera may have a sufficiently large resolution. The resolution may be sufficient to resolve the streak pattern, e.g. about 1/3 of a streak width.
For example, in the autofocus beam path, an image of a pattern mask of e.g. equally spaced, parallel strips may be projected into the sample plane. The back reflection off the slide surface is imaged onto a dedicated camera sensor of the autofocus camera. In the camera’s image, the pattern as well as a dense cell layer is visible (in case of whole blood). Both change contrast as the sample is moved in and out of the focus plane.
The slide imaging apparatus comprises at least one processing unit configured for analyzing the digital autofocus image. The term “processing unit” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to an arbitrary logic circuitry configured for performing basic operations of a computer or system, and/or, generally, to a device which is configured for performing calculations or logic operations. In particular, the processing unit may be configured for processing basic instructions that drive the computer or system. As an example, the processing unit may comprise at least one arithmetic logic unit (ALU), at least one floatingpoint unit (FPU), such as a math coprocessor or a numeric coprocessor, a plurality of registers, specifically registers configured for supplying operands to the ALU and storing results of operations, and a memory, such as an LI and L2 cache memory. In particular, the processing unit may be a multicore processor. Specifically, the processing unit may be or may comprise a central processing unit (CPU). Additionally or alternatively, the processing unit may be or may comprise a microprocessor, thus specifically the processing unit’s elements may be contained in one single integrated circuitry (IC) chip. Additionally or alternatively, the processing unit may be or may comprise one or more application specific-integrated circuits (ASICs) and/or one or more field-programmable gate arrays (FPGAs) or the like. The processing unit may provide one or more hardware elements for performing one or more of the named operations and/or may provide one or more processors with software running thereon for performing one or more of the named operations. The processing unit may comprise one or more programmable devices such as one or more computers, application-specific integrated circuits (ASICs), Digital Signal Processors (DSPs), or Field Programmable Gate Arrays (FPGAs) which are configured to perform the named operations. Additionally or alternatively, however, the processing unit may also fully or partially be embodied by hardware.
The analyzing of the digital autofocus image comprises spatially analyzing of contrast and determining an image location of maximum contrast. The analyzing comprises using at least one digital image filtering method based on convolution of the digital autofocus image with one or more sine and cosine functions. This can allow to extract a measure for the pattern contrast only. This way the detected contrast is largely insensitive to presence of cells in the digital autofocus image.
By applying a tilt to the optical grating and, thus, to the pattern, the imaged pattern is only in focus within a limited region in the digital autofocus image. The location of highest contrast allows deducing a bi-directional signal that can be used as an input to a focus-finder- procedure and/or algorithm. When the slide is moved along the optical axis, i.e. in and out of focus, the focused pattern fraction may move to sides of the autofocus camera. Thus, a direction of shift can indicate the direction of defocus (above/below).
For example, the slide imaging apparatus is configured for performing at least one focus- finder-procedure comprising the following steps:
I. generating at least one pattern by using the pattern illumination source and projecting the pattern into the sample plane;
II. generating a plurality of digital autofocus images at a plurality of different positions of the microscopic slide along the optical axis,
III. for each autofocus image, analyzing the digital autofocus image by spatially analyzing of contrast and determining an image location of maximum contrast;
IV. determining the position of the microscopic slide along the optical axis at which the image location is most central in the digital autofocus image thereby identifying a focus plane.
Step III) of the focus-finder-procedure may comprise the following steps: a) grouping pixels of each of the digital autofocus images into a plurality of image strips; b) cross correlating, per image strip, with a dedicated sine and/or cosine function of the same frequency; c) for each autofocus image, integrating over all envelopes integrals for all image strips and determining the maximum; d) determining a distribution of the maxima as a function of the position of the microscopic slide along the optical axis and applying a linear regression for identifying the focus plane.
The focus-finder-procedure may comprise in step b) applying a linear fit per image strip, thereby subtracting background.
The focus-finder-procedure may comprise determining a focus error by calculating a deviation for each point of the distribution of the maxima as a function of the position of the microscopic slide.
The focus-finder-procedure may performed repeatedly. The focus-finder-procedure may be performed with and/or without the biological specimen on the slide. The slide imaging apparatus comprises the control unit configured for controlling the z-stage for setting a position of the microscopic slide along the optical axis of the slide imaging apparatus considering the determined image location. As further used herein, the term “control unit” generally refers to an arbitrary device configured for performing the named operations, preferably by using at least one processing unit and, more preferably, by using at least one processor and/or at least one application-specific integrated circuit. Thus, as an example, the control unit may comprise at least one processing unit. The control unit may in particular be programmatically arranged, for example for controlling positioning of the slide. The control unit may be a single-part or multi-part device, which is arranged to control and/or regulate, in whole or in part, an operation of the slide imaging apparatus. In particular, the control unit may be arranged to change, in particular to control and/or regulate, one or more operating parameters such as controlling the z-stage or the motorized focusing stage and/or the motorized XY-moving stage and/or imaging.
The control unit may be configured for setting the microscopic slide into the focus plane. In case the determined image location is within a central region of the digital autofocus image, the control unit may maintain the position of the microscopic slide. Otherwise, in case the determined image location is shifted from the central region of the digital autofocus image, the control unit may be configured for adapting the position of the microscopic slide. The control unit is configured for adapting the position of the microscopic slide depending on a direction of shift.
The slide imaging apparatus may comprise a storage device which is loadable with a plurality of slides and configured for storing the slides. The term “storage device” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term may, specifically, refer, without limitation, to a slide repository which is designated for receiving an individual slide holder or, alternatively, more than one slide holder simultaneously, wherein each slide holder is configured to hold more than one slide. The storage device may be selected from a slide tray or a slide rack, however, further types of storage devices may also be feasible. The storage device may be loadable with the at least one slide, preferably, in a manual fashion, wherein, however, an automatic loading of the storage device may also be conceivable.
The slide imaging apparatus may comprise a supply device which is configured to supply the at least one slide from the storage device to the z-stage. The term “supply device” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term may, specifically, refer, without limitation, to a device which is configured to transfer the slide from the storage device to the z-stage. For this purpose, the supply device may comprise a robotic arm.
In a further aspect of the present invention, a method for focus control is proposed. The method comprises using a slide imaging apparatus according to the present invention such as disclosed in an embodiment of the first aspect or as described in more detail below. With respect to definitions and embodiments for the method reference is made to the description of the slide imaging apparatus.
The method comprises the following method steps which, specifically, may be performed in the given order. Still, a different order is also possible. It is further possible to perform two or more of the method steps fully or partially simultaneously. Further, one or more or even all of the method steps may be performed once or may be performed repeatedly, such as repeated once or several times. Further, the method may comprise additional method steps which are not listed.
The method comprises the following steps: i. generating at least one pattern by using the pattern illumination source and projecting the pattern into the sample plane; ii. generating at least one digital autofocus image of the pattern reflected from the sample plane by using the autofocus camera; iii. analyzing the digital autofocus image by spatially analyzing of contrast and determining an image location of maximum contrast; iv. controlling the z-stage for setting a position of the microscopic slide along the optical axis of the slide imaging apparatus considering the determined image location by using the control unit.
The method may be computer-implemented. The term “computer-implemented” as used herein is a broad term and is to be given its ordinary and customary meaning to a person of ordinary skill in the art and is not to be limited to a special or customized meaning. The term specifically may refer, without limitation, to a process which is fully or partially implemented by using a data processing means, such as data processing means comprising at least one processing unit. The term “computer”, thus, may generally refer to a device or to a combination or network of devices having at least one data processing means such as at least one processing unit. The computer, additionally, may comprise one or more further components, such as at least one of a data storage device, an electronic interface or a human-machine interface.
In a further aspect of the present invention, a method for focus finding is proposed. The method comprises using a slide imaging apparatus according to the present invention such as disclosed in an embodiment of the first aspect or as described in more detail below. With respect to definitions and embodiments for the method reference is made to the description of the slide imaging apparatus.
The method comprises the following method steps which, specifically, may be performed in the given order. Still, a different order is also possible. It is further possible to perform two or more of the method steps fully or partially simultaneously. Further, one or more or even all of the method steps may be performed once or may be performed repeatedly, such as repeated once or several times. Further, the method may comprise additional method steps which are not listed.
The method comprising the following steps:
I. generating at least one pattern by using the pattern illumination source and projecting the pattern into the sample plane;
II. generating a plurality of digital autofocus images at a plurality of different positions of the microscopic slide along the optical axis,
III. for each autofocus image, analyzing the digital autofocus image by spatially analyzing of contrast and determining an image location of maximum sharpness;
IV. determining the position of the microscopic slide along the optical axis at which the image location is most central in the digital autofocus image thereby identifying a focus plane.
The method may be computer-implemented.
Further disclosed and proposed herein is a computer program including computer-executable instructions for performing one or both of the methods according to the present invention in one or more of the embodiments enclosed herein when the instructions are executed on a computer or computer network. Specifically, the computer program may be stored on a computer-readable data carrier and/or on a computer-readable storage medium.
As used herein, the terms “computer-readable data carrier” and “computer-readable storage medium” specifically may refer to non-transitory data storage means, such as a hardware storage medium having stored thereon computer-executable instructions. The computer- readable data carrier or storage medium specifically may be or may comprise a storage medium such as a random-access memory (RAM) and/or a read-only memory (ROM).
Thus, specifically, one, more than one or even all of method steps of one or both of the methods as indicated above may be performed by using a computer or a computer network, preferably by using a computer program.
Further disclosed and proposed herein is a computer program product having program code means, in order to perform one or both of the methods according to the present invention in one or more of the embodiments enclosed herein when the program is executed on a computer or computer network. Specifically, the program code means may be stored on a computer-readable data carrier and/or on a computer-readable storage medium.
Further disclosed and proposed herein is a data carrier having a data structure stored thereon, which, after loading into a computer or computer network, such as into a working memory or main memory of the computer or computer network, may execute one or both of the methods according to one or more of the embodiments disclosed herein.
Further disclosed and proposed herein is a non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform one or both of the methods according to one or more of the embodiments disclosed herein.
Further disclosed and proposed herein is a computer program product with program code means stored on a machine-readable carrier, in order to perform one or both of the methods according to one or more of the embodiments disclosed herein, when the program is executed on a computer or computer network. As used herein, a computer program product refers to the program as a tradable product. The product may generally exist in an arbitrary format, such as in a paper format, or on a computer-readable data carrier and/or on a computer- readable storage medium. Specifically, the computer program product may be distributed over a data network.
Finally, disclosed and proposed herein is a modulated data signal which contains instructions readable by a computer system or computer network, for performing one or both of the methods according to one or more of the embodiments disclosed herein. Referring to the computer-implemented aspects of the invention, one or more of the method steps or even all of the method steps of one or both of the methods according to one or more of the embodiments disclosed herein may be performed by using a computer or computer network. Thus, generally, any of the method steps including provision and/or manipulation of data may be performed by using a computer or computer network. Generally, these method steps may include any of the method steps, typically except for method steps requiring manual work, such as providing the samples and/or certain aspects of performing the actual measurements.
Specifically, further disclosed herein are:
- a computer or computer network comprising at least one processor, wherein the processor is adapted to perform one or both of the methods according to one of the embodiments described in this description,
- a computer loadable data structure that is adapted to perform one or both of the methods according to one of the embodiments described in this description while the data structure is being executed on a computer,
- a computer program, wherein the computer program is adapted to perform one or both of the methods according to one of the embodiments described in this description while the program is being executed on a computer,
- a computer program comprising program means for performing one or both of the methods according to one of the embodiments described in this description while the computer program is being executed on a computer or on a computer network,
- a computer program comprising program means according to the preceding embodiment, wherein the program means are stored on a storage medium readable to a computer,
- a storage medium, wherein a data structure is stored on the storage medium and wherein the data structure is adapted to perform one or both of the methods according to one of the embodiments described in this description after having been loaded into a main and/or working storage of a computer or of a computer network, and
- a computer program product having program code means, wherein the program code means can be stored or are stored on a storage medium, for performing one or both of the methods according to one of the embodiments described in this description, if the program code means are executed on a computer or on a computer network.
Summarizing and without excluding further possible embodiments, the following embodiments may be envisaged: Embodiment 1. A slide imaging apparatus for hematological digital morphology, comprising at least one z-stage configured for mounting a microscopic slide at a sample plane, wherein a biological specimen is mountable on the microscopic slide, wherein the z-stage is configured for setting a relative distance between the microscopic slide and an imaging system of the slide imaging apparatus along an optical axis of the slide imaging apparatus; at least one sample beam path comprising at least one sample illumination source configured for illuminating the sample plane and at least one imaging sensor configured for generating at least one digital sample image of at least one part of the microscopic slide upon illumination by the sample illumination source, wherein the sample beam path comprises at least one first transfer system configured for allowing light from the sample plane to propagate to the imaging sensor; at least one autofocus beam path comprising at least one pattern illumination source comprising at least one light emitter and at least one optical grating, wherein the optical grating is tilted with respect to the optical axis of the slide imaging apparatus, wherein the autofocus beam path comprises at least one second transfer system and at least one autofocus camera, wherein the second transfer system is configured for allowing a pattern generated by the pattern illumination source to be projected into the sample plane and for allowing the pattern reflected from the sample plane to propagate to the autofocus camera, wherein the autofocus camera is configured for generating at least one digital autofocus image of the pattern reflected from the sample plane; at least one processing unit configured for analyzing the digital autofocus image, wherein the analyzing comprises spatially analyzing of contrast and determining an image location of maximum contrast; at least one control unit configured for controlling the z-stage for setting a position of the microscopic slide along the optical axis of the slide imaging apparatus considering the determined image location.
Embodiment 2. The slide imaging apparatus according to the preceding embodiment, wherein the slide imaging apparatus is configured for performing at least one focus- finder-procedure comprising the following steps:
I. generating at least one pattern by using the pattern illumination source and projecting the pattern into the sample plane;
II. generating a plurality of digital autofocus images at a plurality of different positions of the microscopic slide along the optical axis, III. for each autofocus image, analyzing the digital autofocus image by spatially analyzing of contrast and determining an image location of maximum contrast;
IV. determining the position of the microscopic slide along the optical axis at which the image location is most central in the digital autofocus image thereby identifying a focus plane.
Embodiment 3. The slide imaging apparatus according to any one of the preceding embodiments, wherein the pattern comprises at least one pattern selected from the group consisting of a streak pattern having a plurality of equally spaced strips; a checker board, a pattern having different line density in a field of view of the autofocus camera.
Embodiment 4. The slide imaging apparatus according to any one of the two preceding embodiments, wherein step III) of the focus-finder-procedure comprises the following steps: a) grouping pixels of each of the digital autofocus images into a plurality of image strips; b) cross correlating, per image strip, with a dedicated sine and/or cosine function of the same frequency; c) for each autofocus image, integrating over all envelopes integrals for all image strips and determining the maximum; d) determining a distribution of the maxima as a function of the position of the microscopic slide along the optical axis and applying a linear regression for identifying the focus plane.
Embodiment 5. The slide imaging apparatus according to the preceding embodiment, wherein the focus-finder-procedure comprises in step b) applying a linear fit per image strip, thereby subtracting background.
Embodiment 6. The slide imaging apparatus according to any one of the two preceding embodiments, wherein the focus-finder-procedure comprises determining a focus error by calculating a deviation for each point of the distribution of the maxima as a function of the position of the microscopic slide. Embodiment 7. The slide imaging apparatus according to any one of the five preceding embodiments, wherein the control unit is configured for setting the microscopic slide into the focus plane.
Embodiment 8. The slide imaging apparatus according to any one of the six preceding embodiments, wherein the focus-finder-procedure is performed repeatedly, wherein the focus-finder-procedure is performed with and/or without the biological specimen on the slide.
Embodiment 9. The slide imaging apparatus according to the preceding embodiments, wherein, in case the determined image location is within a central region of the digital autofocus image, the control unit maintains the position of the microscopic slide, wherein otherwise, in case the determined image location is shifted from the central region of the digital autofocus image, the control unit is configured for adapting the position of the microscopic slide, wherein the control unit is configured for adapting the position of the microscopic slide depending on a direction of shift.
Embodiment 10. The slide imaging apparatus according to any one of the preceding embodiments, wherein the emitter comprises at least one light emitting diode.
Embodiment 11. The slide imaging apparatus according to any one of the preceding embodiments, wherein the pattern is an infrared pattern, wherein projected light beams of the pattern have a wavelength in the near ultraviolet or near infrared spectral region such as from 400 to 300 nm or a spectral region from 760 nm to 1000 pm, more preferably of 780 nm.
Embodiment 12. The slide imaging apparatus according to any one of the preceding embodiments, wherein the optical grating is tilted by 10° to 45° with respect to the optical axis.
Embodiment 13. The slide imaging apparatus according to any one of the preceding embodiments, wherein the first transfer system comprises one or more of at least one objective, at least one spectral filter, at least one tube lens, and at least one beam splitter. Embodiment 14. The slide imaging apparatus according to any one of the preceding embodiments, wherein the second transfer system comprises at least one beam splitter configured for reflecting at least a part of a tilted pattern generated by the pattern illumination source into the sample plane and for transmitting the pattern reflected from the sample plane to propagate to the autofocus camera.
Embodiment 15. The slide imaging apparatus according to any one of the preceding embodiments, wherein the sample beam path is configured for generating a magnified image of the biological specimen on the imaging sensor.
Embodiment 16. The slide imaging apparatus according to any one of the preceding embodiments, wherein the sample beam path comprises a bright field microscope, wherein the sample illumination source is a bright field illumination source and the imaging sensor is a bright field camera.
Embodiment 17. The slide imaging apparatus according to any one of the preceding embodiments, wherein the autofocus camera comprises at least one camera, wherein the camera is a charge-coupled device (CCD) and/or a complimentary metal-oxide semiconductor (CMOS) image sensor.
Embodiment 18. The slide imaging apparatus according to any one of the preceding embodiments, wherein the biological specimen is a blood smear sample, a cerebrospinal fluid, or a pleural fluid, urine, synovial fluid, peritoneal fluid, ascites fluid, bone marrow aspirate.
Embodiment 19. A method for focus control using a slide imaging apparatus according to any one of the preceding embodiments, the method comprising the following steps: i. generating at least one pattern by using the pattern illumination source and projecting the pattern into the sample plane; ii. generating at least one digital autofocus image of the pattern reflected from the sample plane by using the autofocus camera; iii. analyzing the digital autofocus image by spatially analyzing of contrast and determining an image location of maximum contrast; iv. controlling the z-stage for setting a position of the microscopic slide along the optical axis of the slide imaging apparatus considering the determined image location by using the control unit. Embodiment 20. The method according to the preceding embodiment, wherein the method is computer-implemented.
Embodiment 21. A method for focus finding using a slide imaging apparatus according to any one of the preceding embodiments, the method comprising the following steps:
I. generating at least one pattern by using the pattern illumination source and projecting the pattern into the sample plane;
II. generating a plurality of digital autofocus images at a plurality of different positions of the microscopic slide along the optical axis,
III. for each autofocus image, analyzing the digital autofocus image by spatially analyzing of contrast and determining an image location of maximum sharpness;
IV. determining the position of the microscopic slide along the optical axis at which the image location is most central in the digital autofocus image thereby identifying a focus plane.
Embodiment 22. The method according to the preceding embodiment, wherein the method is computer-implemented.
Embodiment 23. A computer program comprising instructions which, when the program is executed by the slide imaging apparatus according to any one of the preceding embodiments referring to a slide imaging apparatus, cause the slide imaging apparatus to perform one or both of the methods according to any one of the preceding embodiments referring to a method.
Embodiment 24. A computer-readable storage medium comprising instructions which, when the instructions are executed by the slide imaging apparatus according to any one of the preceding embodiments referring to a slide imaging apparatus, cause the slide imaging apparatus to perform one or both of the methods according to any one of the preceding embodiments referring to a method.
Embodiment 25. A non-transient computer-readable medium including instructions that, when executed by one or more processors, cause the one or more processors to perform one or both of the methods according to any one of the preceding embodiments referring to a method. Short description of the Figures
Further optional features and embodiments will be disclosed in more detail in the subsequent description of embodiments, preferably in conjunction with the dependent claims. Therein, the respective optional features may be realized in an isolated fashion as well as in any arbitrary feasible combination, as the skilled person will realize. The scope of the invention is not restricted by the preferred embodiments. The embodiments are schematically depicted in the Figures. Therein, identical reference numbers in these Figures refer to identical or functionally comparable elements.
In the Figures:
Figure 1 shows an exemplary embodiment slide imaging apparatus;
Figures 2A and 2B show exemplary pattern;
Figures 3 and 4 show exemplary digital autofocus images;
Figures 5 shows an exemplary analysis of a digital autofocus image;
Figures 6A to 6D show experimental results; and
Figures 7A to 7L show further experimental results.
Detailed description of the embodiments
Figure 1 shows an exemplary embodiment slide imaging apparatus 110 for hematological digital morphology.
The slide imaging apparatus 110 comprises at least one z-stage 112 configured for mounting a microscopic slide 114 at a sample plane. A biological specimen is mountable on the microscopic slide 114. The z-stage 112 is configured for setting a relative distance between the microscopic slide 114 and an imaging system 116 of the slide imaging apparatus 110 along an optical axis 118 of the slide imaging apparatus 110. The slide imaging apparatus 110 comprises two beam paths, i.e. at least one sample beam path and at least one autofocus beam path.
The sample beam path may be a beam path used for magnified imaging of the biological specimen on the microscopic slide 114, e.g. for subsequent assessing of hematological morphology. The sample beam path comprises at least one sample illumination source 120 configured for illuminating the sample plane and at least one imaging sensor 122 configured for generating at least one digital sample image of at least one part of the microscopic slide upon illumination by the sample illumination source 120. For example, the sample beam path may comprise a bright field microscope. The sample illumination source 120 may be a bright field illumination source and the imaging sensor 122 may be a bright field camera.
The sample beam path comprises at least one first transfer system 124 configured for allowing light from the sample plane to propagate to the imaging sensor 122. As shown in Figure 1, the illuminating of the specimen may comprise transillumination of the slide 114. Thus, the slide 114 may be arranged between the sample illumination source and an objective 126. The specimen disposed on the slide 114 may generate in response to the illumination at least one light beam such as by transmission. The first transfer system 124, in the embodiment of Figure 1, further comprises at least one spectral filter 128 and at least one tube lens 130. The first transfer system 124 may be configured such that the imaging sensor 122 is parfocal.
The autofocus beam path comprises at least one pattern illumination source 132 comprising at least one light emitter 134 and at least one optical grating 136. The light emitter 134 may comprise at least one light emitting diode (LED). The light emitter 134 may be configured for generating at least one light beam and the optical grating 136 may be configured for generating the pattern from the at least one impinging light beam. The pattern illumination source 132 may comprise an array of light emitters 134.
For example, the pattern comprises at least one pattern selected from the group consisting of a streak pattern having a plurality of equally spaced strips; a checker board, e.g. a checkered pattern, a pattern having different line density in a field of view of the autofocus camera. Figures 2A and 2B show exemplary pattern. Figure 2A shows a line pattern with constant line density and Figure 2B shows a line pattern with different line density in a field of view of the autofocus camera 138.
The light beams of the pattern may have the same or different wavelengths, wherein wavelength of the complete optical spectral range are possible. For example, the pattern is an infrared pattern. The projected light beams of the pattern have a wavelength in the near ultraviolet or near infrared spectral region such as from 400 to 300 nm or a spectral region from 760 nm to 1000 pm, more preferably of 780 nm. The wavelengths of the light beams forming the pattern may be significantly different from the wavelengths used for the sample beam path.
The optical grating 136 is tilted with respect to the optical axis 118. For example, the optical grating 136 is tilted by 10° to 45° with respect to the optical axis 118.
The autofocus beam path comprises at least one second transfer system 140 and the at least one autofocus camera 138. The second transfer system 140 may be configured such that the autofocus camera 138 is parfocal. The second transfer system 140 is configured for allowing a pattern generated by the pattern illumination source 132 to be projected into the sample plane and for allowing the pattern reflected from the sample plane to propagate to the autofocus camera 138. The second transfer system 140 may comprise at least one beam splitter 142 configured for reflecting at least a part of a tilted pattern generated by the pattern illumination source 132 into the sample plane and for transmitting the pattern reflected from the sample plane to propagate to the autofocus camera 138. In Figure 1, the second transfer system 140 may comprise two beam splitters 142, e.g. a dichroitic beam splitter and a 50/50 beam splitter. The second transfer system 140 may comprise in the embodiment of Figure 1, two achromats 144, e.g. comprising a plurality of lenses configured for minimizing an imaging error between the pattern and the autofocus camera 138. The second transfer system 140 may comprise further elements such as at least one aspheric condenser lens 146.
The autofocus camera 138 is configured for generating at least one digital autofocus image of the pattern reflected from the sample plane. The autofocus camera 138 may comprise at least one camera. The camera may a charge-coupled device (CCD) and/or a complimentary metal-oxide semiconductor (CMOS) image sensor. The camera may have a sufficiently large resolution. The resolution may be sufficient to resolve the streak pattern, e.g. about 1/3 of a streak width.
The slide imaging apparatus 110 comprises at least one processing unit 148 configured for analyzing the digital autofocus image. The analyzing comprises spatially analyzing of contrast and determining an image location of maximum contrast. The analyzing comprises using at least one digital image filtering method based on convolution of the digital autofocus image with one or more sine and cosine functions. This can allow to extract a measure for the pattern contrast only. This way the detected contrast is largely insensitive to presence of cells in the digital autofocus image.
The slide imaging apparatus 110 comprises at least one control unit 150 configured for controlling the z-stage 112 for setting a position of the microscopic slide 114 along the optical axis 118 considering the determined image location.
By applying a tilt to the optical grating 136 and, thus, to the pattern, the imaged pattern is only in focus within a limited region in the digital autofocus image. The location of highest contrast allows to deduce a bi-directional signal that can be used as an input to a focus- finder-procedure and/or algorithm.
When the slide 114 is moved along the optical axis 118, i.e. in and out of focus, the focused pattern fraction may move to sides of the autofocus camera. Thus, a direction of shift can indicate the direction of defocus (above/below).
For example, in the autofocus beam path, an image of a pattern mask of e.g. equally spaced, parallel strips (e.g. a line pattern with 10 lines/mm) may be projected into the sample plane. The back reflection off the slide surface is imaged onto a dedicated camera sensor of the autofocus camera 138. In the camera’s image, the pattern as well as a dense cell layer is visible (in case of whole blood). Both change contrast as the sample is moved in and out of the focus plane. Figures 3 and 4 show experimental results of digital autofocus images. For the experimental setup a chrome-on-glass line chart with 10 lines/mm, a 100 mm achromatic lens, a 780 nm LED source, a broad band beam splitter, a 700 nm dichroic mirror, a 780 nm notch filter, a IMX265 B/W camera and an aspheric condenser lens were used. In Figure 3, the top image shows a digital autofocus image when the slide 114 is in focus. The central region of the digital autofocus image is the image location of maximum contrast. In the center image of Figure 3, a left image region has the maximum contrast. This image relates to a position of the slide 114 above the focus. In the lower image of Figure 3, a right image region has the maximum contrast. This image relates to a position of the slide 114 below the focus.
Figure 4 shows further experimental digital autofocus images using such a line pattern. On the left side of Figure 4 experimental results of a clean slide, i.e. without a sample, and on the right side of Figure 4, experimental results of a whole blood printed slide are shown. In the top row, respective slide 114 is in focus and the central region of the digital autofocus images is the image location of maximum contrast. In the middle row, a left image region has the maximum contrast. This image relates to a position of the slide 114 above the focus. In the lower row of Figure 4, a right image region has the maximum contrast. This image relates to a position of the slide 114 below the focus. The images of the middle and lower row may be obtained at ± 2pm from the focus position.
Figures 5 shows an exemplary analysis of a digital autofocus image, e.g. of the middle row of Figure 4 for the clean slide. Starting from the top left figure of Figure 5, the analysis may comprise calculating a pixel mean of the original image within 10 vertical strips. The contrast may be analyzed on these strips only, e.g. 400x2000pixel are combined to 400x10. Per strip, a linear fit may be applied to remove 0th and 1st order background, see top right figure of Figure 5. Per strip, a cross correlation with sine and cosine functions of the same frequency may be applied, see middle row left figure of Figure 5. This step can effectively remove impact from printed cells as only the specific spatial frequency of the strip pattern is measured. Next, see middle row right figure of Figure 5, integrals over all 10 envelopes are integrated for all strips (one image) at a single z-position. Next, see lower row left figure of Figure 5, maximum, also denoted as central maximum (CM), the vs the actual z position of the objective lens may be plotted, resulting in an error curve around the central strip, e.g. within ± 4 pm around the focus position. Next, see lower row right figure of Figure 5, a linear regression around the focal position may be calculated and a deviation per point form the linear regression may be calculated.
Figures 6A to 6D show further experimental results, in particular a comparison between a clean slide (Fig. 6A and B) and printed slide (with sample) (Fig. 6C and D). In Figures 6A and 6C the sensor position vs the z position is depicted. In Figures 6B and 6D the focus error vs the z position is depicted.
Figures 7 A to 7L show further experimental results of the focus-finder-procedure and/or algorithm for different slide conditions. In Figures 7A, D, G and J, the respective slide condition is shown which relates to the plots of the same row, e.g. Figure 7A shows a clean slide and Figures 7B and 7C relate to results using the clean slide. Figure 7D shows a slide with whole blood, Figure 7G a slide with a high cell content and Figure 7J a slide with low cell content. Figures 7B, 7C, 7E, 7F, 7H, 71, 7K and 7L show repeated focusing movements in which the autofocus determined the focus position and the z stage is moved in accordingly. The x-axis of these plots shows the position measurement of the z-axis, the y-axis of these plots shows the autofocus signal with which the autofocus is moved. Figures 7B, E, H, K show the autofocus signal as a function of the z-position in pm when starting from below the focus position for a number of runs (7 runs for 7B, 3 runs for 7E, 9 runs for 7H and 11 runs for 7K). Figures 7C, F, I, L show the autofocus signal as a function of the z-position in pm when starting from above the focus position for a number of runs (8 runs for 7C, 6 runs for 7F, 10 runs for 71 and 12 runs for 7L). The experimental results show that the focus precision is consistent within < ± 0.1 pm, e.g. a factor of about 5 less than the objective depth of field.
List of reference numbers slide imaging apparatus z- stage slide imaging system optical axis sample illumination source imaging sensor first transfer system objective spectral filter tube lens pattern illumination source light emitter optical grating autofocus camera second transfer system beam splitter achromat aspheric condenser lens processing unit control unit

Claims

Claims
1. A slide imaging apparatus (110) for hematological digital morphology, comprising at least one z-stage (112) configured for mounting a microscopic slide (114) at a sample plane, wherein a biological specimen is mountable on the microscopic slide (114), wherein the z-stage (112) is configured for setting a relative distance between the microscopic slide (114) and an imaging system (116) of the slide imaging apparatus (110) along an optical axis (118) of the slide imaging apparatus (no); at least one sample beam path comprising at least one sample illumination source (120) configured for illuminating the sample plane and at least one imaging sensor (122) configured for generating at least one digital sample image of at least one part of the microscopic slide (114) upon illumination by the sample illumination source (120), wherein the sample beam path comprises at least one first transfer system (128) configured for allowing light from the sample plane to propagate to the imaging sensor (122); at least one autofocus beam path comprising at least one pattern illumination source (132) comprising at least one light emitter (134) and at least one optical grating (136), wherein the optical grating (136) is tilted with respect to the optical axis (118) of the slide imaging apparatus (110), wherein the autofocus beam path comprises at least one second transfer system (140) and at least one autofocus camera (138), wherein the second transfer system (140) is configured for allowing a pattern generated by the pattern illumination source (132) to be projected into the sample plane and for allowing the pattern reflected from the sample plane to propagate to the autofocus camera (138), wherein the autofocus camera (138) is configured for generating at least one digital autofocus image of the pattern reflected from the sample plane; at least one processing unit (148) configured for analyzing the digital autofocus image, wherein the analyzing comprises spatially analyzing of contrast and determining an image location of maximum contrast; at least one control unit (150) configured for controlling the z-stage (112) for setting a position of the microscopic slide (114) along the optical axis (118) of the slide imaging apparatus (110) considering the determined image location.
2. The slide imaging apparatus (110) according to the preceding claim, wherein the slide imaging apparatus (110) is configured for performing at least one focus-finder-proce- dure comprising the following steps:
I. generating at least one pattern by using the pattern illumination source (132) and projecting the pattern into the sample plane;
II. generating a plurality of digital autofocus images at a plurality of different positions of the microscopic slide (114) along the optical axis (118),
III. for each autofocus image, analyzing the digital autofocus image by spatially analyzing of contrast and determining an image location of maximum contrast;
IV. determining the position of the microscopic slide (114) along the optical axis (118) at which the image location is most central in the digital autofocus image thereby identifying a focus plane.
3. The slide imaging apparatus (110) according to any one of the preceding claims, wherein the pattern comprises at least one pattern selected from the group consisting of a streak pattern having a plurality of equally spaced strips; a checker board, a pattern having different line density in a field of view of the autofocus camera.
4. The slide imaging apparatus (110) according to any one of the two preceding claims, wherein step III) of the focus-finder-procedure comprises the following steps: a) grouping pixels of each of the digital autofocus images into a plurality of image strips; b) cross correlating, per image strip, with a dedicated sine and/or cosine function of the same frequency; c) for each autofocus image, integrating over all envelopes integrals for all image strips and determining the maximum; d) determining a distribution of the maxima as a function of the position of the microscopic slide (114) along the optical axis and applying a linear regression for identifying the focus plane.
5. The slide imaging apparatus (110) according to the preceding claim, wherein the fo- cus-finder-procedure comprises in step b) applying a linear fit per image strip, thereby subtracting background.
6. The slide imaging apparatus (110) according to any one of the two preceding claims, wherein the focus-finder-procedure comprises determining a focus error by calculating a deviation for each point of the distribution of the maxima as a function of the position of the microscopic slide (114).
7. The slide imaging apparatus (110) according to the preceding claims, wherein, in case the determined image location is within a central region of the digital autofocus image, the control unit (150) maintains the position of the microscopic slide (114), wherein otherwise, in case the determined image location is shifted from the central region of the digital autofocus image, the control unit (150) is configured for adapting the position of the microscopic slide (114), wherein the control unit (150) is configured for adapting the position of the microscopic slide (114) depending on a direction of shift.
8. The slide imaging apparatus (110) according to any one of the preceding claims, wherein projected light beams of the pattern have a wavelength in the near ultraviolet or near infrared spectral region such as from 400 to 300 nm or a spectral region from 760 nm to 1000 pm, more preferably of 780 nm.
9. The slide imaging apparatus (110) according to any one of the preceding claims, wherein the optical grating (136) is tilted by 10° to 45° with respect to the optical axis.
10. The slide imaging apparatus (110) according to any one of the preceding claims, wherein the sample beam path is configured for generating a magnified image of the biological specimen on the imaging sensor (122), and/or wherein the sample beam path comprises a bright field microscope, wherein the sample illumination source (120) is a bright field illumination source and the imaging sensor (122) is a bright field camera.
11. The slide imaging apparatus (110) according to any one of the preceding claims, wherein the biological specimen is a blood smear sample, a cerebrospinal fluid, or a pleural fluid, urine, synovial fluid, peritoneal fluid, ascites fluid, bone marrow aspirate.
12. A method for focus control using a slide imaging apparatus (110) according to any one of the preceding claims, the method comprising the following steps: i. generating at least one pattern by using the pattern illumination source (132) and projecting the pattern into the sample plane; ii. generating at least one digital autofocus image of the pattern reflected from the sample plane by using the autofocus camera (138); iii. analyzing the digital autofocus image by spatially analyzing of contrast and determining an image location of maximum contrast; iv. controlling the z-stage (112) for setting a position of the microscopic slide (114) along the optical axis (118) of the slide imaging apparatus (110) considering the determined image location by using the control unit (150).
13. The method according to the preceding claim, wherein the method is computer-implemented.
14. A method for focus finding using a slide imaging apparatus (110) according to any one of the preceding claims, the method comprising the following steps:
I. generating at least one pattern by using the pattern illumination source (132) and projecting the pattern into the sample plane;
II. generating a plurality of digital autofocus images at a plurality of different positions of the microscopic slide (114) along the optical axis (118),
III. for each autofocus image, analyzing the digital autofocus image by spatially analyzing of contrast and determining an image location of maximum sharpness;
IV. determining the position of the microscopic slide (114) along the optical axis (118) at which the image location is most central in the digital autofocus image thereby identifying a focus plane.
15. The method according to the preceding claim, wherein the method is computer-implemented.
PCT/EP2024/083288 2023-11-23 2024-11-22 Slide imaging apparatus for hematological digital morphology Pending WO2025109168A1 (en)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
EP23211821.6 2023-11-23
EP23211821 2023-11-23

Publications (1)

Publication Number Publication Date
WO2025109168A1 true WO2025109168A1 (en) 2025-05-30

Family

ID=88964937

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/EP2024/083288 Pending WO2025109168A1 (en) 2023-11-23 2024-11-22 Slide imaging apparatus for hematological digital morphology

Country Status (1)

Country Link
WO (1) WO2025109168A1 (en)

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0787378A (en) * 1993-09-09 1995-03-31 Topcon Corp Focus detection device
US6677565B1 (en) 1998-08-18 2004-01-13 Veeco Tucson Inc. High speed autofocus and tilt for an optical imaging system
US20100033811A1 (en) 2006-06-16 2010-02-11 Carl Zeiss Microimaging Gmbh Autofocus device for microscopy
US8258448B2 (en) * 2008-11-28 2012-09-04 Mitutoyo Corporation Autofocus device with contrast enhancement
US20130062501A1 (en) 2011-09-09 2013-03-14 Carl Zeiss Smt Gmbh Autofocus Device and Autofocussing Method For An Imaging Device
US9229209B2 (en) 2008-01-21 2016-01-05 Carl Zeiss Smt Gmbh Autofocus device and autofocusing method for an imaging device
US9389405B2 (en) 2012-12-13 2016-07-12 Carl Zeiss Microscopy Gmbh Autofocus method for microscope and microscope with autofocus device
EP3175302B1 (en) 2014-08-01 2021-12-29 The Regents of the University of California Device and method for iterative phase recovery based on pixel super-resolved on-chip holography

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH0787378A (en) * 1993-09-09 1995-03-31 Topcon Corp Focus detection device
US6677565B1 (en) 1998-08-18 2004-01-13 Veeco Tucson Inc. High speed autofocus and tilt for an optical imaging system
US20100033811A1 (en) 2006-06-16 2010-02-11 Carl Zeiss Microimaging Gmbh Autofocus device for microscopy
US9229209B2 (en) 2008-01-21 2016-01-05 Carl Zeiss Smt Gmbh Autofocus device and autofocusing method for an imaging device
US8258448B2 (en) * 2008-11-28 2012-09-04 Mitutoyo Corporation Autofocus device with contrast enhancement
US20130062501A1 (en) 2011-09-09 2013-03-14 Carl Zeiss Smt Gmbh Autofocus Device and Autofocussing Method For An Imaging Device
US9389405B2 (en) 2012-12-13 2016-07-12 Carl Zeiss Microscopy Gmbh Autofocus method for microscope and microscope with autofocus device
EP3175302B1 (en) 2014-08-01 2021-12-29 The Regents of the University of California Device and method for iterative phase recovery based on pixel super-resolved on-chip holography

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
D. CLARK ET AL.: "A Rapid Image Acquisition Method for Focus Stacking in Microscopy", MICROSCOPY TODAY, vol. 23, July 2015 (2015-07-01), pages 18 - 25, XP055694503, DOI: 10.1017/S1551929515000577

Similar Documents

Publication Publication Date Title
JP7670886B2 (en) Calibration of light field imaging systems
EP3513241B1 (en) Light-field microscope with selective-plane illumination
CN108780218B (en) Imaging system using auxiliary image detector for sample position
KR102419163B1 (en) Real-time autofocus focusing algorithm
CN111164484B (en) Two-way macro image
US20110157350A1 (en) Arithmetically operating device, arithmetically operating method, arithmetically operating program, and microscope
JP7191047B2 (en) Method and microscope system for capturing images
KR20220092999A (en) A digital scanning apparatus
JP2006184303A (en) Image inspecting device
JPWO2020157681A5 (en)
KR100679643B1 (en) Auto focus control device adopting auto focus control pattern and auto focus control method using it
CN108291870A (en) Light microscopy and method for determining the wavelength-dependent refractive index of a sample medium
WO2025109168A1 (en) Slide imaging apparatus for hematological digital morphology
JP2024169694A (en) microscope
US11971531B2 (en) Method and microscope for determining the thickness of a cover slip or slide
JP7507767B2 (en) Physical Calibration Slide
JP7640821B1 (en) Image acquisition device and image acquisition method
US20240264180A1 (en) Instrument for automatically dissecting a biological specimen on a slide

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24817081

Country of ref document: EP

Kind code of ref document: A1