High-magnification three-dimensional imaging microscope and formation method based on double-light source off-axis illumination
[technical field]:
The invention belongs to the optical instrument technical field, relate to image-forming principle, Flame Image Process and the three-dimensional coordinate computing method of the three-dimensional micro-image of real-time collection high power.
[background technology]:
In optical instrument field, gather in real time and show that high power stereomicroscopy image is one of human difficult problem of exploring microworld always.In order to realize high precision, high-level efficiency, the high reliability operation to small items, people have invented micro-manipulating robot.In the micro-manipulating robot system, the unique channel of feedback three dimensional local information is the micro-image that obtains to have three dimensional local information by micro-vision, through Flame Image Process, pattern-recognition, calculating, obtain the Three-dimension Target coordinate, thus, be the requirement that microscope has proposed real-time feedback small objects three-dimensional coordinate to optical system.Therefore, the difficult point of development high power stereo microscope shows following 5 aspects: 1. the operating distance of high power microcobjective is too short, the space of two microcobjective images acquired of no use; 2. must could obtain high resolving power with the microcobjective of large-numerical aperture in the conventional optical microscope, and numerical aperture is big more, the depth of field is more little, and therefore, field depth is subjected to the restriction of resolution, the ferret out difficulty; 3. a lot of biomedicine experiments are towards small cell manipulation, and the enlargement ratio of stereomicroscope and resolution all can not meet the demands; 4. existing method calculated amount from plane micro-image extraction longitudinal position information is big, exists when being used for position feedback to lag behind, and influences the performance of control system; 5. Laser Scanning Confocal Microscope collection and reconstruction of three-dimensional images can't requirement of real time.
Existing instrument or the method for obtaining the stereomicroscopy image has stereomicroscope, semiaperture stereo microscope, Laser Scanning Confocal Microscope; The method of obtaining three dimensional local information from the micro-image of plane mainly contains fast fourier transform method, image identification method.Can obtain the three dimensional local information of stereo-picture or small items with above instrument or method, but all can't obtain high magnification high-resolution three-dimension video frequency microscopic image, for example:
" novel stereomicroscope " [application number: CN90202741.7], this microscope begins to be divided into two-way light from microcobjective, and in order to satisfy requirements of installation space, microcobjective must have very long operating distance, so, can only use the low power microcobjective in the stereomicroscope.
" semiaperture stereo microscope " [Cheng Yongnan, Ding Xiulan, Cao Yaqin, the semiaperture stereo microscope, cell biology magazine, 13,2:95], the aperture diaphragm of microcobjective is divided into two semiapertures, only enters observer's right eye, only enter observer's left eye by the light beam of right semiaperture by the light beam of left semiaperture.When multiplying power was very high, because used numerical aperture is very big, the depth of field was very short, is not easy to obtain good stereoeffect.For every eyes of observer, the numerical aperture of microcobjective on left and right directions is reduced to original half, and resolving power also is reduced to original half.
" a kind of minisize three-dimensional self-scanning confocal microscope " [application number: CN200510018429.4], by pinhole diaphragm illumination, another pinhole diaphragm imaging by conjugation again, pointwise images acquired, stereomicroscopy image that can the reconstruct ultrahigh resolution.Be subjected to the restriction of scan period, also can't obtain video image at present.
" by the vertical information of micro-image feature extraction acquisition microoperation target " [Zhang Jianxun, Xue Daqing, Lu Guizhang, Li Bin obtains the vertical information of microoperation target, robot by the micro-image feature extraction, 2001,23 (1): 73-77], handle image with fast fourier transform, calculated amount is very big.
" based on the micro manipulator tool depth information extracting method and the device of micro-image processing " [application number: CN200510011629.7] obtains depth information by the relation of setting up point spread function and defocusing amount, calculates about 0.2 second of the time of a depth value.
[summary of the invention]:
Fundamental purpose of the present invention is when gathering high power stereomicroscopy image, it is short to solve high power microcobjective operating distance, the problem that does not have two microcobjectives of enough space mounting, a kind of high magnification stereo microscope that can gather and show in real time high power video stereo-picture is provided, and provide the method for calculating the object dimensional positional information, provide practical window for the mankind observe the dynamic three-dimensional microcosmic world, also provide practical apparatus and method for the micro-manipulating robot system realizes the high accuracy three-dimensional machine vision.
The present invention adopts the optical system of two light illuminations, single channel imaging, replaces the optical system of single light source illumination, double light path imaging in the stereomicroscope.
High-magnification three-dimensional imaging microscope based on double-light source off-axis illumination provided by the invention comprises:
About two groups of light sources: the optical axis of two groups of light sources separation microcobjective optical axis both sides, about the optical axis of two groups of light sources and the angle between the microcobjective optical axis all less than the object space aperture angle of microcobjective, and three optical axis coplanes; Light source is to the object transillumination;
Microcobjective: about the illuminating bundle that produces of two groups of light sources illuminate the whole visual field of microcobjective, and be imaged on 2 points on the entrance pupil face of microcobjective;
Diplopore iris: be arranged at the microcobjective back, and make be imaged on the microcobjective entrance pupil face 2 to overlap with diplopore iris center;
CCD: be installed on the microscopical standard CC D interface image pick-up card that is used to receive the microcobjective imaging and is transported to computing machine.
Two groups of light-source structures are identical about described, and every group of light source comprises a pinhole diaphragm between a bulb or led light source, two groups of condensers and the two groups of condensers successively; Pinhole diaphragm is arranged on the emergent pupil plane of microcobjective, and the aperture diaphragm center is on the intersection point on illuminating bundle optical axis and microcobjective emergent pupil plane, as shown in Figure 1.
Two groups of light sources can also adopt the monochromatic LED of same structure to throw light on about described, and promptly every group of light source comprises a led light source, a pinhole diaphragm and one group of condenser successively.
The step of formation method that obtains the object dimensional positional information with three-dimensional imaging microscope provided by the invention is as follows:
1st, demarcate the relation of microscope defocusing amount and double image spacing
1.1st, use the cross-graduation plate as target, longitudinally be that the microcobjective optical axis direction moves graticule, images acquired, obtain comprising focusing, the set of diagrams picture of out of focus is right up and down, write down the Center Gap d of cross curve on CCD and the along slope coordinate z of graticule in every pair of image, graticule position when promptly focusing on cross curve center, left and right sides coincidence d=0 is the zero point of along slope coordinate, determines the relation that along slope coordinate changes with the Center Gap of cross curve on CCD:
z=f(d)
Wherein d comprises the size and Orientation of cross curve Center Gap;
According to sequencing regulation from left to right, if the cross curve in the image of left light source occurs earlier, d>0 then, be that viewed cross curve is for being imaged on the preceding out of focus situation of CCD top through microcobjective, the left projected image that then left light illumination obtains is to left, and the right projected image that right light illumination obtains is to right translation, and promptly left projected image is on a left side, right projected image is on the right side, defines along slope coordinate z this moment for just;
If the cross curve in the image of right light source occurs earlier, d<0 then, be that viewed cross curve is for being imaged on the back out of focus situation of CCD below through microcobjective, the left projected image that then left light illumination obtains is to right translation, the right projected image that right light illumination obtains is to left, be left projected image on the right side, right projected image is on a left side, this moment, corresponding along slope coordinate z was for negative;
The 1.2nd or according to the Center Gap d of cross curve on CCD and the along slope coordinate z of graticule in every pair of image of last step record, the match defocusing amount is the curve that along slope coordinate z changes with the Center Gap d of cross curve on CCD;
1.3rd, demarcate the horizontal enlargement ratio β and the distortion Q of imaging according to a conventional method;
2nd, images acquired
About two groups of light sources adopt the light of two kinds of different colours to throw light on simultaneously respectively, by the CCD images acquired, this image can become the left and right sides image of band parallax right by color separated;
3rd, Flame Image Process
Become left and right sides image right the 2nd separation of images that goes on foot the two kinds of colors that overlap that obtain; It is right that the correcting distorted respectively back of two width of cloth monochrome images constitutes left and right sides image;
The image of two kinds of colors is separately converted to gray level image, after removing noise processed, calculates gray threshold, again image is converted into bianry image according to feature of image;
4th, Target Recognition
The target different to feature in the image, according to the automatic recognition objective of each target feature separately, and the edge of definite each target image;
5th, calculate two-dimensional coordinate
According to the two-dimensional coordinate of unique point in the image of the left and right sides on each target of edge calculations of the 4th each target image of step,
Remember that the two-dimensional coordinate of target picture on CCD is among the left figure
P
L(x
L,y
L)
The two-dimensional coordinate of target picture on CCD is among the right figure
P
R(x
R,y
R)
Same target is charged to variable S by the regulation of sign in the 1.1st step, promptly arranges by sequencing from left to right, then
6th, be converted into three-dimensional coordinate
Calculate target at the xy of object space coordinate, promptly as the mean value of two-dimensional coordinate divided by β with target in the image of the left and right sides
Plan range with target picture in the image of the left and right sides calculates target at the z of object space coordinate, promptly
Can change the performance of microscope ferret out with structure of the present invention, it is as follows that this optical system is provided with the diplopore iris:
The first, by dwindling the diaphragm bore, the depth of field of increasing is convenient to find target;
The second, dwindle the diaphragm bore after, can reduce the resolving power of microcobjective, open big diaphragm after finding target and focusing, recover the original resolving power of microcobjective, so the bore of this diaphragm must be adjustable, promptly variable diaphragm;
During three, in order to ensure adjusting diaphragm bore, illuminance of image plane is even and constant substantially, iris is arranged on the emergent pupil face of microcobjective, and the Zero-order diffractive hot spot should all pass through iris;
Four, two bundle off-axis illumination light on the microcobjective emergent pupil similarly be about two hot spots separating, so iris has two holes, and bore that can the adjusted in concert diplopore.
The present invention can be with three kinds of method images acquired and with left and right sides separation of images, and concrete grammar and structure are as follows:
One, duochrome method: adopt the LED illumination of different colours in the light source of the left and right sides respectively, on CCD, obtain two kinds of colour superimpositions left and right sides image together, press color component with left and right sides separation of images, obtain the video stereo-picture of flicker free, be converted into gray level image during observation with common CCD;
Two, polarization method: on the light source of the left and right sides, add polaroid respectively, and the polarization direction of left and right sides light source is vertical mutually, between diplopore iris and CCD with polarization spectroscope with left and right sides separation of images, other adds one tunnel common CCD, two-way CCD is images acquired simultaneously, obtain the color solid video image of flicker free, as shown in Figure 4;
Three, LED switches illumination: adopt white light LEDs timesharing illumination, and the left light illumination of odd-numbered frame image for example, the right light illumination of even frame image, the odd-numbered frame image that obtains is formed left road dynamic menu, even frame image composition right wing dynamic menu.
Advantage of the present invention and good effect:
The present invention just can obtain high power stereomicroscopy image by transforming the light source of ordinary optical microscope, for the dynamic microworld of human Real Time Observation provides practical device; Need utilize the system of micro-image feedback small items three dimensional local information for micro-manipulating robot etc., provide the three-dimensional coordinate computing method, compare with existing method, computation process is simpler, and the result is more reliable; With the diplopore iris can be in bigger longitudinal extent ferret out, the original resolving power of microcobjective is recovered in the focusing back; Light source converges to illuminating bundle on the diplopore iris, and when regulating the aperture of the diaphragm, the illumination on the CCD is constant substantially, does not need to regulate light-source brightness repeatedly when using microscope, helps prolonging light source life.
Full accuracy with Measuring Object three-dimensional coordinate of the present invention is better than 0.5 μ m.
[description of drawings]:
Fig. 1 is the optical texture principle schematic of high magnification stereo microscope of the present invention.
Among the figure, the 1, the 5th, incandescent lamp bulb or white LED light source, the 2,4,6, the 8th, condenser, the 3, the 7th, pinhole diaphragm or single slit diaphragm, the 9th, thing, the 10th, microcobjective, the 11st, diplopore iris, the 12nd, CCD.Wherein 1,2,3,4 form left light source, and 5,6,7,8 form right light source.
Fig. 2 is the optical texture principle schematic of simplifying.
Among the figure, 13,16 is respectively red and blue LED, the 14, the 17th, and pinhole diaphragm or single slit diaphragm, the 15, the 18th, condenser, the 19th, thing, the 20th, microcobjective, the 21st, diplopore iris, the 22nd, CCD.Wherein 13,14,15 form left light source, and 16,17,18 form right light source.
Fig. 3 is the imaging schematic diagram of object when different out-of-focus appearance in apparatus of the present invention, and Fig. 3 (a) is a not out of focus of imaging, and Fig. 3 (b) is out of focus upwards, and Fig. 3 (c) is out of focus downwards.
Among the figure, the 23rd, left light source, the 24th, right light source, the 25th, the object plane of CCD conjugation, the 26th, microcobjective, the 27th, diplopore iris, the 28th, CCD, the 29th, be positioned at the thing on the object plane 25, the 30th, be positioned at the picture that CCD (28) goes up thing (29), the 31st, the thing of last out of focus, the 32nd, the picture of thing (31), the 33rd, left light source (23) will be projected in hot spot on the CCD (28) as (32), the 34th, right light source (24) will be projected in hot spot on the CCD (28) as (32), and the 35th, the thing of following out of focus, the 36th, the picture of thing (35), the 37th, left light source (23) will be projected in hot spot on the CCD (28) as (36), and the 38th, right light source (24) will be projected in hot spot on the CCD (28) as (36).
Fig. 4 is an another kind of optical texture synoptic diagram of the present invention.
Among the figure, the 39, the 44th, incandescent lamp or white light LEDs, the 40,42,45, the 47th, condenser, the 41, the 46th, pinhole diaphragm, the 43, the 48th, polaroid, the 49th, thing, the 50th, microcobjective, the 51st, diplopore iris, the 52nd, polarization spectroscope, the 53, the 54th, CCD.Wherein the polarization direction of two polaroids is vertical mutually, and polarization spectroscope is with left and right sides separation of images.
[embodiment]:
Embodiment 1: high-magnification three-dimensional imaging microscope (double lens light source)
As shown in Figure 1, the high-magnification three-dimensional imaging microscope based on double-light source off-axis illumination provided by the invention comprises:
About two groups of light sources that structure is identical, left light source comprises a bulb or 1, two group of condenser 2 of led light source and 4 successively, and a pinhole diaphragm 3 between two groups of condensers; Right light source comprises a bulb or 5, two groups of condensers 6 of led light source and 8 successively, and a pinhole diaphragm 7 between two groups of condensers; Pinhole diaphragm is arranged on the emergent pupil plane of microcobjective, and the aperture diaphragm center is on the intersection point on illuminating bundle optical axis and microcobjective emergent pupil plane.
About the optical axis separation microcobjective optical axis both sides of two groups of light sources, about the optical axis of two groups of light sources and the object space aperture angle that the angle between the microcobjective optical axis all is slightly less than microcobjective, and three optical axis coplanes; Light source is to the object transillumination;
Microcobjective 10: about the illuminating bundle that produces of two groups of light sources illuminate the whole visual field of microcobjective, and be imaged on 2 points on the entrance pupil face of microcobjective;
Diplopore iris 11: be arranged at the microcobjective back, and make be imaged on the microcobjective entrance pupil face 2 to overlap with diplopore iris center;
CCD12: be installed on the microscopical standard CC D interface image pick-up card that is used to receive the microcobjective imaging and is transported to computing machine.
Embodiment 2: high-magnification three-dimensional imaging microscope (simple lens light source)
As shown in Figure 2, of the present invention about two groups of light sources can also adopt the monochromatic LED illumination of same structure, promptly left light source comprises 14, one groups of condensers 15 of 13, one pinhole diaphragms of a monochromatic LED light source successively; Right light source comprises 16, one pinhole diaphragms 17 of a monochromatic LED light source and one group of condenser 18 successively, and all the other structures are with embodiment 1.
Embodiment 3: the formation method that obtains the object dimensional positional information with three-dimensional imaging microscope
With the three-dimensional coordinate of Fig. 1 of the present invention or measurement device object shown in Figure 2 (concrete steps as summary of the invention part as described in), left side light source throws light on red LED, right light source throws light on blue led, the horizontal enlargement ratio of microcobjective is 40 times, the object-image conjugate distance is 195mm, NA=0.65, the effective imaging area 3.3mm * 4.4mm of CCD, 1024 * 768 pixels.
1, demarcates microscope with the cross-graduation plate
Vertically regulate graticule, red Blue-Cross picture is overlapped, determine z axle zero point, x, y axle are taken on the axle of CCD center correspondence zero point, the longitudinal translation graticule, and range of translation ± 10 μ m, step-length 2 μ m, red Blue-Cross is listed in table 1 as spacing.
Table 1 calibration result
| Sequence number |
Defocusing amount (μ m) |
Distance (mm) between the red Blue-Cross picture |
| 1 |
10 |
0.5037 |
| 2 |
8 |
0.4031 |
| 3 |
6 |
0.3025 |
| 4 |
4 |
0.2018 |
| 5 |
2 |
0.1010 |
| 6 |
0 |
0.0000 |
| 7 |
-2 |
-0.1008 |
| 8 |
-4 |
-0.2019 |
| 9 |
-6 |
-0.3030 |
| 10 |
-8 |
-0.4042 |
| 11 |
-10 |
-0.5056 |
In table 1, the distance between the red Blue-Cross line is greater than 0, and expression Red Cross line is on a left side, and the Blue-Cross line is on the right side, and out of focus makes progress; Distance between the red Blue-Cross line is less than 0, and expression Blue-Cross line is on a left side, and the Red Cross line is on the right side, downwards out of focus; Distance between the red Blue-Cross line equals 0, represents red Blue-Cross line overlap, does not have out of focus.
2, the three-dimensional coordinate of measurement target
The red blue two-dimensional coordinate that measures last 3 the target pictures of CCD is listed in the table 2.
Table 2 measurement result unit: mm
Utilize formula (1)~(3) to calculate three Three-dimension Target coordinates, result of calculation is listed in the table 3.
The result of calculation unit of table 3 three-dimensional coordinate: μ m
| Target sequence number |
1 |
2 |
3 |
| Coordinate |
(-45.4,-1.7,1.6) |
(0.5,-1.8,0.0) |
(28.5,-1.7,8.9) |