CN101099673A - Surgical instrument positioning method using infrared reflective ball as marker point - Google Patents

Abstract

The invention relates to a positioning method for surgical instruments using infrared reflective balls as marker points, respectively determining the internal parameters and first-order radial distortion parameters of the left and right cameras, calculating the rotation matrix and translation vector representing the relative positional relationship between the left and right cameras, and then using The infrared light source irradiates the navigation bracket equipped with three infrared reflective balls, and two images of elliptical images containing infrared reflective balls are captured and corrected for distortion; by fitting the elliptical images of the left and right image planes after distortion correction, the direct linear The projection points of the three infrared reflective spheres are accurately calculated; the three center projection points on the left and right images are matched, and then the coordinates of the three infrared reflective spheres are reconstructed; finally, the tip of the surgical instrument is obtained through coordinate transformation The coordinates of the working part. The invention accurately determines the projection point of the center of the infrared reflective sphere, so that the positioning accuracy of the surgical instrument does not change with the change of its position, and at the same time expands the effective working range of the surgical instrument.

Description

Surgical instrument positioning method using infrared reflective ball as marker point

technical field

The invention relates to a surgical instrument positioning method using an infrared reflective ball as a mark point, which can be applied to a surgical instrument tracking and positioning system in a surgical navigator, and can be extended to all optical positioning systems that use an infrared reflective ball as a mark point. The invention belongs to the field of advanced medical equipment and technology.

Background technique

So far, surgical operations mainly rely on doctors to judge the location of the lesion with the naked eye, so as to perform operations such as surgical resection. The disadvantage of this method is that the surgical opening is large, the bleeding is much, and the postoperative recovery time is long. With the development of computer imaging technology, virtual reality technology and computer vision theory, people pay more and more attention to Surgical Navigation System. The surgical navigation system can provide doctors with an intuitive and visualized surgical operation platform, making minimally invasive surgery a reality and greatly reducing the pain of patients. In western developed countries, minimally invasive surgery and even remote surgery assisted by surgical navigation systems are gradually being promoted.

In order to achieve surgical navigation, accurate tracking and positioning of surgical instruments is an indispensable step. The navigation locator produced by Canadian NDI company is the mainstream product in the international market at present. However, due to the defects of existing positioning algorithms, the effective working range of almost all navigation and positioning equipment is relatively narrow, which often cannot meet the requirements of arbitrarily moving surgical instruments during the operation. At present, people often use the elliptical projection center of the infrared reflective marker ball to replace the projection point of the reflective ball center, but this premise is only approximately true when the surgical instrument is in the middle of the camera's field of view. However, during the operation, it is very likely that the surgical instrument needs to be moved in a relatively large range. Once the surgical instrument deviates from the central area of the camera's field of view, the positioning accuracy will decrease, which may lead to risky operations or even medical accidents. At present, common navigation products avoid risks by prompting users to exceed the scope of use, but this often causes inconvenience to use. For example, NDI has strict regulations on the use of its products (POLARIS User Manual, http://www.ndigital.com ). At the same time, the existing technology requires reducing the diameter of the reflective ball as much as possible, but when the spot is small, the spot boundary is easily disturbed by various noises on the image, resulting in relatively large errors.

In short, in order to expand the effective working range of the navigator, it is necessary to accurately determine the projection point of the center of the infrared reflective ball, and then determine the spatial coordinates of the center of the reflective ball, instead of replacing the projection point of the center of the ball with the center of the ellipse image of the reflective ball. Therefore, an algorithm that can obtain the projection point of the center of the reflective sphere with a small calculation cost has high practical value.

Contents of the invention

The purpose of the present invention is to address the deficiencies of the prior art, to propose a surgical instrument positioning method using infrared reflective balls as marker points, to expand the effective working range of the surgical navigation system, and to improve the positioning accuracy of the surgical instrument.

The above-mentioned purpose of the present invention is achieved through the following technical solutions: use the single camera calibration method to determine the internal parameters and first-order radial distortion parameters of the left and right cameras respectively, and use the stereo camera calibration method to calculate the relative positional relationship between the left and right cameras The rotation matrix and translation vector of ; use the infrared light source to irradiate the surgical instrument navigation bracket with 3 infrared reflective balls that are not on the same straight line, and then use the left and right cameras to shoot the navigation bracket at the same time, so as to obtain two left and right ellipses containing 3 reflective balls The image of the image; the distortion correction is performed on the two images so that they do not contain distortion information; the boundary of each reflective spherical ellipse image is detected on the distortion-corrected left and right images, and the ellipse equation is fitted by using the boundary pixels. In this way, three mathematical representation matrices of ellipse images are obtained respectively on the left and right images; the dual matrix of each matrix is obtained respectively, and the position of the projection point of the center of the reflective sphere is calculated by using the corresponding constraint relationship; The projected points of the center of the sphere are matched to get 3 point pairs, and then they are used to reconstruct the spatial coordinates of the sphere center of the 3 reflective spheres; finally, a series of coordinate transformations can be performed to calculate the spatial orientation of the working part of the surgical instrument tip.

The positioning method involved in the present invention includes the following specific steps:

(1) Camera calibration: Use the single camera calibration method for the left and right cameras respectively to determine the left camera’s internal parameter matrix A _l , the left camera’s first-order radial distortion parameter S _l , the right camera’s internal parameter matrix A _r , and the right camera’s The first-order radial distortion parameter S _r ; use the stereo camera calibration method to calculate the rotation matrix R ₀ and translation vector t ₀ representing the relative position relationship between the left and right cameras;

(2) Distortion correction: Use an infrared light source to irradiate the navigation bracket with three infrared reflective balls that are not on the same straight line, and place the left and right cameras near the navigation bracket to ensure that the navigation bracket is within the common field of view of the left and right cameras ; Use the left and right cameras to shoot the navigation bracket simultaneously, thereby obtaining an image of an ellipse image comprising three infrared reflective spheres at the left camera, and at the same time, obtaining an image of an ellipse image comprising three infrared reflective spheres at the right camera; The first-order radial distortion parameter S _l of the left camera performs distortion correction on the left image to obtain the left image without distortion information; at the same time, the right image is corrected for distortion by using the first-order radial distortion parameter S _r of the right camera to obtain The right image with distortion information;

(3) Boundary detection and ellipse fitting: Use the canny operator to detect the boundaries of the elliptical images of the reflective ball on the two image planes after distortion correction respectively, so that three groups of elliptical images are obtained on the left and right images respectively The point pairs formed by the coordinates of the boundary pixels; use the three sets of point pairs of the left image to fit the matrices C _l1 , C _l2 , and C _l3 representing the three elliptical images on the left, and then use the three sets of point pairs of the right image respectively , to fit the matrices C _r1 , C _r2 , C _r3 representing the three elliptical images on the right;

(4) Calculating the projection point of the center of the sphere: calculating the dual matrix C _li ^* of C _li and the dual matrix C _ri ^* of C _ri respectively, where i=1, 2, 3;

Use the following formula:

${\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; li</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; li</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; li</span>}}<span\; class="notranslate">\; -</span>{{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; li</span>}}}^{\mathrm{<span\; class="notranslate">\; T</span>}}$

${\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; the\; ri</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; the\; ri</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; the\; ri</span>}}{{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; the\; ri</span>}}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; ,</span>$

Solve k _li , k _ri , v _li , v _ri ; where k _li , k _ri are scale factors, i=1, 2, 3; the three-dimensional column vector v _li indicates that the center of the infrared reflective sphere corresponding to the ellipse C _li is at The homogeneous coordinates of the projection point on the left image, the three-dimensional column vector v _ri represents the homogeneous coordinates of the projection point of the center of the infrared reflective sphere corresponding to the ellipse C _ri on the right image; thus three infrared The homogeneous coordinates of the projection point of the center of the reflective sphere;

(5) Point matching and reconstruction: point matching is performed on the three spherical center projection points on the left image and the three spherical center projection points on the right image to obtain three sets of corresponding points; use the internal parameter matrices A _l and A _r and the rotation matrix R ₀ and the translation vector t ₀ are used for point reconstruction, and the space coordinates of the centers of the three infrared reflective spheres in the coordinate system of the left camera are obtained;

(6) Coordinate transformation: when installing the navigation bracket on the surgical instrument, randomly select the line connecting the centers of two infrared reflective balls as the X-axis, and the vertical line from the center of the third infrared reflective ball to the X-axis is the Y-axis. The vertical point is the coordinate origin, and the coordinate system of the navigation support is established according to the right-hand rule, and the coordinates of the working part of the tip of the surgical instrument in this coordinate system are determined at the same time; the spatial coordinates of the center of the three infrared reflective spheres in the left camera coordinate system are obtained, Calculate the rotation matrix R and translation vector t between the navigation bracket coordinate system and the left camera coordinate system; use the rotation matrix R and translation vector t to calculate the coordinates of the tip working part of the surgical instrument in the left camera coordinate system, thus completing Positioning of surgical instruments.

Compared with the existing method, the present invention accurately determines the projection point of the center of the infrared reflective sphere, so that the positioning accuracy of the surgical positioning instrument does not vary with the position of the surgical instrument. At the same time, the effective working range of surgical instruments is expanded, and the movement restrictions on surgical operations are greatly reduced. The invention can be used to improve the existing navigation equipment without changing the hardware configuration, and is beneficial to the transition upgrade of the existing products.

Description of drawings

Fig. 1 is a schematic diagram of the surgical instrument positioning method using infrared reflective balls as marker points according to the present invention.

Fig. 2 is a schematic diagram of the establishment and coordinate transformation of the coordinate system of the navigation support.

Detailed ways

In order to better understand the technical solutions of the present invention, the implementation manners of the present invention will be described in detail below in conjunction with the accompanying drawings and examples.

FIG. 1 is a schematic diagram of the positioning method of the present invention, in which a binocular stereo vision system with any configuration is shown. O _l X _l Y _l Z _l and Or _{X r} Y _r Z _r are the left and right camera coordinate systems respectively, and o _lu u _l v _l and or _{u r} v _r are the left and right image coordinate systems in units of pixels. A navigation bracket equipped with three infrared reflective balls that are not on the same straight line is placed within the common field of view of the left and right cameras. Use the left and right cameras to shoot the navigation bracket at the same time. At this time, three ellipses are obtained on the left and right images respectively, which are the projections of the three infrared reflective balls on the image.

Describe in detail the implementation steps of the inventive method below:

1. For the left and right cameras, use the single camera calibration method (Z. Zhang, Flexible camera calibration by viewing a plane from unknown orientations, proceedings of the Fifth International Conference on Computer Vision, 1999, pp.666-673) to determine the left camera The internal parameter matrix A _l , the first-order radial distortion parameter S _l of the left camera, the internal parameter matrix A _r of the right camera, and the first-order radial distortion parameter S _r of the right camera; using the stereo camera calibration method (Jean-Yves Bouguet, Camera Calibration Toolbox for Matlab, MRL-Intel Incorp.), calculate the rotation matrix R ₀ and translation vector t ₀ representing the relative positional relationship between the left and right cameras. Among them, the form of the internal parameter matrix is

${\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}& {\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; l</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; l</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$ ${\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}& {\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; r</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; r</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; .</span>$

2. Use an infrared light source to irradiate the navigation bracket with three infrared reflective balls that are not on the same straight line, and place the left and right cameras near the navigation bracket to ensure that the navigation bracket is within the common field of view of the left and right cameras; use the left and right cameras Simultaneously shoot the navigation bracket, so that an image of an ellipse image containing three infrared reflective balls is obtained at the left camera, and at the same time, an image of an ellipse image containing three infrared reflective balls is also obtained at the right camera; Distortion correction is performed on the left image using the first-order radial distortion parameter S _l to obtain the left image without distortion information, denoted as π _l ; at the same time, the distortion correction is performed on the right image by using the first-order radial distortion parameter S _r of the right camera to obtain The right image without distortion information is denoted as π _r ;

其归一化的图像坐标为

它们对应的不含畸变信息的图像点分别记为(u，v)和(x，y)。根据文献(D.C.Brown，Close-range camera calibration，Photogram-metric Engineering，37(8)：855-866，1971)，有The specific correction process is: for the image on the left, set the coordinates of an image point containing distortion information in the image coordinate system in units of pixels as

Its normalized image coordinates are

Their corresponding image points without distortion information are denoted as (u, v) and (x, y) respectively. According to the literature (DCBrown, Close-range camera calibration, Photogram-metric Engineering, 37(8):855-866, 1971), there are

$\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; x</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>$

$\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; +</span>\mathrm{<span\; class="notranslate">\; the\; y</span>}<span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>$

Use the coordinate transformation formula

$\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; u</span>}\\ \mathrm{<span\; class="notranslate">\; v</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; =</span>\left[\begin{array}{ccc}{\mathrm{<span\; class="notranslate">\; \&alpha;</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}& {\mathrm{<span\; class="notranslate">\; \&gamma;</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}& {\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; l</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& {\mathrm{<span\; class="notranslate">\; \&beta;</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}& {\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; l</span>}}\\ \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 0</span>}& \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]\left[\begin{array}{c}\mathrm{<span\; class="notranslate">\; x</span>}\\ \mathrm{<span\; class="notranslate">\; the\; y</span>}\\ \mathrm{<span\; class="notranslate">\; 1</span>}\end{array}\right]<span\; class="notranslate">\; ,</span>$

$\left[\begin{array}{c}\hat{u}\\ \hat{v}\\ 1\end{array}\right]=A_l\left[\begin{array}{c}\hat{x}\\ \widehat{\mathrm{the\; y}}\\ 1\end{array}\right]=\left[\begin{array}{ccc}{\mathrm{\&alpha;}}_l& {\mathrm{\&gamma;}}_l& u_{0l}\\ 0& {\mathrm{\&beta;}}_l& v_{0l}\\ 0& 0& 1\end{array}\right]\left[\begin{array}{c}\hat{x}\\ \widehat{\mathrm{the\; y}}\\ 1\end{array}\right],$ Among them, S _l is the first-order radial distortion parameter of the left camera, and A _l is the internal parameter matrix of the left camera, and both of them have been determined through calibration.

can get

$\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>$

$\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; =</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\mathrm{<span\; class="notranslate">\; the\; y</span>}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>$

Since the above equations are nonlinear equations, in order to simplify the solution process, the above equations can be approximated as (Janne Heikkila, Geometric Camera Calibration Using Circular Control Points, IEEE.0n Pattern Analysis and Machine Intelligence, vol.22, no.10, 1066- 1077)

$\mathrm{<span\; class="notranslate">\; u</span>}<span\; class="notranslate">\; =</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; u</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; u</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; (</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>$

$\mathrm{<span\; class="notranslate">\; v</span>}<span\; class="notranslate">\; =</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; -</span><span\; class="notranslate">\; (</span>\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; v</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; 0</span>}\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; [</span>{\mathrm{<span\; class="notranslate">\; S</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}<span\; class="notranslate">\; (</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; x</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{\stackrel{<span\; class="notranslate">\; ^</span>}{\mathrm{<span\; class="notranslate">\; the\; y</span>}}}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; ]</span>$

Using the above two formulas, distortion correction can be performed on each image point on the left image, so as to obtain an image π _l without distortion information. For the image on the right, the correction method is exactly the same as for the image on the left.

3. Use the canny operator to detect the boundaries of the ellipse images of the infrared reflective ball on the two image planes of π _l and π _r respectively, so that three groups of coordinates composed of the pixel points of the ellipse image boundaries are obtained on the left and right images respectively point pair; use the three groups of point pairs in the left image to fit the matrices C _l1 , C _l2 , C _l3 representing the three elliptical images on the left, and then use the three groups of point pairs in the right image to fit the three Matrices C _r1 , C _r2 , C _r3 of ellipse images. For specific methods, see (AW Fitzgibbon, M. Pilu, and RB Fisher, "Direct Least-Squares Fitting of Ellipses", IEEE Trans. Pattern Analysis and Machine Intelligence, vol.14, no.2, pp.239-256).

4. Calculate the dual matrix C _li ^* of C _li and the dual matrix C _ri ^* of C _ri respectively, where i=1, 2, 3;

Use the following formula:

${\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; li</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; li</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; l</span>}}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; li</span>}}{{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; li</span>}}}^{\mathrm{<span\; class="notranslate">\; T</span>}}$

${\mathrm{<span\; class="notranslate">\; k</span>}}_{\mathrm{<span\; class="notranslate">\; the\; ri</span>}}{\mathrm{<span\; class="notranslate">\; C</span>}}_{\mathrm{<span\; class="notranslate">\; the\; ri</span>}}^{<span\; class="notranslate">\; *</span>}<span\; class="notranslate">\; =</span>{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}{{\mathrm{<span\; class="notranslate">\; A</span>}}_{\mathrm{<span\; class="notranslate">\; r</span>}}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; the\; ri</span>}}{{\mathrm{<span\; class="notranslate">\; v</span>}}_{\mathrm{<span\; class="notranslate">\; the\; ri</span>}}}^{\mathrm{<span\; class="notranslate">\; T</span>}}<span\; class="notranslate">\; ,</span>$

Solve k _li , k _ri , v _li , v _ri ; where k _li , k _ri are scale factors, i=1, 2, 3; the three-dimensional column vector v _li indicates that the center of the infrared reflective sphere corresponding to the ellipse C _li is at The homogeneous coordinates of the projection point on the left image, the three-dimensional column vector v _ri represents the homogeneous coordinates of the projection point of the center of the infrared reflective sphere corresponding to the ellipse C _ri on the right image; thus three infrared The homogeneous coordinates of the projection point of the center of the reflective sphere;

_The specific method is: calculate the generalized eigenvalues of matrix A _l A _l ^T and ^matrix C _li ^* , then the unknown ^scale factor k _li is their double _{generalized eigenvalues ;} Value decomposition, let the eigenvector corresponding to its non-zero eigenvalue be β _i , then $v_{\mathrm{li}}=\frac{{\mathrm{\&beta;}}_i}{\left|\right|{\mathrm{\&beta;}}_i\left|\right|},$ Thus, the homogeneous coordinates of the projection point of the center of the infrared reflective sphere corresponding to the ellipse C _li on the left image are calculated; the image on the right is processed in the same way.

5. Point matching and reconstruction: point matching is performed on the three spherical projection points on the left image and the three spherical projection points on the right image to obtain three sets of corresponding points; use the internal parameter matrices A _l and A of the left and right cameras _r and the rotation matrix R ₀ and the translation vector t ₀ are used for point reconstruction, and the space coordinates of the centers of the three infrared reflective spheres in the left camera coordinate system are obtained;

The specific method is: since the left and right cameras are close to each other and are on the same side of the navigation bracket, the positions of the three projection points of the center of the sphere on the left and right images are similar. Scan row by row and column by row on the left and right image planes respectively, and directly match the left and right spherical center projection points in similar positions.

Using the spatial point reconstruction method, when the image coordinates of the two projected points and the internal parameter matrices A _l and A _r of the left and right cameras are known, as well as the rotation matrix R ₀ and translation vector t ₀ representing the relative position relationship between the left and right cameras, it can be reconstructed The space coordinates of the center of the sphere in the coordinate system of the left camera (Ma Songde, Zhang Zhengyou, Computer Vision---Basics of Computing Theory and Algorithms, Science Press, 1999).

(6) When installing the navigation bracket on the surgical instrument, randomly select the line connecting the centers of two infrared reflective balls as the X-axis, the vertical line from the center of the third infrared reflective ball to the X-axis is the Y-axis, and the vertical point is The origin of the coordinates, establish the coordinate system of the navigation bracket according to the right-hand rule; at the same time, determine the coordinates of the working part of the tip of the surgical instrument in this coordinate system; use the obtained space coordinates of the centers of the three infrared reflective spheres in the coordinate system of the left camera to calculate the navigation Rotation matrix R and translation vector t between the bracket coordinate system and the left camera coordinate system; use the rotation matrix R and translation vector t to calculate the coordinates of the tip working part of the surgical instrument in the left camera coordinate system, thereby completing the alignment of the surgical instrument positioning.

Figure 2 is a schematic diagram of the establishment of the coordinate system of the navigation support and its coordinate transformation. It is advisable to choose the connection line between the center of the sphere O _A and O _B as the X-axis, draw the vertical line from the center of the sphere O _C to the X-axis, take the vertical point as the origin of the coordinates, and denote it as O _w , and establish the coordinate system O of the navigation support according to the right-hand rule _w -X _w Y _w Z _w ; through precise installation, determine the coordinates of the operating part O _D of the tip of the surgical instrument in the coordinate system of the navigation support, which is recorded as α _D = [X _wD Y _wD Z _wD ] ^T ; set the center of the sphere The coordinates of O _A , O _B , and O _C in the left camera coordinate system are α _A ＝[X _cA Y _cA Z _cA ] ^T , α _B ＝[X _cB Y _cB Z _cB ] ^T , α _C ＝[X _cC Y _cC Z _cC ] ^T ; the rotation matrix between the navigation bracket coordinate system O _w -X _w Y _w Z _w and the left camera coordinate system O _l -X _l Y _l Z _l is R, and the translation vector is t; then the coordinate origin The coordinates of O _w in the left camera coordinate system are ${\mathrm{\&alpha;}}_o=\left[\begin{array}{c}{x}_{\mathrm{cO}}\\ Y_{\mathrm{cO}}\\ Z_{\mathrm{cO}}\end{array}\right]=\left[\begin{array}{c}{x}_{\mathrm{cA}}\\ Y_{\mathrm{cA}}\\ Z_{\mathrm{cA}}\end{array}\right]-\mathrm{\&lambda;}(\left[\begin{array}{c}{x}_{\mathrm{cB}}\\ Y_{\mathrm{cB}}\\ Z_{\mathrm{cB}}\end{array}\right]-\left[\begin{array}{c}{x}_{\mathrm{cA}}\\ Y_{\mathrm{cA}}\\ Z_{\mathrm{cA}}\end{array}\right]),$ in

$\mathrm{<span\; class="notranslate">\; \&lambda;</span>}<span\; class="notranslate">\; =</span>\frac{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; cB</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; cA</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; cA</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; cB</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; cA</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; cA</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; +</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; cB</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; cA</span>}}<span\; class="notranslate">\; )</span><span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; C</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; cA</span>}}<span\; class="notranslate">\; )</span>}{{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; cB</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; x</span>}}_{\mathrm{<span\; class="notranslate">\; cA</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; cB</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Y</span>}}_{\mathrm{<span\; class="notranslate">\; cA</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}<span\; class="notranslate">\; +</span>{<span\; class="notranslate">\; (</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; cB</span>}}<span\; class="notranslate">\; -</span>{\mathrm{<span\; class="notranslate">\; Z</span>}}_{\mathrm{<span\; class="notranslate">\; cA</span>}}<span\; class="notranslate">\; )</span>}^{\mathrm{<span\; class="notranslate">\; 2</span>}}}$

Then the rotation matrix $R=\left[\begin{array}{ccc}\frac{{\mathrm{\&alpha;}}_A-{\mathrm{\&alpha;}}_o}{\left|\right|{\mathrm{\&alpha;}}_A-{\mathrm{\&alpha;}}_o\left|\right|}& \frac{{\mathrm{\&alpha;}}_C-{\mathrm{\&alpha;}}_o}{\left|\right|{\mathrm{\&alpha;}}_C-{\mathrm{\&alpha;}}_o\left|\right|}& \frac{{\mathrm{\&alpha;}}_A-{\mathrm{\&alpha;}}_o}{\left|\right|{\mathrm{\&alpha;}}_A-{\mathrm{\&alpha;}}_o\left|\right|}\&times;\frac{{\mathrm{\&alpha;}}_C-{\mathrm{\&alpha;}}_o}{\left|\right|{\mathrm{\&alpha;}}_C-{\mathrm{\&alpha;}}_o\left|\right|}\end{array}\right]$

Translation vector t=α _A -λ(α _B -α _A )

The coordinate of the tip working part of the surgical instrument in the left camera coordinate system O _l -X _l Y _l Z _l is Rα _D +t, so far, the positioning of the surgical instrument is completed.

Claims (1)

1, a kind of infrared reflecting ball that adopts is the surgical instrument positioning method of index point, it is characterized in that comprising following steps:

(1) camera calibration: left and right cameras is utilized single camera calibration method respectively, determine the intrinsic parameter matrix A of left video camera _l, left video camera single order radial distortion parameter S _l, right video camera the intrinsic parameter matrix A _r, right video camera single order radial distortion parameter S _rUtilize the stereo camera scaling method, calculate the spin matrix R that characterizes the left and right cameras relative position relation ₀With translation vector t ₀

(2) near distortion correction: with infrared light supply irradiation the not navigation support of three infrared reflecting balls on same straight line is installed, and left and right cameras is placed on the navigation support, guarantee that this navigation support is in the public view field scope of left and right cameras; Utilize left and right cameras to take the navigation support simultaneously, thereby obtain the image that a width of cloth comprises the oval image of three infrared reflecting balls, simultaneously, also obtain the image that a width of cloth comprises the oval image of three infrared reflecting balls at right video camera at left video camera; Utilize the single order radial distortion parameter S of left video camera _lLeft image is carried out distortion correction, do not contained the left image of distortion information; Simultaneously, utilize the single order radial distortion parameter S of right video camera _rRight image is carried out distortion correction, do not contained the right image of distortion information;

(3) border detection and ellipse fitting: utilize the canny operator, detect the border of the oval image of witch ball on two width of cloth planes of delineation behind the distortion correction respectively, thereby it is right to obtain the point that 3 groups of coordinates by oval image boundary pixel constitute on the image of the right and left respectively; Utilize 3 groups of points of left image right respectively, simulate the Matrix C of three the oval images in the expression left side _L1, C _L2, C _L3, utilize 3 groups of points of right image right more respectively, simulate the Matrix C of three the oval images in expression the right _R1, C _R2, C _R3

(4) calculate the gnomonic projection point: calculate C respectively _LiThe antithesis Matrix C _Li ^*, C _RiThe antithesis Matrix C _Ri ^*, i=1 wherein, 2,3;

Utilize following formula:

$k_{\mathrm{li}}{C}_{\mathrm{li}}^{*}=A_l{A_l}^T-v_{\mathrm{li}}{v_{\mathrm{li}}}^T$

$k_{\mathrm{ri}}{C}_{\mathrm{ri}}^{*}=A_r{A_r}^T-v_{\mathrm{ri}}{v_{\mathrm{ri}}}^T,$

Solve k _Li, k _Ri, v _Li, v _RiK wherein _Li, k _RiBe scale factor, i=1,2,3; Three dimensional vector v _LiExpression and oval C _LiThe homogeneous coordinates of the subpoint on the corresponding infrared reflecting ball centre of sphere on the left side image, three dimensional vector v _RiExpression and oval C _RiThe corresponding infrared reflecting ball centre of sphere is the homogeneous coordinates of the subpoint on the image on the right; Thereby on the image of the left and right sides, obtain the homogeneous coordinates of the subpoint of three infrared reflecting ball centre ofs sphere respectively;

(5) coupling and rebuild: three gnomonic projection points on the left image and three gnomonic projection points on the right image are carried out a coupling, obtain 3 groups of corresponding point; Utilize the intrinsic parameter matrix A of left and right cameras _l, A _rAnd spin matrix R ₀With translation vector t ₀Carry out a reconstruction, obtain three space coordinatess under the infrared reflecting ball centre of sphere on the left side camera coordinate system;

(6) Coordinate Conversion: when the navigation support is installed on operating theater instruments, the centre of sphere line of choosing 2 infrared reflecting balls arbitrarily is as X-axis, the centre of sphere of the 3rd infrared reflecting ball is a Y-axis to the vertical line of X-axis, the point that hangs down is zero, set up navigation support coordinate system according to right-hand rule, determine the coordinate of the most advanced and sophisticated working position of operating theater instruments under this coordinate system simultaneously; Space coordinates under three infrared reflecting ball centre of sphere on the left side camera coordinate systems that utilization obtains calculates spin matrix R and translation vector t between navigation support coordinate system and the left side camera coordinate system; Utilize spin matrix R and translation vector t, calculate the coordinate of the most advanced and sophisticated working position of operating theater instruments under left camera coordinate system, finish location thus operating theater instruments.

Images