WO2017018748A1 - Optical tracking marker, optical tracking system, and optical tracking method - Google Patents

Abstract

The present invention provides a marker comprising an image formed to have a random shape so that a plurality of feature points can be arbitrarily selected, and a lens for enlarging and transmitting the image, and provides an optical tracking system and a tracking method for tracking the location or posture of the marker. The location and posture, in the present invention, may be determined on the basis of the plurality of feature points which have been arbitrarily selected from the image having the random shape. In addition, the present invention provides a method for forming, on a marker, an image having a random shape from which a plurality of feature points can be arbitrarily selected.

Description

Marker, optical tracking system and optical tracking method for optical tracking

The present invention relates to a marker for optical tracking, an optical tracking system and an optical tracking method.

The present invention also relates to a method of forming an image on an optical tracking marker.

An optical tracking system can be used as one method for tracking the object. Specifically, an optical tracking system may be used to track the location or coordinates and posture or orientation of the affected area or surgical robot or surgical tool. The position of the object may be defined as spatial coordinates such as, for example, coordinates on the X, Y, and Z axes in the rectangular coordinate system. The posture of the target may be defined as a roll, pitch, yaw. For accurate tracking of the object, it is important to know exactly the position and posture corresponding to the six degrees of freedom of the object, as described above.

In an optical tracking system, for example, a reference object called a marker is attached to an object, and then the position and attitude of the object can be measured by tracking the marker. For example, a structure having three or more markers attached thereto may be attached to the object to measure the position and posture of the object. The method of using three or more markers has a problem that miniaturization of the tracking reference body is difficult. In particular, when a plurality of markers are attached to a tool used directly by a person, such as a surgical tool, the size and weight thereof may interfere with the use of the user's tool or the user may feel uncomfortable.

The problem to be solved by the present invention is to provide an optical tracking marker that can be reduced in weight or downsized.

In addition, the problem to be solved by the present invention is to provide an optical tracking system and method that can measure both the position and the posture of the object using a lightweight or compact marker. For example, the present invention provides an optical tracking system and tracking method for tracking the position and posture of an object to which at least one marker is attached.

Another object of the present invention is to provide a method for forming an image having at least two feature points on a marker for an optical tracking system having a three-dimensional shape.

The marker for measuring position and posture according to an embodiment of the present invention may include an image having a random shape formed so that a plurality of feature points can be arbitrarily selected, and a lens for enlarging and transmitting the image.

According to an embodiment, the image may be located on a surface on which the focal point of the lens is formed.

According to one embodiment, the lens may be coated such that only light of a specific wavelength range may be transmitted.

According to an embodiment, the plurality of feature points in the image may be formed of a light emitting material.

According to an embodiment, the image may be formed of two or more materials having different reflectances for light incident to the marker, and the plurality of feature points in the image may reflect a difference in reflectances of the two or more materials. It can be formed using.

According to an embodiment, the image may be formed of two or more materials having different transmittances to light incident on the marker, and the plurality of feature points in the image may indicate differences in transmittances of the two or more materials. It can be formed using.

According to an embodiment, when a plurality of markers are used, the random shape may be unique for each marker.

According to an embodiment, the surface where the focal point of the lens is formed may be an outer surface of the surface reflected from the lens after light is incident on the lens.

According to an embodiment, the surface where the focal point of the lens is formed may be spaced apart from the lens, and the image may be spaced apart from a material having a refractive index different from that of the lens.

According to an embodiment, the image may be formed on an outer portion of a surface where light is reflected after the light is incident on the lens and passes through the material having the different refractive index.

According to an embodiment, the surface on which the focal point of the lens is formed may be formed inside the lens.

According to an embodiment of the present disclosure, light may be irradiated onto the image by including a light source that is brighter than the inside of the marker.

According to an embodiment, the image may be a predetermined shape instead of a random shape.

According to an embodiment, the image may be formed by printing an image material.

According to an embodiment, the image may be formed by lithography using an image material.

An optical tracking system according to an embodiment of the present invention includes a first image having a random shape and a first lens for enlarging and transmitting the first image having a random shape formed so that a plurality of feature points can be arbitrarily selected. And an imaging unit including one or more imaging elements for forming the first image into a second image at a focal length of the second lens and the second lens, and optionally selecting a plurality of feature points from the first image. The object to which the marker is attached can be tracked.

In example embodiments, the first image may be positioned on a surface on which the focal point of the first lens is formed.

According to an embodiment, the optical tracking system may determine a posture based on first information related to the first image and second information related to the second image, and the first information is included in the first image. The coordinates may be coordinates of two or more feature points among the plurality of feature points, and the second information may be detected in the second image as coordinates corresponding to the two or more feature points.

According to an embodiment, the first image may be formed of two or more materials having different reflectances to light, and the plurality of feature points included in the first image may indicate a difference in reflectances of the two or more materials. It can be formed using.

In example embodiments, the first image may be formed of two or more materials having different transmittances to light, and the plurality of feature points included in the first image may indicate a difference between transmittances of the two or more materials. It can be formed using.

According to an embodiment, when the marker is attached to a plurality of objects, the random shape of the first image may be unique for each marker.

According to an embodiment, the first image may be a predetermined shape instead of the random shape.

The optical tracking method according to an embodiment of the present invention comprises the steps of: imaging the first image of a random shape included in a marker that is extended and transmitted through the first lens into a second image at a focal length of the second lens; And determining a posture of the marker based on the first information related to the first image and the second information related to the second image, wherein the randomly shaped first image has a plurality of feature points selected arbitrarily. It may be formed to be able to be, and to track a target to which the marker is attached by selecting a plurality of arbitrary feature points from the first image.

According to an embodiment, the determining of the pose of the marker based on the first information related to the first image and the second information related to the second image may include determining coordinates of two or more feature points from the second image. Detecting as second information and detecting coordinates of a feature point corresponding to two or more feature points detected from the second image as the first information among the plurality of feature points included in the first image stored in advance; And determining the posture from a relational expression between the coordinates as the first information and the coordinates as the second information.

The method of forming an image on a marker for an optical tracking system according to an embodiment of the present invention may include preparing a marker including a lens and forming an image on the marker. Forming may include forming a portion of the image with a material having a first reflectance and forming a remainder of the image with a material having a second reflectance different from the first reflectance in the marker, The image may be formed in a random shape such that a plurality of feature points may be arbitrarily selected from the difference between the first reflectance and the second reflectance, and the plurality of feature points may be arbitrarily detectable by the optical tracking system.

According to an embodiment, the forming of the image on the marker may be forming the image on a surface on which the focal point of the lens is formed.

In example embodiments, a portion formed of a material having the first reflectance or the second reflectance may form the feature point.

The marker according to an embodiment of the present invention may be implemented in a small size.

In addition, according to the optical tracking system using a marker according to an embodiment of the present invention, the position and posture corresponding to six degrees of freedom of the object to which the marker is attached may be measured using at least one marker.

In addition, according to the optical tracking system using the marker according to an embodiment of the present invention, the trackable distance can be increased.

In addition, according to the optical tracking system using a marker according to an embodiment of the present invention, a high speed tracking is possible.

In addition, according to the method for forming an image on a marker for an optical tracking system according to an embodiment of the present invention, an image having at least two feature points may be simply formed on a small marker having a three-dimensional shape.

1 is a view showing a surgical scene as an application of the optical tracking system according to an embodiment of the present invention.

2 is a diagram showing an example in which a marker according to an embodiment of the present invention is attached to a surgical tool and used.

3 is a block diagram of an optical tracking system including a marker according to an embodiment of the present invention.

4 is a diagram illustrating a process of transmitting image information included in a marker to the outside according to an embodiment of the present invention.

FIG. 5 is a diagram for describing a case where a focus is formed according to characteristics of a ball lens when a marker according to an embodiment of the present invention is implemented using a ball lens.

6 is a view for explaining the position of the surface on which the image is formed on the marker according to the focal position of the ball lens when implementing the marker according to an embodiment of the present invention.

7 is a lens for a marker according to an embodiment of the present invention, which shows a lens combining two hemispherical lenses.

8 is a lens for a marker according to an embodiment of the present invention, which shows a lens combining two identical hemispherical lenses.

FIG. 9 illustrates a drum lens as a lens for a marker according to an exemplary embodiment of the present invention.

FIG. 10 is a view illustrating a lens combining two spherical dome lenses as lenses for a marker according to an embodiment of the present invention.

FIG. 11 illustrates a lens for a marker according to an embodiment of the present invention, in which two spherical dome lenses having the same inner radius are combined.

12 is a lens for a marker according to an embodiment of the present invention, which shows a lens combining two spherical dome lenses having the same outer radius.

FIG. 13 is a view illustrating a lens combining two spherical dome lenses having the same shape as a lens for a marker according to an embodiment of the present invention.

FIG. 14 is a view illustrating a lens combining two spherical dome lenses and one ball lens as a lens for a marker according to an embodiment of the present invention.

FIG. 15 is a lens for a marker according to an embodiment of the present invention, which shows two spherical dome lenses and one ball lens which are not spherical.

FIG. 16 is a lens for a marker according to an embodiment of the present invention, which illustrates a lens combining one ball lens and one optical medium having a concentric surface.

FIG. 17 is a lens for a marker according to an embodiment of the present invention, which illustrates a lens combining one ball lens and one spherical dome lens.

FIG. 18 is a lens for a marker according to an embodiment of the present invention, which illustrates a lens combining one ball lens and one spherical dome lens instead of the hemisphere.

19 is a view showing a GRIN ball lens as a lens for a marker according to an embodiment of the present invention.

20 is a diagram illustrating an image formed on one surface of a ball lens and feature points included in the image according to an embodiment of the present invention.

21 is a diagram illustrating a case in which an image included in a marker has a predetermined shape according to an embodiment of the present invention.

22 is a diagram illustrating a marker including a light source therein according to one embodiment of the present invention.

23 is a diagram illustrating a marker including a light source therein according to one embodiment of the present invention.

24 is a view illustrating a positional relationship between a lens and an imaging device in an imaging unit in an optical tracking system according to an exemplary embodiment of the present invention.

25 is a diagram for describing a process of detecting a posture of a marker, according to an embodiment of the present invention.

FIG. 26 is a diagram exemplarily illustrating a change in a feature point in an image of a marker formed in an image forming unit as a marker moves in an optical tracking system according to an exemplary embodiment of the present invention.

27 is a block diagram of an optical tracking system in which an imaging unit includes two imaging devices in order to measure the position of a marker in the optical tracking system according to an exemplary embodiment of the present invention.

28 is a flowchart illustrating a method of determining a pose of a marker in an optical tracking system according to an embodiment of the present invention.

29 is a flowchart illustrating a method of determining a posture of a marker from image information included in the marker in the optical tracking system according to an embodiment of the present invention.

30 is a flowchart illustrating a method of determining the position of a marker in an optical tracking system according to an embodiment of the present invention.

FIG. 31 is a view illustrating a method of forming an image on a marker for an optical tracking system according to an embodiment of the present invention. The image is formed on the marker by a spraying technique.

32 is a flowchart illustrating a method of forming an image on a marker for an optical tracking system according to an embodiment of the present invention.

FIG. 33 is a diagram illustrating a method of forming an image on a marker for an optical tracking system according to an embodiment of the present invention, and illustrating an image on a marker by a printing technique.

FIG. 34 is a view illustrating a method of forming an image on a marker for an optical tracking system according to an embodiment of the present invention and forming an image on the marker by a technique of injecting a substance.

FIG. 35 is a view illustrating a method of forming an image on a marker for an optical tracking system according to an embodiment of the present invention. The image is formed on the marker by a stencil technique.

FIG. 36 is a view illustrating a method of forming an image on a marker for an optical tracking system according to an embodiment of the present invention, wherein the image is formed on the marker by a stamping technique.

FIG. 37 is a view illustrating a method of forming an image on a marker for an optical tracking system according to an embodiment of the present invention, and implementing the marker using a 3D printer.

FIG. 38 illustrates a method of forming an image on a marker for an optical tracking system according to an embodiment of the present invention, and illustrates a method of forming an image on a marker by a lithography technique.

FIG. 39 is a view illustrating a method of forming an image on a marker for an optical tracking system according to an embodiment of the present invention, and a method of forming an image on the marker by a lithography technique.

40 is a view illustrating a method of forming an image on a marker for an optical tracking system according to an embodiment of the present invention, and a method of forming an image on the marker by a lithography technique.

Hereinafter, exemplary embodiments of the present invention will be described with reference to the accompanying drawings. However, in the following description, when there is a risk of unnecessarily obscuring the gist of the present invention, a detailed description of well-known functions and configurations will be omitted.

< Optical Tracking  System>

1 illustrates a surgical scene as an application of an optical tracking system according to an embodiment of the present invention. In the optical tracking system, the markers 110 and 130 are attached to an object such as an affected part or a surgical tool, and then the markers 110 and 130 of the markers 110 and 130 are based on the information about the markers 110 and 130 obtained through the imaging unit 150. By determining the position and attitude, the object to which the markers 110 and 130 are attached can be tracked. In general, in order to determine both the position and the posture corresponding to the six degrees of freedom of the object to be tracked, a structure having three or more markers attached thereto may be attached to the object. In the optical tracking system according to an embodiment of the present invention, all six degrees of freedom of the object to which the marker is attached may be determined by only attaching one marker.

2 shows a state in which the marker 210 according to an embodiment of the present invention is attached to the surgical tool 230 and used. The marker may be attached or detached to the object to be tracked. When using a marker for medical use, since the marker must be disinfected, a removable method may be convenient. In the case of the marker 210 according to an embodiment of the present invention, since a small number of markers can be attached to and removed from a target object, the detaching process may be simple. In addition, because there are fewer objects to sterilize, the time and effort required to disinfect the markers can be reduced. If there are many objects to which the markers are to be attached, an optical tracking system that can use a small number of markers as in the present invention can be conveniently used. Hereinafter, the optical tracking system of the present invention briefly described with reference to FIGS. 1 and 2 will be described in more detail.

3 is a block diagram of an optical tracking system 300 in accordance with an embodiment of the present invention. The optical tracking system 300 may include a marker 310, an image forming unit 330, a light source 350, and a processor 370. The marker 310 may include an image 312 and a lens 314 for enlarging and transmitting the image 312. The imaging unit 330 may include an imaging device 334 and a lens 332 capable of imaging an image transmitted from the marker 310.

According to an embodiment, the imaging unit 330 may include two or more imaging devices 334. Alternatively, two or more imaging units 330 may be included in the optical tracking system 300. The image 312 transmitted from the marker 310 may pass through the lens 332 and may be formed into another image in the imaging device 334.

The light source 350 may be arranged to irradiate light toward the marker 310 to help the image 312 included in the marker 310 form an image in the image forming unit 330. The light source 350 may be disposed outside the marker 310, and in this case, the marker 310 may operate as a passive marker. According to another embodiment, the light source 350 may be disposed inside the marker 310, in which case the marker 310 may operate as an active marker.

The processor 370 is hardware or software capable of calculating the position and posture of the marker 310 by receiving image information of the marker 310 formed by the imaging unit 330. For example, the processor 370 may be a CPU (central processing unit). According to an embodiment, the processor 370 may be used independently of the optical tracking system 300 or installed in the imaging unit 330. According to another embodiment, the processor 370 may distribute functions for calculating the position and attitude of the marker 310 to various components such as the image forming unit 330 in order to reduce the load on the calculation amount. According to another embodiment, the processor 370 may be located at a different place far from where the optical tracking system 300 is installed, and may exchange data with the optical tracking system 300 through a wired or wireless network. . As such, when the functions of the processor 370 are distributed, the burden on the amount of computation or the speed of the optical tracking system 300 itself may be reduced. In this case, the high speed processor 370 may not be required.

< Marker (310)>

The marker 310 is an object to be measured position and posture through the optical tracking system 300. By measuring the position and posture of the marker 310, the position and posture of the object to which the marker 310 is attached may be measured. Hereinafter, the marker 310 to be used in the present invention will be described in more detail.

4 illustrates a process of transmitting image information included in a marker to the outside through the lens 314 according to an embodiment of the present invention. The marker 310 according to an embodiment of the present invention may operate in a passive manner or an active manner. Either way, by causing the light emitted from the light source 350 to be reflected on the image 312 included in the marker 310, the image 312 information may be transmitted to the outside of the marker 310 via the lens 314. . By allowing the light emitted from the light source 350 to be reflected on the image 312 included in the marker 310, the discriminating power of the image 312 of the marker 310 formed in the image forming unit 330 may be increased. According to another embodiment, the light irradiated to the image 312 is transmitted through the image 312, so that the image 312 information may be transmitted to the outside via the lens 314.

In order to reflect or transmit light in image 312, image 312 may be formed of a material having a high or low reflectivity or a material having a high or low transmittance for light emitted to image 312. Here, the reflectance may mean the ratio of the energy of the reflected light and the energy of the incident light, and the transmittance may mean the ratio of the energy of the transmitted light and the energy of the reflected light. The reason for distinguishing the reflectance from the transmittance is to consider the case where the energy of light is absorbed in the image.

The marker 310 may include a lens 314 that enlarges the image 312 and transmits the image to the outside. The type of the lens 314 is not particularly limited, and any type of lens capable of magnifying and transferring the image 312 included in the marker 310 may be used. According to one embodiment, the image 312 may be positioned on the marker 310 where the image 312 may be identified outside the marker 310 through the lens 314. In detail, the lens 314 and the image 312 may be disposed such that the image 312 may be transmitted to the outside of the marker 310 in the form of parallel emission light. This is a condition that can be combined with the optical system of the image forming unit 330 side of the optical tracking system 300 to implement an infinite optical system. When the infinite optical system is implemented, through this, the imaging unit 330 may obtain an imaging image of the enlarged marker 310. Therefore, even if the position of the marker 310 is far from the image forming unit 330, it may be easy to obtain information on the image 312 included in the marker 310 from the image forming image. Hereinafter, the arrangement relationship between the image 312 and the lens 314 in the marker 310 will be described in more detail.

5 illustrates various focal positions of the ball lens 500 when the lens 314 is used as the ball lens 500 according to an exemplary embodiment of the present invention. As shown in FIG. 5, the ball lens 500 focuses the light incident on the ball lens 500 on the surface P ₀ , inside P ₂ , or outside P ₁ of the ball lens 500. Can bear. The position where the focal point of the lens 314 is formed may vary depending on the material or appearance of the lens 314. When the image 312 is positioned on the surface in which the lens 314 is in focus in accordance with the focus characteristic of the lens 314, the image 312 may be transmitted in the form of parallel output light through the lens 314. .

6 illustrates a position of a surface on which an image 312 is to be formed according to a position at which the ball lens 500 is focused when the lens 314 is used as the ball lens 500 according to an exemplary embodiment. According to an embodiment, if the focal point of the ball lens 500 is focused on the surface P ₀ of the ball lens 500, the image 312 may be formed on the surface 510 of the ball lens 500. Specifically, the image 312 may be formed on the outer surface of the surface reflected from the lens 314 after light is incident on the lens 314.

According to another embodiment, if the focal point of the ball lens 500 is confined to the exterior P ₁ spaced apart from the ball lens 500, the image 312 is a focal plane 530 spaced apart from the ball lens 500. Can be formed on. In this case, the focal plane 530 may form a concentric circle with the ball lens 500. The surface on which the image 312 is formed and the spaced space on the lens 312 may be filled with a material having a refractive index different from that of the lens 312. Spaces may be filled with gas. By adjusting the refractive index of the material to be filled in the spaced space, it is possible to adjust the recognition rate for the image 312 on the image forming unit 330 side. It is also possible to adjust the separation distance between the lens 314 and the image 312.

According to another embodiment, if the focal point of the ball lens 500 is in the interior P ₂ of the ball lens 500, the image 312 may be formed on the inner surface 550 of the ball lens 500. have. The inner surface 550 may be concentric with the ball lens 500. Meanwhile, embodiments of the marker 310 using various kinds of lenses may be considered based on the arrangement relationship between the lens 314 and the image 312.

According to one embodiment, the lens 314 may be implemented by combining two optical media. For example, as shown in FIG. 7, the lens 314 may be implemented by combining two hemispherical lenses 510c and 530c. In detail, the lens 314 may be implemented by combining a hemispherical lens 510c having a radius of R1 and a hemispherical lens 530c having a radius of R2. According to one embodiment, the two hemispherical lenses 510c and 530c may be concentric with each other.

According to another embodiment, as shown in FIG. 8, the lens 314 may be implemented by combining two identical hemispherical lenses 510c and 530c. According to an embodiment, at least one of the two hemispherical lenses 510c and 530c may have a shape that forms a part of the ball lens instead of the lens of the hemispherical shape. In this case, the two lenses to be combined may be located on concentric circles with each other. The focus of light irradiated to the hemispherical lens 510c from the outside may be formed on the inside, the surface, or the outside of the hemispherical lens 530c, and the image 312 may be formed in any one of them. According to one embodiment, image 312 may be formed at any location within hemispherical ball lens 530c.

According to one embodiment, the refractive indices of the two optical media to be combined may be the same or different from each other, and the focal position of the incident light may be determined according to the refractive indices. Thus, by changing the refractive index, the position where the image 312 is to be formed in the marker 310 can be adjusted, and also the recognition rate for the image 312 can be adjusted when the image 312 is imaged.

According to one embodiment, an optical adhesive may be used to couple between the first optical medium and the second optical medium, such as between hemispherical lenses 510c and 530c or lenses that form part of a ball lens. According to another embodiment, by placing another optical medium between the first optical medium and the second optical medium, it is possible to combine between the optical media. 9 illustrates an embodiment in which portions 510d and 530d of a ball lens and another optical medium 550d are combined to form a lens 314. According to an exemplary embodiment, the portions 510d and 530d of the ball lenses have the same radius, that is, the same radius of curvature, and may be located on concentric circles with each other.

According to another embodiment, portions 510d and 530d of the ball lens and another optical medium 550d may be formed integrally. In this case, the lens 314 may be a drum lens. According to one embodiment, the drum lens can be made by machining the ball lens into a cylindrical shape, or by processing the optical medium to have two curved surfaces of the same radius of curvature and to have a thickness twice the radius of curvature.

With respect to the location where the image 312 is formed, if light is incident on a portion 510d of the ball lens, the image 312 may be formed on a portion 530d of the other ball lens. Specifically, if light is incident on the portion 510d of the ball lens, the image 312 is inside, on the surface of, the portion 530d of the ball lens focused by the portion 510d of the ball lens and the optical medium 550d. Or it may be formed on at least one of the outside. At least one of the refractive indexes of the portions 510d and 530d of the ball lens and the optical medium 550d may be different or the same. By adjusting the refractive indices of the optical media, the position at which the image 312 will be formed in the marker 310 can be adjusted. Also, the recognition rate for the image 312 when the image 312 is imaged may be adjusted.

According to one embodiment, the optical medium 550d may further include a convex portion or groove for easily attaching the marker 310 to the object to be tracked.

According to an embodiment, the lens 314 of the marker 310 may be implemented by combining two spherical dome lenses. FIG. 10 illustrates a lens 314 implemented by combining hemispherical dome lenses 510e and 530e among spherical dome lenses according to an embodiment of the present invention. According to an embodiment, the hemispherical dome lens 510e may have a shape having an inner radius of R1 and an outer radius of R2, and the hemispherical dome lens 530e may have a shape having an inner radius of R3 and an outer radius of R4. At this time, as in the embodiment shown in FIG. 11, the inner radii R1 and R3 of the two hemispherical dome lenses may be the same, or as in the embodiment shown in FIG. 12, the outer radii R2 and R4 may be the same. According to another embodiment, as shown in FIG. 13, the two hemispherical dome lenses 510e and 530e may have the same shape. These spherical dome lenses 510e and 530e may be located on concentric circles. According to one embodiment, the two spherical dome lenses are not limited to hemispherical dome lenses but may be any dome lens that forms part of a sphere.

With respect to the location where the image 312 is formed, if light is incident on the hemispherical dome lens 510e, the image 312 may be formed on another hemispherical dome lens 530e. Specifically, if light is incident on the hemispherical dome lens 510e, the image 312 may be formed on at least one of the surface, inside or outside of the hemispherical dome lens 530e to which the hemispherical dome lens 510e is focused. . In this case, the refractive indices of the hemispherical dome lens 510e and the hemispherical dome lens 530e may be different or the same. By adjusting the refractive indices of these optical media, the position where the image 312 is to be formed in the marker 310 can be adjusted, and also the recognition rate for the image 312 when the image 312 is imaged can be adjusted.

According to one embodiment, when two spherical dome lenses are combined to form a lens 314, the space between the two spherical dome lenses may be filled with a material having a refractive index different from at least one of the refractive indices of the two spherical dome lenses. have. For example, the material to be filled may be a gas or an optical resin. According to one embodiment, it may have another optical medium between two spherical dome lenses. FIG. 14 shows a lens 314 with a ball lens 550f between two hemispherical dome lenses 510f and 530f as two spherical dome lenses, as a lens for a marker according to an embodiment of the present invention. Since the ball lens 550f is interposed, the two hemispherical dome lenses 510f and 530f may have the same inner radius. According to one embodiment, the hemispherical dome lenses 510f and 530f and the ball lens 550f may be located on concentric circles.

According to another embodiment of the present invention, the dome lens to be coupled to the ball lens 550f may be any spherical dome lens that is not hemispherical. FIG. 15 shows a lens for a marker according to an embodiment of the present invention, in which two spherical dome lenses 510f and 530f and one ball lens 550f are combined.

With respect to the location at which image 312 is formed, if light is incident on hemispherical dome lens 510f, image 312 may be formed on another hemispherical dome lens 530f. Specifically, if light is incident on the hemispherical dome lens 510f, the image 312 is either of the surface, interior or exterior of the hemispherical dome lens 530f focused by the hemispherical dome lens 510f and the ball lens 550f. It may be formed in at least one. In this case, at least one of the refractive indices of the hemispherical dome lenses 510f and 530f and the ball lens 550f may be different or the same. By adjusting the refractive indices of these optical media, the position where the image 312 is to be formed in the marker 310 can be adjusted, and the recognition rate for the image 312 when the image 312 is imaged can be adjusted.

As another embodiment, FIG. 16 illustrates a combination of one ball lens 510g and one optical medium 530g having a sphere surface 550g concentrically spaced apart from the ball lens 510g. The lens 312 is shown. According to an exemplary embodiment, the spaced space between the ball lens 510g and the optical medium 530g may be filled with a material having a refractive index different from that of at least one of the ball lens 510g or the optical medium 530g. For example, the material to be filled may be a gas or an optical resin. According to another embodiment of the present invention, the surface of the ball lens 510g may be in contact with the spherical surface 550g of the optical medium 530g without a space between the ball lens 510g and the optical medium 530g. have.

With respect to the location where the image 312 is formed, if light is incident on the ball lens 510g, the image 312 may be formed on the optical medium 530g. Specifically, if light is incident on the ball lens 510g, the image 312 may be formed on at least one of the spherical surface 550g, the inside or the outside of the optical medium 530g to which the ball lens 510g is focused. . In this case, the refractive indexes of the ball lens 510g and the optical medium 530g may be different or the same.

According to an embodiment, the optical medium 530g may be filled with a material having a refractive index different from that of the ball lens 510g or the optical medium 530g in a space spaced between the ball lens 510g and the optical medium 530g. The location at which the image 312 is formed can be adjusted. According to another embodiment, by adjusting the refractive index of the ball lens 510g, the position where the image 312 is to be formed in the optical medium 530g may be adjusted. By adjusting the refractive index in this manner, not only the position where the image 312 is to be formed can be adjusted, but also the recognition rate for the image 312 when the image 312 is imaged can be adjusted.

According to an embodiment, the optical medium 530g may include convex portions 570g and 572g for coupling with the ball lens 510g. In addition, the optical medium 530g may include convex portions or grooves that can easily attach to the object to be tracked to the marker 310.

17 shows a lens 312 in which one ball lens 550h and a hemispherical dome lens 530h constituting a concentric circle are combined. The hemispherical dome lens 530h may have an inner radius R1 and an outer radius R2. According to an embodiment, the radius of the ball lens 550h may be the inner radius R1 of the hemispherical dome lens 530h, and the ball lens 550h may contact the surface of the hemispherical dome lens 530h. According to another embodiment of the present invention, the ball lens 550h and the hemispherical dome lens 530h may be spaced apart. In this case, an optical medium having a refractive index different from at least one of the ball lens 550h or the hemispherical dome lens 530h may be included between the spaces.

According to another embodiment, the spherical dome lens to be combined with the ball lens 550h may be any spherical dome lens 530h that is not hemispherical. Ball lens 550h and any spherical dome lens 530h may be located concentrically. FIG. 18 is a lens for a marker according to an embodiment of the present invention, and shows a lens 314 implemented by combining one ball lens 550h and one spherical dome lens 530h that is not hemispherical.

Regarding the location where the image 312 is formed, if light is incident on the ball lens 550h, the image 312 may be formed in the hemispherical dome lens 530h. Specifically, if light is incident on the ball lens 550h, the image 312 may be formed on at least one of the surface, inside or outside of the hemispherical dome lens 530h to which the ball lens 550h is focused. According to one embodiment, the surface of the hemispherical dome lens 530h may be a surface formed by the inner radius R1 or the outer radius R2. The refractive indexes of the ball lens 550h and the hemispherical dome lens 530h may be the same or different. By adjusting the refractive indices of these optical media, the position at which the image 312 is to be formed in the marker 310 can be adjusted, and the recognition rate for the image 312 when the image 312 is imaged can be adjusted.

19 illustrates a case in which the lens 314 is a GRIN (Gradation Refractive Index) ball lens according to an embodiment of the present invention. The GRIN ball lens may refer to a ball lens in which layers of different refractive indices are stacked 510i, 530i, 550i, and 570i. In FIG. 19, it is assumed that the GRIN ball lens is implemented with materials having four refractive indices, but the number of materials may vary depending on the lens. In addition, according to one embodiment, each material forming the GRIN ball lens may have a different color.

Using a GRIN ball lens, the image 312 can be located on either the surface, inside or outside of the GRIN ball lens to which the GRIN ball lens is focused, as when using a ball lens. For example, the surface of the GRIN ball lens may be an outer surface of the surface reflected from the GRIN ball lens after light enters the GRIN ball lens. According to one embodiment, the inside or outside of the GRIN ball lens to which the GRIN ball lens is focused may be concentric with the GRIN ball lens. Since the GRIN ball lens can make a spot size smaller than a general spherical lens, when the image 312 is formed, the resolution of the formed image can be increased.

Next, the image 312 included in the marker 310 will be described. The image 312 may provide information for measuring the position and pose of the marker 310. In one embodiment, the image 312 is a shape including a plurality of feature points, and the shape may be a random shape or a predetermined shape. Specifically, the image 312 in the present invention may be formed so that two or more feature points can be detected from the detected image when the image 312 is detected by the imaging unit 330. This is because information about at least two of the plurality of feature points included in the image 312 is required as the minimum information for measuring the position and the attitude corresponding to the six degrees of freedom of the marker 310.

20 illustrates an image 312 formed on either surface of the ball lens 500 according to an embodiment of the present invention. In one embodiment, the image 312 may have a random shape. The feature points in the present invention may be represented by points on the image 312. In addition, the feature points in the present invention may represent points 610, 612, 614, 616 that represent the characteristic regions, as shown in FIG. 20. According to one embodiment, the feature points 610, 612, 614, 616 may be the field of view of the characteristic region.

The feature points 610, 612, 614, and 616 exemplarily shown in FIG. 20 may be formed of a material having a high reflectance with respect to light irradiated to the image 312 in order to increase discrimination on the image 312. According to another embodiment, the feature points 610, 612, 614, and 616 may be formed of a material having a high transmittance for light emitted to the image 312. As another embodiment, the feature points 610, 612, 614, 616 may be formed of a light emitting material. In this case, the feature points 610, 612, 614, and 616 may be detected at the image forming unit 330 even in a dark environment or a condition in which there is no light source irradiated to the image 312.

According to another embodiment, the image 312 may be formed of two or more materials having different reflectances for light irradiated with the marker 310. In this case, the feature points 610, 612, 614, and 616 in the image 312 may be formed using differences in reflectances of two or more materials. In one embodiment, the feature points 610, 612, 614, 616 are formed of one material having a high reflectance for light irradiated with the marker 310, and the rest of the image 312 has a high transmittance so that light is transmitted. The amount to be formed can be formed with one more material.

In another embodiment, the feature points 610, 612, 614, and 616 are formed of one material having a higher transmittance with respect to the light irradiated with the marker 310 and having a larger amount of light transmitted therethrough, and the image 312. The rest of can be formed of a material with high reflectance. When the feature points are formed by maximizing the difference between the reflectance and the transmittance of the materials used to form the image 312, the identification of the feature points in the image formed in the image forming unit 330 may be increased.

FIG. 21 illustrates a case where an image 312 included in the marker 310 has a predetermined shape according to an embodiment of the present invention. Like the image shown in FIG. 20, the marker 310 of the present invention may have an image having a random shape, and a predetermined shape having a regular pattern or the like may be formed as in the image shown in FIG. 21. In any case, when the image 312 formed on the marker 310 is detected by the imaging unit 330, at least two feature points of the plurality of feature points included in the image 312 may be detected.

When tracking an object having a plurality of objects to be tracked, such as a surgical robot, a plurality of markers may be used since the marker 310 should be attached to each object. In addition, two or more markers may be attached to one object for the purpose of broadening the tracking range or angle. In this case, it is necessary to distinguish between a plurality of markers attached to the object. According to an embodiment, different markers may be formed by differently forming images for each marker. If the markers are manufactured so that the image has a random shape, it may be possible to distinguish between the markers by using random characteristics of the image formed on each marker.

In order to determine the position and posture of the marker 310, when the image 312 included in the marker 310 is formed in the image forming unit 330, information about a plurality of feature points included in the image formed is needed. According to an embodiment, the information about the plurality of feature points may be coordinates of the feature points. Therefore, the surface on which the image 312 is formed may be shaped to minimize distortion of coordinates of feature points detected from the image formed when the image 312 is formed in the image forming unit 330. For example, the shape of the surface on which the image 312 is formed may be a spherical surface or a curved surface close to the spherical surface.

According to an embodiment, when the marker 310 is used as the active marker, the light source 350 may be included in the marker 310. Although the light source 350 is included in the marker 310, its role is not significantly different from the case where the light source 350 is located outside the marker 310. Accordingly, the above descriptions regarding the marker 310 may be equally applied even when the light source 350 is inside the marker 310.

FIG. 22 illustrates that the light emitted from the light source 350 passes through the image 312 included in the marker 310, so that the information of the plurality of feature points included in the image 312 may be changed into a lens. 314) is transmitted to the outside. The plurality of feature points included in the image 312 may be formed of a material that transmits or reflects light emitted from the light source 350. This depends on whether the image of the feature point is bright or dark when the image 312 is imaged on the image forming unit 330 side. On the contrary, FIG. 23 illustrates that the light irradiated from the light source 350 is reflected from the image 312 according to an embodiment of the present invention, so that information of a plurality of feature points included in the image 312 may be applied to the lens 314. It is shown to be transmitted to outside. In this case, the plurality of feature points included in the image 312 may be formed of a material that transmits or reflects light emitted from the light source 350.

There is no restriction | limiting in particular about the wavelength of the light of the light source 350 used in FIGS. 22 and 23 or the light irradiated to the marker 310 from the outside of the marker 310. Depending on the purpose for which the optical tracking system 300 is used, the wavelength of the light can be a wavelength in the visible region where the user can recognize a particular color or a wavelength in the infrared band where the user can not recognize the presence of light. It may be. For example, if the optical tracking system 300 is used for medical purposes, the wavelength of the infrared band may be used so as not to obstruct the view of the doctor or nurse in the surgical environment.

According to one embodiment, the surface of the lens may be coated such that only light of a specific wavelength range is transmitted to the lens 314 used for the marker 310 to reduce the influence on other external light. For example, if light is irradiated to the marker 310 using the light source 350 in the infrared band, the lens 314 may be coated on the surface of the lens 314 such that only light of the wavelength in the infrared band is transmitted. . According to one embodiment, the marker 310 may further include an additional optical filter such that only light of a specific wavelength can convey image 312 information included in the marker 310.

< Image (330)>

According to an embodiment, in order to detect information about a plurality of feature points included in the image 312, the image 312 included in the marker 310 is included in the image forming unit 330 after passing through the lens 314. It may be formed in the imaging device 334. The imaging unit 330 may be implemented with a lens 332 and an imaging device 334. The image 312 of the marker 310 may be imaged in the imaging device 334 via the lens 332. The imaging unit 330 may be implemented using a camera, a camcorder, or the like as an image forming apparatus. The imaging device 334 is a device for converting image information transmitted through light into an electrical signal, and may be implemented using a CMOS image sensor or a CCD.

24 illustrates a positional relationship between the lens 332 and the imaging device 334 in the imaging unit 330 according to an exemplary embodiment. In general, the imaging unit 330 may obtain a clear image by detecting an image formed at a distance apart from the lens 332 by a focal length. Therefore, the imaging unit 330 may adjust or focus the distance between the imaging device 334 and the lens 332 such that the imaging device 334 is positioned at the focal length of the lens 332 when the image is detected. . However, if the imaging unit 330 repeats the focus adjustment every time the marker 310 moves to detect the image 312 included in the marker 310, the detection speed of the image 312 may be slowed down. In addition, in order to accurately detect the feature points of the image 312 included in the marker of very small size and to increase the recognition rate, it is necessary to form an image 312 in an enlarged form in the image forming unit 330.

In order to solve this problem, in the imaging unit 330 according to the exemplary embodiment, the imaging device 334 may be fixed to the focal length of the lens 332. If the imaging device 334 is positioned at the infinite focus which is a focal point formed by the lens 314 of the marker 310 and the lens 332 of the imaging unit 330, the change in distance according to the movement of the marker 310 may occur. Regardless, the image 312 may be enlarged and imaged in the imaging device 334. As shown in FIG. 24, the -f _c point, which is the focal length of the lens 332 in which the imaging device 334 is located, may be a position at which infinite focus is provided.

According to one embodiment, to form an optical system with infinite focus, the image 312 in the marker 310 may be located on the surface where the focus of the lens 314 is formed. Through this optical system, the imaging unit 330 may image the image 312 included in the marker 310 without adjusting the focus even if the marker 310 moves, and also may include an image included in the small marker 310. 312) can be formed in an enlarged form. Therefore, the imaging unit 330 may accurately detect the image 312 included in the small size marker 310 and the plurality of feature points included therein at high speed. In addition, since the burden on the size of the image 312 included in the marker 310 can be reduced, it is possible to miniaturize the marker 310 attached to the object.

<Processor 370>

The processor 370 of the optical tracking system 300 according to an embodiment of the present invention may obtain information related to the image 312 included in the marker 310 and the image 312 by forming an image on the image forming unit 330. The posture of the marker 310 may be determined based on the information related to the. According to an embodiment, the information related to the image 312 may be coordinates (first coordinates) on the marker 310 of the plurality of feature points included in the image 312. The information related to the imaged image may be coordinates (fourth coordinates) on the imaged image corresponding to the plurality of feature points included in the image 312. The processor 370 may determine the posture of the marker 310 based on a relation between the first coordinate and the fourth coordinate. To this end, the first coordinate may be obtained in advance through a calibration process of the marker 310, and then stored in a memory accessible by the processor 370. Hereinafter, how the processor 370 determines the posture of the marker 310 based on the relational expression of the first coordinate and the fourth coordinate will be described. A detailed description of the algorithm for determining the pose of the marker 310 is disclosed in Korean Patent Application No. 10-2014-0065168, which is incorporated by reference.

FIG. 25 is a diagram for describing a process of detecting a posture of the marker 310 based on a relational expression between first coordinates and fourth coordinates, according to an exemplary embodiment. Point A represents the first coordinate, point B is the coordinate (second coordinate) for lens 314 of the first coordinate, and point C is the coordinate (third coordinate) for lens 332 of the second coordinate. , Point D represents the fourth coordinate. The point O is a reference point of the first to fourth coordinates and is a point indicating the center of the lens 332. According to an embodiment, the transformation matrix for converting the first coordinate to the second coordinate is defined as C, the transformation matrix for converting the third coordinate to the fourth coordinate is defined as A, and the matrix representing the attitude of the marker is R If defined as, the matrix R can be calculated through the following equations (1) to (3).

<Equation 1>

For flat images:

,

), ..., (

,

)은 제1 좌표, (

,

), ..., (

,

)은 제4 좌표)((

,

), ..., (

,

) Is the first coordinate, (

,

), ..., (

,

) Is the fourth coordinate)

If an image is formed on the ball lens:

은 제1좌표,

은 볼렌즈의 반지름, (

,

), ..., (

,

)은 제4 좌표)(

Is the first coordinate,

Is the radius of the ball lens, (

,

), ..., (

,

) Is the fourth coordinate)

<Equation 2>

,

)는 이미지(312)의 중심에 대응하는 결상부(330)에서 검출된 이미지 상의 좌표,

는 결상소자(334)가 렌즈(332)로부터 이격된 거리,

는 결상된 이미지의 픽셀의 폭,

는 결상된 이미지의 픽셀의 높이)((

,

) Is the coordinate on the image detected by the imaging unit 330 corresponding to the center of the image 312,

Is the distance that the imaging element 334 is separated from the lens 332,

Is the width of the pixels in the image,

Is the height of the pixels in the image

<Equation 3>

For flat images:

,

)는 마커(310)의 이미지(312)의 중심의 좌표,

는 렌즈(314)의 초점거리)((

,

) Is the coordinate of the center of the image 312 of the marker 310,

Is the focal length of the lens 314)

If an image is formed on the ball lens:

In order for the processor 370 to calculate the roll, pitch, and yaw of the marker 310 using Equations 1 to 3, at least two of the plurality of feature points of the image 312 may be formed in the image forming unit 330. have. As the number of feature points detected on the formed image increases, an error between the calculated posture of the marker 310 and the actual posture of the marker 310 may be reduced.

According to another embodiment of the present invention, the matrix R can also be calculated through the following equation (4).

<Equation 4>

For flat images:

,

), ..., (

,

)은 제1 좌표, (

,

), ..., (

,

)은 제4 좌표, (

,

)는 이미지(312)의 중심에 대응하는 결상부(330)에서 검출된 이미지 상의 좌표,

는 결상소자(334)가 렌즈(332)로부터 이격된 거리,

는 결상된 이미지의 픽셀의 폭,

는 결상된 이미지의 픽셀의 높이)((

,

), ..., (

,

) Is the first coordinate, (

,

), ..., (

,

) Is the fourth coordinate, (

,

) Is the coordinate on the image detected by the imaging unit 330 corresponding to the center of the image 312,

Is the distance that the imaging element 334 is separated from the lens 332,

Is the width of the pixels in the image,

Is the height of the pixels in the image

If an image is formed on the ball lens:

은 제1좌표,

은 볼렌즈의 반지름, (

,

), ..., (

,

)은 제4 좌표, (

,

)는 이미지(312)의 중심에 대응하는 결상부(330)에서 검출된 이미지 상의 좌표,

는 결상소자(334)가 렌즈(332)로부터 이격된 거리,

는 결상된 이미지의 픽셀의 폭,

는 결상된 이미지의 픽셀의 높이)(

Is the first coordinate,

Is the radius of the ball lens, (

,

), ..., (

,

) Is the fourth coordinate, (

,

) Is the coordinate on the image detected by the imaging unit 330 corresponding to the center of the image 312,

Is the distance that the imaging element 334 is separated from the lens 332,

Is the width of the pixels in the image,

Is the height of the pixels in the image

As described above, the processor 370 may use a relationship between a feature point on the image 312 and the feature point formed in the image forming unit 330 corresponding to the feature point to determine the posture of the marker 310. In the related art, the pose of the marker 310 may be determined by detecting a change in the geometric relationship between the markers in a structure to which three or more markers are attached. In contrast, according to the present invention, the pose of the marker may be determined using only one marker by determining the pose through an equation based on the information of the feature points of the image 312. Therefore, in the optical tracking system 300, even if only one marker is attached to the object, all posture calculations of the object can be performed, and thus the number of markers to be used can be reduced even if there are many tracking objects. In addition, when a large number of markers are attached, such as a surgical tool used directly by a doctor, when the user may be uncomfortable, the marker 310 of the present invention may be conveniently used.

Meanwhile, in the case of the optical tracking system 300, the processor 370 may not calculate the posture of the marker 310 based on the same feature point included in the image 312 while tracking the marker 310. FIG. 26 exemplarily illustrates changes in feature points when the optical tracking system 300 according to an exemplary embodiment detects an image 312 included in the marker 310 according to the movement of the marker 310. Indicates. As shown in FIG. 26, when the image 312 of the marker 310 is detected while the marker 310 is moving, four feature points 1110, 1112, 1114, and 1116 are detected at time t ₁ , and t ₂ hours, and the detection of three feature points (1130, 1132, 1134), t ₃ is the time can be detected five feature points (1150, 1152, 1154, 1156, 1158). As such, while the marker 310 moves, the information of the feature point obtained for each image detection time point in the image forming unit 330 may vary.

The change in the information of the feature point obtained for each detection time may mean that the marker 310 is moving, and that only the information of the same feature point is detected may mean that the marker 310 is not moving. However, some of the feature points detected at each view point may be the same, and only two or more of the feature points changed in this way may be determined to determine the posture of the marker 310 at each view point. As such, the optical tracking system 300 may not calculate the posture of the marker 310 based only on the same feature points detected while the marker 310 is moving. Accordingly, since the optical tracking system 300 does not need to keep track of the same object change as in the conventional optical tracking system, the marker 310 can be stably tracked even when a problem such as a part of the object is blocked. have.

According to an embodiment, the optical tracking system 300 may determine the position of the marker 310 using triangulation. In this case, in the optical tracking system 300, the imaging unit 330 may include two or more imaging devices or two or more imaging units 330. This is because in order to use the trigonometry, information related to the image of the marker 310 formed at different positions or the image 312 included in the marker 310 is required.

27 is a block diagram of an optical tracking system 300 in which the imaging unit 330 includes two imaging devices 334 and 1234 according to an embodiment of the present invention. According to an embodiment, the imaging elements 334 and 1234 may image the marker 310 or the image 312 included in the marker 310 at different positions. This information is passed to the processor 370, which can determine the location of the marker 310 through trigonometry based on information obtained from the transferred image.

According to another embodiment, the processor 370 may coordinate with the two imaging elements and the coordinates on the marker 310 of the plurality of feature points included in the image 312, similar to the method of determining the pose of the marker 310 described above. The position of the marker 310 may be determined based on a relationship between coordinates of the plurality of feature points detected from the images formed at 334 and 1234. A detailed description thereof is disclosed in Korean Patent Application No. 10-2014-0065178, which is incorporated by reference.

According to an embodiment, the pose of the marker 310 may also be determined based on the images detected by the two imaging devices 334 and 1234. The processor 370 detects coordinates of two or more arbitrary feature points corresponding to each other from the images formed by the two imaging devices 334 and 1234, and is included in the detected coordinates and the previously stored image 312. The attitude of the marker 310 may be determined through a relationship with the coordinates of the plurality of feature points. When the posture of the marker 310 is determined based on the coordinates of the feature points detected using the plurality of imaging elements, an error between the determined posture and the posture of the actual marker 310 may be reduced. A detailed description thereof is disclosed in Korean Patent Application No. 10-2014-0065178, which is incorporated by reference.

< Of marker 310  How to determine position and posture>

28 is a flowchart illustrating a method of determining the pose of the marker 310 in the optical tracking system 300 according to an embodiment of the present invention. Hereinafter, a method of determining the pose of the marker 310 will be described in more detail with reference to the drawings. First, in step S1310. The first image having a random shape included in the marker may be enlarged and transmitted through the first lens. For example, referring to FIG. 3, an image 312 having a random shape included in the marker 310 may be enlarged and transmitted through the lens 314. The image 312 may be formed on the surface where the lens 314 is focused in the marker 310. For example, the image 312 may be formed inside or on the surface of the lens 314, or may be formed at a location remote from the lens 314, depending on where the lens 314 is to be in focus. . According to one embodiment, to deliver the image 312 to the outside through the lens 314, the optical tracking system 300 may further include a light source 350 for irradiating light to the image 312. The light source 350 may be located inside or outside the marker 310.

According to an embodiment, the image 312 may include a plurality of feature points. The plurality of feature points may be formed of a material that reflects or transmits light emitted to the image 312. According to another embodiment, the image 312 may be formed of two or more materials having different reflectances for light irradiated onto the image 312. The plurality of feature points included in the image 312 may be formed using differences in reflectances of two or more different materials. The plurality of feature points may be formed of a material having a high reflectance to reflect light irradiated onto the image 312, or may be formed of a material having a low reflectance to transmit light emitted to the image 312. According to one embodiment, the image 312 may have a predetermined shape instead of a random shape.

When the image 312 is transferred to the outside of the marker 310 through step S1310, in step S1330, the first image may pass through the second lens and may be imaged as a second image at a position spaced apart from the focus of the second lens. . For example, referring to FIG. 3, the image 312 may be imaged by the imaging device 334 at a position spaced apart from the focal point of the lens 332 after passing through the lens 332. According to an embodiment, the imaging element 334 may be positioned at an infinite focal point formed by the lens 314 and the lens 332 to form an image 312 included in the marker 310. Since the imaging element 334 is positioned at the infinite focus formed by the lens 314 and the lens 332, the imaging unit 330 focuses whenever imaging the image 312 included in the moving marker 310. no need. Thus, regardless of the movement of the marker 310, the information contained in the image 312 can be obtained accurately and quickly. In addition, when the optical system is used, the imaging unit 330 may enlarge the image 312 to form an image. Therefore, even when the small marker 310 is used, it is easy to obtain information on the feature point. The image information formed by the imaging unit 330 may then be transferred to the processor 370.

When the first image is formed as a second image in the image forming unit 330 through step S1330, in step S1350, the processor 370 may be configured based on the first information related to the first image and the second information related to the second image. Determine your posture. For example, referring to FIG. 3, the processor 370 may determine a posture based on first information related to the image 312 and second information related to the image formed by the image forming unit 330.

29 illustrates a specific method of determining the pose of the marker 310 based on the information related to the image 312 and the information related to the image formed by the image 312. First, in step S1351, the processor 370 may detect coordinates of two or more feature points from the second image as second information. Subsequently, in step S1352, the processor 370 may detect, as the first information, coordinates of the feature points corresponding to two or more feature points detected from the second image among the plurality of feature points included in the first image stored in advance. have. For example, the processor 370 may detect, as information related to the image 312, coordinates of the feature points corresponding to the coordinates of the feature points detected from the image formed among the coordinates of the plurality of feature points included in the image 312. have.

After the coordinates of two or more arbitrary feature points respectively corresponding to each other are detected from the image 312 and the image formed by the image forming unit 330, in operation S1353, the processor 370 determines the first information associated with the first image and The posture may be determined based on the second information related to the second image. For example, the processor 370 may determine the marker 310 based on a relationship between the coordinates of two or more arbitrary feature points corresponding to each other extracted from the image 312 and the image formed by the image forming unit 330. Determine your posture.

30 is a flowchart illustrating a method of determining the position of the marker 310 in the optical tracking system 300 according to an embodiment of the present invention. First, in step S1510, a first image of a random shape included in a marker may be imaged at two different positions. For example, referring to FIG. 27, the imager elements 334 and 1234 may image the images 312 at different positions. Thereafter, the image information imaged at each different position is transferred to the processor 370. In operation S1530, the processor 370 may determine the position of the object to which the marker is attached through trigonometry based on the two imaged images. According to an embodiment, trigonometry based on information about an image in which an image of the marker 310 itself is formed may be used instead of the image 312.

According to one embodiment, at least one marker 310 may be attached to the object in an exchangeable form. In order to widen the tracking range or angle of the marker 310, two or three or more markers may be attached to the object. When there are many objects to be tracked, such as a surgical robot, a plurality of markers may be used. In the case of an optical tracking system using a plurality of markers, it is necessary to distinguish between the markers. According to an embodiment, the markers may be distinguished from each other by having a unique random shape image for each marker.

< On the marker 310  How to Form Image 312>

In the optical tracking system 300 according to an embodiment of the present invention, the image 312 may include a plurality of feature points, and the processor 370 may be configured to form an image 312 in the image forming unit 330. Any two or more feature points can be detected. Hereinafter, a method of forming such an image 312 on the marker 310 is described.

According to one embodiment, as shown in FIG. 31, a randomly shaped image 312 may be formed on the marker 310 using a spraying technique. The marker 310 may be the ball lens 1610 itself, in which case an image 312 may be formed on the surface of the ball lens 1610.

According to one embodiment, by spraying a spray 1630 on the surface of the ball lens 1610 material having a high reflectance for the light irradiated to the ball lens 1610, the image of the random shape on the surface of the ball lens 1610 312 may be formed. This is based on the granularity of the material, and the image 312 having a random shape may be formed by simply spraying the spray 1630 with a material having a high reflectance on the surface of the ball lens 1610. In this case, regions 1612, 1614, 1616, and 1618 on which particles with high reflectance are deposited may be characteristic points.

According to another embodiment, the ball lens 1610 is sprayed with a spray 1630 spraying on the surface of the ball lens 1610 material having a low reflectance for light incident to the ball lens 1610 and having a high property of transmitting the light. A random shape image 312 may be formed on the surface. In this case, regions 1612, 1614, 1616, and 1618 on which a material having a low reflectance (ie, a material having a high transmittance) are deposited may be characteristic points.

According to another embodiment, a material having a high reflectance for light incident on the ball lens 1610 is sprayed on the surface of the ball lens 1610 through a spray 1630, and after being dried and deposited, the material having a low reflectance By spraying through spray 1630, an image 312 of random shape may be formed. Also, as an image forming method using the granularity of the material, the features 1616, 1614, 1616, and 1618 may be regions in which a material having high reflectance is deposited or regions in which a material having low reflectance is deposited. The low reflectance material may be sprayed first with a spray 1630 and then the high reflectance material may be sprayed with a spray 1630. When the materials having different reflectances are used at the same time, the recognition rate of the feature points included in the image formed by the imaging unit 330 may be increased by using the difference in reflectance between these materials.

According to one embodiment, the size of the particles of material to be sprayed with the spray 1630 is small enough that feature points can be formed when sprayed on the marker 310, and proportionally according to the size of the marker 310. Can change. For example, larger markers 310 may be used for larger particles, and smaller markers 310 may be used for smaller particles. According to an embodiment, the lens 314 may be any lens surface capable of magnifying and transferring the image 312 formed on the marker 310 to the outside. According to an embodiment, a ball lens may be used as shown in the example of FIG. 31, and in particular, a marker 310 having an image 312 formed on a surface of the lens 314 may be implemented using a ball lens having a refractive index of 2. have.

32 is a flowchart illustrating a method of forming an image 312 on a marker 310 according to an embodiment of the present invention. First, in step S1710, a marker including a lens may be prepared. For example, referring to FIG. 3, a marker 310 including a lens 314 may be prepared. According to an embodiment, when the image 312 is formed in the lens 314, the lens 314 itself may be the marker 310. When the image 312 is formed to be spaced apart from the lens 314, the marker 310 may further include a surface on which the image 312 is to be formed in addition to the lens 314.

Thereafter, in step S1730, an image may be formed on the prepared marker. In more detail with reference to step S1730, a portion of the image 312 may be formed of a material having a first reflectance in step S1750. In this case, a region in which a part of the image 312 is formed may be a feature point, and conversely, a region in which a material having a first reflectance is not coated may be a feature point. According to an embodiment, the first reflectance may be a material that mainly reflects the irradiated light due to its high reflectance with respect to the light irradiated onto the marker 310, or may be a material that mainly transmits the irradiated light with a low reflectance. .

In operation S1750, an image 312 having a random shape having a plurality of feature points may be formed in the marker 310. However, in operation S1770, the image may be formed as a material having a second reflectance different from the first reflectance in the marker 310. The remainder of 312 may be further formed. According to one embodiment, if the first reflectance is a reflectance for the purpose of reflecting light, the second reflectance may be a low reflectance for the purpose of transmitting light. Conversely, if the first reflectance is a reflectance for the purpose of transmitting light, the second reflectance may be a high reflectance for the purpose of reflecting light. By forming the image 312 on the marker 310 by using two materials having a large difference in reflectance through steps S1750 and S1770, the recognition rate of the feature points included in the formed image may be further increased.

According to an embodiment, the surface on which the image 312 is formed on the marker 310 may be a surface on which the lens 314 is focused.

According to one embodiment, after the image 312 is formed of a material having different reflectances in steps S1750 and / or S1770, the drying and / or deposition step may be further subjected to the material and formation method used. Hereinafter, other embodiments of forming the image 312 on the marker 310 based on the manufacturing method described with reference to FIG. 32 will be described.

33 illustrates a method of forming an image 312 on the marker 310. The image 312 is formed on the marker 310 by a printing technique. A material having a first reflectance or a material having a second reflectance may be printed on the surface of the marker 310 such as a ball lens using an inkjet printer or the like. If the printing method is used as described above, an image having a random shape or a predetermined shape may be freely formed on the surface of the marker 310.

34 illustrates a method of forming an image 312 on the marker 310. The image 312 is formed on the marker 310 by a technique of injecting a substance. After filling the syringe or the like with a material having a first reflectance or a material having a second reflectance, the materials 312 are displayed on the surface of the marker 310 by scanning the materials through the needle 1910 onto the surface of the marker 310. ) May be formed. If such a scanning method is used, a precise image representing a random shape or a predetermined shape may be formed on the surface of the marker 310. In addition, the scanning method may be a method suitable for forming an image on the surface of the marker 310 when the size of the marker 310 is small.

FIG. 35 illustrates a method of forming the image 312 on the marker 310. The image 312 is formed on the marker 310 by a stencil technique. Stencil frame 2010 having a random shape or a predetermined shape is prepared in advance so that it can be attached to the outline of the marker 310, and then sprayed on the surface of the marker 310 to spray 2030 or the like thereon. By applying a material having a first reflectance or a material having a second reflectance, a desired image 312 can be formed on the surface of the marker 310. The stencil technique may be in a manner suitable for forming the same image 312 continuously on the surface of the marker 310.

FIG. 36 illustrates a method of forming the image 312 on the marker 310. The image 312 is formed on the marker 310 by a stamping technique. By embedding a material having a first reflectance or a material having a second reflectance on the flexible stamp 2110 and stamping the material on the surface of the marker 310, a random shape or a predetermined shape is formed on the surface of the three-dimensional marker 310. Image 312 may be formed. If the image 312 is formed on the surface of the marker 310 in such a stamping manner, the image 312 may be formed on the surface of the marker 310 in a large area at one time.

FIG. 37 illustrates a method of forming an image 312 on the marker 310, wherein the marker 310 itself including the image 312 and the lens 314 is integrally formed using the 3D printer 2210. Indicates. If the 3D printer 2210 is used, the shape of the marker 310, the shape of the image 312, and the like may be freely expressed. In addition, a material having a first reflectance or a material having a second reflectance may be easily formed on the marker 310. In addition, the image 312 may be formed inside the marker 310 or the lens 314 without any special assembly process.

38 to 40 illustrate a method of forming an image 312 on the marker 310, and show an image of forming an image 312 on the marker 310 by lithography. As shown in FIG. 38, a material 2314 having a high reflectance or high transmittance may be first deposited on a surface 2312 on which an image 312 is to be formed. Then, as shown in FIG. 39, material 2314 may be removed at a location 2316 where the image 312 will not be formed. Only the process up to FIG. 39 may form an image 312 for tracking on the marker 310.

According to an embodiment, as shown in FIG. 40, a material having a reflectance or transmittance different from that of the material 2314 at a location 2316 where the image 312 is not formed to increase the recognition rate of the image 312 ( 2318 may be filled. Using this lithographic technique, an image 312 having two or more feature points using the difference in reflectance or transmittance of the material 2314 and the material 2318 may be formed. In addition, with lithographic techniques, a very sophisticated image 312 can be formed on the surface of the marker 310.

While the invention has been described and illustrated by way of preferred embodiments, it will be apparent to those skilled in the art that various changes and modifications can be made therein without departing from the spirit and scope of the appended claims.

Claims (27)

Marker for position and posture measurement, A randomly shaped image formed such that a plurality of feature points can be arbitrarily selected; And Lens for enlarging and transmitting the image Marker for position and pose measurement comprising a. The method of claim 1, And the image is located on a surface on which the focal point of the lens is formed. The method according to claim 1 or 2, Wherein the lens is coated such that only light in a specific wavelength range can be transmitted. The method according to claim 1 or 2, And the plurality of feature points in the image are formed of a luminescent material. The method according to claim 1 or 2, The image is formed of two or more materials having different reflectances for light incident on the marker, And the plurality of feature points in the image are formed using a difference in reflectance of the two or more materials. The method according to claim 1 or 2, The image is formed of two or more materials having different transmittances for light incident to the marker, And the plurality of feature points in the image are formed using a difference in transmittance of the two or more materials. The method according to claim 1 or 2, Wherein a plurality of markers are used, wherein the random shape is unique for each marker. The method of claim 2, And the surface on which the focal point of the lens is formed is an outer surface of the surface reflected from the lens after light is incident on the lens. The method of claim 2, The surface on which the focal point of the lens is formed is spaced apart from the lens, And the image is spaced apart by a material having a refractive index different from that of the lens. The method of claim 9, Wherein the image is formed on an outer portion of a surface where light is incident upon the lens and is reflected after passing through the material having the different refractive index. The method of claim 2, A marker for measuring position and attitude, wherein the surface on which the focal point of the lens is formed is formed inside the lens. The method according to claim 1 or 2, A marker for measuring position and attitude, including a light source that is brighter than the outside inside the marker to illuminate the image. The method according to claim 1 or 2, Wherein said image is a predetermined shape instead of a random shape. The method according to claim 1 or 2, The image is a marker for measuring position and posture, characterized in that formed on the marker by printing (printing) an image material. The method according to claim 1 or 2, The image is a marker for measuring position and posture, characterized in that formed by lithography (lithography) using an image material. As an optical tracking system, A marker including a first image having a random shape formed to allow a plurality of feature points to be arbitrarily selected and a first lens to enlarge and transmit the first image; And An imaging unit including at least one imaging device for forming the first image as a second image at a focal length of the second lens and the second lens. Including, And optionally selecting a plurality of feature points from the first image to track an object to which the marker is attached. The method of claim 16, And the first image is located on a plane on which the focal point of the first lens is formed. The method according to claim 16 or 17, Determine a posture based on first information related to the first image and second information related to the second image, The first information is a coordinate of two or more feature points among the plurality of feature points included in the first image, And the second information is coordinates corresponding to the two or more feature points, detected in the second image. The method of claim 18, The first image is formed of two or more materials having different reflectances to light. And the plurality of feature points included in the first image are formed using a difference in reflectance of the two or more materials. The method of claim 18, The first image is formed of two or more materials having different transmittances to light. And the plurality of feature points included in the first image are formed using a difference in transmittance of the two or more materials. The method of claim 16, And when the marker is attached to a plurality of objects, the random shape of the first image is unique to the marker. The method of claim 16, And the first image is a predetermined shape instead of the random shape. As an optical tracking method, Imaging the first image having a random shape included in the marker enlarged and transmitted through the first lens into a second image at a focal length of the second lens; And Determining a posture of the marker based on first information related to the first image and second information related to the second image. Including, The random shape first image is formed such that a plurality of feature points can be arbitrarily selected, And selecting a plurality of arbitrary feature points from the first image to track an object to which the marker is attached. The method of claim 23, wherein Determining the pose of the marker based on the first information related to the first image and the second information related to the second image may include Detecting coordinates of at least two feature points from the second image as the second information; Detecting coordinates of a feature point corresponding to two or more feature points detected from the second image as the first information among the plurality of feature points included in the first image stored in advance; And Determining the posture from a relational expression between the coordinates as the first information and the coordinates as the second information Including, optical tracking method. A method of forming an image on a marker for an optical tracking system, Preparing a marker comprising a lens; And Forming an image on the marker Including, Forming an image on the marker Forming a portion of the image with a material having a first reflectance; And Forming a rest of the image on the marker with a material having a second reflectance different from the first reflectance Including; The image is formed in a random shape such that a plurality of feature points can be arbitrarily selected from the difference between the first reflectance and the second reflectance, so that the plurality of feature points can be arbitrarily detected by the optical tracking system. How to form an image on a marker for dragon. The method of claim 25, And forming an image on the marker is forming the image on a surface on which the focal point of the lens is formed. The method of claim 25, And a portion formed of the material having the first reflectance or the second reflectance constitutes the feature point.

Images