FR3161098A1 - DEVICE AND METHOD FOR CONTACTLESS RECALIBRATION BETWEEN A ROBOT, A PATIENT AND MEDICAL IMAGING - Google Patents

Abstract

TITLE OF THE INVENTION: DEVICE AND METHOD FOR CONTACTLESS REGISTERING BETWEEN A ROBOT, A PATIENT, AND A MEDICAL IMAGING SYSTEM The device (10) for contactless registration between a robot (17), a patient (12), and a medical imaging system comprises: - a means (27) for projecting a laser line (21) onto an area of interest (22) of the patient's body, - a means (25) for capturing geometric coordinates of points on the laser line to form a point cloud of the area of interest, and - a means (13) for matching a point cloud of the medical imaging system and the point cloud of the area of interest. Figure for abstract: Figure 1

Description

CONTACTLESS DEVICE AND METHOD FOR REGISTERING BETWEEN A ROBOT, A PATIENT AND MEDICAL IMAGING Technical field of the invention

The present invention relates to a device and a method for contactless registration between a robot, a patient and a medical image. It applies, in particular, to the field of robot-assisted surgery and more specifically to the contactless and non-invasive registration between a robotic arm (active or passive), a patient to be operated on and at least one medical image (from a magnetic resonance imaging, acronym MRI, or a CT scan, for example) of a part of the patient's body to be operated on.

State of the art

The approaches described in this section are approaches that could be pursued, but not necessarily approaches that have been previously conceived or pursued. Therefore, unless otherwise stated, it should not be assumed that any of the approaches described in this section constitutes prior art simply because of its inclusion in this section.

During surgery using a robotic aiming system, geometric registration is necessary between the robotic aiming system, the surgical site on the patient's body, and their medical imaging. Registration establishes a relationship between the different geometric coordinate systems, enabling them to interact. Various registration techniques exist today. There are "invasive" and "non-invasive" techniques for human anatomy. One non-invasive technique is described in document FR 2 963 693. It uses a laser pointer to scan the anatomical surface by emitting a series of light points onto the area of the patient's body to be operated on and then mapping their three-dimensional geometric coordinates within the robot's reference frame. This type of technique has the disadvantage of being time-consuming, due to the time it takes the laser pointer to scan the entire area of interest (between 15 and 25 minutes per re-alignment attempt). Furthermore, the accuracy obtained is highly dependent on the precision of the operator's manipulations.

The general concept of the invention consists of a non-contact registration system preferably employing a robotic arm carrying a registration unit that forms, on the patient's body area of interest, a point from a laser pointer and a line whose width is adjustable within this area of interest. The robotic arm is preferably operated by the operator in a cooperative mode, under planar/axial constraint, to scan, in a single linear movement, the area of interest to be registered with the surface extracted from the medical imaging of said anatomy.

The cooperative mode is implemented via at least one force sensor placed at the end of the robotic arm, or directly in the joints of the arm, if the robotic arm is a so-called "active" arm.

Brief description of the figures

Other advantages, purposes and special features of the invention will become apparent from the following non-limiting description of at least one particular embodiment of the device and method of the present invention, with reference to the accompanying drawings, in which:

FIG. 1 represents, schematically and in side view, a first particular embodiment of the device that is the subject of the invention, during a positioning step of the process that is the subject of the invention,

FIG. 2 represents, schematically and in side view, the device illustrated in FIG. 1 , during a scanning step of the process that is the subject of the invention,

FIG. 3 is an example of a point cloud resulting from a three-dimensional laser scan obtained through the implementation of a device that is the subject of the invention,

FIG. 4 is an example of a point cloud resulting from a surface extraction of a three-dimensional medical image,

FIG. 5 represents, in the form of a flowchart, the steps of a particular embodiment of the process that is the subject of the invention, and

FIG. 6 represents, schematically and in side view, a second particular embodiment of the device which is the subject of the invention, during a scanning step of the process which is the subject of the invention.

The present description is given by way of non-limiting attribution, each feature of an embodiment being able to be advantageously combined with any other feature of any other embodiment.

It should be noted from the outset that the figures are not to scale.

As can be understood from this description, various inventive concepts can be implemented by one or more of the methods or devices described below, several examples of which are provided herein. The actions or steps performed in implementing the method or device can be ordered in any appropriate manner. Consequently, it is possible to construct embodiments in which the actions or steps are performed in a different order than illustrated, which may include performing certain acts simultaneously, even if they are presented as sequential acts in the illustrated embodiments.

The indefinite articles "un" and "une", as used in the description, should be understood as meaning "at least one", unless clearly stated otherwise.

The expression "and/or", as used in this document, is to be understood as meaning "either or both" of the elements thus joined, that is, elements that are present conjunctively in some cases and disjunctively in others. Multiple elements listed with "and/or" are to be interpreted in the same way, that is, "one or more" of the elements thus joined. Other elements may possibly be present, other than those specifically identified by the "and/or" clause, whether or not they are related to those specifically identified elements. Thus, by way of non-limiting example, a reference to "A and/or B", when used in conjunction with an open language such as "comprising", may refer, in one embodiment, to A only (possibly including elements other than B); in another embodiment, to B only (possibly including elements other than A); in yet another embodiment, at A and B (possibly including other elements); etc.

As used here in the description, "or" should be understood inclusively.

As used in this description, the expression "at least one," when referring to a list of one or more items, should be understood as meaning at least one item chosen from one or more items in the list of items, but not necessarily including at least one of each item specifically listed in the list of items and not excluding any combination of items in the list of items. This definition also allows for the optional presence of items other than those specifically identified in the list of items to which the expression "at least one" refers, whether or not they are related to those specifically identified items. Thus, by way of non-limiting example, "at least one of A and B" (or, equivalently, "at least one of A or B", or, equivalently, "at least one of A and/or B") may refer, in one embodiment, to at least one, possibly including more than one, A, without B present (and possibly including elements other than B); in another embodiment, to at least one, possibly including more than one, B, without A present (and possibly including elements other than A); in yet another embodiment, to at least one, possibly including more than one, A, and at least one, possibly including more than one, B (and possibly including other elements); etc.

In the description below, all transitive expressions such as "comprising", "including", "carrying", "having", "containing", "implying", "holding", "composed of", and others, should be understood as open, that is, as meaning including but not limited to. Only the transitive expressions "consisting of" and "consisting essentially of" should be understood as closed or semi-closed transitive expressions, respectively.

Throughout this description, the terms "upper" and "top" refer to what is at the top when the device of the present invention is in its operational configuration. The terms "lower" and "bottom" refer to what is at the bottom when the device of the present invention is in its operational configuration. The term "inside" refers to what is inside the device. The term "outside" refers to what is outside the device.

We observe, on the FIG. 1 A non-contact registration device 10, which is not to scale, is used to connect a robot 11, a patient 12, and medical imaging stored in memory, for example, in a memory 13 of the robot 11. The patient 12 is positioned on a surgical table 14, in preparation for surgery. The robot includes a mobile trolley 15 equipped with wheels 16, supporting an anthropomorphic robotic arm 17 with joints 18. The robotic arm 17 supports, at its free end, a non-contact registration unit 19 comprising a means 27 for projecting a laser line 21 onto the surgical intervention area on the patient 12, here the head 22 of the patient 12. The registration unit 19 also includes a means for projecting a laser point 20, also called a laser pointer, configured to measure a distance in a known manner.

The unit 19 is also equipped with a distance sensor 25 for the points of the laser line 21 and a handle 24 with which the operator 23 can manipulate this unit 19. The distance sensor 25 includes, for example, an image sensor and a means for measuring distance by triangulation, according to known techniques.

The registration unit 19 can be installed by the operator 23 at the free end of the arm 17 as a "tool." Alternatively, the registration unit 19 is integrated into this end of the arm 17. The advantage of integration is that it avoids having to remove and replace the registration unit 19 during a surgical procedure, in case the registration is lost during the procedure. This integration also prevents assembly errors by the operator 23 when installing the registration unit 19 at the end of the arm 17, as these assembly errors could lead to registration inaccuracies.

We find, in FIG. 2 the same elements as in FIG. 1 during a scanning step of the patient's head 22. Arrow 26 represents the direction of scanning of the user's head by the laser line 21.

FIG. 3 is an example of a point cloud 30 of points 31 obtained from a three-dimensional laser scan using the device of the invention. The point cloud 30 of points 31 is generated from a three-dimensional surface image captured by the unit 19 through the projection of a laser line 21 and the distance sensor 25. Each point 31 of this point cloud 30 is located according to a geometric coordinate system linked to the conditions under which the three-dimensional image was captured and therefore to the position of the robot 11.

FIG. 4 is an example of a cloud of 40 points extracted from a medical image. Each point 41 of this cloud is located according to a geometric coordinate system linked to the conditions under which the medical image was acquired.

In order for the robot 11 to virtually position the medical image 40 in space relative to the actual space of the patient 12, a mapping operation (also called "registration") of the point clouds 30 and 40 of points 31 and 41 is necessary. The medical image 40 is thus located in the geometric coordinate system of the robot 11.

To this end, the implementation of device 10 follows the following steps of process 50 illustrated in FIG. 5 .

During step 51, the robot 11, and in particular the carriage 15, is positioned opposite the area of interest, i.e., the surgical intervention area on the patient's body 12 (the head in Figures 1 and 2). The carriage 15 remains stationary after step 51 until the end of the surgical procedure. Preferably, the carriage 15 is equipped with jack legs (not shown) which are deployed during step 51 to prevent any movement of the carriage 15. At the end of step 51, the robot, and in particular its robotic arm 17, is started.

During step 52, the robotic arm 17 performs an automatic pre-positioning so that its free end is near the area of interest, in a position where the operator 23 can easily install the registration unit 19 if it is not integrated into the arm 17. During this step 52, the robotic arm moves from a so-called "storage" position in which it is connected above the carriage 15 to a deployed position allowing the easy installation of the registration unit 19 at the free end of the arm 17. If the unit 19 is integrated into the arm 17, the position reached by the arm 17 is a position allowing the operator 23 to grasp the handle 24 to move it, in cooperative mode, to the area of interest to be scanned.

During step 53, in the event that the recalibration unit 19 is not integrated into the arm 17, it is installed at the free end of the robotic arm 17.

During a step 54, the operator 23 roughly positions the non-contact registration unit 19 parallel to the area of interest, using the handle 24 of the registration unit 19 and the cooperative mode of the robotic arm 17. Alternatively, this rough positioning is carried out automatically by the robotic arm 17.

The cooperative mode is implemented via at least one force sensor located at the end of the robotic arm 17, or directly in the joints of the robotic arm, if it is an "active" arm. In this cooperative mode, the arm can apply constraints to the movements of its extremity. For example, the only movements permitted, in cooperative mode, at the free end of the robotic arm 17 are in a plane roughly parallel to the area of interest or along a straight line roughly parallel to an axis of symmetry of the area of interest, for example, the patient's face 12. Alternatively, the plane is parallel to the upper surface of the operating table 14 and/or the axis is parallel to the longitudinal axis of the operating table 14. This is the configuration during step 54, which is illustrated in Figure 54. FIG. 1 .

During step 55, the laser pointer provides a distance between the registration unit 19 and the area of interest. Based on this initially measured distance, the distance is adjusted automatically or manually by moving the registration unit 19. This adjustment is made taking into account that a distance that is too small can cause difficulties in terms of the distance measurement area covered by the registration unit 19 and in terms of the sterility of the surgical field. Conversely, a distance that is too large increases the uncertainty in distance measurements. Typically, the adjusted distance is on the order of 40 centimeters.

In step 56, the projection angle of the planar laser beam, which projects a line of light onto the patient's skin in the area of interest, is adjusted. The length of the laser beam line is adjusted via a user interface (not shown), for example, a touchscreen mounted on a cart other than cart 15. This sizing of the light line on the scanned surface avoids illuminating external elements, such as a stereotactic frame. Alternatively, the laser pointer is used to define the edges of the projection field to be covered.

During step 57, the laser registration unit is switched on and the coordinates of the starting position of the scan of the area of interest with the laser light line are recorded in the robot's geometric coordinate system, via a control button (not shown) on the handle of the registration unit or the user interface.

During step 58, the robotic arm 17 switches to cooperative mode under axial or planar constraint parallel to the area of interest.

During step 59, the operator 23 moves the robotic arm 17 under axial or planar constraint parallel to the area of interest in order to scan this area of interest and measure the distance to the registration unit 19 of each point illuminated by the light line. The configuration at the beginning of step 59 is illustrated in FIG. 2 .

In step 60, a computing unit (not shown) extracts a point cloud (a 3D surface similar to point cloud 30) as the output data from the laser scan. It should be noted that this three-dimensional output data from the laser scan can be used in so-called "open" surgery, i.e., by exposing the bony part of the area concerned (e.g., part of the spine, a knee, or a hip) to the air, without "intraoperative" (during the operation) imaging, but with "preoperative" (pre-op) imaging.

During step 61, the computing unit performs a point cloud extraction from the medical imaging, similar to point cloud 40 of 41.

During step 62, the computing unit uses a point cloud matching algorithm, for example, an ICP (Iterative Closest Point) algorithm, to calculate the transformation matrix between the real-space reference frame of the area of interest (in robot 10's geometric coordinates) and the associated medical imaging reference frame. This aligns the point clouds in the robot's geometric coordinate system, minimizing the error between the two clouds. Preferably, before using this algorithm, the operator performs a quick "manual" coarse registration step. This step accelerates the convergence of the automatic ICP algorithm. To this end, the operator identifies equivalent anatomical areas (or points) in the point clouds of the laser scan and the medical imaging. A first, so-called "coarse" registration is then performed based on these. The ICP algorithm uses this coarse registration as input data.

Preferably, a confidence level is assigned to the final match, for example by measuring an average distance between matched points, and, if the confidence level is too low (for example, this average distance is greater than a predetermined limit), steps 55 to 61 are repeated. Two examples of confidence levels are described below:

1. An overall average RMS (root mean square error) index is calculated for the entire cloud. The resulting RMS value (e.g., 0.26 mm) is displayed on the user interface using a two-color code (green acceptable, red unacceptable). If the index is green, the operator can continue the procedure. A red index on the user interface requires the operator to perform another surface laser scan.

2. A point index (distance between two points matched between the two point clouds) that can be represented as a color map directly on the 3D extraction of the medical image or on the 3D surface obtained via laser scanning. If the percentage of points farther apart than a predetermined limit (e.g., one millimeter) exceeds a limit value (e.g., 10%), or if a distance between corresponding points exceeds another predetermined limit (e.g., three millimeters), a red index on the user interface requires the operator to perform a new surface laser scan.

In step 63, the computing unit creates a no-go zone for the robotic arm 17 and its instrument holder from the scanned 3D surface and/or the 3D reconstruction of the medical image registered with the laser-scanned 3D surface. This no-go zone prohibits collisions between the robotic arm 17 and the patient's body 12, at least within the area of interest (the entire volume of the head 22 in the case shown in Figures 1 and 2).

In the second embodiment of device 70, the object of the invention is represented in FIG. 6 a patient 12 on an operating table 14, and an operator 23. However, unlike the first embodiment, no robotic arm is used to follow the movement of a non-contact registration unit 71 comprising a means for projecting a laser point 20 and a laser line 21 formed on the area of interest on the body of the patient 12, here his head 22. The unit 71 is also equipped with a distance sensor 25 and a handle 72 with which the operator 23 can manipulate this unit 71. The distance sensor 25 comprises, for example, an image sensor and a means for measuring distance by triangulation, according to known techniques.

A navigation camera 77 is mounted on an articulated positioning arm 76, itself supported by a camera support carriage 75. To locate the registration unit 71 in space, this unit 71 is equipped with at least one navigation reference 74, which is fixed to the unit 71. The optical field 79 of the camera 77 covers the different positions of the positioning reference 74 during the movement of the unit 71.

Alternatively, to avoid operator 23 having to hold the registration unit 71 at arm's length and to ensure that the movement of the registration unit 71 is a translation parallel to the longitudinal axis of the operating table 14, a sliding support 78 is removable and mounted on the operating table 14. For example, this support 78 has, at its upper end, two horizontal rails parallel to the longitudinal axis of the operating table 14, positioned on either side of the patient's head 22. The unit 71 slides or rolls on casters (not shown) along these rails in translation. This movement is initiated by operator 23 and, if necessary, braked at the casters to ensure sufficiently dense point capture on the surface of interest.

Thanks to the geometric positioning of the registration unit 71, performed by the navigation camera 77 via each navigation reference 74, the coordinates of the points in the area of interest within the operating room space can be obtained. Once a robotic arm is in position, its free end is, in turn, located within this space, for example, by implementing at least one other navigation reference. To this end, registration between the robotic arm and the navigation camera must be performed as follows. A navigation reference is positioned at the free end of the robotic arm, either as a tool to be worn or by being integrated into this end of the arm. The robotic arm then moves automatically to different known positions in space. These different positions are recorded by the camera and located in its geometric reference frame. The positions seen by the camera can then be linked to the positions of the joints (via the joint encoders) of the robotic arm. Alternatively, to avoid the time required to set up the navigation reference (which can lead to incorrect setups and therefore inaccuracies) and to move the arm to different positions, a navigation reference can be permanently mounted on the robotic arm's carriage. Since the location of this reference relative to the robotic arm's base (and therefore its frame of reference) is inherently known, the camera simply needs to locate this reference to align it with the robotic arm's frame of reference. The three-dimensional coordinates of the points in the area of interest can then be matched with the coordinates of the robotic arm and with the previously stored medical images.

The registration unit 71 includes the same means for adjusting the projection angle of the laser line 21 or the length of the laser line 21 as the registration unit 19 described opposite Figures 1 and 2.

Although, in the embodiments of the device described above, the distance sensor 25 is mounted on the robotic arm 17, in other embodiments, this distance sensor is mounted on a carriage other than the robotic arm. Naturally, the geometric reference frame of this distance sensor is then fixed, and the coordinate transformation matrix from this reference frame to coordinates in the robot's reference frame is then predetermined.

Presentation of the invention

The present invention aims to remedy all or part of the drawbacks of the state of the art.

To this end, according to a first aspect, the present invention relates to a contactless registration device between a robot, a patient and medical imaging, which comprises:
- a means of projecting a laser line onto an area of interest on the patient,
- a means of capturing geometric coordinates of points on the laser line to form a point cloud of the area of interest, and
- a means of matching a point cloud from medical imaging and the point cloud of the area of interest.

Thus, the projection device forms, on the area of interest of the patient's body, at least one line allowing the simultaneous capture of three-dimensional geometric coordinates of points within that area. A single linear movement of the projection device thus captures the three-dimensional geometric coordinates of the entire area of interest, which must then be aligned with the surface extracted from the medical imaging of said anatomy.

The implementation of the present invention enables geometric registration between the robot, the area of interest on the patient's body, and the medical imaging. This registration is performed without contact with the patient's body, and is therefore non-invasive. The implementation of the invention significantly reduces the time required to acquire a point cloud of the anatomical surface of interest. Typically, this time is reduced from 15 to 20 minutes in the prior art using laser pointer registration to less than five minutes. This time saving reduces the operating room time and the duration of anesthesia, and its potentially harmful consequences for the patient, especially if the registration needs to be repeated during surgery if it is lost, for example, if the robot has moved. The invention also reduces the operator's dependence on registration accuracy and improves the operator's user experience.

In some embodiments, the laser line projection means further includes a means for adjusting the projection angle of the laser line and/or the length of the laser line.

The laser line is sized accordingly on the scanned surface to avoid illuminating elements external to the patient, such as a stereotactic frame. This prevents artifacts from capturing the geometric coordinates of points on the laser line within the area of interest.

In some embodiments, the device of the invention further comprises a laser pointer for measuring the distance between the geometric coordinate capture unit and the area of interest.

This adjustment is made taking into account that a distance that is too short can pose difficulties in terms of the area covered by the capture device for distance measurement and in terms of the sterility of the operating field. Conversely, a distance that is too long increases the uncertainty in distance measurements.

In some embodiments, the device of the invention further comprises the robot equipped with an arm having at least one force sensor, the robot being configured to operate in cooperative mode.

The movements of the operator using the device that is the subject of the invention are measured by force sensors, allowing the robot to determine the geometric coordinates of the capture unit in its own geometric reference frame. The geometry of the capture unit is known, by design, relative to the robotic arm. The capture unit is mounted at the end of the arm or directly integrated into the arm. The encoders of the robotic arm's joints provide the coordinates of the arm end in the robot's reference frame. Knowing the geometry of the capture unit, its coordinates relative to the coordinates of the arm end can be deduced.

In some embodiments, the robot is configured to operate in cooperative mode, constraining the movement of the capture means in a plane or along an axis.

The operator implementing the device that is the subject of the invention is thus helped by the robotic arm to follow a movement facilitating the 3D acquisition of the surface of interest.

In some embodiments, the robot is a aiming assistance robot.

Thus, the robot's reference frame, calibrated both with respect to the area of interest and with respect to the medical imaging, allows the robotic arm to position a surgical instrument guide very precisely (to within one millimeter) and stably in real space, enabling the surgeon to target, via the insertion of the surgical instrument into the guide carried by the robot, an anatomical area in real space from a target area and an entry area that he will have previously identified in the patient's medical imaging.

In some embodiments, the device of the invention further comprises a navigation camera, and a calibration unit equipped with a fixed reference relative to the calibration unit.

This navigation camera can thus determine the geometric coordinates of the capture means, in its geometric reference frame, and therefore of the area of interest also in this reference frame.

According to a second aspect, the present invention relates to a contactless registration method between a robot, a patient, and medical imaging, which comprises:
- a step involving the projection of a laser line onto an area of interest on the patient's body,
- a step of capturing geometric coordinates of points on the laser line to form a point cloud of the area of interest, and
- a step of matching a point cloud from the medical imaging and the point cloud of the area of interest.

The advantages, purposes and particular characteristics of this process being similar to those of the device which is the subject of the invention, they are not recalled here.

In embodiments, the method of the invention further comprises a step of creating a geometrically forbidden zone for the robotic arm from the point cloud of the area of interest and/or from medical imaging registered with the point cloud of the area of interest.

This ensures better patient safety, thanks to the creation of a "NoGo" zone around the patient's body.

Claims (12)

A contactless registration device (10, 70) between a robot (17), a patient (12) and a medical imaging system, characterized in that it comprises:
- a means (27) for projecting a laser line (21) onto an area of interest (22) on the patient's body,
- a means (25) for capturing geometric coordinates of points on the laser line to form a point cloud (40) of points (41) of the area of interest, and
- a means (13) of matching a cloud (30) of points (31) of the medical imaging and the point cloud of the area of interest. Device (10, 70) according to claim 1, wherein the means (27) for projecting the laser line (21) further comprises a means for adjusting the projection angle of the laser line and/or the length of the laser line. Device (10, 70) according to claim 1 or 2, further comprising a laser pointer for measuring the distance between the geometric coordinate capture unit and the area of interest. Device (10) according to any one of claims 1 to 3, further comprising the robot (11) equipped with an arm (17) having at least one force sensor, the robot being configured to operate in cooperative mode Device (10) according to claim 4, wherein the robot (17) is configured to operate in cooperative mode constraining the movement of the capture means (25) in a plane or along an axis. Device (10) according to any one of claims 4 or 5, wherein the robot (17) is a aiming assistance robot. Device (10, 70) according to any one of claims 1 to 6, further comprising a navigation camera (77), and a recalibration unit (19, 71) provided with a reference (74) fixed relative to the recalibration unit. Method (50) for contactless registration between a robot (17), a patient (12) and a medical imaging system, characterized in that it comprises:
- a step (59) of projecting a laser line (21) onto an area of interest (22) on the patient's body (12),
- a step (59, 60) of capturing geometric coordinates of points on the laser line to form a point cloud (40) of points (41) of the area of interest, and
- a step (62) of matching a cloud (30) of points (31) of the medical imaging and the point cloud of the area of interest. Method (50) according to claim 8, which further comprises a step (56) of adjusting the projection angle of the laser line (21) and/or the length of the laser line. Method (50) according to any one of claims 8 or 9, further comprising a step (55) of adjusting the distance between a means (27) for projecting the laser line (21) and the area of interest (22). Method (50) according to any one of claims 8 to 10, wherein, during the capture step (59, 60), the robot (17) operates in cooperative mode constraining the movement of a capture means (25) in a plane or along an axis. Method (50) according to any one of claims 8 to 11, which further comprises a step (63) of creating a geometric forbidden zone for the robotic arm (17) from the point cloud (30) (31) of the area of interest (22) and/or from the medical imaging (40) registered with the point cloud of the area of interest.