[go: up one dir, main page]

WO2025019569A1 - Imagerie peropératoire pour navigation assistée par robot - Google Patents

Imagerie peropératoire pour navigation assistée par robot Download PDF

Info

Publication number
WO2025019569A1
WO2025019569A1 PCT/US2024/038352 US2024038352W WO2025019569A1 WO 2025019569 A1 WO2025019569 A1 WO 2025019569A1 US 2024038352 W US2024038352 W US 2024038352W WO 2025019569 A1 WO2025019569 A1 WO 2025019569A1
Authority
WO
WIPO (PCT)
Prior art keywords
coordinate system
fiducial
flexible elongate
sensor
processors
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/US2024/038352
Other languages
English (en)
Inventor
Troy K. ADEBAR
Federico Barbagli
Sungwon YOON
Hui Zhang
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Intuitive Surgical Operations Inc
Original Assignee
Intuitive Surgical Operations Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Intuitive Surgical Operations Inc filed Critical Intuitive Surgical Operations Inc
Publication of WO2025019569A1 publication Critical patent/WO2025019569A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2048—Tracking techniques using an accelerometer or inertia sensor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2051—Electromagnetic tracking systems
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2059—Mechanical position encoders
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
    • A61B2034/2046—Tracking techniques
    • A61B2034/2061—Tracking techniques using shape-sensors, e.g. fiber shape sensors with Bragg gratings
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • A61B2090/374—NMR or MRI
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • A61B2090/376—Surgical systems with images on a monitor during operation using X-rays, e.g. fluoroscopy
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • A61B2090/376—Surgical systems with images on a monitor during operation using X-rays, e.g. fluoroscopy
    • A61B2090/3762—Surgical systems with images on a monitor during operation using X-rays, e.g. fluoroscopy using computed tomography systems [CT]
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • A61B2090/376—Surgical systems with images on a monitor during operation using X-rays, e.g. fluoroscopy
    • A61B2090/3762—Surgical systems with images on a monitor during operation using X-rays, e.g. fluoroscopy using computed tomography systems [CT]
    • A61B2090/3764—Surgical systems with images on a monitor during operation using X-rays, e.g. fluoroscopy using computed tomography systems [CT] with a rotating C-arm having a cone beam emitting source
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/37—Surgical systems with images on a monitor during operation
    • A61B2090/378—Surgical systems with images on a monitor during operation using ultrasound
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/39—Markers, e.g. radio-opaque or breast lesions markers
    • A61B2090/3966—Radiopaque markers visible in an X-ray image
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots

Definitions

  • Disclosed examples relate to planning and/or navigating minimally invasive medical procedures and, more specifically, to registration between coordinates of a robotic-assisted system actuating an elongate device and coordinates of intraoperative images.
  • Minimally invasive medical techniques are intended to reduce the amount of tissue that is damaged during medical procedures, thereby reducing patient recovery time, discomfort, and harmful side effects.
  • Such minimally invasive techniques may be performed through natural orifices in a patient anatomy or through one or more surgical incisions. Through these natural orifices or incisions, physicians may insert minimally invasive medical instruments (including surgical, diagnostic, therapeutic, and/or biopsy instruments) to reach a target tissue location.
  • minimally invasive medical instruments including surgical, diagnostic, therapeutic, and/or biopsy instruments
  • One such minimally invasive technique is to use a flexible and/or steerable elongate device, such as a flexible catheter, that can be inserted into anatomic passageways and navigated toward a region of interest within the patient anatomy.
  • a combination of localization sensors disposed at a flexible elongate device and intraoperative imaging can greatly aid in planning and navigating a minimally invasive procedure.
  • combining sensor data with intraoperative images can enable accurate determination of a position, orientation, and/or pose of the flexible elongate device within the patent anatomy.
  • accurate and fast registration between a coordinate system of the sensors and a coordinate system of the intraoperative images remains a challenge using current techniques.
  • a tangible, non-transitory, computer readable medium stores instructions that, when executed by one or more processors, cause the one or more processors to receive indications of positions, in a first coordinate system, of a first sensor and a second sensor disposed at a flexible elongate medical device.
  • the instructions may further cause the one or more processors to determine, based on intra-operative imaging data, an indication of a position, in a second coordinate system, of a fiducial of the flexible elongate device, wherein the fiducial is disposed at a separation distance, along the flexible elongate device, from the first sensor. Still further, the instructions may cause the one or more processors to register the first coordinate system to the second coordinate system based at least in part on the indications of the positions of the first sensor and the second sensor in the first coordinate system, the indication of the position of the fiducial in the second coordinate system, and the separation distance.
  • a medical device comprises a flexible elongate body having an axis and a position sensor disposed at the flexible elongate body of the medical device, the position sensor configured to generate an indication of position in a first coordinate system.
  • the medical device further comprises a fiducial disposed at the flexible elongate body of the device and having a rotationally asymmetrical shape with respect to the axis of the flexible elongate body of the device.
  • a medical system comprises a flexible elongate device, a first sensor and a second sensor disposed at the flexible elongate device, and a fiducial disposed at the flexible elongate device at a separation distance from the first sensor.
  • the system further comprises one or more processors configured to receive indications of positions, in a first coordinate system, of the first sensor and the second sensor.
  • the one or more processors are further configured to determine, based on intra-operative imaging data, a position of the fiducial in a second coordinate system.
  • the one or more processors are configured to register the first coordinate system to the second coordinate system based on the received indications of positions of the first sensor and the second sensor in the first coordinate system, the determined position of the fiducial in the second coordinate system, and the separation distance.
  • a method comprises receiving, by one or more processors, indications of positions, in a first coordinate system, of a first sensor and a second sensor disposed at a flexible elongate medical device. The method further comprises determining, based on intra-operative imaging data and by the one or more processors, an indication of position, in a second coordinate system, of a fiducial of the flexible elongate device, wherein the fiducial is disposed at the flexible elongate device at a separation distance from the first sensor.
  • the method comprises registering, by the one or more processors, the first coordinate system to the second coordinate system based at least in part on the indications of positions of the first sensor and the second sensor in the first coordinate system, the indication of position of the fiducial in the second coordinate system, and the separation distance.
  • a tangible, non-transitory, computer readable medium stores instructions that, when executed by one or more processors, cause the one or more processors to receive an indication of a position, in a first coordinate system, of a position sensor disposed at a flexible elongate body of a medical device.
  • the instructions may further cause the one or more processors to determine, based on intra-operative imaging data, an indication of a position, in a second coordinate system, of a fiducial disposed at the flexible elongate body and having a rotationally asymmetrical shape with respect to an axis of the flexible elongate body.
  • the instructions may cause the one or more processors to register the first coordinate system to the second coordinate system based at least in part on the indication of the position of the position sensor and the indication of the position of the fiducial in the second coordinate system.
  • a method comprises receiving, by one or more processors, an indication of a position, in a first coordinate system, of a position sensor disposed at a flexible elongate body of a medical device.
  • the method further comprises determining, based on intraoperative imaging data and by the one or more processors, an indication of a position, in a second coordinate system, of a fiducial disposed at the flexible elongate body and having a rotationally asymmetrical shape with respect to an axis of the flexible elongate body.
  • the method comprises registering, by the one or more processors, the first coordinate system to the second coordinate system based at least in part on the indication of the position of the position sensor and the indication of the position of the fiducial in the second coordinate system.
  • FIG. 1 A depicts an example system for navigating during a medical procedure within an operating environment.
  • FIG. 1 B is a simplified diagram of a flexible elongate device disposed within an anatomical structure.
  • FIG. 2A depicts a rigid body in two example coordinate systems.
  • FIG. 2B schematically illustrates an example coordinate registration process.
  • FIGS. 3A-E schematically illustrate example configurations of sensors and a fiducial disposed at a flexible elongate device.
  • FIGS. 4A-D schematically illustrate example configurations of a fiducial having a rotationally asymmetrical shape with respect to an axis of a flexible elongate body of a device.
  • FIGS. 5A and 5B schematically illustrate example configurations of a fiducial removably attached to a flexible elongate device.
  • FIGS. 6A and 6B are block diagrams of methods for implementing the techniques of this disclosure.
  • FIG. 7 is a simplified diagram of a medical system according to some examples.
  • FIG. 8A is a simplified diagram of a medical instrument system according to some examples.
  • FIG. 8B is a simplified diagram of a medical instrument including a medical tool within an elongate device according to some examples.
  • FIGS. 9A and 9B are simplified diagrams of side views of a patient coordinate space including a medical instrument mounted on an insertion assembly according to some examples.
  • position refers to the location of an object or a portion of an object in a three-dimensional space (e.g., three degrees of translational freedom along Cartesian x-, y-, and z-coordinates).
  • orientation refers to the rotational placement of an object or a portion of an object (e.g., one or more degrees of rotational freedom such as roll, pitch, and yaw).
  • the term “pose” refers to the position of an object or a portion of an object in at least one degree of translational freedom and to the orientation of that object or portion of the object in at least one degree of rotational freedom (e.g., up to six total degrees of freedom).
  • the term “shape” refers to a set of poses, positions, and/or orientations measured along an object.
  • distal refers to a position that is closer to a procedural site and the term “proximal” refers to a position that is further from the procedural site. Accordingly, the distal portion or distal end of an instrument is closer to a procedural site than a proximal portion or proximal end of the instrument when the instrument is being used as designed to perform a procedure.
  • the disclosure generally relates to systems and methods that facilitate user (e.g., physician) planning of, and/or user navigation during, a medical procedure (e.g., an endoluminal medical procedure).
  • a medical procedure e.g., an endoluminal medical procedure.
  • These systems and methods can provide precise and accurate registration between a first coordinate system (e.g., a coordinate system of a robotic-assisted system actuating a medical instrument) and a second coordinate system (e.g., a coordinate system of intraoperative images), to improve procedure accuracy and efficiency.
  • a first coordinate system e.g., a coordinate system of a robotic-assisted system actuating a medical instrument
  • a second coordinate system e.g., a coordinate system of intraoperative images
  • the methods and systems of this disclosure use data from one or more sensors (e.g., point sensors) disposed at a flexible elongate device, such as a catheter, to identify positions and/or orientations of the sensor(s) in the coordinate system of the robotic system (i.e., using robotic system coordinates) and use data from intraoperative images in the image coordinate system (i.e., using imaging system coordinates) to register the two coordinate systems with each other.
  • the registration process may include determining a one-to-one point correspondence between the two coordinate systems and/or a mathematical transformation to compute coordinates of a point in one coordinate system based on the coordinates of the same point in the other coordinate system.
  • the registration may be updated, as needed, during the course of the procedure, for example, in response to a change in one of the coordinate systems.
  • a change in the physical environment may precipitate a change in a coordinate system by altering magnetic fields sensed by the one or more sensors, for example.
  • Some previous approaches to registration focus on obtaining a multi-point (e.g., dozens of points) description of a shape of the flexible elongate device in the robotic system coordinates as well as a multi-point description of the shape in the image coordinates (e.g., by segmenting the flexible elongate device from an intra-operative image).
  • a fiber-optic shape sensor may be disposed along the flexible elongate device.
  • a computing system may perform segmentation of the flexible elongate device from image data.
  • the shape-based coordinate transformation may have certain limitations. For example, a flexible elongate device may not have an accompanying fiber-optic shape sensor.
  • segmenting the flexible elongate device from image data may require user intervention to identify pixels or voxels associated with the device.
  • a system may segment an image portion corresponding to (e.g., showing at least a portion of) the flexible elongate device from the remainder of an intraoperative image, and identify, in the segmented portion, position(s) and/or orientation(s) of at least one EM sensor disposed at the flexible elongate device.
  • DOFs degrees of freedom
  • the system may use multiple (at least two) EM sensors to obtain the missing DOFs in the robot system coordinates.
  • a well-resolved image of one of the EM sensors may be sufficient to obtain the six DOFs in the image coordinates.
  • the EM sensor package may not be sufficiently visible or well-resolved in the image, or may have non- resolvable symmetries, confounding orientation information. That is, there are cases where the package of the EM sensor is an unsatisfactory fiducial for determining the position and/or orientation of the sensor in the image coordinate systems.
  • An additional fiducial may be disposed at the flexible elongate device to identify position(s) and/or orientation(s) in image coordinates with better accuracy and precision.
  • the fiducial need not coincide with a position of one of the sensors.
  • the fiducial may be disposed at the flexible elongate device a known distance along the body of the flexible elongate device from one of the sensors. If the separation between sensors along the body of the flexible elongate device is also known, the position of the fiducial with respect to one sensor indicates positions (along the body of the flexible elongate device) with respect to the other one(s).
  • the fiducial may be distributed along the flexible elongate device.
  • an image of the fiducial may be indicative of positions of multiple points along the flexible elongate device. That is, multiple fiducial elements, disposed at the flexible elongate device with known relative positions with respect to the device, and whether connected or disconnected, may be thought of as a single fiducial.
  • Such a fiducial may include elements of a variety of materials such as metals, plastics, confined fluids (e.g., bubbles), etc.
  • the fiducial may include multiple circular or elliptical rings encircling a flexible elongate device with a substantially circular or elliptical cross-section.
  • the elements of the fiducial may be disposed to create a pattern that, in a sense, encodes specific locations along the device.
  • the fiducial may include axial elements oriented parallel to the axis, or the long dimension, of the elongate device, to aid, for example, in identifying orientation in the image coordinates.
  • the fiducial may be removably disposed at the flexible elongate device, for example, by way of being disposed at a removable sleeve or stylet that may be removably attached to a flexible elongate device.
  • the registration between coordinate systems need not solely rely on the positions of point sensors and the imaged fiducial for registration.
  • constraints may aid registration.
  • the disclosed registration methods may use constraints based on the mechanics of the flexible elongate device, such as an assumption of curvature between consecutive points of the device, orientation constraints among the robotic system, the imaging system, and patient anatomy, etc.
  • the operating environment e.g., horizontal patient bed or operating table
  • the methods may include a variety of specific combinations of the numbers and types of point sensors and fiducial shapes. For example, L-shape fiducial elements, unevenly spaced ring elements, etc. may be included in the fiducial.
  • a system may use one, two, three, or any other suitable number of sensors to obtain six DOFs information and/or redundant position information and thereby improve registration accuracy.
  • the system may generate a graphical representation of the flexible elongate device in a graphical user interface (GUI).
  • GUI graphical user interface
  • the system may overlay the graphical representation of the flexible elongate device on or with a model of the anatomy within which the flexible elongate device is disposed.
  • the system may display the sensor positions and/or fiducial in a joint coordinate system.
  • the system may display one or more of the sensor positions along with the portion of the flexible elongate device at which the one or more of the sensors and/or the fiducial are disposed.
  • FIG. 1 A depicts an example system 100 for navigating during a medical procedure within an operating environment 101.
  • the system 100 may obtain images from a portion of the operating environment 101 disposed within a field of view F (approximately demarcated by dashed lines) of an imaging unit 1 10. To that end, the system 100 may be in communicative connection with the imaging unit 1 10.
  • the imaging unit 110 may generate two-dimensional or volumetric images of at least a portion of the operating environment using a color imaging device, an infrared imaging device, an ultrasound imaging device, an x-ray imaging device, a fluoroscopic imaging device, a computed tomography (CT) imaging device, a magnetic resonance imaging (MRI) imaging device, or some other type of imaging device.
  • CT computed tomography
  • MRI magnetic resonance imaging
  • the system 100 includes a processing unit 120 and a display unit 130 in communicative connection with each other. Although in FIG. 1 A the imaging unit 1 10 and the sensing unit 115 are depicted as being distinct from the system 100, in other examples, the system 100 may include the imaging unit 1 10 and/or the sensing unit 115. In any case, one or more processors of the processing unit 120 of the system 100 may be configured to receive images and/or processed image information from the imaging unit 1 10 and to receive data from the one or more sensors by way of the sensing unit 115. [0037] Throughout the disclosure, the descriptions of example operations performed by the processing unit 120 below are to be understood to be executed by the one or more processors of the processing unit 120.
  • the one or more processors may include hardware specifically configured (e.g., hardwired or programmable) to carry out at least a portion of the example operations described in this disclosure. Additionally or alternatively, the one or more processors may be configured to carry out at least a portion of the example operations described in this disclosure by carrying out a set of software instructions.
  • the system 100 may include or be communicatively connected to a tangible, non-transitory, computer readable medium. The medium may store instructions which, when executed by the processing unit 120, perform any one or more of the example operations described below.
  • the instructions may cause the processing unit 120 to perform image processing operations on the images received from the imaging unit 110 and/or to perform computations (e.g., for coordinate registration) based on the data received by way of the sensing unit 115.
  • the instructions may cause the processing unit 120 to cause the display unit 130 to display, via a graphical user interface, information based on the processing of images received from the imaging unit 1 10 and the processing of data received by way of the sensing unit 115.
  • the processing unit 120 may send the information, or send data representing the entire graphical user interface including the information, to the display unit 130.
  • An operator e.g., a physician, another medical practitioner, or a fully-automated robotic surgery system
  • a medical procedure e.g., endoscopy, biopsy, pharmacological treatment, and/or treatment, such as ablation
  • the operator may control a flexible elongate device 140 inserted through an orifice O into an anatomical structure A of a patient P disposed at a table T.
  • the medical procedure may include navigating the flexible elongate device 140 (indicated with solid lines outside and dashed lines inside the patient P) toward a region of interest R within the anatomical structure A with the aid of information displayed at the display unit 130.
  • the region R for example, may be a designated procedure site for examination, biopsy, treatment, or any other medical procedure.
  • a fiducial 142 may be disposed at (e.g., physically contacting, integrated within, fixedly attached to, or removably attached to in a manner that, during operation/use, forms a rigid relationship with) the flexible elongate device 140.
  • the fiducial 142 is configured to be visible in images obtained by the imaging unit 110.
  • the fiducial 142 may include elements of a variety of materials and/or structures such as metals, plastics, etched glass, dyes, radioactive or fluorescent markings, confined fluids (e.g., bubbles), etc.
  • the fiducial 142 need not be localized (e.g., to within, 1 , 2, 5, 10 mm) at a certain point along the flexible elongate device and may, instead, have elements distributed along the length of the flexible elongate device 140.
  • the distributed fiducial 142 may span a length anywhere from a few millimeters to tens of centimeters. Though spanning a certain length, the fiducial 142 may leave a large portion (e.g., 20-99%) of the span along the flexible elongate device 140 unobstructed.
  • At least some of the elements of the fiducial 142 may be integrated (e.g., etched, deposited, painted, or otherwise fixedly attached) onto the flexible elongate device 140.
  • the fiducial 142 may include elements removably disposed at the flexible elongate device 140.
  • the fiducial 142 may be integrated onto a removable structure, such as a sleeve or a stylet, which in turn may be removably attached to the flexible elongate device 140.
  • the removable structure may include elements that allow registering the removable structure to the flexible elongate device 140, for example, at a fixed distance from the distal end of the flexible elongate device 140.
  • Sensors 144a, b may also be disposed at the flexible elongate device.
  • the sensors 144a, b may be mechanical sensors, optical sensors, electromagnetic (EM) sensors, or any other suitable sensors.
  • the sensors 144a, b may be disposed at the flexible elongate device in a known geometrical relationship with respect to the fiducial 142.
  • the sensors 144a, b may be integrated into the flexible elongate device 140, or removably attached to the flexible elongate device 140.
  • the sensors 144a, b may be configured to communicate with the sensing unit 115.
  • the sensors 144a, b are active sensors, configured to transmit electromagnetic (e.g., optical, radio, low-frequency magnetic) or radioactive radiation.
  • the sensing unit 115 may include components to receive the radiation emitted by the sensors 144a, b and triangulate or trilaterate locations of the sensors 144a, b in the sensor system coordinates.
  • the sensor system coordinates may be the coordinates of the robotic-assisted system configured to manipulate, control, or guide the flexible elongate device 140.
  • the sensors 144a, b are passive and emit no radiation.
  • the passive sensors 144a, b may sense radiation emitted by components of the sensing unit 115 disposed within the operating environment 101 .
  • one or more electrified coils may be disposed within the operating environment 101 to generate static or dynamic magnetic fields.
  • the sensors 144a, b may be configured to pick up variations in the dynamic magnetic fields or, as the sensors move 144a, b, variations in the sensed static or quasi-static magnetic field and convert the sensed variations into electrical currents that are received by the sensing unit 115.
  • the sensing unit 115 may compute indications of positions of the sensors 144a, b from the received electrical currents.
  • the point sensors may include accelerometers, gyroscopes, and/or magnetometers.
  • the point sensors may include inertial measurement units (IMUs) that combine multiple sensors (e.g., accelerometers, gyroscopes) and/or inertial and magnetic measurement units (IMMlls) that combine multiple sensors (e.g., accelerometers, gyroscopes, magnetometers).
  • IMUs and/or IMMUs may generate signals indicative of orientation of the flexible elongate device at a given point with respect to gravitational field and/or magnetic field (e.g., of Earth).
  • additional magnetic fields may be introduced to the sensor environment.
  • IMUs and/or IMMUs may generate signals indicative of motion of the flexible elongate device (e.g., caused by motion of an anatomical structure due to breathing and/or other factors, and/or independent motion of the flexible elongate device within the anatomical structure).
  • the sensing unit 115 may combine indications of orientation (e.g., up to three degrees of freedom) from IMUs with indications of position from other (e.g., EM) sensors to generate more complete data indicative of pose of the flexible elongate device.
  • the sensing unit 115 may combine data from sensors in multiple sensor coordinate systems.
  • the sensing unit 115 and/or the processing unit 120 may register multiple sensor coordinate systems with each other.
  • FIG. 1 B is a simplified diagram of the flexible elongate device disposed within the anatomical structure A.
  • FIG. 1 B is included to give an expanded and more detailed view of a portion of the operating environment 101 disposed within the field of view F.
  • the anatomical structure A may be a lung of the patient P.
  • the flexible elongate device 140 may be inserted into and navigated by the operator toward the region R, for example, for the purpose of investigating or treating a pathology in the region R.
  • the techniques described in the present disclosure can facilitate the navigation process by generating and displaying timely and accurate sensing (e.g., imaging) and detection (e.g., identification) of the flexible elongate device 140.
  • These techniques incorporate location data in one coordinate system obtained using the sensors 144a, b, and location data obtained using the fiducial 142 in another coordinate system.
  • the techniques use the combined data to more accurately and quickly sense changes in the position and/or orientation of at least a portion (e.g., a distal end or another suitable portion) of the flexible elongate device 140 than using data only from the imaging unit 1 10 or the sensing unit 115.
  • the combined data may improve speed, accuracy, reliability and/or safety of a medical procedure.
  • the processing unit 120 may combine the data from the imaging unit 110 and/or the sensing unit 115 to quickly and accurately determine the position, orientation, and/or pose of at least a portion of the flexible elongate device 140 within reference to the anatomical structure A of the patient P.
  • the processing unit 1 0 may generate a graphical user interface (GUI) or update GUI data for display on the display device 130 to aid an operator with the medical procedure.
  • GUI graphical user interface
  • the processing unit 120 may generate data and/or control signals for a control unit of a robotic system configured to manipulate and/or navigate the flexible elongate device 140.
  • the processing unit 120 may be configured to generate, based on the combined imaging and sensing data, one or more alerts.
  • the alerts may include, for example, an alert indicating proximity to the region R, an alert indicating a potential navigation error, and alert indicating that confidence in location of the tip of the flexible elongate device 140 fell below a threshold level, etc.
  • combining data from the imaging unit 110 and the sensing unit 115 entails registering two coordinate systems with respect to each other, as described with reference to FIGS. 2A and B below.
  • FIG. 2A depicts a rigid body 202 in two example coordinate systems: a first coordinate system 204 and a second coordinate system 206.
  • the first coordinate system 204 may correspond to coordinates of the sensors 144a, b obtained or generated by the sensing unit 115.
  • the sensing unit 115 may be a part of a robotic unit configured to actuate and/or manipulate the flexible elongate device 140.
  • the first coordinate system 204 may therefore represent the coordinate system of the robotic unit.
  • the second coordinate system 206 may correspond to coordinates of images obtained or generated by the imaging system 120.
  • the terms “robotic coordinate system” and “sensor (or sensing) coordinate system” may be used interchangeably and in contrast to the term “imaging coordinate system.”
  • the rigid body 202 may represent a portion (e.g., a segment of length) of the flexible elongate device 140. Although the flexible elongate device 140 is flexible, a short portion (e.g., an infinitesimal segment) may be considered rigid for all practical purposes.
  • the depicted rigid body 202 has neither translational nor rotational symmetries.
  • a segment of the flexible elongate device 140 may, at least to an approximate limitation of imaging, have a rotational symmetry (continuous or discrete) along an axis representing a local center line along the long dimension of the flexible elongate device 140.
  • a segment of the flexible elongate device 140 may be substantially indistinguishable (e.g., within an image) from an adjacent segment. That is, the flexible elongate device 140 effectively has a degree of translational symmetry along the axis of the flexible elongate device 140.
  • the techniques of this disclosure are, at least in part, directed to resolving the state (e.g., pose) of the flexible elongate device 140 in view of the aforementioned symmetries.
  • a state (e.g., position and orientation) of the rigid body 202 in three-dimensional space can be described using the first coordinate system 204 and/or the second coordinate system 206.
  • a rigid body without symmetries e.g., body 202
  • the rigid body 202 may have coordinates (x, y, z, 0, , a), where x, y and z are position coordinates of a center 208 of the rigid body 202 along the axes of the first coordinate system 204 with respect to origin O.
  • the position coordinates may be designated for any point within the rigid body 202, or, in fact, any point in a rigidly defined geometric relationship to the body 202.
  • Coordinates 0, and a can describe orientation of the body 202 with the aid of an orientation vector 209, which, in the example of FIG. 2A, originates in the center 208 and goes through the middle of one of the facets of the rigid body 202.
  • 0 may be an elevation angle with respect to z-axis
  • ) may be an azimuthal angle parallel to the xy-plane
  • a may be the angle of rotation of the rigid body 202 around the orientation vector 209.
  • the three orientation coordinates may be roll, pitch and yaw of the rigid body 202 with respect to any suitable reference direction.
  • ), a) of first coordinate system 204 coordinates (x’, y’, z’, 0’, ’, a’) of the second coordinate system 206 describe position (x’, y’, z’) with respect to origin O’ and orientation (O’, ⁇ ]>’, a’) with respect to, for example, z’-axis of the rigid body 202.
  • Registering the first coordinate system 204 with the second coordinate system 206, at least in the vicinity of the rigid body 202 includes finding a mapping (e.g., a transformation, a mathematical relationship, etc.) at least between the coordinates (x, y, z, 0, ⁇ f>, a) and the coordinates (x’, y’, z’, 0’, ’, a’).
  • a mapping e.g., a transformation, a mathematical relationship, etc.
  • Another way of looking at the registration is that the mapping defines a corresponding point (u, v, w) in the vicinity of position (x, y, z) for any point (u’, v’, w’) in the vicinity of position (x’, y’, z’).
  • the first coordinate system 204 and the second coordinate system 206 have the same scale. That is, a unit of length is the same along corresponding axes of the coordinate systems 204 and 206. In other words, the transformation from the first coordinate system 204 to the second coordinate system 206 is rigid. In some examples, however, the mapping from the first coordinate system 204 to the second coordinate system 206 may include one or more scaling factors for the axes. Thus, the mapping may include three translation variables, three rotation variables, and/or three scaling variables. Furthermore, each of the variables may depend on position.
  • each of the coordinate systems 204 and 206 are independently calibrated to have accurate and consistent scaling within a shared operating volume.
  • Coordinate registration process may then be defined in terms of three translation constants and three rotation constants for the shared operating volume.
  • gradual variations in scaling within at least one of the coordinate systems 204 and/or 206 may necessitate use of up to three translation variables and up to three rotation variables, each a function of location within the shared operating volume.
  • FIG. 2B schematically illustrates an example coordinate registration process.
  • a processing unit e.g., the processing unit 120
  • the processing unit may then generate a mapping, M, between the two coordinate systems 204 and 206.
  • the processing unit may be configured to map a new position (u’, v’, w’) within the second coordinate system 206 on to a corresponding position (u, v, w) within the first coordinate system 204. Additionally or alternatively, the processing unit may be configured to map coordinates from the first coordinate system 204 to the second coordinate system 206. The mapping may be only valid in a region around (x, y, z). By collecting rigid body coordinates in the two coordinate systems 204 and 206, the processing unit may extend the validity of mapping over any portion of a shared operating volume of the two coordinate systems 204 and 206.
  • the processing unit may implement the coordinate registration as a linear mapping M of Equation 1 : 0 sin /?' 1 0 1 0 0 cos a (1 ) 0 cos/?. .0 sin a
  • a, p and y are rotation parameters (e.g., roll, pitch and yaw)
  • Sn, S22 and S33 are scaling parameters (which may be unity, as discussed above)
  • di, d2 and ds are displacement factors.
  • the linear mapping may be a function of the input position coordinates (u’, v’, w’).
  • the processing unit may store and/or access a lookup table find an entry for mapping parameters corresponding to the input position coordinates. Because the lookup table can only have a limited number of recorded input coordinates (herein, recorded coordinates), the system may use the entry corresponding to the recorded coordinates nearest to the input coordinates. Alternatively, the system may interpolate mapping parameters corresponding to a set of recorded coordinates near the input coordinates. In other examples, the processing unit may store and/or accessa polynomial, spline, or another suitable fit function relating input coordinates to the mapping parameters.
  • the transformation M may vary as a function of time.
  • vibrations of any structure within the operating environment may shift the coordinate system of an imaging system (e.g., the second coordinate system 206) with respect to the coordinate system of a sensing or a robotic system (e.g., the first coordinate system 204).
  • certain sensing systems may have associated instabilities.
  • a baseline field sensed by passive EM sensors may be distorted by moving metallic objects in the measurement scope of the EM sensing system.
  • the processing unit may update at least portions of the mapping M at suitable time intervals (e.g., every 1 , 2, 5, 10, 20, 50, 200, 500 or any other suitable number of seconds).
  • the processing unit may obtain new images including images of a fiducial (e.g., fiducial 142) from the imaging unit 120 and/or obtain data from the sensing unit 1 15 indicative of coordinates of sensors (e.g., sensors 144a, b) in the sensing coordinate system. Registering the imaging coordinate system to the sensing coordinate system is discussed in more detail below, with reference to FIGS. 4a-e and 5a-d.
  • a fiducial e.g., fiducial 142
  • the sensing unit 1 15 indicative of coordinates of sensors (e.g., sensors 144a, b) in the sensing coordinate system. Registering the imaging coordinate system to the sensing coordinate system is discussed in more detail below, with reference to FIGS. 4a-e and 5a-d.
  • FIGS. 3A-E schematically illustrate example configurations of sensors and one or more fiducial elements disposed at a flexible elongate device. It should be noted, that in each of the FIGS. 3A-E, the shown fiducial elements may be the whole or a part of a single fiducial.
  • FIG. 3A illustrates sections 340a-c of a flexible elongate device. Fiducial elements 342a and b (which may each be the fiducial 142) are disposed, respectively, at the sections 340c and 340b.
  • the fiducial element 342a and b each have a rotationally asymmetrical shape with respect to an axis of the corresponding section (i.e., sections 340b and c) of the flexible elongate device.
  • Example asymmetrical shapes of fiducial elements are described in more detail with reference to FIGS. 4a-d.
  • Sensors 344a and b are disposed, respectively, at the sections 340a and 340b. At the section 340b, the fiducial element 342b and the sensor 344b are disposed substantially at the same location.
  • a processing unit may receive an indication of position of the sensor 344b in a first coordinate system (e.g., sensor or robotic coordinate system), for example, from the sensing unit 115.
  • the processing unit may obtain one or more intraoperative images (e.g., from the imaging unit 110) and determine, based on the intra-operative imaging data, an indication of position, in a second coordinate system, of the fiducial element 342b.
  • the processing unit may register the first coordinate system to the second coordinate system based at least in part on the indications of position of the sensor 344b and the fiducial element 342b.
  • the processing unit may receive the indication of position and orientation of the sensor 344b including 6DOFs and determine (based on one or more intraoperative images) the indication of position and orientation of the fiducial element 342b likewise including 6DOFs.
  • the processing unit may register the first coordinate system to the second coordinate system, at least in the vicinity of the coincident location of the fiducial element 342b and the sensor 344b, solely based on the indications of positions and orientations received of the fiducial element 342b and the sensor 344b.
  • the processing unit may need additional constraints to perform the registration. Such constraints may, for example, be obtained from additional sensors and/or elements of a distributed fiducial.
  • a coincident position of the fiducial element 342b and the sensor 344b may lead to errors in either an indication of position or orientation of the fiducial element 342b or an indication of position or orientation of the sensor 344b.
  • the sensor 344b may interfere with accurately segmenting the fiducial element 342b from an intra-operative image.
  • the fiducial element 342b may interfere with a sensing unit accurately obtaining position or orientation of the sensor 344b.
  • the fiducial element 342b may include a metallic element that distorts a magnetic field sensed by the sensor 344b, which may be an EM sensor. Therefore, it may be advantageous to geometrically/spatially separate fiducial elements from sensors disposed at the flexible elongate device.
  • FIG. 3B illustrates a configuration with a fiducial element 342c and two sensors 344c and d disposed at different locations along a segment 340d of a flexible elongate device, with the fiducial element 342c disposed between the sensors 344c and d along the segment 340d.
  • the sensor 344c is disposed at a separation distance Li from the fiducial element 342c and the sensor 344d is disposed at a separation distance L 2 from the fiducial element 342c.
  • the separation distances Li and L 2 need not be the same, and each may be 2, 5, 10, 20, 50 mm or any other suitable length. Furthermore, in some examples one of the separation distances Li and L 2 may be zero, leading to a location overlap between the fiducial element 342c and one of the sensors 344c and d.
  • neither of the sensors 344c and d is configured to provide six
  • the sensors 344c and d may provide the six DOFs.
  • one of the sensors 344c and d may provide at least three DOFs while the other may provide at least four DOFs, one being a rotation angle with respect to the segment 340d.
  • the processing unit may assume that the segment 340d maintains rigidity between the two sensors 344c and d.
  • the position in the first coordinate system corresponding to the position of the fiducial element 342c may be interpolated (by the processing unit) from the positions of the sensors 344c and d.
  • the interpolation may assume rigidity of the segment 340d between the sensors 344c and d.
  • the processing unit needs not assume rigidity between the sensors 344c and d.
  • FIG. 30 illustrates a configuration with a fiducial element 342d and two sensors 344e and f disposed at different locations along a segment 340d of a flexible elongate device, with the fiducial element 342d disposed between the sensors 344e and f along the segment 340e.
  • the configuration is analogous to the configuration in FIG. 3B, but with a bend in the segment 340e.
  • the processing unit may determine, based on the obtained indications of positions of the sensors 344e and f, a separation distance r. Based on the separation distance r and the distance between the sensors 344e and f (e.g., Li + l_ 2 , if the configuration in FIG. 3C is the configuration in FIG.
  • the processing unit may estimate the bend angle. To that end, the processing unit may assume that the bend follows a circular arc. The processing unit may, therefore, interpolate along the arc the position of the fiducial element 342d in the sensing coordinate system, aiding the registration between the sensing coordinate system and the image coordinate system.
  • the processing unit may obtain orientation coordinates (which need not include rotation around the axes of the segment 340e) for the sensors 344e and f, as indicated by lines 349a and b.
  • the processing unit may compute, in the sensing coordinate system, whether the lines 349a and b intersect or, at least, nearly intersect to evaluate validity of the assumption of a smooth bend in the section 340e.
  • the processing unit may generate an alert indicating a level of uncertainty in the registration between the sensing and the image in coordinate systems.
  • FIG. 3D illustrates a configuration with a multi-element portion of a fiducial 342e and two sensors 344g and h disposed at different locations along a segment 340f of a flexible elongate device.
  • the fiducial 342e is distributed along the length of the segment 340f. That is, the fiducial 342e includes elements at two distinct positions along the length of the segment 340f.
  • the processing unit may determine, based on intra-operative imaging data, indications of the two positions of the fiducial 342e.
  • the processing unit may register the sensing and imaging coordinate systems with each other based at least in part on the indications of the two positions of the fiducial 342e.
  • the sensor 344h is disposed between the two elements of the fiducial 342e, while the sensor 344g is not disposed between the two elements of the fiducial 342e.
  • the processing unit may therefore uniquely identify the sensor 344h, distinguishing it from the sensor 344g.
  • fiducial elements can encode locations of sensors along a flexible elongate device.
  • fiducial elements may include different geometrical features, as illustrated did by the fiducial 342e of FIG. 3D.
  • distributed elements of a fiducial may improve the accuracy of registration between two coordinate systems by providing additional position and/or orientation information.
  • FIG. 3E illustrates a configuration with a multi-element portion of a fiducial 342f and two sensors 344i and j disposed at different locations along a segment 340g of a flexible elongate device.
  • the distributed fiducial 342f is indicative of two distinct distances l_3 and l_4 along the segment 340g by virtue of having three unevenly spaced elements. Any number of unevenly spaced elements of a fiducial may encode longitudinal location along a flexible elongate device, may aid in identifying evenly spaced sensors along the flexible elongate device, and, generally, improve accuracy in the registration of the sensing and the imaging coordinate systems.
  • the processing unit may use data indicative of positions of the elements of the fiducial 342f separated by L 3 to interpolate the position of the sensor 344i in the imaging system coordinates.
  • the interpolation method may be similar to the interpolation methods discussed with reference to FIGS. 3B and C, with the coordinate systems and the roles of fiducial elements and sensors reversed.
  • FIGS. 4A-D schematically illustrate example configurations of fiducials that have rotationally asymmetrical shapes with respect to an axis of a flexible elongate body of a device and are disposed at segments of the flexible elongate body of the device.
  • FIG. 4A illustrates a segment 440a of a device having a flexible elongate body with an axis 445 and a fiducial element 442a with a rotationally asymmetrical elliptical shape with respect to the axis 445. It should be noted that, for a curving flexible elongate device, an axis orientation changes along the length of the device.
  • a fiducial element 442b includes a ring and an attached segment parallel with the axis of a segment 440b.
  • the fiducial element 442b has an L-shaped or a T-shaped projection when viewed orthogonally to the axis of the segment 440b.
  • a fiducial element 442c includes an open ring disposed at a segment 440c. The opening may subtend any suitable angular extent of the ring.
  • a fiducial element 442d disposed at a segment 440d has a ring element and a dot or a ball element detached from the ring element.
  • Fiducial elements 442a-d of the FIGS. 4A-D are examples of fiducial elements which, similarly to the rigid body 202 in FIG. 2, have uniquely defined six DOFs of position and orientation in space. It should be noted that a fiducial having an asymmetrical shape with respect to an axis on a flexible elongate body and at a certain position along the length of the flexible elongate body may include many other configurations of fiducial elements. For example, portions of the fiducial elements 442a-d (e.g., rings, open rings, ellipses, dots, and/or lines) may be reconfigured into many alternative fiducial elements without introducing rotational symmetries.
  • portions of the fiducial elements 442a-d e.g., rings, open rings, ellipses, dots, and/or lines
  • FIGS. 5A and 5B schematically illustrate example configurations of a fiducial removable attached to a flexible elongate device.
  • a fiducial element 542a may be fixedly attached to a sleeve 545, which, in turn, may be removably attached to a flexible elongate device 540a.
  • portions of a fiducial element 542b are fixedly attached to stylets 546a and b, which may, in turn, be removably attached (e.g., inserted in or through) to a flexible elongate device.
  • the sleeve 545 or the stylets 546a may be mechanically or optically registered to the corresponding flexible elongate device 540a or b.
  • the registration of the removable sleeve 545 and/or the removable stylets 546a and b may be with respect to the distal end or any other suitable point of the corresponding flexible elongate device 540a or 540b.
  • FIG. 6A is a block diagram of a method 600a for implementing the techniques of this disclosure.
  • the method 600a includes receiving, by one or more processors (e.g., of the processing unit 120), indications of positions, in a first coordinate system, of a first sensor and a second sensor (e.g., sensors 144a and b, 344c and d, 344e and f, 344g and h, or 344 i and j) disposed at a flexible elongate device (e.g., device 140, or any device represented by segments 340a-g, 440a-d, or 540a, b).
  • a flexible elongate device e.g., device 140, or any device represented by segments 340a-g, 440a-d, or 540a, b.
  • the method 600a includes determining, based on intra-operative imaging data and by the one or more processors, an indication of position, in a second coordinate system, of a fiducial of the flexible elongate device.
  • the fiducial may be disposed at the flexible elongate device at a separation distance (e.g., Li) from the first sensor (e.g., sensor 344c).
  • the method 600a may include, more generally, determining, based on intra-operative images, any suitable number of DOFs for any fiducial element disposed at the flexible elongate device using any of the techniques discussed above with reference to FIGS. 2A and B, 3A-E, and 4A-D.
  • a suitably configured imaging unit may generate the intraoperative imaging data.
  • the intra-operative imaging data may include, for example, cone-bean tomography data.
  • the imaging unit may include a C-arm CBCT imaging system.
  • the intra-operative imaging data may include fluoroscopy X-ray data (e.g., generated by a C-arm X-ray device), tomosynthesis data (reconstructed from 2D x-ray images, such as fluoroscopy images, into a 3D volume), and/or MRI data.
  • the intraoperative imaging data may be obtained using thermography, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging or any other suitable imaging technique.
  • the method 600a may include registering, by the one or more processors, the first coordinate system to the second coordinate system based at least in part on the indications of positions of the first sensor and the second sensor in the first coordinate system, the indication of position of the fiducial in the second coordinate system, and the separation distance.
  • the method may use, for example, any of the techniques discussed above with reference to FIGS. 2A and B, 3A-E, and 4A-D.
  • the registration may be based on any suitable number of sensors (e.g., 2, 3, 4, 5, etc.).
  • the sensors may have specified positions along the flexible elongate device, as described above.
  • specified positions of sensors along the flexible elongate device relative to the distal tip of the device may indicate separation distances between any pairs of sensors.
  • the separation distances along the flexible elongate device may effectively encode or identify the sensors.
  • the fiducial may be at least in part integrated into the sensors. That is, at least a portion of one or more sensors may be identifiable and localizable within intraoperative imaging data.
  • the system may use three sensors with 3 DOF each and match the positions of each of the sensors in the sensor coordinate system to the corresponding positions in the intra-operative imaging coordinate system.
  • a portion of the flexible elongate device with the three sensors may be segmented from the intraoperative image and the positions of the sensors may be identified using the fiducial markings.
  • the registration may be accomplished using an iterative closest point (ICP) or another suitable algorithm.
  • ICP iterative closest point
  • one 5 DOF sensor and one 3 DOF sensor at known positions with respect to the tip of the flexible elongate device may be matched with the pose of the corresponding segments along the flexible elongate device extracted from the intra-operative images.
  • a portion of the fiducial may be poorly resolved in intra-operative imaging data.
  • the fiducial may include redundant structures at different points along the flexible elongate device, fully defining image coordinates around those points.
  • the system may compute several coordinate mappings between the first coordinate system and the second coordinate system. The system may then generate the coordinate registration based on the weighted (e.g., based on quality or confidence in individual coordinate mappings) mapping functions or generate a space-varying (e.g., non-rigid) registration between the first and the second coordinate system.
  • additional sensors and fiducial elements may improve robustness of the registration methods in view of noisy or poorly resolved intra-operative imaging and/or sensor data.
  • the method 600a may optionally include generating, based at least in part on registering the first coordinate system to the second coordinate system on a display unit and by the one or more processors, a graphical user interface displaying positions of the first sensor, the second sensor and the fiducial in a joint coordinate system.
  • Generating the graphical user interface may include initially generating the graphical user interface, and/or updating the graphical user interface from a previously-generated graphical user interface.
  • the method 600a may further include generating a metric of uncertainty of registration, comparing the metric to a threshold, and generating an alert when the metric of uncertainly exceeds the threshold. Additionally or alternatively, the method 600a may generate one or more alerts indicative of position, orientation, and/or pose of any segment of the flexible elongate device.
  • the method 600a may be implemented by one or more processors (e.g., of the processing unit 120). Instructions for implementing the method 600a by the one or more processors may be stored on tangible, non-transitory, computer readable medium (CRM).
  • processors e.g., of the processing unit 120. Instructions for implementing the method 600a by the one or more processors may be stored on tangible, non-transitory, computer readable medium (CRM).
  • CRM computer readable medium
  • FIG. 6B is a block diagram of an alternative method 600b for implementing the techniques of this disclosure.
  • the method 600b includes receiving, by one or more processors (e.g., of the processing unit 120), an indication of a position, in a first coordinate system, of a position sensor (e.g., any one of sensors 144a and b, 344c and d, 344e and f, 344g and h, or 344i and j) disposed at a flexible elongate body of a medical device (e.g., device 140, or any device represented by segments 340a-g, 440a-d, or 540a, b).
  • a position sensor e.g., any one of sensors 144a and b, 344c and d, 344e and f, 344g and h, or 344i and j
  • a medical device e.g., device 140, or any device represented by segments 340a-g, 440a-d, or 540
  • indications of a position of one or more additional position sensors may be received at block 610b.
  • the method 600b includes determining, based on intra-operative imaging data and by the one or more processors, an indication of a position, in a second coordinate system, of a fiducial disposed at the flexible elongate body and having a rotationally asymmetrical shape with respect to the axis of the flexible elongate device.
  • the fiducial may have multiple rotationally asymmetrical elements to improve robustness of coordinate registration in view of the possibility that portions of the fiducial may be poorly resolved in imaging data.
  • the fiducial may be disposed at the flexible elongate body at a separation distance from the sensor (e.g., any one of sensors 144a and b, 344c and d, 344e and f, 344g and h, or 344 i and j).
  • the method 600b may include, more generally, determining, based on intra-operative images, any suitable number of DOFs for any fiducial element disposed at the flexible elongate device using any of the techniques discussed above with reference to FIGS. 2A and B, 3A-E, and 4A-D.
  • the intra-operative imaging data may be obtained as described above, for example with reference to FIGS. 1 and 6A.
  • the method 600b may include registering, by the one or more processors, the first coordinate system to the second coordinate system based at least in part on the indication of the position of the position sensor and the indication of the position of the fiducial in the second coordinate system.
  • the fiducial may include two or more portions physically detached from each other and disposed along the length of the flexible elongate device.
  • the method may use, for example, any of the techniques discussed above with reference to FIGS. 2A and B, 3A-E, and 4A-D.
  • a 6 DOF sensor may be registered with a fiducial that has a rotationally asymmetrical shape with respect to the flexible elongate device.
  • the fiducial may thereby provide 6 DOF at a point along the flexible elongate device.
  • the fiducial location is sufficiently close to the flexible elongate device that (e.g., assuming rigidity of the flexible elongate device between the fiducial and the 6 DOF sensor) the position and orientation of the sensor in the first coordinate system and the position and orientation of the fiducial in the second coordinate system may enable registration of the two coordinate systems at least in the vicinity of the sensor.
  • the method 600b may optionally include generating, based at least in part on registering the first coordinate system to the second coordinate system, on a display unit and by the one or more processors, a graphical user interface displaying the position of the sensor and the position of the fiducial in a joint coordinate system.
  • Generating the graphical user interface may include initially generating the graphical user interface, and/or updating the graphical user interface from a previously-generated graphical user interface.
  • the method 600b may further include generating a metric of uncertainty of registration, comparing the metric to a threshold, and generating an alert when the metric of uncertainly exceeds the threshold. Additionally or alternatively, the method 600b may generate one or more alerts indicative of position, orientation, and/or pose of any segment of the flexible elongate device.
  • the method 600b may be implemented by one or more processors (e.g., of the processing unit 120). Instructions for implementing the method 600b by the one or more processors may be stored on tangible, non-transitory, computer readable medium (CRM).
  • processors e.g., of the processing unit 120. Instructions for implementing the method 600b by the one or more processors may be stored on tangible, non-transitory, computer readable medium (CRM).
  • CRM computer readable medium
  • FIGS. 7-9B depict diagrams of a medical system that may be used for manipulating a medical instrument that includes a flexible elongate device according to any of the methods and systems described above, in some examples.
  • each reference above to the “system” may refer to a system (e.g., system 700) discussed below, or to a subsystem thereof.
  • FIG. 7 is a simplified diagram of a medical system 700 according to some examples.
  • the medical system 700 may include at least portions of the system 100 described with reference to FIG. 1 .
  • the medical system 700 may be suitable for use in, for example, surgical, diagnostic (e.g., biopsy), or therapeutic (e.g., ablation, electroporation, etc.) procedures. While some examples are provided herein with respect to such procedures, any reference to medical or surgical instruments and medical or surgical methods is non-limiting.
  • the systems, instruments, and methods described herein may be used for animals, human cadavers, animal cadavers, portions of human or animal anatomy, non-surgical diagnosis, as well as for industrial systems, general or special purpose robotic systems, general or special purpose teleoperational systems, or robotic medical systems.
  • medical system 700 may include a manipulator assembly 702 that controls the operation of a medical instrument 704 in performing various procedures on a patient (e.g., patient P on table T, as in FIG. 1).
  • the medical instrument 704 may include the flexible elongated device 140 of FIG. 1.
  • Medical instrument 704 may extend into an internal site within the body of patient P via an opening in the body of patient P.
  • the manipulator assembly 702 may be teleoperated, non-teleoperated, or a hybrid teleoperated and non-teleoperated assembly with one or more degrees of freedom of motion that may be motorized and/or one or more degrees of freedom of motion that may be non-motorized (e.g., manually operated).
  • the manipulator assembly 702 may be mounted to and/or positioned near patient table T.
  • a master assembly 706 allows an operator O (e.g., a surgeon, a clinician, a physician, or other user, as described above) to control the manipulator assembly 702.
  • the master assembly 706 allows the operator O to view the procedural site or other graphical or informational displays.
  • the manipulator assembly 702 may be excluded from the medical system 700 and the instrument 704 may be controlled directly by the operator O.
  • the manipulator assembly 702 may be manually controlled by the operator O. Direct operator control may include various handles and operator interfaces for hand-held operation of the instrument 704.
  • the master assembly 706 may be located at a surgeon’s console which is in proximity to (e.g., in the same room as) the patient table T on which patient P is located, such as at the side of the patient table T. In some examples, the master assembly 706 is remote from the patient table T, such as in in a different room or a different building from the patient table T.
  • the master assembly 706 may include one or more control devices for controlling the manipulator assembly 702.
  • the control devices may include any number of a variety of input devices, such as joysticks, trackballs, scroll wheels, directional pads, buttons, data gloves, trigger-guns, handoperated controllers, voice recognition devices, motion or presence sensors, and/or the like.
  • the manipulator assembly 702 supports the medical instrument 704 and may include a kinematic structure of links that provide a set-up structure.
  • the links may include one or more non-servo controlled links (e.g., one or more links that may be manually positioned and locked in place) and/or one or more servo controlled links (e.g., one or more links that may be controlled in response to commands, such as from a control system 712).
  • the manipulator assembly 702 may include a plurality of actuators (e.g., motors) that drive inputs on the medical instrument 704 in response to commands, such as from the control system 712.
  • the actuators may include drive systems that move the medical instrument 704 in various ways when coupled to the medical instrument 704.
  • one or more actuators may advance medical instrument 704 into a naturally or surgically created anatomic orifice.
  • Actuators may control articulation of the medical instrument 704, such as by moving the distal end (or any other portion) of medical instrument 704 in multiple degrees of freedom.
  • degrees of freedom may include three degrees of linear motion (e.g., linear motion along the X, Y, Z Cartesian axes) and in three degrees of rotational motion (e.g., rotation about the X, Y, Z Cartesian axes).
  • One or more actuators may control rotation of the medical instrument about a longitudinal axis.
  • Actuators can also be used to move an articulable end effector of medical instrument 704, such as for grasping tissue in the jaws of a biopsy device and/or the like, or may be used to move or otherwise control tools (e.g., imaging tools, ablation tools, biopsy tools, electroporation tools, etc.) that are inserted within the medical instrument 704.
  • medical instrument 704 such as for grasping tissue in the jaws of a biopsy device and/or the like
  • move or otherwise control tools e.g., imaging tools, ablation tools, biopsy tools, electroporation tools, etc.
  • the control system 704 may include at least portions of the processing unit 120. Additionally or alternatively, the control system 704 may be in communicative connection with the processing unit 120. In some examples, the output of the processing unit 120 according to the techniques described above may cause the control system 704 to autonomously (without input from the operator O) control certain movements of the instrument 104.
  • the medical system 700 may include a sensor system 708 (which may include at least a portion of the sensor unit 115) with one or more sub-systems for receiving information about the manipulator assembly 702 and/or the medical instrument 704.
  • Such sub-systems may include a position sensor system (e.g., that uses electromagnetic (EM) sensors or other types of sensors that detect position or location); a shape sensor system for determining the position, orientation, speed, velocity, pose, and/or shape of a distal end and/or of one or more segments along a flexible body of the medical instrument 704; a visualization system (e.g., using a color imaging device, an infrared imaging device, an ultrasound imaging device, an x-ray imaging device, a fluoroscopic imaging device, a computed tomography (CT) imaging device, a magnetic resonance imaging (MRI) imaging device, or some other type of imaging device or imaging unit 1 10) for capturing images, such as from the distal end of medical instrument 704 or from some other location; and/or actuator position sensors
  • the positions and orientations of sensors in the sensor system 708 may be determined in the sensor coordinate system.
  • the sensor coordinate system is integrated with or identical to the coordinate system of the manipulator assembly 702.
  • the medical system 700 may include a display system 710 for displaying an image or representation of the procedural site and the medical instrument 704.
  • Display system 710 and master assembly 706 may be oriented so physician O can control medical instrument 704 and master assembly 706 with the perception of telepresence.
  • the display system 710 may include at least portions of the display unit 130.
  • the medical instrument 704 may include a visualization system, which may include an image capture assembly that records a concurrent or real-time image of a procedural site and provides the image to the operator O through one or more displays of display system 710.
  • the image capture assembly may include various types of imaging devices.
  • the concurrent image may be, for example, a two-dimensional image or a three- dimensional image captured by an endoscope positioned within the anatomical procedural site.
  • the visualization system may obtain intra-operative images in image system coordinates, distinct from the sensor system coordinates.
  • the visualization system may include endoscopic components that may be integrally or removably coupled to medical instrument 704. Additionally or alternatively, a separate endoscope, attached to a separate manipulator assembly, may be used with medical instrument 704 to image the procedural site.
  • the visualization system may be implemented as hardware, firmware, software or a combination thereof which interact with or are otherwise executed by one or more computer processors, such as of the control system 712.
  • Display system 710 may also display an image of the procedural site and medical instruments, which may be captured by the visualization system.
  • the medical system 700 provides a perception of telepresence to the operator O.
  • images captured by an imaging device at a distal portion of the medical instrument 704 may be presented by the display system 710 to provide the perception of being at the distal portion of the medical instrument 704 to the operator O.
  • the input to the master assembly 706 provided by the operator O may move the distal portion of the medical instrument 704 in a manner that corresponds with the nature of the input (e.g., distal tip turns right when a trackball is rolled to the right) and results in corresponding change to the perspective of the images captured by the imaging device at the distal portion of the medical instrument 704.
  • the perception of telepresence for the operator O is maintained as the medical instrument 704 is moved using the master assembly 706.
  • the operator O can manipulate the medical instrument 704 and hand controls of the master assembly 706 as if viewing the workspace in substantially true presence, simulating the experience of an operator that is physically manipulating the medical instrument 704 from within the patient anatomy.
  • the display system 710 may present virtual images of a procedural site that are created using image data recorded pre-operatively (e.g., prior to the procedure performed by the medical instrument system 200) or intra-operatively (e.g., concurrent with the procedure performed by the medical instrument system 200), such as image data created using computed tomography (CT), magnetic resonance imaging (MRI), positron emission tomography (PET), fluoroscopy, thermography, ultrasound, optical coherence tomography (OCT), thermal imaging, impedance imaging, laser imaging, nanotube X-ray imaging, and/or the like.
  • CT computed tomography
  • MRI magnetic resonance imaging
  • PET positron emission tomography
  • fluoroscopy thermography
  • ultrasound ultrasound
  • OCT optical coherence tomography
  • thermal imaging impedance imaging
  • laser imaging laser imaging
  • nanotube X-ray imaging and/or the like.
  • the virtual images may include two-dimensional, three-dimensional, or higher-dimensional (e.g., including, for example, time based
  • display system 710 may display a virtual image that is generated based on tracking the location of medical instrument 704.
  • the tracked location of the medical instrument 704 may be registered (e.g., dynamically referenced) with the model generated using the pre-operative or intra-operative images, with different portions of the model correspond with different locations of the patient anatomy.
  • the registration is used to determine portions of the model corresponding with the location and/or perspective of the medical instrument 704 and virtual images are generated using the determined portions of the model. This may be done to present the operator O with virtual images of the internal procedural site from viewpoints of medical instrument 704 that correspond with the tracked locations of the medical instrument 704.
  • the display system 710 may include the display unit 130 and may display images including the position, orientation, and/or pose of the medical instrument 704 in a joint coordinate system based on registering the sensor coordinate system with the imaging coordinate system according to the techniques described above with reference to FIGS. 2A-6.
  • the medical system 700 may also include the control system 712, which may include processing circuitry (e.g., the processing unit 120) that implements the some or all of the methods or functionality discussed herein.
  • the control system 712 may include at least one memory and at least one processor for controlling the operations of the manipulator assembly 702, the medical instrument 704, the master assembly 706, the sensor system 708, and/or the display system 710.
  • Control system 712 may include instructions (e.g., a non-transitory machine-readable medium storing the instructions) that when executed by the at least one processor, configures the one or more processors to implement some or all of the methods or functionality discussed herein. While the control system 712 is shown as a single block in FIG.
  • control system 712 may include two or more separate data processing circuits with one portion of the processing being performed at the manipulator assembly 702, another portion of the processing being performed at the master assembly 706, and/or the like.
  • control system 71 may include other types of processing circuitry, such as application-specific integrated circuits (ASICs) and/or field-programmable gate array (FPGAs).
  • ASICs application-specific integrated circuits
  • FPGAs field-programmable gate array
  • the control system 712 may be implemented using hardware, firmware, software, or a combination thereof.
  • control system 712 may receive feedback from the medical instrument 704, such as force and/or torque feedback. Responsive to the feedback, the control system 712 may transmit signals to the master assembly 706. In some examples, the control system 712 may transmit signals instructing one or more actuators of the manipulator assembly 702 to move the medical instrument 704. In some examples, the control system 712 may transmit informational displays regarding the feedback to the display system 710 for presentation or perform other types of actions based on the feedback.
  • the control system 712 may include a virtual visualization system to provide navigation assistance to operator O when controlling the medical instrument 704 during an image-guided medical procedure.
  • Virtual navigation using the virtual visualization system may be based upon an acquired pre-operative or intra-operative dataset of anatomic passageways of the patient P.
  • the control system 712 or a separate computing device may convert the recorded images, using programmed instructions alone or in combination with operator inputs, into a model of the patient anatomy.
  • the model may include a segmented two-dimensional or three-dimensional composite representation of a partial or an entire anatomic organ or anatomic region.
  • An image data set may be associated with the composite representation.
  • the virtual visualization system may obtain sensor data from the sensor system 708 that is used to compute an (e.g., approximate) location of the medical instrument 704 with respect to the anatomy of patient P.
  • the sensor system 708 may be used to register and display the medical instrument 704 together with the pre-operatively or intra-operatively recorded images.
  • PCT Publication WO 2016/191298 published December 1 , 2016 and titled “Systems and Methods of Registration for Image Guided Surgery”
  • the registration may be based on the techniques discussed above with reference to FIGS. 2A-6.
  • the sensor system 708 may be used to compute the (e.g., approximate) location of the medical instrument 704 with respect to the anatomy of patient P.
  • the location can be used to produce both macro-level (e.g., external) tracking images of the anatomy of patient P and virtual internal images of the anatomy of patient P.
  • the system may include one or more electromagnetic (EM) sensors, fiber optic sensors, and/or other sensors to register and display a medical instrument together with pre-operatively recorded medical images.
  • EM electromagnetic
  • Medical system 700 may further include operations and support systems (not shown) such as illumination systems, steering control systems, irrigation systems, and/or suction systems.
  • the medical system 700 may include more than one manipulator assembly and/or more than one master assembly.
  • the exact number of manipulator assemblies may depend on the medical procedure and space constraints within the procedural room, among other factors. Multiple master assemblies may be co-located or they may be positioned in separate locations. Multiple master assemblies may allow more than one operator to control one or more manipulator assemblies in various combinations.
  • FIG. 8A is a simplified diagram of a medical instrument system 800 according to some examples.
  • the medical instrument system 800 includes a flexible elongate device 802 (e.g., device 140), also referred to as elongate device 802, a drive unit 804, and a medical tool 826 that collectively is an example of a medical instrument 704 of a medical system 700.
  • the medical system 700 may be a teleoperated system, a non-teleoperated system, or a hybrid teleoperated and non-teleoperated system, as explained with reference to FIG. 7.
  • a visualization system 831 , tracking system 830, and navigation system 832 are also shown in FIG. 8A and are example components of the control system 712 of the medical system 700.
  • the medical instrument system 800 may be used for non-teleoperational exploratory procedures or in procedures involving traditional manually operated medical instruments, such as endoscopy.
  • the medical instrument system 800 may be used to gather (e.g., measure) a set of data points corresponding to locations within anatomic passageways of a patient, such as patient P.
  • the elongate device 802 is coupled to the drive unit 804.
  • the elongate device 802 includes a channel 821 through which the medical tool 826 may be inserted.
  • the elongate device 802 navigates within patient anatomy to deliver the medical tool 826 to a procedural site.
  • the elongate device 802 includes a flexible body 816 having a proximal end 817 and a distal end 818.
  • the flexible body 816 may have an approximately 3 mm outer diameter. Other flexible body outer diameters may be larger or smaller.
  • Medical instrument system 800 may include the tracking system 830 for determining the position, orientation, speed, velocity, pose, and/or shape of the flexible body 816 at the distal end 818 and/or of one or more segments 824 along flexible body 816, as will be described in further detail below.
  • the tracking system 830 may include one or more sensors and/or imaging devices.
  • the flexible body 816 such as the length between the distal end 818 and the proximal end 817, may include multiple segments 824.
  • the tracking system 830 may be implemented using hardware, firmware, software, or a combination thereof. In some examples, the tracking system 830 is part of control system 712 shown in FIG. 7.
  • the tracking system 830 may implement at least some of the techniques described with reference to FIGS. 1 A-6, and, to that end, may include at least portions of or be in communicative connection with the processing unit 120 of FIG. 1 A.
  • T racking system 830 may track the distal end 818 and/or one or more of the segments 824 of the flexible body 816 using a shape sensor 822.
  • the shape sensor 822 may be omitted.
  • the shape sensor 822 may include an optical fiber aligned with the flexible body 816 (e.g., provided within an interior channel of the flexibly body 816 or mounted externally along the flexible body 816).
  • the optical fiber may have a diameter of approximately 800 pm. In other examples, the diameter may be larger or smaller.
  • the optical fiber of the shape sensor 822 may form a fiber optic bend sensor for determining the shape of flexible body 816.
  • Optical fibers including Fiber Bragg Gratings may be used to provide strain measurements in structures in one or more dimensions.
  • FBGs Fiber Bragg Gratings
  • Various systems and methods for monitoring the shape and relative position of an optical fiber in three dimensions are described in U.S. Patent Application Publication No. 2006/0013523 (filed July 13, 2005 and titled “Fiber optic position and shape sensing device and method relating thereto”); U.S. Patent No. 7,772,541 (filed on March 12, 2008 and titled “Fiber Optic Position and/or Shape Sensing Based on Rayleigh Scatter”); and U.S. Patent No. 8,773,650 (filed on Sept.
  • Sensors in some examples may employ other suitable strain sensing techniques, such as Rayleigh scattering, Raman scattering, Brillouin scattering, and Fluorescence scattering.
  • the shape of the flexible body 816 may be determined using other techniques. For example, a history of the position and/or pose of the distal end 818 of the flexible body 816 can be used to reconstruct the shape of flexible body 816 over an interval of time (e.g., as the flexible body 816 is advanced or retracted within a patient anatomy).
  • the tracking system 830 may alternatively and/or additionally track the distal end 818 of the flexible body 816 using a position sensor system 820.
  • Position sensor system 820 may be a component of an EM sensor system with the position sensor system 820 including one or more position sensors.
  • the position sensor system 820 is shown as being near the distal end 818 of the flexible body 816 to track the distal end 818, the number and location of the position sensors of the position sensor system 820 may vary to track different regions along the flexible body 816.
  • the position sensors include conductive coils that may be subjected to an externally generated electromagnetic field. Each coil of position sensor system 820 may produce an induced electrical signal having characteristics that depend on the position and orientation of the coil relative to the externally generated electromagnetic field.
  • the position sensor system 820 may measure one or more position coordinates and/or one or more orientation angles associated with one or more portions of flexible body 816.
  • the position sensor system 820 may be configured and positioned to measure six degrees of freedom, e.g., three position coordinates X, Y, Z and three orientation angles indicating pitch, yaw, and roll of a base point. In some examples, the position sensor system 820 may be configured and positioned to measure five degrees of freedom, e.g., three position coordinates X, Y, Z and two orientation angles indicating pitch and yaw of a base point. Further description of a position sensor system, which may be applicable in some examples, is provided in U.S. Patent No. 6,380,732 (filed August 11 , 1999 and titled “Six-Degree of Freedom T racking System Having a Passive Transponder on the Object Being Tracked”), which is incorporated by reference herein in its entirety.
  • a processing unit may enhance the accuracy of positions obtained by the sensor system 820 by combing data obtained by the sensor system 820 with data obtained by an external imaging system (e.g., by way of the imaging unit 110) according to the techniques of this disclosure described above with reference to FIGS. 2A-6.
  • the tracking system 830 may alternately and/or additionally rely on a collection of pose, position, and/or orientation data stored for a point of an elongate device 802 and/or medical tool 826 captured during one or more cycles of alternating motion, such as breathing. This stored data may be used to develop shape information about the flexible body 816.
  • a series of position sensors such as EM sensors like the sensors in position sensor 820 or some other type of position sensors may be positioned along the flexible body 816 and used for shape sensing.
  • a history of data from one or more of these position sensors taken during a procedure may be used to represent the shape of elongate device 802, particularly if an anatomic passageway is generally static.
  • FIG. 8B is a simplified diagram of the medical tool 826 within the elongate device 802 according to some examples.
  • the flexible body 816 of the elongate device 802 may include the channel 821 sized and shaped to receive the medical tool 826.
  • the medical tool 826 may be used for procedures such as diagnostics, imaging, surgery, biopsy, ablation, illumination, irrigation, suction, electroporation, etc.
  • Medical tool 826 can be deployed through channel 821 of flexible body 816 and operated at a procedural site within the anatomy.
  • Medical instrument 826 may be, for example, an image capture probe, a biopsy tool (e.g., a needle, grasper, brush, etc.), an ablation tool (e.g., a laser ablation tool, radio frequency (RF) ablation tool, cryoablation tool, thermal ablation tool, heated liquid ablation tool, etc.), an electroporation tool, and/or another surgical, diagnostic, or therapeutic tool.
  • the medical tool 826 may include an end effector having a single working member such as a scalpel, a blunt blade, an optical fiber, an electrode, and/or the like.
  • Other end types of end effectors may include, for example, forceps, graspers, scissors, staplers, clip appliers, and/or the like.
  • Other end effectors may further include electrically activated end effectors such as electrosurgical electrodes, transducers, sensors, and/or the like.
  • the medical tool 826 may be a biopsy tool used to remove sample tissue or a sampling of cells from a target anatomic location.
  • the biopsy tool is a flexible needle.
  • the biopsy tool may further include a sheath that can surround the flexible needle to protect the needle and interior surface of the channel 821 when the biopsy tool is within the channel 821 .
  • the medical tool 826 may be an image capture probe that includes a distal portion with a stereoscopic or monoscopic camera that may be placed at or near the distal end 818 of flexible body 816 for capturing images (e.g., still or video images).
  • the captured images may be processed by the visualization system 831 for display and/or provided to the tracking system 830 to support tracking of the distal end 818 of the flexible body 816 and/or one or more of the segments 824 of the flexible body 816.
  • the image capture probe may include a cable for transmitting the captured image data that is coupled to an imaging device at the distal portion of the image capture probe.
  • the image capture probe may include a fiber-optic bundle, such as a fiberscope, that couples to a more proximal imaging device of the visualization system 831 .
  • the image capture probe may be single-spectral or multi-spectral, for example, capturing image data in one or more of the visible, near-infrared, infrared, and/or ultraviolet spectrums.
  • the image capture probe may also include one or more light emitters that provide illumination to facilitate image capture.
  • the image capture probe may use ultrasound, x-ray, fluoroscopy, CT, MRI, or other types of imaging technology.
  • the image capture probe is inserted within the flexible body 816 of the elongate device 802 to facilitate visual navigation of the elongate device 802 to a procedural site and then is replaced within the flexible body 816 with another type of medical tool 826 that performs the procedure.
  • the image capture probe may be within the flexible body 816 of the elongate device 802 along with another type of medical tool 826 to facilitate simultaneous image capture and tissue intervention, such as within the same channel 821 or in separate channels.
  • a medical tool 826 may be advanced from the opening of the channel 821 to perform the procedure (or some other functionality) and then retracted back into the channel 821 when the procedure is complete.
  • the medical tool 826 may be removed from the proximal end 817 of the flexible body 816 or from another optional instrument port (not shown) along flexible body 816.
  • the elongate device 802 may include integrated imaging capability rather than utilize a removable image capture probe.
  • the imaging device (or fiber-optic bundle) and the light emitters may be located at the distal end 818 of the elongate device 802.
  • the flexible body 815 may include one or more dedicated channels that carry the cable(s) and/or optical fiber(s) between the distal end 818 and the visualization system 831 .
  • the medical instrument system 800 can perform simultaneous imaging and tool operations.
  • the medical tool 826 is capable of controllable articulation.
  • the medical tool 826 may house cables (which may also be referred to as pull wires), linkages, or other actuation controls (not shown) that extend between its proximal and distal ends to controllably bend the distal end of medical tool 826, such as discussed herein for the flexible elongate device 802.
  • the medical tool 826 may be coupled to a drive unit 804 and the manipulator assembly 702.
  • the elongate device 802 may be excluded from the medical instrument system 800 or may be a flexible device that does not have controllable articulation. Steerable instruments or tools, applicable in some examples, are further described in detail in U.S. Patent No.
  • the flexible body 816 of the elongate device 802 may also or alternatively house cables, linkages, or other steering controls (not shown) that extend between the drive unit 804 and the distal end 818 to controllably bend the distal end 818 as shown, for example, by broken dashed line depictions 819 of the distal end 818 in FIG. 2A.
  • the flexible elongate device 802 may be a steerable catheter.
  • steerable catheters are described in detail in PCT Publication WO 2019/018736 (published Jan. 24, 2019 and titled “Flexible Elongate Device Systems and Methods”), which is incorporated by reference herein in its entirety.
  • the drive unit 804 may include drive inputs that removably couple to and receive power from drive elements, such as actuators, of the teleoperational assembly.
  • the elongate device 802 and/or medical tool 826 may include gripping features, manual actuators, or other components for manually controlling the motion of the elongate device 802 and/or medical tool 826.
  • the elongate device 802 may be steerable or, alternatively, the elongate device 802 may be non-steerable with no integrated mechanism for operator control of the bending of distal end 818.
  • one or more channels 821 (which may also be referred to as lumens), through which medical tools 826 can be deployed and used at a target anatomical location, may be defined by the interior walls of the flexible body 816 of the elongate device 802.
  • the medical instrument system 800 may include a flexible bronchial instrument, such as a bronchoscope or bronchial catheter, for use in examination, diagnosis, biopsy, and/or treatment of a lung.
  • a flexible bronchial instrument such as a bronchoscope or bronchial catheter
  • the medical instrument system 800 may also be suited for navigation and treatment of other tissues, via natural or surgically created connected passageways, in any of a variety of anatomic systems, including the colon, the intestines, the kidneys and kidney calices, the brain, the heart, the circulatory system including vasculature, and/or the like.
  • the information from the tracking system 830 may be sent to the navigation system 832, where the information may be combined with information from the visualization system 831 and/or pre-operatively obtained models to provide the physician, clinician, surgeon, or other operator with real-time position information.
  • the tracking system 830, the navigation system 832, and the visualization system 831 may cooperatively implement, at least partially, the functionality of the system 100 in implementing the techniques described with reference to FIGS. 1 A-6.
  • the real-time position information may be displayed on the display system 710 for use in the control of the medical instrument system 800.
  • the navigation system 832 may utilize the position information as feedback for positioning medical instrument system 800.
  • FIGS. 9A and 9B are simplified diagrams of side views of a patient coordinate space including a medical instrument mounted on an insertion assembly according to some examples.
  • a surgical environment 900 may include the patient P positioned on the patient table T.
  • Patient P may be stationary within the surgical environment 900 in the sense that gross patient movement is limited by sedation, restraint, and/or other means. Cyclic anatomic motion, including respiration and cardiac motion, of patient P may continue.
  • a medical instrument 904 is used to perform a medical procedure which may include, for example, surgery, biopsy, ablation, illumination, irrigation, suction, or electroporation.
  • the medical instrument 904 may also be used to perform other types of procedures, such as a registration procedure to associate the position, orientation, and/or pose data captured by the sensor system 708 to a desired (e.g., anatomical or system) reference frame.
  • the medical instrument 904 may be, for example, the medical instrument 704.
  • the medical instrument 904 may include an elongate device 910 (e.g., a catheter) coupled to an instrument body 912.
  • Elongate device 910 may be the elongate device 140 of FIG. 1.
  • Elongate device 910 includes one or more channels sized and shaped to receive a medical tool.
  • Elongate device 910 may also include one or more sensors (e.g., components of the sensor system 708).
  • a shape sensor 914 may be fixed at a proximal point 916 on the instrument body 912.
  • the proximal point 916 of the shape sensor 914 may be movable with the instrument body 912, and the location of the proximal point 916 with respect to a desired reference frame may be known (e.g., via a tracking sensor or other tracking device).
  • the shape sensor 914 may measure a shape from the proximal point 916 to another point, such as a distal end 918 of the elongate device 910.
  • the shape sensor 914 may be aligned with the elongate device 910 (e.g., provided within an interior channel or mounted externally).
  • the shape sensor 914 may optical fibers used to generate shape information for the elongate device 910.
  • position sensors e.g., EM sensors
  • a series of position sensors may be positioned along the flexible elongate device 910 and used for shape sensing.
  • Position sensors may be used alternatively to the shape sensor 914 or with the shape sensor 914, such as to improve the accuracy of shape sensing or to verify shape information.
  • Elongate device 910 may house cables, linkages, or other steering controls that extend between the instrument body 912 and the distal end 918 to controllably bend the distal end 918.
  • at least four cables are used to provide independent up-down steering to control a pitch of distal end 918 and left-right steering to control a yaw of distal end 918.
  • the instrument body 912 may include drive inputs that removably couple to and receive power from drive elements, such as actuators, of a manipulator assembly.
  • the instrument body 912 may be coupled to an instrument carriage 906.
  • the instrument carriage 906 may be mounted to an insertion stage 908 that is fixed within the surgical environment 900.
  • the insertion stage 908 may be movable but have a known location (e.g., via a tracking sensor or other tracking device) within surgical environment 900.
  • Instrument carriage 906 may be a component of a manipulator assembly (e.g., manipulator assembly 702) that couples to the medical instrument 904 to control insertion motion (e.g., motion along an insertion axis A) and/or motion of the distal end 918 of the elongate device 910 in multiple directions, such as yaw, pitch, and/or roll.
  • the instrument carriage 906 or insertion stage 908 may include actuators, such as servomotors, that control motion of instrument carriage 906 along the insertion stage 908.
  • a sensor device 920 which may be a component of the sensor system 708, may provide information about the position of the instrument body 912 as it moves relative to the insertion stage 908 along the insertion axis A.
  • the sensor device 920 may include one or more resolvers, encoders, potentiometers, and/or other sensors that measure the rotation and/or orientation of the actuators controlling the motion of the instrument carriage 906, thus indicating the motion of the instrument body 912.
  • the insertion stage 908 has a linear track as shown in FIGS. 9A and 9B.
  • the insertion stage 908 may have curved track or have a combination of curved and linear track sections.
  • FIG. 9A shows the instrument body 912 and the instrument carriage 906 in a retracted position along the insertion stage 908.
  • the proximal point 916 is at a position L0 on the insertion axis A.
  • the location of the proximal point 916 may be set to a zero value and/or other reference value to provide a base reference (e.g., corresponding to the origin of a desired reference frame) to describe the position of the instrument carriage 906 along the insertion stage 908.
  • the distal end 918 of the elongate device 910 may be positioned just inside an entry orifice of patient P.
  • the instrument body 912 and the instrument carriage 906 have advanced along the linear track of insertion stage 908, and the distal end 918 of the elongate device 910 has advanced into patient P.
  • the proximal point 916 is at a position L1 on the insertion axis A.
  • the rotation and/or orientation of the actuators measured by the sensor device 920 indicating movement of the instrument carriage 906 along the insertion stage 908 and/or one or more position sensors associated with instrument carriage 906 and/or the insertion stage 908 may be used to determine the position L1 of the proximal point 916 relative to the position L0.
  • the position L1 may further be used as an indicator of the distance or insertion depth to which the distal end 918 of the elongate device 910 is inserted into the passageway(s) of the anatomy of patient P.
  • control system 712 may be implemented in software for execution on one or more processors of a computer system.
  • the software may include code that when executed by the one or more processors, configures the one or more processors to perform various functionalities as discussed herein.
  • the code may be stored in a non-transitory computer readable storage medium (e.g., a memory, magnetic storage, optical storage, solid-state storage, etc.).
  • the computer readable storage medium may be part of a computer readable storage device, such as an electronic circuit, a semiconductor device, a semiconductor memory device, a read only memory (ROM), a flash memory, an erasable programmable read only memory (EPROM); a floppy diskette, a CD-ROM, an optical disk, a hard disk, or other storage device.
  • the code may be downloaded via computer networks such as the Internet, Intranet, etc. for storage on the computer readable storage medium.
  • the code may be executed by any of a wide variety of centralized or distributed data processing architectures.
  • the programmed instructions of the code may be implemented as a number of separate programs or subroutines, or they may be integrated into a number of other aspects of the systems described herein.
  • the components of the computing systems discussed herein may be connected using wired and/or wireless connections.
  • the wireless connections may use wireless communication protocols such as Bluetooth, near-field communication (NFC), Infrared Data Association (IrDA), home radio frequency (HomeRF), IEEE 802.11 , Digital Enhanced Cordless Telecommunications (DECT), and wireless medical telemetry service (WMTS).
  • wireless communication protocols such as Bluetooth, near-field communication (NFC), Infrared Data Association (IrDA), home radio frequency (HomeRF), IEEE 802.11 , Digital Enhanced Cordless Telecommunications (DECT), and wireless medical telemetry service (WMTS).
  • NFC near-field communication
  • IrDA Infrared Data Association
  • HomeRF home radio frequency
  • IEEE 802.11 wireless medical telemetry service
  • WMTS wireless medical telemetry service
  • Various general-purpose computer systems may be used to perform one or more processes, methods, or functionalities described herein. Additionally or alternatively, various specialized computer systems may be used to perform one or more processes, methods, or

Landscapes

  • Health & Medical Sciences (AREA)
  • Surgery (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Engineering & Computer Science (AREA)
  • Molecular Biology (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • General Health & Medical Sciences (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Animal Behavior & Ethology (AREA)
  • Gynecology & Obstetrics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Pathology (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Robotics (AREA)
  • Apparatus For Radiation Diagnosis (AREA)

Abstract

Sont divulgués des systèmes et des procédés de navigation pendant un acte médical. Une unité de traitement reçoit d'un système de détection des indications de positions, dans un premier système de coordonnées, d'un premier capteur et d'un deuxième capteur disposés au niveau d'un dispositif allongé flexible. L'unité de traitement reçoit en outre des données d'imagerie provenant d'une unité d'imagerie et détermine, en fonction des données d'imagerie, une indication de position, dans un deuxième système de coordonnées, d'un repère du dispositif allongé flexible. Le repère est disposé au niveau du dispositif allongé flexible à une distance de séparation du premier capteur. L'unité de traitement enregistre ensuite le premier système de coordonnées dans le deuxième système de coordonnées en fonction des données d'imagerie reçues et des données de système de capteurs reçues.
PCT/US2024/038352 2023-07-18 2024-07-17 Imagerie peropératoire pour navigation assistée par robot Pending WO2025019569A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363514191P 2023-07-18 2023-07-18
US63/514,191 2023-07-18

Publications (1)

Publication Number Publication Date
WO2025019569A1 true WO2025019569A1 (fr) 2025-01-23

Family

ID=92296003

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2024/038352 Pending WO2025019569A1 (fr) 2023-07-18 2024-07-17 Imagerie peropératoire pour navigation assistée par robot

Country Status (1)

Country Link
WO (1) WO2025019569A1 (fr)

Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US6380732B1 (en) 1997-02-13 2002-04-30 Super Dimension Ltd. Six-degree of freedom tracking system having a passive transponder on the object being tracked
US20060013523A1 (en) 2004-07-16 2006-01-19 Luna Innovations Incorporated Fiber optic position and shape sensing device and method relating thereto
US7316681B2 (en) 1996-05-20 2008-01-08 Intuitive Surgical, Inc Articulated surgical instrument for performing minimally invasive surgery with enhanced dexterity and sensitivity
US7772541B2 (en) 2004-07-16 2010-08-10 Luna Innnovations Incorporated Fiber optic position and/or shape sensing based on rayleigh scatter
US8773650B2 (en) 2009-09-18 2014-07-08 Intuitive Surgical Operations, Inc. Optical position and/or shape sensing
US20140275997A1 (en) * 2013-03-15 2014-09-18 Intuitive Surgical Operations, Inc. Shape sensor systems for tracking interventional instruments and mehods of use
US8900131B2 (en) 2011-05-13 2014-12-02 Intuitive Surgical Operations, Inc. Medical system providing dynamic registration of a model of an anatomical structure for image-guided surgery
US9259274B2 (en) 2008-09-30 2016-02-16 Intuitive Surgical Operations, Inc. Passive preload and capstan drive for surgical instruments
WO2016191298A1 (fr) 2015-05-22 2016-12-01 Intuitive Surgical Operations, Inc. Systèmes et procédés d'alignement pour chirurgie guidée par image
WO2019018736A2 (fr) 2017-07-21 2019-01-24 Intuitive Surgical Operations, Inc. Systèmes et procédés de dispositif allongé flexible
AU2021283341A1 (en) * 2020-06-03 2022-12-22 Noah Medical Corporation Systems and methods for hybrid imaging and navigation
US20230000566A1 (en) * 2021-07-01 2023-01-05 Remedy Robotics, Inc. Image space control for endovascular tools
US20230210604A1 (en) * 2021-12-31 2023-07-06 Auris Health, Inc. Positioning system registration using mechanical linkages

Patent Citations (13)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US7316681B2 (en) 1996-05-20 2008-01-08 Intuitive Surgical, Inc Articulated surgical instrument for performing minimally invasive surgery with enhanced dexterity and sensitivity
US6380732B1 (en) 1997-02-13 2002-04-30 Super Dimension Ltd. Six-degree of freedom tracking system having a passive transponder on the object being tracked
US20060013523A1 (en) 2004-07-16 2006-01-19 Luna Innovations Incorporated Fiber optic position and shape sensing device and method relating thereto
US7772541B2 (en) 2004-07-16 2010-08-10 Luna Innnovations Incorporated Fiber optic position and/or shape sensing based on rayleigh scatter
US9259274B2 (en) 2008-09-30 2016-02-16 Intuitive Surgical Operations, Inc. Passive preload and capstan drive for surgical instruments
US8773650B2 (en) 2009-09-18 2014-07-08 Intuitive Surgical Operations, Inc. Optical position and/or shape sensing
US8900131B2 (en) 2011-05-13 2014-12-02 Intuitive Surgical Operations, Inc. Medical system providing dynamic registration of a model of an anatomical structure for image-guided surgery
US20140275997A1 (en) * 2013-03-15 2014-09-18 Intuitive Surgical Operations, Inc. Shape sensor systems for tracking interventional instruments and mehods of use
WO2016191298A1 (fr) 2015-05-22 2016-12-01 Intuitive Surgical Operations, Inc. Systèmes et procédés d'alignement pour chirurgie guidée par image
WO2019018736A2 (fr) 2017-07-21 2019-01-24 Intuitive Surgical Operations, Inc. Systèmes et procédés de dispositif allongé flexible
AU2021283341A1 (en) * 2020-06-03 2022-12-22 Noah Medical Corporation Systems and methods for hybrid imaging and navigation
US20230000566A1 (en) * 2021-07-01 2023-01-05 Remedy Robotics, Inc. Image space control for endovascular tools
US20230210604A1 (en) * 2021-12-31 2023-07-06 Auris Health, Inc. Positioning system registration using mechanical linkages

Similar Documents

Publication Publication Date Title
US12150718B2 (en) Systems and methods of registration compensation in image guided surgery
US12245833B2 (en) Systems and methods of continuous registration for image-guided surgery
US12121204B2 (en) Systems and methods of registration for image guided surgery
US12343094B2 (en) Systems and methods for using tracking in image-guided medical procedure
US12369980B2 (en) Systems and methods for intelligently seeding registration
US20220346886A1 (en) Systems and methods of pose estimation and calibration of perspective imaging system in image guided surgery
US20210100627A1 (en) Systems and methods related to elongate devices
US20210259783A1 (en) Systems and Methods Related to Registration for Image Guided Surgery
US20230030727A1 (en) Systems and methods related to registration for image guided surgery
WO2025019569A1 (fr) Imagerie peropératoire pour navigation assistée par robot
WO2025171214A2 (fr) Filtre directionnel et/ou contraintes de translation pour enregistrement de coordonnées

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24754450

Country of ref document: EP

Kind code of ref document: A1