[go: up one dir, main page]

WO2025059207A1 - Appareil médical doté d'une structure de support et son procédé d'utilisation - Google Patents

Appareil médical doté d'une structure de support et son procédé d'utilisation Download PDF

Info

Publication number
WO2025059207A1
WO2025059207A1 PCT/US2024/046243 US2024046243W WO2025059207A1 WO 2025059207 A1 WO2025059207 A1 WO 2025059207A1 US 2024046243 W US2024046243 W US 2024046243W WO 2025059207 A1 WO2025059207 A1 WO 2025059207A1
Authority
WO
WIPO (PCT)
Prior art keywords
catheter
continuum robot
operate
section
sleeves
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/US2024/046243
Other languages
English (en)
Inventor
Benedict Shia
Anne Yujin Edge
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Canon USA Inc
Original Assignee
Canon USA Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Canon USA Inc filed Critical Canon USA Inc
Publication of WO2025059207A1 publication Critical patent/WO2025059207A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J18/00—Arms
    • B25J18/06—Arms flexible
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00006—Operational features of endoscopes characterised by electronic signal processing of control signals
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A61B1/000096—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope using artificial intelligence
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00147—Holding or positioning arrangements
    • A61B1/0016—Holding or positioning arrangements using motor drive units
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/005—Flexible endoscopes
    • A61B1/0051—Flexible endoscopes with controlled bending of insertion part
    • A61B1/0055—Constructional details of insertion parts, e.g. vertebral elements
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/005—Flexible endoscopes
    • A61B1/0051—Flexible endoscopes with controlled bending of insertion part
    • A61B1/0057—Constructional details of force transmission elements, e.g. control wires
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A61B34/71—Manipulators operated by drive cable mechanisms
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/06—Programme-controlled manipulators characterised by multi-articulated arms
    • B—PERFORMING OPERATIONS; TRANSPORTING
    • B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
    • B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
    • B25J9/00—Programme-controlled manipulators
    • B25J9/10—Programme-controlled manipulators characterised by positioning means for manipulator elements
    • B25J9/104—Programme-controlled manipulators characterised by positioning means for manipulator elements with cables, chains or ribbons
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B2034/301—Surgical robots for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B2034/303—Surgical robots specifically adapted for manipulations within body lumens, e.g. within lumen of gut, spine, or blood vessels
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/30—Surgical robots
    • A61B2034/305—Details of wrist mechanisms at distal ends of robotic arms
    • A61B2034/306—Wrists with multiple vertebrae
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/70—Manipulators specially adapted for use in surgery
    • A61B34/71—Manipulators operated by drive cable mechanisms
    • A61B2034/715—Cable tensioning mechanisms for removing slack

Definitions

  • the present disclosure generally relates to imaging and/or medical devices and, more particularly, to an apparatus, method, and storage medium for making, and/or use with, a robotic catheter tip and/or a continuum robot applicable to guide interventional tools and instruments in medical procedures, including after or while implementing robotic control for all sections of a catheter or imaging device/ apparatus or system to match a state or states when each section reaches or approaches a same or similar, or approximately a same or similar, state or states of a first section of the catheter or imaging device, apparatus, or system.
  • the present disclosure generally relates to imaging and, more particularly, to bronchoscope(s), robotic bronchoscope(s), robot apparatus(es), method(s), and storage medium(s) that operate to image a target, object, or specimen (such as, but not limited to, a lung, a biological object or sample, tissue, etc.).
  • a target, object, or specimen such as, but not limited to, a lung, a biological object or sample, tissue, etc.
  • a target, object, or specimen such as, but not limited to, a lung, a biological object or sample, tissue, etc.
  • a target, object, or specimen such as, but not limited to, a lung, a biological object or sample, tissue, etc.
  • a target, object, or specimen such as, but not limited to, a lung, a biological object or sample, tissue, etc.
  • One or more bronchoscopic, endoscopic, medical, camera, catheter, or imaging devices, systems, and methods and/or storage mediums for use with same, are discussed here
  • Medical imaging is used with equipment to diagnose and treat medical conditions. Endoscopy, bronchoscopy, catheterization, and other medical procedures facilitate the ability to look inside a body.
  • a flexible medical tool may be inserted into a patient’s body, and an instrument may be passed through the tool to examine or treat an area inside the body.
  • a scope can be used with an imaging device that views and/or captures objects or areas. The imaging can be transmitted or transferred to a display for review or analysis by an operator, such as a physician, clinician, technician, medical practitioner or the like.
  • the scope can be an endoscope, bronchoscope, or other type of scope.
  • a bronchoscope is an endoscopic instrument to look or view inside, or image, the airways in a lung or lungs of a patient.
  • the bronchoscope may be put in the nose or mouth and moved down the throat and windpipe, and into the airways, where views or imaging may be made of the bronchi, bronchioles, larynx, trachea, windpipe, or other areas.
  • Catheters and other medical tools may be inserted through a tool channel in the bronchoscope to provide a pathway to a target area in the patient for diagnosis, planning, medical procedure(s), treatment, etc.
  • Robotic bronchoscopes, robotic endoscopes, or other robotic imaging devices may be equipped with a tool channel or a camera and biopsy tools, and such devices (or users of such devices) may insert/ retract the camera and biopsy tools to exchange such components.
  • the robotic bronchoscopes, endoscopes, or other imaging devices may be used in association with a display system and a control system.
  • An imaging device such as a camera, may be placed in the bronchoscope, the endoscope, or other imaging device/ system to capture images inside the patient and to help control and move the bronchoscope, the endoscope, or the other type of imaging device, and a display or monitor may be used to view the captured images.
  • An endoscopic camera that may be used for control may be positioned at a distal part of a catheter or probe (e.p., at a tip section).
  • the display system may display, on the monitor, an image or images captured by the camera, and the display system may have a display coordinate used for displaying the captured image or images.
  • the control system may control a moving direction of the tool channel or the camera. For example, the tool channel or the camera may be bent according to a control by the control system.
  • the control system may have an operational controller (such as, but not limited to, a joystick, a gamepad, a controller, an input device, etc.) , and physicians may rotate or otherwise move the camera, probe, catheter, etc. to control same.
  • an operational controller such as, but not limited to, a joystick, a gamepad, a controller, an input device, etc.
  • physicians may rotate or otherwise move the camera, probe, catheter, etc. to control same.
  • control methods or systems are limited in effectiveness.
  • information obtained from an endoscopic camera at a distal end or tip section may help decide which way to move the distal end or tip section, such information does not provide details on how the other bending sections or portions of the bronchoscope, endoscope, or other type of imaging device may move to best assist the navigation.
  • a continuum robot or snake may include a plurality of bending sections having a flexible structure, wherein the shape of the continuum robot is controlled by deforming the bending sections.
  • the snake has two advantages over existing robots including rigid links. The first advantage is that the snake can move along a curve in a narrow space or in an environment with scattered objects in which the rigid link robot may get stuck. The second advantage is that it is possible to operate the snake without damaging surrounding fragile elements because the snake has intrinsic flexibility.
  • minimally invasive medical care with which burden on a patient can be reduced and the quality of life (QOL) after the treatment or inspection can be improved, has been attracting attention.
  • QOL quality of life
  • a surgery or inspection using an endoscope is a typical example of minimally invasive medical care. For example, a laparoscopic surgery is advantageous over a conventional abdominal surgery in that it can be performed with a smaller surgical wound/insertion site, which results in a shorter stay in the hospital and less damage to an appearance of the insertion site.
  • Endoscopes used for the minimally invasive medical care are roughly divided into rigid endoscopes and soft endoscopes.
  • a rigid endoscope although clear images can be obtained, the direction in which an observation target can be observed is limited.
  • an insertion portion of the rigid endoscope presses the organ and causes pain for the patient.
  • a soft endoscope includes an insertion portion formed of a bendable member, so that a large area can be observed in detail by adjusting the bending angle of the distal end of the endoscope.
  • the endoscope can be inserted to a deep area of the body without causing the endoscope to come into contact with tissue even when the insertion path has a complex curved shape.
  • a pose of the endoscope does not necessarily match the shape of the passageway. This mismatch can cause the endoscope to experience friction against the airway wall. This friction can cause the endoscope to twist or prolapse, making navigation more difficult or impossible. In addition, this can impart a lot of force into the target, the object, or an anatomy of a patient/ subject, and potentially cause trauma to the patient/subject. This phenomena is not unique to multisection robotic endoscopes, but it could be a more frequent occurrence since multi-section robotic endoscopes can take more complex shapes.
  • the amount of operating force that can be applied to the drive w ires is limited by the critical buckling force of the specific wire diameter and material.
  • the design of the bendable medical device include continuous support around the drive wires throughout the entire length of the bendable medical device.
  • Attachment points can introduce: ledges where wires can catch, different material friction, irregular channel sizes, bad alignment of channels, or gaps where the wire is unsupported. Locations where the path of the wires curves also increase the risk of buckling, such as the transition from a pushing mechanism in a hub body from a large diameter tapering to a bendable medical device with a very small diameter.
  • a robotic catheter and/or autonomous robot may be used in one or more embodiments of the present disclosure to address the above issues, and robotic catheter and/or autonomous robot apparatuses, systems, methods, storage mediums, and/or other related features may be used to increase maneuverability into an object, sample, or target (e.p., a patient, an organ of a patient, a tissue being imaged or evaluated, a lung, a lumen, a vessel, another part of a patient, etc.), while preserving visualization and catheter stability and while refining and advancing bendable medical devices to reduce or eliminate incidence of buckling of the bendable body, including, but not limited to, reducing or eliminating buckling of the junction of a hub of a bendable device with a smaller diameter shaft.
  • an object, sample, or target e.p., a patient, an organ of a patient, a tissue being imaged or evaluated, a lung, a lumen, a vessel, another part of a patient, etc.
  • refining and advancing bendable medical devices
  • imaging c.g., computed tomography (CT), Magnetic Resonance Imaging (MR1), etc.
  • CT computed tomography
  • MR1 Magnetic Resonance Imaging
  • apparatuses, systems, methods, and storage mediums for using a navigation and/or control method or methods (manual or automatic) in one or more apparatuses or systems (e.g., an imaging apparatus or system, a catheter, an autonomous catheter or robot, an endoscopic imaging device or system, etc.).
  • CT computed tomography
  • MR1 Magnetic Resonance Imaging
  • One or more embodiments of an apparatus of the present disclosure may include or have: a continuum robot including one or more bending sections, wherein the one or more bending sections is/ are bent by one or more driving wires; and one or more support structures or sleeves that operate to support one or more portions of the one or more driving wires.
  • the one or more support sleeves operate to one or more of the following: (i) support the one or more portions of the one or more driving wires to prevent wire buckling; (ii) provide smooth and continuous support for the one or more driving wires to prevent buckling at a transition point where the one or more driving wires slide through one body and into a second body (e.g., where no unsupported gaps or ledges may exist for the one or more driving wires to catch on); and/ or (iii) reduce friction of the one or more driving wires.
  • the one or more driving wires may terminate at a distal end of the one or more bending sections.
  • the one or more support sleeves may be made of or include stainless steel, and the one or more support sleeves may each have an inside diameter between one or more of the following: o.oi inches and 0.02 inches, and 0.01 inches and 0.015 inches. In one or more embodiments, an inside diameter of each of the one or more support sleeves may be larger than an outside diameter of the one or more driving wires (e.g., the one or more support sleeves may have corresponding one or more driving wires passing through a respective support sleeve of the one or more support sleeves).
  • One or more embodiments may further include: an actuator that drives the one or more driving wires; a transition area between the actuator and the one or more bending sections; and an operational console or one or more processors that operate to control a movement of the actuator, based on an input, wherein the support sleeve is disposed or situated in the transition area and at least partially extends into a proximal end of the one or more bending sections.
  • the transition area may include a hub.
  • a proximal end of the support sleeve may be bonded to one or more portions of the transition area.
  • the one or more bending sections may include a first section proximal to the transition area and a second section attached to the first section and distal to the transition area.
  • the first section of the one or more bending sections and second section of the one or more bending sections may be constructed of thermoplastic material, and/or the first section of the one or more bending sections and the second section of the one or more bending sections may be constructed of or include thermoplastic material having different durometer scales or values.
  • the one or more driving wires may be bent or operate to bend at least in the transition area.
  • the transition area may include or comprise of a first section proximal to the one or more bending sections and a second section attached to the first section and distal to the one or more bending sections.
  • the first section of the transition area and the section of the transition area may include or be constructed of thermoplastic material having different durometer scales or values.
  • the continuum robot may include or comprise a steerable catheter, and the hub may be connected to the catheter body.
  • the hub may be bonded to the catheter or continuum robot at one or more attachment points.
  • multiple support sleeves of the one or more support sleeves and a tube of a tool channel may all be bonded to the hub and the steerable catheter or continuum robot.
  • One or more embodiments may use or include an outer jacket that operates to add robustness to the catheter/continuum robot-hub connection.
  • One or more embodiments may have multiple push/pull drive/driving w ires for the one or more driving wires, and the multiple push/pull driving wires may be protected at or in a transition area, via the one or more support sleeves, from a body of the hub to a shaft of the catheter or continuum robot.
  • An inner diameter (ID) of the one or more support sleeves may be sized and shaped to support the one or more drive wires and prevent the one or more drive wires from buckling.
  • the drive wires may terminate in a distal bending end of the shaft of the catheter or continuum robot.
  • a proximal end of each of the one or more support sleeves may be bonded to the hub body.
  • the one or more support sleeves may be bonded into sub-lumens of the catheter body or continuum robot, and/or the one or more support structures or sleeves operate to be made of or include stainless steel.
  • the one or more support sleeves may have a specific orientation or alignment with respect to the hub and catheter/continuum robot.
  • a shaft of the catheter body or continuum robot may have multiple sub-lumens with two different diameters.
  • a diameter of the proximal sub-lumen(s) may be larger at a proximal end of the catheter shaft or continuum robot to fit the one or more support sleeves, and larger sublumens may terminate at a set distance into the catheter shaft or continuum robot.
  • a diameter of the distal sub-lumen(s) may be smaller to support the one or more driving wires, and may start after the end (e.g., a distal end) of the one or more support sleeves and may extend up to the distal end of the catheter shaft or continuum body.
  • the lumens or sub-lumens that operate to support the one or more driving wires may extend through and from an inside of the hub body and may extend into and through one or more portions of the catheter shaft (e.g., from the proximal end of the catheter shaft to the distal end of the catheter shaft, from the proximal end of the catheter shaft and extend along the catheter shaft for a set or predetermined distance, from the proximal end of the catheter shaft and extend along and through a portion of the catheter shaft, etc.).
  • One or more embodiments may operate to provide or have ease of assembly. For example, using the one or more support sleeves to connect the hub to the catheter automatically aligns the lumen/drive wire positions/states/poses/etc. as well as a distance between the hub and the catheter or continuum robot in one or more embodiments. Multiple assembly steps may be combined into a single reflow step in a case where using thermoplastic materials.
  • the apparatus may further include a tool channel extending a length of the one or more bending sections.
  • a tool channel or tool channel tube may run through the hub body and catheter shaft.
  • the catheter shaft or continuum robot may be bonded to the tool channel or tool channel tube to create an additional attachment to the hub.
  • One or more methods may be used to assemble such structure.
  • two polymer or thermoplastic multilumen extrusions may be used where the two multi-lumen extrusions may have different lumen diameters by reflowing such thermoplastic extrusions together.
  • the tool channel or tool channel tube may also be a thermoplastic material and may be reflowed in the same step or in another step.
  • one or more embodiments may use a lower durometer extrusion for the proximal catheter shaft, which may allow' the one or more support sleeves to enter the catheter shaft or continuum robot at a larger angle from a diameter of the hub.
  • An outer jacket tube may be added to the outer diameter of the catheter or continuum robot to reinforce an outer wall thickness of the reflowed transition area.
  • one or more embodiments of the present disclosure may include a two part catheter body or continuum robot where the proximal catheter shaft section has angled sublumens for the one or more support sleeves.
  • a storage medium stores instructions or a program for causing one or more processors of an apparatus or system to perform a method of manufacturing a robotic catheter or imaging apparatus, where the method may include using one or more attachment or reflow technique(s) as discussed herein.
  • an apparatus for performing navigation control and/ or for controlling, manufacturing, or using a catheter and/or continuum robot may include a flexible medical device or tool; and one or more processors that operate to: bend a distal portion of the flexible medical device or tool; and advance the flexible medical device or tool through a pathway, wherein the flexible medical device or tool may be advanced through the pathway in a substantially centered manner.
  • the flexible medical device or tool may have multiple bending sections, and the one or more processors may further operate to control or command the multiple bending sections of the flexible medical device or tool using one or more of the following modes: a Follow the Leader (FTL) mode, a Reverse Follow the Leader (RFTL) mode, a Hold the Line mode, a Close the Gap mode, and/or a Stay the Course mode.
  • the flexible medical device or tool may include a catheter or scope and the catheter or scope may be part of, include, or be attached to an imaging apparatus, such as, but not limited to, an endoscope, a catheter, a probe, a bronchoscope, or any other imaging device discussed herein or known to those skilled in the art.
  • a method for controlling an apparatus including a flexible medical device or tool that operates to perform navigation control and/or for controlling, manufacturing, or using a catheter and/or one or more support sleeves may include: bending a distal portion of the flexible medical device or tool; and advancing the flexible medical device or tool through a pathway, wherein the flexible medical device or tool may be advanced through the pathway in a substantially centered manner.
  • the flexible medical device or tool may have multiple bending sections, and the method may further include controlling or commanding the multiple bending sections of the flexible medical device or tool using one or more of the following modes: a follow the Leader (FTL) process or mode, a Reverse Follow the Leader (RFTL) process or mode, a Hold the Line process or mode, a Close the Gap process or mode, and/or a Stay the Course process or mode.
  • FTL Follow the Leader
  • RFTL Reverse Follow the Leader
  • a non-transitory computer-readable storage medium storing at least one program for causing a computer to execute a method for controlling an apparatus including a flexible medical device or tool that operates to perform navigation control and/ or for controlling, manufacturing, or using a catheter and/or one or more support sleeves, where the method may include: bending a distal portion of the flexible medical de vice or tool; and advancing the flexible medical device or tool through a pathway, wherein the flexible medical device or tool may be advanced through the pathway in a substantially centered manner.
  • the method may include any other feature discussed herein.
  • a continuum robot for performing robotic control may include: one or more processors that operate to: instruct or command a first bending section or portion of a catheter or a probe of the continuum robot such that the first bending section or portion achieves, or is disposed at, a pose, position, or state at a position along a path, the catheter or probe of the continuum robot having a plurality of bending sections or portions and a base; instruct or command each of the other bending sections or portions of the plurality of bending sections or portions of the catheter or probe to match, substantially match, or approximately match the pose, position, or state of the first bending section or portion at the position along the path in a case where each section or portion reaches or approaches a same, similar, or approximately similar state or states at the position along the path; and instruct or command the plurality of bending sections or portions such that the first bending section or portion or a Tip or distal bending section or portion is located in a predetermined pose, position, or state at or
  • a first bending section or portion or the Tip or distal bending section or portion may include a camera, an endoscopic camera, a sensor, or other imaging device or system to obtain one or more images of or in a target, sample, or object; and the one or more processors may further operate to command the camera, sensor, or other imaging device or system to obtain the one or more images of or in the target, sample, or object at the predetermined pose, position, or state, and the one or more processors operate to receive the one or more images and/or display the one or more images on a display.
  • the method(s) may further include any of the features discussed herein that may be used in the one or more apparatuses of the present disclosure.
  • a non-transitory computer-readable storage medium may store at least one program for causing a computer to execute a method for performing robotic control, and may use any of the method feature(s) discussed herein.
  • apparatuses and systems, and methods and storage mediums for performing navigation, movement, and/or control, and/or for controlling, manufacturing, or using a catheter and/or the one or more support sleeves/structure may operate to characterize biological objects, such as, but not limited to, blood, mucus, tissue, etc.
  • One or more embodiments of the present disclosure may be used in clinical application(s), such as, but not limited to, intervascular imaging, intravascular imaging, bronchoscopy, atherosclerotic plaque assessment, cardiac stent evaluation, intracoronary’ imaging using blood clearing, balloon sinuplasty, sinus stenting, arthroscopy, ophthalmology, ear research, veterinary use and research, etc.
  • one or more technique(s) discussed herein may be employed as or along with features to reduce the cost of at least one of manufacture and maintenance of the one or more apparatuses, devices, systems, and storage mediums by reducing or minimizing a number of optical and/or processing components and by virtue of the efficient techniques to cut down cost e.g., physical labor, mental burden, fiscal cost, time and complexity, etc.) of use/manufacture of such apparatuses, devices, systems, and storage mediums.
  • a different or similar reference number and/or character may be used for the same or similar feature(s) in one or more other embodiments, and/or the reference numeral(s) or character(s) may be altered or modified to indicate one or more secondary elements and/or references of a same or similar nature and/or kind (such as, but not limited to, using 168’ compared to 168).
  • the one or more features shown in the figures are illustrative, and it is intended that changes and modifications may be made to the one or more embodiments without departing from the scope and spirit of the subject disclosure.
  • FIG. 1 illustrates at least one embodiment of an imaging, continuum robot/ steerable catheter, or endoscopic apparatus or system in accordance with one or more aspects of the present disclosure
  • FIG. 2 is a schematic diagram showing at least one embodiment of an imaging, steerable catheter, or continuum robot apparatus or system in accordance with one or more aspects of the present disclosure
  • FIGS. 3A-3B illustrate at least one embodiment example of a steerable catheter, continuum robot, and/or medical device that may be used with one or more technique(s), including support sleeve/structure features and/or technique(s), in accordance with one or more aspects of the present disclosure
  • FIGS. 3C-3D illustrate one or more principles of steerable catheter or continuum robot tip manipulation by actuating one or more bending segments of a continuum robot or steerable catheter 104 of FIGS. 3A-3B in accordance with one or more aspects of the present disclosure
  • FIG. 4 is a schematic diagram showing at least one embodiment of an imaging, continuum robot, steerable catheter, or endoscopic apparatus or system in accordance with one or more aspects of the present disclosure
  • FIG. 5 is a flowchart of at least one embodiment of a method for planning an operation of at least one embodiment of a continuum robot or steerable catheter apparatus or system in accordance with one or more aspects of the present disclosure
  • FIG. 6 is a block diagram of at least one embodiment of a bendable device/apparatus or system incorporating one or more features and/or technique(s) in accordance w ith one or more aspects of the present disclosure
  • FIG. 7 illustrates a kinematic model of a bendable portion of one or more embodiments of a steerable catheter or continuum robot in accordance w ith one or more aspects of the present disclosure
  • FIG. 8 provides an overall side view of one or more embodiments of a steerable catheter or continuum robot in accordance with one or more aspects of the present disclosure
  • FIGS. 9A-9B show a side image and a cross-sectional image, respectively, of one or more embodiments of a steerable catheter or continuum robot in accordance with one or more aspects of the present disclosure
  • FIG. 10 shows a side view of one or more embodiments of a steerable catheter or continuum robot in accordance with one or more aspects of the present disclosure
  • FIG. 11 shows a cross-sectional view of one or more embodiments of a steerable catheter or continuum robot in accordance with one or more aspects of the present disclosure
  • FIG. 12 shows a side view of one or more embodiments of a steerable catheter or continuum robot in accordance with one or more aspects of the present disclosure
  • FIGS. 13A and 13B show cross-sectional views of one or more embodiments of a steerable catheter or continuum robot at a proximal catheter shaft and a distal catheter shaft, respectively, in accordance with one or more aspects of the present disclosure
  • FIG. 14 illustrates a focused side view of one or more embodiments of a steerable catheter or continuum robot pre-reflow and post-reflow in accordance with one or more aspects of the present disclosure
  • FIGS. 15A and 15B show side view of one or more embodiments of a steerable catheter or continuum robot using support sleeve(s) in accordance with one or more aspects of the present disclosure
  • FIG. 16 shows a close-up, side view’ of one or more embodiments of a steerable catheter or continuum robot in accordance with one or more aspects of the present disclosure
  • FIGS. 17A and 17B show cross-sectional views of one or more embodiments of a steerable catheter or continuum robot at a proximal catheter shaft and a distal catheter shaft, respectively, in accordance with one or more aspects of the present disclosure
  • FIGS. 18A-18C show close-up side views of one or more embodiments of a steerable catheter or continuum robot using support sleeve(s) in accordance with one or more aspects of the present disclosure
  • FIG. 19 shows a close-up, side view of one or more embodiments of a steerable catheter or continuum robot in accordance with one or more aspects of the present disclosure
  • FIGS. 20A and 20B show’ cross-sectional views of one or more embodiments of a steerable catheter or continuum robot at a proximal catheter shaft and a distal catheter shaft, respectively, in accordance with one or more aspects of the present disclosure
  • FIG. 21 illustrates a flowchart for at least one method embodiment for performing correction, adjustment, and/or smoothing for a catheter or probe of a continuum robot device or system that may be used with one or more tip feature(s) and/or technique(s) in accordance with one or more aspects of the present disclosure
  • FIG. 22 shows a schematic diagram of an embodiment of a computer or console that may be used with one or more embodiments of an apparatus or system, or one or more methods, discussed herein in accordance with one or more aspects of the present disclosure
  • FIG. 23 shows a schematic diagram of at least an embodiment of a system using a computer or processor, a memory, a database, and input and output devices in accordance with one or more aspects of the present disclosure
  • FIG. 24 shows a created architecture of or for a regression model(s) that may be used for autonomous navigation, movement detection, and/or control techniques and/or any other technique or feature discussed herein in accordance with one or more aspects of the present disclosure
  • FIG. 25 shows a convolutional neural network architecture that may be used for autonomous navigation, movement detection, and/or control techniques and/or any other technique or feature discussed herein in accordance with one or more aspects of the present disclosure
  • FIG. 26 shows a created architecture of or for a regression model(s) that may be used for autonomous navigation, movement detection, and/or control techniques and/or any other technique or feature discussed herein in accordance with one or more aspects of the present disclosure
  • FIG. 27 is a schematic diagram of or for a segmentation model(s) that may be used for any feature(s) and/or technique(s) discussed herein in accordance with one or more aspects of the present disclosure.
  • CT computed tomography
  • MR1 Magnetic Resonance Imaging
  • OCT Optical Coherence Tomography
  • NIRF Near infrared fluorescence
  • One or more embodiments of the present disclosure avoid the aforementioned issues by providing one or more simple, efficient, cost-effective, and innovative structure that may be used w ith catheter or probe control technique(s) (including, but not limited to, robotic control technique(s)) as discussed herein and/or support structure feature(s) and/or technique(s) as discussed herein.
  • the robotic control techniques may be used w ith a co-registration (e.g., computed tomography (CT) co-registration, conebeam CT (CBCT) co-registration, etc.) to enhance a successful targeting rate for a predetermined sample, target, or object (e.g., a lung, a portion of a lung, a vessel, a nodule, an organ of a patient, a patient, tissue, etc.) by minimizing human error.
  • CT computed tomography
  • CBCT conebeam CT
  • CBCT may be used to locate a target, sample, or object (e.g., the lesion(s) or nodule(s) of a lung or airways, plaque or other tissue in one or more samples or in a patient(s), a set or predetermined target in tissue or in a patient, etc.) along with an imaging device (e.g., a steerable catheter, a continuum robot, etc.) and to co-register the target, sample, or object (e.g., the lesions or nodules, plaque or other tissue in one or more samples or in a patient(s), a set or predetermined target in tissue or in a patient, etc.) with the device shown in an image to achieve proper guidance.
  • a target, sample, or object e.g., the lesion(s) or nodule(s) of a lung or airways, plaque or other tissue in one or more samples or in a patient(s), a set or predetermined target in tissue or in a patient, etc.
  • an imaging device
  • imaging e.g., computed tomography (CT), Magnetic Resonance Imaging (MRI), etc.
  • MRI Magnetic Resonance Imaging
  • apparatuses, systems, methods, and storage mediums for using a navigation and/or control method or methods (manual or automatic) and/or for using support structure feature(s) and/or technique(s) in one or more apparatuses or systems (e.g., an imaging apparatus or system, an endoscopic imaging device or system, a bronchoscope, etc.).
  • CT computed tomography
  • MRI Magnetic Resonance Imaging
  • CT computed tomography
  • MRI Magnetic Resonance Imaging
  • Details of the present disclosure may cover the one or more mechanisms or features of a continuum robot, followed by details or features of support structure for reducing buckling, as well as the apparatuses/systems, methods/procedures/techniques, and/or other hardware/ software (e.g., storage medium(s), processor(s), etc.) that maybe used with the one or more continuum robots and the one or more support structures.
  • hardware/ software e.g., storage medium(s), processor(s), etc.
  • the navigation and/ or control may be employed so that an apparatus or system having multiple portions or sections (e.g., multiple bending portions or sections) operates to: (i) keep track of a path of a portion (e.g., a tip) or of each of the multiple portions or sections of an apparatus or system; ( ii) have a state or states of each of the multiple portions or sections match a state or states of a first portion or section of the multiple portions or sections in a case where each portion or section reaches or approaches a same, similar, or approximately similar state (e.g., a position or other state(s) in a target, object, or specimen; a position or other state(s) in a patient; a target position or state(s) in an image or frame; a set or predetermined position or state(s) in an image or frame; a set or predetermined position or state(s) in an image or frame where the first portion or section reaches or approaches the set or predetermined position or state(s) at one point
  • an orientation, pose, or state may include one or more degrees of freedom.
  • two (2) degrees of freedom may be used, which may include an angle for a magnitude of bending and a plane for a direction of bending.
  • matching state(s) may involve matching, duplicating, mimicking, or otherwise copying other characteristics, such as, but not limited to, vectors for each section or portion of the one or more sections or portions of a probe or catheter, for different portions or sections of the catheter or probe.
  • a transition or change from a base angle/plane to a target angle/plane may be set or predetermined using transition values (e.g., while not limited hereto, a base orientation or state may have a stage at o mm, an angle at o degrees, and a plane at o degrees whereas a target orientation or state may have a stage at 20mm, an angle at 90 degrees, and a plane at 180 degrees.
  • the intermediate values for the stage, angle, and plane may be set depending on how many transition orientations or states may be used).
  • a continuum robot or steerable catheter may include one or more of the following: (i) a distal bending section or portion, wherein the distal bending section or portion is commanded or instructed automatically or based on an input of a user of the continuum robot or steerable catheter; (ii) a plurality of bending sections or portions including a distal or most distal bending portion or section and the rest of the plurality of the bending sections or portions; and/or (iii) the one or more processors further operate to instruct or command the forward motion, or the motion in the set or predetermined direction, of a motorized linear stage (or other structure used to map path or path-like information) and/or of the continuum robot or steerable catheter automatically and/or based on an input of a user of the continuum robot.
  • a continuum robot or steerable catheter may further include: a base and an actuator that operates to bend the plurality of the bending sections or portions independently; and a motorized linear stage and/or a sensor that operates to move the continuum robot or steerable catheter forward and backward, and/or in the predetermined or set direction or directions, wherein the one or more processors operate to control the actuator and the motorized linear stage and/ or the sensor.
  • the plurality of bending sections or portions may each include driving wires that operate to bend a respective section or portion of the plurality of sections or portions, wherein the driving wires are connected to an actuator so that the actuator operates to bend one or more of the plurality of bending sections or portions using the driving wires.
  • One or more embodiments may include a user interface of or disposed on a base, or disposed remotely from a base, the user interface operating to receive an input from a user of the continuum robot or steerable catheter to move one or more of the plurality of bending sections or portions and/or a motorized linear stage and/or a sensor, wherein the one or more processors further operate to receive the input from the user interface, and the one or more processors and/or the user interface operate to use a base coordinate system.
  • One or more displays may be provided to display a path (e.g., a control path) of the continuum robot or steerable catheter.
  • the continuum robot may further include an operational controller or joystick that operates to issue or input one or more commands or instructions as an input to one or more processors, the input including an instruction or command to move one or more of a plurality of bending sections or portions and/or a motorized linear stage and/ or a sensor;
  • the continuum robot may further include a display to display one or more images taken by the continuum robot; and/ or
  • the continuum robot may further include an operational controller or joystick that operates to issue or input one or more commands or instructions to one or more processors, the input including an instruction or command to move one or more of a plurality of bending sections or portions and/or a motorized linear stage and/or a sensor, and the operational controller or joystick operates to be controlled by a user of the continuum robot.
  • the continuum robot or the steerable catheter may include a plurality of bending sections or portions and may include an endoscope camera, wherein one or more processors operate or further operate to receive one or more endoscopic images from the endoscope camera, and wherein the continuum robot further comprises a display that operates to display the one or more endoscopic images.
  • Any discussion of a state, pose, position, orientation, navigation, path, or other state type discussed herein is discussed merely as a non-limiting, non-exhaustive embodiment example, and any state or states discussed herein may be used interchangeably/alternatively or additionally with the specifically mentioned type of state.
  • Driving and/or control technique(s) may be employed to adjust, change, or control any state, pose, position, orientation, navigation, path, or other state type that may be used in one or more embodiments for a continuum robot or steerable catheter.
  • Physicians or other users of the apparatus or system may have reduced or saved labor and/or mental burden using the apparatus or system due to the navigation, control, and/or orientation (or pose, or position, etc.) feature(s) of the present disclosure. Additionally, one or more features of the present disclosure may achieve a minimized or reduced interaction with anatomy (e.g., of a patient), object, or target (e.g., tissue, one or more lungs, one or more airways, etc.) during use, which may reduce the physical and/or mental burden on a patient or target.
  • anatomy e.g., of a patient
  • object e.g., tissue, one or more lungs, one or more airways, etc.
  • a labor of a user to control and/or navigate e.g., rotate, translate, etc.
  • the imaging apparatus or system or a portion thereof e.g., a catheter, a probe, a camera, one or more sections or portions of a catheter, probe, camera, etc.
  • a labor of a user to control and/or navigate is saved or reduced via use of the navigation and/ or control technique(s) of the present disclosure.
  • an imaging device or system or a portion of the imaging device or system (e.g., a catheter, a probe, etc.), the continuum robot, and/or the steerable catheter may include multiple sections or portions, and the multiple sections or portions may be multiple bending sections or portions.
  • the imaging device or system may include manual and/or automatic navigation and/or control features.
  • Navigation, control, and/or orientation feature(s) may include, but are not limited to, implementing mapping of a pose (angle value(s), plane value(s), etc.) of a first portion or section (e.g., a tip portion or section, a distal portion or section, a predetermined or set portion or section, a user selected or defined portion or section, etc.) to a stage position/state (or a position/state of another structure being used to map path or path-like information), controlling angular position(s) of one or more of the multiple portions or sections, controlling rotational orientation or position(s) of one or more of the multiple portions or sections, controlling (manually or automatically (e.g., robotically)) one or more other portions or sections of the imaging device or system (e.g., continuum robot, steerable catheter, etc.) to match or substantially or approximately match (or be close to or similar to) the navigation/orientation/position/pose of the first portion or section in a case where the one or more other portions or
  • an imaging device or system may enter a target along a path where a first section or portion of the imaging device or system (or portion of the device or system) is used to set the navigation, control, or state path and state(s)/position(s), and each subsequent section or portion of the imaging device or system (or portion of the device or system) is controlled to follow the first section or portion such that each subsequent section or portion matches (or is similar to, approximate to, substantially matching, etc.) the orientation, position, state, etc. of the first section or portion at each location along the path.
  • each section or portion of the imaging device or system is controlled to match (or be similar to, be approximate to, be substantially matching, etc.) the prior orientation, position, state, etc. (for each section or portion) for each of the locations along the path.
  • each section or portion of the device or system may follow a leader (or more than one leader) or may use one or more RFTL and/or FTL technique(s) discussed herein.
  • an imaging or continuum robot device or system or catheter, probe, camera, etc.
  • a target, an object, a specimen, a patient e.g., a lung of a patient, an esophagus of a patient, a spline, another portion of a patient, another organ of a patient, a vessel of a patient, etc.
  • a patient e.g., a lung of a patient, an esophagus of a patient, a spline, another portion of a patient, another organ of a patient, a vessel of a patient, etc.
  • the navigation, control, orientation, and/or state feature(s) are not limited thereto, and one or more devices or systems of the present disclosure may include any other desired navigation, control, orientation, and/or state specifications or details as desired for a given application or use.
  • the first portion or section may be a distal or tip portion or section of the imaging or continuum robot device or system.
  • the first portion or section may be any predetermined or set portion or section of the imaging or continuum robot device or system, and the first portion or section may be predetermined or set manually by a user of the imaging or continuum robot device or system or may be set automatically by the imaging device or system (or by a combination of manual and automatic control).
  • a “change of orientation” or a “change of state” may be defined in terms of direction and magnitude.
  • each interpolated step may have a same direction, and each interpolated step may have a larger magnitude as each step approaches a final orientation. Due to kinematics of one or more embodiments, any motion along a single direction may be the accumulation of a small motion in that direction. The small motion may have a unique or predetermined set of wi re position or state changes to achieve the orientation change. Large or larger motion(s) in that direction may use a plurality of the small motions to achieve the large or larger motion(s).
  • Dividing a large change into a series of multiple changes of the small or predetermined/set change may be used as one way to perform interpolation.
  • Interpolation may be used in one or more embodiments to produce a desired or target motion, and at least one way to produce the desired or target motion may be to interpolate the change of wire positions or states.
  • an apparatus or system may include one or more processors that operate to: instruct or command a distal bending section or portion of a catheter or a probe of the continuum robot such that the distal bending section or portion achieves, or is disposed at, a bending pose or position, the catheter or probe of the continuum robot having a plurality of bending sections or portions and a base; store or obtain the bending pose or position of the distal bending section or portion and store or obtain a position or state of a motorized linear stage (or other structure used to map path or path-like information) that operates to move the catheter or probe of the continuum robot in a case where the one or more processors instruct or command forward motion, or a motion in a set or predetermined direction or directions, of the motorized linear stage (or other predetermined or set structure for mapping path or path-like information); generate a goal or target bending pose or position for each corresponding section or portion of the catheter or probe from, or based on, the previous
  • the navigation, movement, and/ or control may occur such that any intermediate orientations of one or more of the plurality of bending sections or portions is guided towards respective desired, predetermined, or set orientations (e.p., such that the steerable catheter, continuum robot, or other imaging device or system may reach the one or more targets).
  • FIG. 1 illustrates a simplified representation of a medical environment, such as an operating room, where a robotic catheter system 1000 may be used.
  • FIG. 2 illustrates a functional block diagram that may be used in at least one embodiment of the robotic catheter system 1000.
  • FIGS. 3A-3D represent at least one embodiment of the catheter 104 (see FIGS. 3A-3B) and bending for the catheter 104 (as shown in FIGS. 3C-3D).
  • FIG. 4 illustrates a logical block diagram that may be used for the robotic catheter system 1000.
  • the system 1000 may include a computer cart (see e.g., the controller too, 102 in FIG. 1) operatively connected to a steerable catheter or continuum robot 104 via a robotic platform 108.
  • the robotic platform 108 includes one or more than one robotic arm 132 and a rail 110 (see e.g., FIGS. 1-2) and/or linear translation stage 122 (see e.g., FIG. 2).
  • one or more embodiments of a system 1000 for performing robotic control may include one or more of the following: a display controller too, a display 101-1, a display 101-2, a controller 102, an actuator 103, a continuum device (also referred to herein as a “steerable catheter” or “an imaging device”) 104, an operating portion 105, a tracking sensor 106 e.g., an electromagnetic (EM) tracking sensor) or camera, a catheter tip position/orientation/pose/state detector 107, and a rail 110 (which may be attached to or combined with a linear translation stage 122) (for example, as shown in at least FIGS.
  • EM electromagnetic
  • the system 1000 may include one or more processors, such as, but not limited to, a display controller too, a controller 102, a console or computer 1200, a CPU 1201, any other processor or processors discussed herein, etc., that operate to execute a software program, to control the one or more control technique(s), support structure feature(s)r and/or technique(s), or other feature(s) or technique(s) discussed herein, and to control display of a navigation screen on one or more displays 101-1, 101-2, etc.
  • processors such as, but not limited to, a display controller too, a controller 102, a console or computer 1200, a CPU 1201, any other processor or processors discussed herein, etc.
  • the one or more processors may generate a three dimensional (3D) model of a structure (for example, a branching structure like airway of lungs of a patient, an object to be imaged, tissue to be imaged, etc.) based on images, such as, but not limited to, CT images, MRI images, etc.
  • the 3D model may be received by the one or more processors (e.q., the display controller 100, the controller 102, the console or computer 1200, the CPU 1201, any other processor or processors discussed herein, etc.) from another device.
  • a two-dimensional (2D) model may be used instead of 3D model in one or more embodiments.
  • the 2D or 3D model may be generated before a navigation starts.
  • the 2D or 3D model may be generated in real-time (in parallel with the navigation).
  • examples of generating a model of branching structure are explained.
  • the models may not be limited to a model of branching structure.
  • a model of a route direct to a target may be used instead of the branching structure.
  • a model of a broad space may be used, and the model may be a model of a place or a space where an observation or a work is performed by using a continuum robot 104 explained below.
  • a user U may control the robotic catheter system 1000 via a user interface unit (operation unit) to perform an intraluminal procedure on a patient P positioned on an operating table B.
  • the user interface may include at least one of a main or first display 101-1 (a first user interface unit), a second display 101-2 (a second user interface unit), and a handheld controller 105 (a third user interface unit).
  • the main or first display 101-1 may include, for example, a large display screen attached to the system 1000 and/or the controllers 101, 102 of the system 1000 or mounted on a wall of the operating room and may be, for example, designed as part of the robotic catheter system 1000 or may be part of the operating room equipment.
  • a secondary display 101-2 that is a compact (portable) display device configured to be removably attached to the robotic platform 108.
  • the second or secondary display 101-2 may include, but are not limited to, a portable tablet computer, a mobile communication device (a cellphone), a tablet, a laptop, etc.
  • the steerable catheter 104 may be actuated via an actuator unit 103.
  • the actuator unit 103 may be removably attached to the robotic platform 108 or any component thereof (e.g., the robotic arm 132, the rail 110, and/or the linear translation stage 122).
  • the handheld controller 105 may include a gamepad-like controller with a joystick havi ng shift levers and/ or push buttons, and the controller 105 may be a one-handed controller or a two-handed controller.
  • the actuator unit 103 may be enclosed in a housing having a shape of a catheter handle.
  • One or more access ports 126 may be provided in or around the catheter handle. The access port 126 may be used for inserting and/or withdrawing end effector tools and/or fluids when performing an interventional procedure of the patient P.
  • the system 1000 includes at least a system controller 102, a display controller too, and the main display 101-1.
  • the main display 101-1 may include a conventional display device such as a liquid crystal display (LCD), an OLED display, a QLED display, any other display discussed herein, any other display known to those skilled in the art, etc.
  • the main display 101-1 may provide or display a graphic interface unit (GUI) configured to display one or more views. These views may include a live view image 134, an intraoperative image 135, a preoperative image 136, and other procedural information 138. Other views that may be displayed include a model view, a navigational information view, and/or a composite view.
  • GUI graphic interface unit
  • the live image view 134 may be an image from a camera at the tip of the catheter 104.
  • the live image view 134 may also include, for example, information about the perception and navigation of the catheter 104.
  • the preoperative image 136 may include pre-acquired 3D or 2D medical images of the patient P acquired by conventional imaging modalities such as computer tomography (CT), magnetic resonance imaging (MRI), ultrasound imaging, or any other desired imaging modality.
  • CT computer tomography
  • MRI magnetic resonance imaging
  • ultrasound imaging or any other desired imaging modality.
  • the intraoperative image 135 may include images used for image guided procedure such images may be acquired by fluoroscopy or CT imaging modalities (or another desired imaging modality)-
  • the intraoperative image 135 may be augmented, combined, or correlated with information obtained from a sensor, camera image, or catheter data.
  • the sensor may be located at the distal end of the catheter 104.
  • the catheter tip tracking sensor 106 may be, for example, an electromagnetic (EM) sensor. If an EM sensor is used, a catheter tip position detector 107 may be included in the robotic catheter system 1000; the catheter tip position detector 107 may include an EM field generator operatively connected to the system controller 102.
  • EM electromagnetic
  • One or more other embodiments of the catheter/continuum robot 104 may not include or use the EM tracking sensor 106.
  • Suitable electromagnetic sensors for use with a steerable catheter may be used with any feature of the present disclosure, including the sensors discussed, for example, in U.S. Pat. No. 6,201,387 and in International Pat. Pub. WO 2020/194212 Al, which are incorporated by reference herein in their entireties.
  • the element 106 may be a camera or other imaging device/feature.
  • the display controller 100 may acquire position/orientation/navigation/pose/state (or other state) information of the continuum robot 104 from a controller 102.
  • the display controller 100 may acquire the position/orientation/navigation/pose/state (or other state) information directly from a tip position/orientation/navigation/pose/state (or other state) detector 107.
  • the continuum robot 104 maybe a catheter device (e.g., a steerable catheter or probe device).
  • the continuum robot 104 may be attachable/detachable to the actuator 103, and the continuum robot 104 may be disposable.
  • FIG. 2 illustrates the robotic catheter system 1000 including the system controller 102 operatively connected to the display controller too, which is connected to the first display 101-1 and to the second display 101-2.
  • the system controller 102 is also connected to the actuator 103 via the robotic platform 108 or any component thereof (e.g., the robotic arm 132, the rail 110, and/or the linear translation stage 122).
  • the actuator unit 103 may include a plurality of motors 144 that operate to control a plurality of drive wires 160 (while not limited to any particular number of drive wires 160, FIG. 2 shows that six (6) drive wires 160 are being used in the subject embodiment example).
  • the drive wires 160 travel through the steerable catheter or continuum robot 104.
  • One or more access ports 126 may be located on the catheter 104 (and may include an insertion/extraction detector 109).
  • the catheter 104 may include a proximal section 148 located between the actuator 103 and the proximal bending section 152, where the drive wires 160 operate to actuate the proximal bending section 152.
  • Three of the six drive wires 160 continue through the distal bending section 156 where the drive wires 160 operate to actuate the distal bending section 156 and allow for a range of movement.
  • FIG. 2 is shown with two bendable sections 152, 156 (although one or more bendable sections may be used in one or more embodiments).
  • Other embodiments as described herein may have three bendable sections (see e.g., FIGS. 3A-3D).
  • a single bending section may be provided, or alternatively, four or more bendable sections may be present in the catheter 104.
  • FIGS. 3A-3B show at least one embodiment of a continuum robot 104 that may be used in the system 1000 or any other system discussed herein.
  • FIG. 3A shows at least one embodiment of a steerable catheter 104.
  • the steerable catheter 104 may include a non- steerable proximal section 148, a steerable distal section 156, and a catheter tip 320.
  • the proximal section 148 and distal bendable section 156 (including portions 152, 154, and 156 in FIG. 3A) are joined to each other by a plurality of drive wires 160 arranged along the wall of the catheter 104.
  • the proximal section 148 is configured w ith through-holes (or thru-holes) or grooves or conduits to pass drive wires 160 from the distal section 152, 154, 156 to the actuator unit 103.
  • the distal section 152, 154, 156 is comprised of a plurality of bending segments including at least a distal segment 156, a middle segment 154, and a proximal segment 152. Each bending segment is bent by actuation of at least some of the plurality of drive wires 160 (driving members).
  • the posture of the catheter 104 may be supported by supporting wires (support members) also arranged along the wall of the catheter 104 (as discussed in U.S. Pat. Pub.
  • Each bending segment is formed by a plurality of ring-shaped components (rings) with through-holes (or thru-holes), grooves, or conduits along the wall of the rings.
  • the ringshaped components are defined as wire-guiding members 162 or anchor members 164 depending on a respective function(s) within the catheter 104.
  • the anchor members 164 are ring-shaped components onto w hich the distal end of one or more drive wires 160 are attached in one or more embodiments.
  • the wire-guiding members 162 are ring-shaped components through which some drive wires 160 slide through (without being attached thereto).
  • FIG. 3B detail “A” obtained from the identified portion of FIG. 3A illustrates at least one embodiment of a ring-shaped component (a wire-guiding member 162 or an anchor member 164).
  • Each ring-shaped component 162, 164 may include a central opening which may form a tool channel 168 and may include a plurality of conduits 166 (grooves, sub-channels, or through-holes (or thru-holes)) arranged lengthwise (and which may be equidistant from the central opening) along the annular wall of each ring-shaped component 162, 164.
  • an inner cover such as is described in U.S. Pat. Pub.
  • the non-steerable proximal section 148 may be a flexible tubular shaft and may be made of extruded polymer material.
  • the tubular shaft of the proximal section 148 also may have a central opening or tool channel 168 and plural conduits 166 along the wall of the shaft surrounding the tool channel 168.
  • An outer sheath may cover the tubular shaft and the steerable section 152, 154, 156. In this manner, at least one tool channel 168 formed inside the steerable catheter 104 proUdes passage for an imaging device and/or end effector tools from the insertion port 126 to the distal end of the steerable catheter 104.
  • the actuator unit 103 may include, in one or more embodiments, one or more servo motors or piezoelectric actuators.
  • the actuator unit 103 may operate to bend one or more of the bending segments of the catheter 104 by applying a pushing and/or pulling force to the drive wires 160.
  • each of the three bendable segments of the steerable catheter 104 has a plurality of drive wires 160. If each bendable segment is actuated by three drive wires 160, the steerable catheter 104 has nine driving wires arranged along the wall of the catheter 104. Each bendable segment of the catheter 104 is bent by the actuator unit 103 by pushing or pulling at least one of these nine drive wires 160. Force is applied to each individual drive wire in order to manipulate/ steer the catheter 104 to a desired pose.
  • the actuator unit 103 assembled with steerable catheter 104 may be mounted on the robotic platform 108 or any component thereof (e.p., the robotic arm 132, the rail 110, and/or the linear translation stage 122).
  • the robotic platform 108, the rail 110, and/or the linear translation stage 122 may include a slider and a linear motor.
  • the robotic platform 108 or any component thereof e.g., the robotic arm 132, the rail 110, and/or the linear translation stage 122 is motorized, and may be controlled by the system controller 102 to insert and remove the steerable catheter 104 to/from the target, sample, or object (e.g., the patient, the patient’s bodily lumen, one or more airways, a lung, a tissue, an organ, a target or object, a specimen, etc.).
  • An imaging device 180 that may be inserted through the tool channel 168 includes an endoscope camera (videoscope) along with illumination optics (e.g., optical fibers or LEDs) (or any other camera or imaging device, tool, etc. discussed herein or known to those skilled in the art).
  • the illumination optics provide light to irradiate the lumen and/or a lesion target which is a region of interest within the target, sample, or object (e.g., in a patient).
  • End effector tools may refer to endoscopic surgical tools including clamps, graspers, scissors, staplers, ablation or biopsy needles, and other similar tools, which serve to manipulate body parts (organs or tumorous tissue) during imaging, examination, or surgery.
  • the imaging device 180 may be what is commonly known as a chip-on-tip camera and may be color (e.g., take one or more color images) or black-and-white (e.g., take one or more black-and-white images).
  • a camera may support color and black-and-white images.
  • the imaging device 180 or camera may use fluoroscopy (N1RF, N1RAF, any other fluoroscopy discussed herein or known to those skilled in the art, etc.) in addition to another imaging modality (e.g., OCT, IVUS, any other imaging modality discussed herein or known to those skilled in the art, etc.) or may be any other camera discussed herein.
  • a tracking sensor e.g., an EM tracking sensor
  • the steerable catheter 104 and the tracking sensor 106 may be tracked by the tip position detector 107.
  • the tip position detector 107 detects a position of the tracking sensor 106, and outputs the detected positional information to the system controller 102.
  • the system controller 102 receives the positional information from the tip position detector 107, and continuously records and displays the position of the steerable catheter 104 w ith respect to the coordinate system of the target, sample, or object (e.g., a patient, a lung, an airway(s), a vessel, etc.).
  • the system controller 102 operates to control the actuator unit 103 and the robotic platform 108 or any component thereof (e.g., the robotic arm 132, the rail 110, and/or the linear translation stage 122) in accordance with the manipulation commands input by the user U via one or more of the input and/or display devices (e.g., the handheld controller 105, a GUI at the main display 101-1, touchscreen buttons at the secondary display 101-2, etc.).
  • the input and/or display devices e.g., the handheld controller 105, a GUI at the main display 101-1, touchscreen buttons at the secondary display 101-2, etc.
  • FIG. 3C and FIG. 3D show exemplary catheter tip manipulations by actuating one or more bending segments of the steerable catheter 104.
  • manipulating only the most distal segment 156 of the steerable section may change the position and orientation of the catheter tip 320.
  • manipulating one or more bending segments (152 or 154) other than the most distal segment may affect only the position of catheter tip 320, but may not affect the orientation of the catheter tip 320.
  • actuation of distal segment 156 changes the catheter tip from a position Pl having orientation 01, to a position P2 having orientation 02, to position P3 having orientation O3, to position P4 having orientation O4, etc.
  • FIG. 1 shows exemplary catheter tip manipulations by actuating one or more bending segments of the steerable catheter 104.
  • manipulating only the most distal segment 156 of the steerable section may change the position and orientation of the catheter tip 320.
  • manipulating one or more bending segments (152 or 154) other than the most distal segment may affect only the position of catheter tip 320
  • actuation of the middle segment 152 and/or the middle segment 154 may change the position of the catheter tip 320 from a position Pl having orientation 01 to a position P2 and position P3 having the same orientation 01.
  • exemplary catheter tip manipulations shown in FIG. 3C and FIG. 3D may be performed during catheter navigation (e.g., while inserting the catheter 104 through tortuous anatomies, one or more targets, one or more lungs or other organs, one or more airways, samples, objects, a patient, etc.).
  • 3D may apply namely to the targeting mode applied after the catheter tip 320 has been navigated to a predetermined distance (a targeting distance) from the target, sample, or object.
  • the actuator 103 may proceed or retreat along a rail 110 (e.p., to translate the actuator 103, the continuum robot/catheter 104, etc.), and the actuator 103 and continuum robot 104 may proceed or retreat in and out of the patient’s body or other target, object, or specimen (e.g., tissue).
  • the catheter device 104 may include a plurality of driving backbones and may include a plurality of passive sliding backbones.
  • the catheter device 104 may include at least nine (9) driving backbones and at least six (6) passive sliding backbones.
  • the catheter device 104 may include an atraumatic tip at the end of the distal section of the catheter device 104 and/or may include support structure for reducing or avoiding buckling as further discussed below.
  • FIG. 4 illustrates that a system 1000 may include the system controller 102 which may operate to execute software programs and control the display controller too to display a navigation screen (e.g., a live view image 134) on the main display 101-1 and/or the secondary display 101-2.
  • the display controller 100 may include a graphics processing unit (GPU) or a video display controller (VDC) (or any other suitable hardware discussed herein or known to those skilled in the art).
  • GPU graphics processing unit
  • VDC video display controller
  • the system controller 102 and/or the display controller 100 may include one or more computer or processing components or units, such as, but not limited to, the components, processors, or units shown in at least FIG. 22 discussed further below.
  • the system controller 102 and the display controller 100 may be configured separately.
  • the system controller 102 and the display controller too may be configured as one device. In either case, the system controller 102 and the display controller too may include substantially the same components in one or more embodiments. For example, as shown in FIG.
  • the system controller 102 and the display controller too may include a central processing unit (CPU 1201) (which may be comprised of one or more processors (microprocessors)), a random access memory (RAM 1203) module, an input/output or communication (I/O 1205) interface, a read only memory (ROM 1202), and data storage memory (e.g., a hard disk drive 1204 or solid state drive (SSD) 1204) (see e.g., also data storage 150 of FIG. 4).
  • the navigation screen is a graphical user interface (GUI) generated by a software program, but it may also be generated by firmware, or a combination of software and firmware.
  • GUI graphical user interface
  • a Solid State Drive (SSD) 1204 may be used instead of HDD 1204 as the data storage 150.
  • the one or more processors, and/or the display controller too and/or the controller 102 may include structure as shown in FIG. 22 and/or in FIGS. 23-27 as further discussed below.
  • the system controller 102 may control the steerable catheter 104 based on any known kinematic algorithms applicable to continuum or steerable catheter robots.
  • the segments or portions of the steerable catheter 104 may be controlled individually to direct the catheter tip with a combined actuation of all bendable segments or sections.
  • a controller 102 may control the catheter 104 based on an algorithm known as follow the leader (FTL) algorithm.
  • FTL follow the leader
  • the RFTL algorithm may be used.
  • a reverse FTL (RFTL) process may be implemented. This may be implemented using inverse kinematics.
  • the RFTL mode may automatically control all sections of the steerable catheter 104 to retrace the pose (or state) from the same position along the path made during insertion e.g., in a reverse or backwards order or manner).
  • the display 7 controller too may' acquire position information of the steerable catheter 104 from system controller 102. Alternatively, the display controller too may acquire the position information directly from the tip position detector 107.
  • the steerable catheter 104 may be a single-use or limited-use catheter device. In other words, the steerable catheter 104 may be attachable to, and detachable from, the actuator unit 103 to be disposable.
  • the tool may be a medical tool such as an endoscope camera, forceps, a needle, or other biopsy or ablation tools.
  • the tool may be described as an operation tool or working tool.
  • the working tool is inserted or removed through the working tool access port 126.
  • at least one embodiment of using a steerable catheter 104 to guide a tool to a target is explained.
  • the tool may include an endoscope camera or an end effector tool, which may be guided through a steerable catheter under the same principles. In a procedure there is usually a planning procedure, a registration procedure, a targeting procedure, and an operation procedure.
  • the one or more processors may generate and output a navigation screen to the one or more displays 101-1, 101-2 based on the 2D/3D model and the position/orientation/navigation/pose/state (or other state) information by executing the software.
  • the navigation screen may indicate a current position/orientation/navigation/pose/state (orother state) of the continuum robot 104 on the 2D/3D model.
  • a user may recognize the current position/orientation/navigation/pose/state (or other state) of the continuum robot 104 in the branching structure.
  • Any feature of the present disclosure may be used with any navigation/pose/state feature(s) or other feature(s) discussed in PCT App. No. PCT/US2024/031766, filed May 30, 2024, the disclosures of which are incorporated by reference herein in their entireties.
  • a user may recognize the current position of the steerable catheter 104 in the branching structure.
  • one or more end effector tools may be inserted through the access port 126 at the proximal end of the catheter 104, and such tools may be guided through the tool channel 168 of the catheter body to perform an intraluminal procedure from the distal end of the catheter 104.
  • the ROM 1202 and/ or HDD 1204 may operate to store the software in one or more embodiments.
  • the RAM 1203 may be used as a work memory.
  • the CPU 1201 may execute the software program developed in the RAM 1203.
  • the I/O or communication interface 1205 may operate to input the positional (or other state) information to the display controller too (and/or any other processor discussed herein) and to output information for displaying the navigation screen to the one or more displays 101-1, 101-2.
  • the navigation screen may be generated by the software program. In one or more other embodiments, the navigation screen may be generated by a firmware.
  • One or more devices or systems may include a tip position/orientation/navigation/pose/state (or other state) detector 107 that operates to detect a position/orientation/navigation/pose/state (or other state) of the EM tracking sensor 106 and to output the detected positional (and/or other state) information to the controller 100 or 102 (e.g., as shown in FIGS. 1-2), or to any other processor(s) discussed herein.
  • a tip position/orientation/navigation/pose/state (or other state) detector 107 that operates to detect a position/orientation/navigation/pose/state (or other state) of the EM tracking sensor 106 and to output the detected positional (and/or other state) information to the controller 100 or 102 (e.g., as shown in FIGS. 1-2), or to any other processor(s) discussed herein.
  • the controller 102 may operate to receive the positional (or other state) information of the tip of the continuum robot 104 from the tip position /orientation / navigation/pose/ state (or any other state discussed herein) detector 107.
  • the controller 100 and/ or the controller 102 operates to control the actuator 103 in accordance with the manipulation by a user (e.g., manually), and/or automatically (e.g., by a method or methods run by one or more processors using software, by the one or more processors, using automatic manipulation in combination with one or more manual manipulations or adjustments, etc.) via one or more operation/operating portions or operational controllers 105 (e.g., such as, but not limited to a joystick as shown in FIGS.
  • the one or more displays 101-1, 101-2 and/or operation portion or operational controllers 105 may be used as a user interface 3000 (also referred to as a receiving device) (e.g., as shown diagrammatically in FIG. 4).
  • a user interface 3000 also referred to as a receiving device
  • the system(s) 1000 may include, as an operation unit, the display 101-1 e.g., such as, but not limited to, a large screen user interface w ith a touch panel, first user interface unit, etc.), the display 101-2 (e.g., such as, but not limited to, a compact user interface w ith a touch panel, a second user interface unit, etc.) and the operating portion 105 (e.g., such as, but not limited to, a joystick shaped user interface unit having shift lever/ button, a third user interface unit, a gamepad, or other input device, etc.).
  • the display 101-1 e.g., such as, but not limited to, a large screen user interface w ith a touch panel, first user interface unit, etc.
  • the display 101-2 e.g., such as, but not limited to, a compact user interface w ith a touch panel, a second user interface unit, etc.
  • the operating portion 105 e.g., such as, but
  • the controller 100 and/or the controller 102 may control the continuum robot 104 based on an algorithm known as follow the leader (FTL) algorithm and/or the RFTL algorithm.
  • the FTL algorithm may be used in addition to the robotic control features of the present disclosure.
  • the middle section and the proximal section (following sections) of the continuum robot 104 may move at a first position (or other state) in the same or similar way as the distal section moved at the first position (or other state) or a second position (or state) near the first position (or state) (e.g., during insertion of the continuum robot/catheter 104, by using the navigation, movement, and/or control feature(s) of the present disclosure, etc.).
  • the middle section and the distal section of the continuum robot 104 may move at a first position or state in the same/similar/ approximately similar way as the proximal section moved at the first position or state or a second position or state near the first position (e.g., during removal of the continuum robot/catheter 104).
  • the continuum robot/catheter 104 may be removed by automatically and/or manually moving along the same or similar, or approximately same or similar, path that the continuum robot/catheter 104 used to enter a target (e.g., a body of a patient, an object, a specimen (e.g., tissue), etc.) using the FTL algorithm, including, but not limited to, using FTL with the one or more control, support structure (e.g., kink avoidance/reduction, buckling reduction/ayoidance, etc.), or other technique(s) discussed herein.
  • any feature of the present disclosure may be used with features, including, but not limited to, training feature! s), autonomous navigation feature(s), artificial intelligence feature(s), etc., as discussed in PCT App. No. PCT /US2024/037935, filed July 12, 2024, the disclosures of which are incorporated by reference herein in their entireties.
  • any of the one or more processors may be configured as one device (for example, the structural attributes of the controller too and the controller 102 may be combined into one controller or processor, such as, but not limited to, the one or more other processors discussed herein (e.y., computer, console, or processor 1200, etc.).
  • the system 1000 may include a tool channel 126 for a camera, biopsy tools, or other types of medical tools (as shown in FIGS. 1-2).
  • the tool may be a medical tool, such as an endoscope, a forceps, a needle, or other biopsy tools, etc.
  • the tool may be described as an operation tool or working tool (e.g., an imaging device, a camera, etc.).
  • the working tool may be inserted or removed through a working tool insertion slot 126 (as shown in FIGS. 1-2). Any of the features of the present disclosure may be used in combination with any of the features, including, but not limited to, the tool insertion slot, as discussed in U.S. Pat. App. No. 18/477,081, filed September 28, 2023, the disclosure of which is incorporated by reference herein in its entirety.
  • FIG. 5 is a flowchart showing steps of at least one planning procedure of an operation of the continuum robot/catheter device 104.
  • One or more of the processors discussed herein may execute the steps shown in FIG. 5, and these steps may be performed by executing a software program read from a storage medium, including, but not limited to, the ROM 1202 or HDD/SSD 1204, by CPU 1201 or by any other processor discussed herein.
  • One or more methods of planning using the continuum robot/ catheter device 104 may include one or more of the following steps: (i) In step s6ot, one or more images such, as CT or MRI images, may be acquired; (ii) In step S602, a three dimensional model of a branching structure (for example, an airway model of lungs or a model of an object, specimen or other portion of a body) may be generated based on the acquired one or more images; (iii) In step S603, a target on the branching structure may be determined (e.g., based on a user instruction, based on preset or stored information, etc.); (iv) In step S604, a route of the continuum robot/catheter device 104 to reach the target (e.g., on the branching structure) may be determined (e.g., based on a user instruction, based on preset or stored information, based on a combination of user instruction and stored or preset information, etc.); and/or (v)
  • embodiments of using a catheter device/continuum robot 104 are explained, such as, but not limited to features for performing navigation, movement, and/or robotic control technique(s), performing or using support structure (e.g., wire kink avoidance/reduction, buckling reduction/avoidance, etc.) feature(s) and/ or technique(s), or any other feature(s) and/ or technique(s) discussed herein.
  • support structure e.g., wire kink avoidance/reduction, buckling reduction/avoidance, etc.
  • Pose or state information may be stored in a lookup table or tables, and the pose or state information for one or more sections of the catheter or probe may be updated in the lookup table based on new information (e.g., environmental change(s) for the catheter or probe, movement of a target or sample, movement of a patient, user control, relaxation state changes, etc.).
  • new information e.g., environmental change(s) for the catheter or probe, movement of a target or sample, movement of a patient, user control, relaxation state changes, etc.
  • the new information or the updated information may be used to control the one or more sections of the catheter or probe more efficiently during navigation (forwards and/or backwards).
  • the previously stored pose or state may not be ideal or may work less efficiently as compared with an updated pose or state modified or updated in view of the new information (e.g., the movement, in this example).
  • one or more embodiments of the present disclosure may update or modify the pose or state information such that robotic control of the catheter or probe may work efficiently in view of the new information, movement, relaxation, and/or environmental change(s).
  • the update or change may also affect a number of other points (e.g., all points in a lookup table or tables, all points forward beyond the initially changed point, one or more future points or points beyond the initially changed point as desired, etc.).
  • the transform (or difference, change, update, etc.) between the previous pose or state and the new or updated post or state may be propagated to all points going forw ard or may be propagated to one or more of forward points (e.g., for a predetermined or set range, for a predetermined or set distance, etc.). Doing so in one or more embodiments may operate to shift all or part of the future path based on how the pose or state of the catheter or probe was adjusted, using that location as a pivot point.
  • Such update(s) may be obtained from one or more internal sources (e.g., one or more processors, one or more sensors, combination(s) thereof, etc.) or may be obtained from one or more external sources (e.g., one or more other processors, one or more external sensors, combination(s) thereof, etc.).
  • a difference between a real-time target, sample, or object (e.g., an airway) and the previous target, sample, or object (e.g., a previous airway) may be detected using machine vision (of the endoscope image) or using multiple medical images.
  • Body, target, object, or sample divergence may also be estimated from other sensors, like one measuring breathing or the motion of the body (or another predetermined or set motion or change to track).
  • an amount of transform, update, and/ or change may be different for each point, and/or may be a function of, for example, a distance from a current point.
  • One or more methods of controlling or using a continuum robot/catheter device may use one or more Hold the Line techniques, Close the Gap techniques, and/or Stay the Course techniques, such as, but not limited to, the techniques discussed in U.S. Pat. App. No. 63/585,128 filed on September 25, 2023, the disclosure of which is incorporated herein by reference in its entirety.
  • a catheter or probe may be controlled to stay the desired course.
  • a pose, position, or state of a section or sections, or of a portion or portions, of the catheter or probe may be adjusted to minimize any deviation of a pose, position, or state of one or more next (e.p., subsequent, following, proximal, future, Middle/proximal, etc.) sections out of the predetermined, targeted, desired trajectory and maximum motion along the trajectory.
  • the coordinates and the trajectory of subsequent/following/next/future sections may be known, set, or determined, and information for one or more prior sections may be known, set, or determined.
  • the system controller 102 may operate to perform a robotic control mode and/or an autonomous navigation mode.
  • the autonomous navigation mode may include or comprise: (1) a perception step, (2) a planning step, and (3) a control step.
  • the system controller 102 may receive an endoscope view (or imaging data) and may analyze the endoscope view (or imaging data) to find addressable airways from the current position/orientation of the steerable catheter 104. At an end of this analysis, the system controller 102 identifies or perceives these addressable airways as paths in the endoscope view (or imaging data).
  • the planning step is a step to determine a target path, which is the destination for the steerable catheter 104. While there are a couple of different approaches to select one of the paths as the target path, the present disclosure uniquely includes means to reflect user instructions concurrently for the decision of a target path among the identified or perceived paths. Once the system 1000 determines the target paths while considering concurrent user instructions, the target path is sent to the next step, i.e., the control step.
  • the control step is a step to control the steerable catheter 104 and the linear translation stage 122 (or any other portion of the robotic platform 108) to navigate the steerable catheter 104 to the target path, pose, state, etc. This step may also be performed as an automatic step.
  • the system controller 102 operates to use information relating to the real time endoscope view (e.g., the view 134), the target path, and an internal design & status information on the robotic catheter system 1000.
  • the real-time endoscope view r 134 may be displayed in a main display 101-1 (as a user input/output device) in the system 1000.
  • the user may see the airw ays in the real-time endoscope view 134 through the main display 101-1.
  • This real-time endoscope view 134 may also be sent to the system controller 102.
  • the system controller 102 may process the real-time endoscope view 134 and may identify path candidates by using image processing algorithms.
  • the system controller 102 may select the paths with the designed computation processes, and then may display the paths with a circle, octagon, or other geometric shape with the real-time endoscope view 134, for example, as discussed in PCT App. No. PCT/US2024/037935, filed July 12, 2024, the disclosures of which are incorporated by reference herein in their entireties.
  • the system controller 102 may provide a cursor so that the user may indicate the target path by moving the cursor w ith the joystick 105.
  • the system controller 102 operates to recognize the path with the cursor as the target path.
  • system controller 102 may pause the motion of the actuator unit 103 and the linear translation stage 122 while the user is moving the cursor so that the user may select the target path w ith a minimal change of the real-time endoscope view 134 and paths since the system 1000 would not move in such a scenario.
  • the system controller 102 may control the steerable catheter 104 based on any known kinematic algorithms applicable to continuum or snake-like catheter robots.
  • the system controller controls the steerable catheter 104 based on an algorithm known as follow the leader (FTL) algorithm or on the RFTL algorithm.
  • FTL leader
  • the most distal segment 156 is actively controlled with forward kinematic values, while the middle segment 154 and the other middle or proximal segment 152 (following sections) of the steerable catheter 104 move at a first position in the same way as the distal section moved at the first position or a second position near the first position.
  • any other algorithm may be applied to control a continuum robot or catheter/probe.
  • Orientations mapped to two subsequent stage positions/states may have a specific orientation difference bet veen the orientations.
  • the later (or second) stage position/state (or position/state of the another structure) has a same change in orientation that the earlier (or first) stage position/state (or position/state of the another structure) received such that the pose/ state difference did not change.
  • the smoothing process may include an additional step of a “small motion”, yvhich operates to cause the pose/state difference to change by an amount of that small motion.
  • the small motion step operates to direct that orientation/ state in a table towards a proper (e.g., set, desired, predetermined, selected, etc.) direction, while also maintaining a semblance or configuration of the prior path/state before the smoothing process was applied. Therefore, in one or more embodiments, it may be most efficient and effective to combine and compare wi re positions or states to or w ith prior orientations or states while using a smoothing process to maintain the pre-existing orientation changes.
  • a proper e.g., set, desired, predetermined, selected, etc.
  • a catheter or probe may transition, move, or adjust using a shortest possible volume.
  • using the shortest possible volume my reduce or minimize an amount of disruption to positions or states of one or more (or all) of the distal/following sections or portions of the catheter or probe.
  • a process or algorithm may perform the transitioning, moving, or adjusting process more efficiently than computing a transformation stackup of each section or portion of the catheter or probe.
  • each interpolated step aims tow ards the final orientation in a desired direction such that any prior orientation which the interpolated step is combined with will also aim towards the desired direction to achieve the final orientation.
  • an apparatus or system may include one or more processors that operate to: receive or obtain an image or images showing pose or position (or other state) information of a tip section of a catheter or probe having a plurality of sections including at least the tip section; track a history of the pose or position (or other state) information of the tip section of the catheter or probe during a period of time; and use the history' of the pose or position (or other state) information of the tip section to determine how to align or transition, move, or adjust (e.p., robotically, manually, automatically, etc.) each section of the plurality of sections of the catheter or probe.
  • processors that operate to: receive or obtain an image or images showing pose or position (or other state) information of a tip section of a catheter or probe having a plurality of sections including at least the tip section; track a history of the pose or position (or other state) information of the tip section of the catheter or probe during a period of time; and use the history' of the pose or position (or other state) information of the tip section to determine how
  • one or more additional image or images may be received or obtained to show the catheter or probe after each section of the plurality of sections of the catheter or probe has been aligned or adjusted e.g., robotically, manually, automatically, etc.) based on the history of the pose or position (or other state) information of the tip section.
  • the apparatus or system may include a display to display the image or images showing the aligned or adjusted sections of the catheter or probe.
  • the pose or position (or other state) information may include, but is not limited to, a target pose or position (or other state) or a final pose or position (or other state) that the tip section is set to reach, an interpolated pose or position (or other state) of the tip section (e.g., an interpolation of the tip section between two positions or poses (or other states) (e.g., between pose or position (or other state) A to pose or position (or other state) B) where the apparatus or system sends pose (or other state) change information in steps based on a desired, set, or predetermined speed; between poses or positions where each pose or position (or other state) of the catheter or probe takes or is disposed is tracked during the transition; etc.), and a measured pose or position (or other state) (e.g., using tracked poses or positions (or other states), using encoder positions (or other states) of each wire motor, etc.) where the one or more processors may further operate to calculate or derive a current position or position
  • each pose or position may be converted (e.g., via the one or more processors) between the following formats: Drive Wire Positions (or state(s)); and/or Coordinates (three-dimensional (3D) Position and Orientation (or other state(s))).
  • an apparatus or system may include a camera deployed at a tip of a catheter or probe and may be bent w ith the catheter or probe, and/or the camera may be detachably attached to, or removably inserted into, the steerable catheter or probe.
  • an apparatus or system may include a display controller, or the one or more processors may display the image or images for display on a display.
  • imaging modalities including, for example, CT (computed tomography), MRI (magnetic resonance imaging), NIRF (near infrared fluorescence), NIRAF (near infrared auto-fluorescence), OCT (optical coherence tomography), SEE (spectrally encoded endoscope), IVUS (intravascular ultrasound), PET (positron emission tomography), X-ray imaging, combinations or hybrids thereof, other imaging modalities discussed herein, any combination thereof, or any modality known to those skilled in the art.
  • CT computed tomography
  • MRI magnetic resonance imaging
  • NIRF near infrared fluorescence
  • NIRAF near infrared auto-fluorescence
  • OCT optical coherence tomography
  • SEE spectrally encoded endoscope
  • IVUS intravascular ultrasound
  • PET positron emission tomography
  • X-ray imaging combinations or hybrids thereof, other imaging modalities discussed herein, any combination thereof, or any modality known to those skilled in the
  • configurations are described as a robotic bronchoscope (or other scope, such as, but not limited to, an endoscope, other scopes discussed herein, or known to those skilled in the art) arrangement or a continuum robot arrangement that may be equipped with a tool channel for an imaging device and medical tools, where the imaging device and the medical tools may be exchanged by inserting and retracting the imaging device and/or the medical tools via the tool channel (see e.g., tool channel 126 in FIGS. 1-2, see e.g., imaging device or tool (e.g., camera) 160, and see e.g., medical tool 133 in FIG. 1).
  • a robotic bronchoscope or other scope, such as, but not limited to, an endoscope, other scopes discussed herein, or known to those skilled in the art
  • a continuum robot arrangement that may be equipped with a tool channel for an imaging device and medical tools, where the imaging device and the medical tools may be exchanged by inserting and retracting the imaging device and/or the medical tools via the
  • the imaging device can be a camera or other imaging device
  • the medical tool can be a biopsy tool or other medical device.
  • Configurations may facilitate placement of medical tools, catheters, needles or the like, and may be free standing, cart mounted, patient mounted, movably mounted, combinations thereof, or the like.
  • the present disclosure is not limited to any particular configuration.
  • the robotic catheter or steerable catheter arrangement may be used in association with one or more displays and control devices and/or processors, such as those discussed herein (see e.g., one or more device or system configurations shown in one or more of FIGS. 1-27 of the present disclosure).
  • the display device may display, on a monitor, an image captured by the imaging device, and the display device may have a display coordinate used for displaying the captured image.
  • a display coordinate used for displaying the captured image.
  • top, bottom, right, and left portions of the monitor(s) or display(s) may be defined by axes of the displaying coordinate system/grid, and a relative position of the captured image or images against the monitor may be defined on the displaying coordinate system/grid.
  • the robotic catheter or scope arrangement may use one or more imaging devices e.g., a catheter or probe 104, a camera, a sensor, any other imaging device discussed herein, etc.) and one or more display devices ⁇ e.g., a display 101-1, a display 101-2, a screen 1209, any other display discussed herein, etc.) to facilitate viewing, imaging, and/or characterizing tissue, a sample, or other object using one or a combination of the imaging modalities described herein.
  • imaging devices e.g., a catheter or probe 104, a camera, a sensor, any other imaging device discussed herein, etc.
  • display devices e.g., a display 101-1, a display 101-2, a screen 1209, any other display discussed herein, etc.
  • a control device or a portion of a catheter, imaging device, robotic catheter, etc. may control a moving direction of the tool channel (e.g., the tool channel 126) or the camera (e.g., the camera 180).
  • the tool channel or the camera may be bent according to a control by the system (such as, but not limited to, the features discussed herein and shown in at least FIGS. 3A-3D).
  • the system may have an operational controller (for example, a gamepad, a joystick 105 (see e.g., FIGS. 1-2), etc.) and a control coordinate.
  • the control coordinate system/grid may define a moving (or bending) direction of the tool channel or the camera in one or more embodiments, including, but not limited to, in a case where a particular command is input by the operational controller. For example, in a case where a user inputs an “up” command via the operational controller, then the tool channel or the camera moves toward a direction which is defined by the control coordinate system/grid as an upward direction.
  • a calibration may be performed.
  • a direction to which the tool channel or the camera moves or is bent according to a particular command up, down, turn right, or turn left; alternatively, a command set may include a first direction, a second direction opposite or substantially opposite from or to the first direction, a third direction that is about or is 90 degrees from or to the first direction, and a fourth direction that is opposite or substantially opposite from or to the third direction) is adjusted to match a direction (top, bottom, right or left) on a display (or on the display coordinate).
  • the calibration is performed so that an upward of the displayed image on the display coordinate corresponds to an upward direction on the control coordinate (a direction to which the tool channel or the camera moves according to an “up” command).
  • first, second, third, and fourth directions on the display correspond to the first, second, third, and fourth directions of the control coordinate (e.g., of the tool channel or camera).
  • the tool channel or the camera is bent to an upward or first direction on the control coordinate.
  • the direction to which the tool channel or the camera is bent corresponds to an upward or first direction of the capture image displayed on the display.
  • a rotation function of a display of the captured image on the display coordination may be performed.
  • the orientation of the camera view (top, bottom, right, and/or left) should match with a conventional orientation of the camera view that physicians or other medical personnel typically see in their normal catheter, imaging device, scope, etc. procedure: for example, for a bronchoscope, the right and left main bronchus may be displayed horizontally on a monitor or display (e.g., the display 101-1, the display 101-2, the display or screen 1209, etc.).
  • a user may rotate the captured image on the display coordinate so that the right and left main bronchus are displayed horizontally on the monitor or display (e.g., the display 101-1, the display 101-2, the display or screen 1209, etc.).
  • the captured image is rotated on the display coordinate after a calibration is performed, a relationship between the top, bottom, right, and left (or first, second, third, and/or fourth directions) of the displayed image and top, bottom, right, and left (or corresponding first, second, third, and/or fourth directions) of the monitor may be changed.
  • the tool channel or the camera may move or may be bent in the same way regardless of the rotation of the displayed image when a particular command is received (for example, a command to let the tool channel or the camera (or a capturing direction of the camera) move upward, downward, right, or left or to move in the first direction, second direction, third direction, or fourth direction).
  • a calibration or arrangement of the imaging device, scope, catheter or probe may use an orientation feature 406 discussed below.
  • a particular command for example, tilting a joystick to up, down, right, or left; tilting the joystick in a first direction, the second direction, the third direction, or the fourth direction; etc.
  • the tool channel or the camera may not be bent to the direction corresponding to the direction of the top (or of the first direction) of the monitor but may be bent to a direction to a diagonally upward of the monitor. This may complicate user interaction.
  • an operator may map or calibrate the orientation of the camera view, the user interface device, and the robot endeffector.
  • bronchoscopists may not be enough for bronchoscopists, in one or more situations, because (1) the right and left main bronchus may be displayed in arbitrary direction in this case, and (2) bronchoscopists rely on how the bronchi look to navigate a bronchoscope and bronchoscopists typically confirm the location of the bronchoscope using or based on how the right and left main bronchus look like.
  • a direction to which a tool channel or a camera moves or is bent is corrected automatically in a case where a displayed image is rotated.
  • the robot configurational embodiments described below enable to keep a correspondence between a direction on a monitor (top, bottom, right, or left of the monitor; a first, second, third, or fourth direction(s) of the monitor, etc.), a direction the tool channel or the camera moves on the monitor or display (e.q., the display 101-1, the display 101-2, the display or screen 1209, etc.) according to a particular directional command (up, down, turn right, or turn left; first direction, second direction, third direction, or fourth direction, etc.), and a user interface device even in a case where the displayed image is rotated.
  • medical image processing implements functioning through use of one or more processes, techniques, algorithms, or other steps discussed herein, that operate to provide rapid, accurate, cost-effective, and minimally invasive structure and manufacture/use techniques for structure of a catheter and/or a catheter tip.
  • one or more configurations are described that find use in therapeutic or diagnostic procedures in anatomical regions including the respiratory system, the digestive system, the bronchus, the lung, the liver, esophagus, stomach, colon, urinary tract, or other areas.
  • a medical apparatus or system provides advantageous features to continuum robots or steerable catheters by providing rapid, accurate, cost-effective, and minimally invasive structure and manufacture/ use techniques for structure of a catheter and/ ora catheter tip and providing work efficiency to physicians during a medical procedure and rapid, accurate, and minimally invasive techniques for patients.
  • a medical apparatus or system 1000 may be provided in the form of a continuum robot or steerable catheter (or other imaging device) assembly or configuration that provides medical imaging with rapid, accurate, cost-effective, and minimally invasive structure and manufacture/use techniques for structure of a catheter and/ or a catheter tip according to one or more embodiments.
  • FIGS. 2-4, 6-20B, and 22 show one or more hardw are configurations of the system 1000 as discussed above for FIG. 1.
  • the system 1000 (or any other system discussed herein) may include one or more medical tools 133 and one or more medical devices or catheters/probes 104 (see e.g., as shown in FIG. 1).
  • the medical tool 133 may be referred to as a “biopsy tool”, a “medical procedure tool”, an “imaging tool”, etc. in one or more embodiments and the medical device 104 is referred to as a “catheter” (or probe, continuum robot, steerable catheter, etc.). That said, the medical tool 133 and the medical device 104 are not limited thereto, and a variety of other types of tools, devices, configurations, or arrangements also falls within the scope of the present disclosure, including, but not limited to, for example, a bronchoscope, catheter, robotic bronchoscope, robotic catheter, endoscope, colonoscope, ablation device, sheath, guidewire, needle, probe, forceps, another medical tool, a camera, an imaging device, etc.
  • the controller or joystick 105 may have a housing with an elongated handle or handle section which may be manually grasped, and one or more input devices including, for example, a lever or a button or another input device that allows a user, such as a physician, nurse, technician, etc., to send a command to the medical apparatus or system 1000 (or any other system or apparatus discussed herein) to move the catheter 104.
  • input devices including, for example, a lever or a button or another input device that allows a user, such as a physician, nurse, technician, etc., to send a command to the medical apparatus or system 1000 (or any other system or apparatus discussed herein) to move the catheter 104.
  • the controller or joystick 105 may execute software, computer instructions, algorithms, etc., so the user may complete all operations with the hand-held controller 105 by holding it with one hand, and/ or the controller or joystick 105 may operate to communicate with one or more processors or controllers (e.g., processor 1200, controller 102, display controller too, any other processor, computer, or controller discussed herein or known to those skilled in the art, etc.) that operate to execute software, computer instructions, algorithms, methods, other features, etc., so the user may complete any and/or all operations.
  • processors or controllers e.g., processor 1200, controller 102, display controller too, any other processor, computer, or controller discussed herein or known to those skilled in the art, etc.
  • the medical device 104 may be configured as or operate as a bronchoscope, catheter, endoscope, or another type of medical device.
  • the system 1000 (or any other system discussed herein) may use an imaging device, where the imaging device may be a mechanical, digital, or electronic device configured to record, store, or transmit visual images, e.g., a camera, a camcorder, a motion picture camera, etc.
  • the display controller too, the controller 102, a processor (such as, but not limited to, the processor 1200, any other processor discussed herein, etc.), etc. may operate to execute software, computer instructions, algorithms, methods, etc., and control a display of a navigation screen on the display 101-1, other types of imagery or information on the minidisplay or other display 101-2, a display on a screen 1209, etc.
  • the display controller 100, the controller 102, a processor (such as, but not limited to, the processor 1200, any other processor discussed herein, etc.), etc.
  • the 3D model may be received by the display controller too, the controller 102, a processor (such as, but not limited to, the processor 1200, any other processor discussed herein, etc.), etc. from another device.
  • the display controller too, the controller 102, a processor such as, but not limited to, the processor 1200, any other processor discussed herein, etc.
  • the display controller too, the controller 102, a processor may acquire catheter position information from the tracking sensor 106 (e.g., an electromagnetic (EM) tracking sensor) and/or from the catheter tip position/orientation/pose/state detector 107.
  • EM electromagnetic
  • the display controller too, the controller 102, a processor (such as, but not limited to, the processor 1200, any other processor discussed herein, etc.), etc. may generate and output a navigation screen to any of the displays 101-1, 101-2, 1209, etc. based on the 3D model and the catheter position information by executing the software and/ or by performing one or more algorithms, methods, and/or other features of the present disclosure.
  • One or more of the displays 101-1, 101-2, 1209, etc. may display a current position of the catheter 103 on the 3D model, and/or the display controller 100, the controller 102, a processor (such as, but not limited to, the processor 1200, any other processor discussed herein, etc.), etc. may execute a correction of the acquired 3D model based on the catheter position information so as to minimize a divergence between the catheter position and a path mapped out on the 3D model.
  • the display controller 100, the controller 102, a processor such as, but not limited to, the processor 1200, any other processor discussed herein, etc.), etc. and/or any console thereof may include one or more or a combination of levers, keys, buttons, switches, a mouse, a keyboard, etc., to control the elements of the system 1000 (or any other system or apparatus discussed herein) and each may have configurational components, as shown in FIGS. 4 and 10 as aforementioned, and may include other elements or components as discussed herein or know n to those skilled in the art.
  • a processor such as, but not limited to, the processor 1200, any other processor discussed herein, etc.
  • any console thereof may include one or more or a combination of levers, keys, buttons, switches, a mouse, a keyboard, etc., to control the elements of the system 1000 (or any other system or apparatus discussed herein) and each may have configurational components, as shown in FIGS. 4 and 10 as aforementioned, and may include other elements or components as discussed herein or know
  • the components of the system 1000 may be interconnected with medical instruments or a variety of other devices, and may be controlled independently, externally, or remotely by the display controller too, the controller 102, a processor (such as, but not limited to, the processor 1200, any other processor discussed herein, etc.), etc.
  • a sensor such as, but not limited to, the tracking sensor 106, a tip position detector 107, any other sensor discussed herein, etc. may monitor, measure or detect various types of data of the system 1000 (or any other apparatus or system discussed herein), and may transmit or send the sensor readings or data to a host through a network.
  • the I/O interface or communication 1205 may interconnect various components with the medical apparatus or system 1000 to transfer data or information, or facilitate communication, to or from the apparatus or system 1000.
  • a power source may be used to provide power to the medical apparatus or system 1000 (or any other apparatus or system discussed herein) to maintain a regulated power supply, and may operate in a power-on mode, a power-off mode, and/or other modes.
  • the power source may include or comprise a battery contained or included in the medical apparatus or system 1000 (or other apparatus or system discussed herein) and/or may include an external power source such as line power or AC power from a power outlet that may interconnect with the medical apparatus or system 1000 (or other system or apparatus of the present disclosure) through an AC/DC adapter and a DC/DC converter, or an AC/DC converter (or using any other configuration discussed herein or known to those skilled in the art) in order to adapt the power voltage from a source into one or more voltages used by components in the medical apparatus or system 1000 (and/or any other system or apparatus discussed herein).
  • an external power source such as line power or AC power from a power outlet that may interconnect with the medical apparatus or system 1000 (or other system or apparatus of the present disclosure) through an AC/DC adapter and a DC/DC converter, or an AC/DC converter (or using any other configuration discussed herein or known to those skilled in the art) in order to adapt the power voltage from a source into one or more voltages used by components in the medical
  • any of the sensors or detectors discussed herein, including, but not limited to, the sensor 106, the detector 107, etc. may include one or more or a combination of a processor, detection circuitry, memory, hardware, software, firmware, and may include other circuitry, elements, or components.
  • Any of the sensors or detectors discussed herein, including, but not limited to, the sensor 106, the detector 107, etc. may be a plurality of sensors and may acquire sensor information output from one or more sensors that detect force, motion, current
  • any of the sensors or detectors discussed herein, including, but not limited to, the sensor 106, the detector 107, etc. may include a multi-axis acceleration or accelerometer sensor and a multi-axis gyroscope sensor, may be a combination of an acceleration and gyroscope sensors, may include other sensors, and may be configured through the use of a piezoelectric transducer, a mechanical switch, a single axis accelerometer, a multi-axis accelerometer, or other types of configurations. Any? of the sensors or detectors discussed herein, including, but not limited to, the sensor 106, the detector 107, etc.
  • the medical apparatus or system 1000 may monitor, detect, measure, record, or store physical, operational, quantifiable data or other characteristic parameters of the medical apparatus or system 1000 (or any other system or apparatus discussed herein) including one or more or a combination of a force, impact, shock, drop, fall, movement, acceleration, deceleration, velocity, rotation, temperature, pressure position, orientation, motion, or other types of data of the medical apparatus or system 1000 (and/or other apparatus or system discussed herein) in multiple axes, in a multi-dimensional manner, along an x axis, y axis, z axis, or any combination thereof, and may generate sensor readings, information, data, a digital signal, an electronic signal, or other types of information corresponding to the detected state.
  • the medical apparatus or system 1000 may transmit or send the sensor reading data wirelessly or in a wired manner to a remote host or server.
  • Any of the sensors or detectors discussed herein, including, but not limited to, the sensor 106, the detector 107, etc. may? be interrogated and may generate a sensor reading signal or information that may be processed in real time, stored, post processed at a later time, or combinations thereof.
  • the information or data that is generated by? any? of the sensors or detectors discussed herein, including, but not limited to, the sensor 106, the detector 107, etc. may be processed, demodulated, filtered, or conditioned to remove noise or other types of signals.
  • any of the sensors or detectors discussed herein, including, but not limited to, the sensor 106, the detector 107, etc. may include one or more or a combination of a force sensor, an acceleration, deceleration, or accelerometer sensor, a gyroscope sensor, a power sensor, a battery’ sensor, a proximity sensor, a motion sensor, a position sensor, a rotation sensor, a magnetic sensor, a barometric sensor, an illumination sensor, a pressure sensor, an angular position sensor, a temperature sensor, an altimeter sensor, an infrared sensor, a sound sensor, an air monitoring sensor, a piezoelectric sensor, a strain gauge sensor, a sound sensor, a vibration sensor, a depth sensor, and may include other types of sensors.
  • the acceleration sensor may sense or measure the displacement of mass of a component of the medical apparatus or system 1000 with a position or sense the speed of a motion of the component of the medical apparatus or system 1000 (or other apparatus or system).
  • the gyroscope sensor may sense or measure angular velocity or an angle of motion and may measure movement of the medical apparatus or system 1000 in up to six total degrees of freedom in three-dimensional space including three degrees of translation freedom along cartesian x, y, and z coordinates and orientation changes between those axes through rotation along one or more or of a yaw axis, a pitch axis, a roll axis, and a horizontal axis.
  • Yaw is when the component of the medical apparatus or system 1000 (or other apparatus or system) tw ists left or right on a vertical axis. Rotation on the front-to-back axis is called roll. Rotation from side to side is called pitch.
  • the acceleration sensor may include, for example, a gravity sensor, a drop detection sensor, etc.
  • the gyroscope sensor may include an angular velocity sensor, a handshake correction sensor, a geomagnetism sensor, etc.
  • the position sensor may be a global positioning system (GPS) sensor that receives data output from a GPS.
  • GPS global positioning system
  • the longitudinal and latitude of a current position may be obtained from access points of a radio frequency identification device (RFID) and a WiFi device and information output from wireless base stations, for example, so that these detections may be used as position sensors.
  • RFID radio frequency identification device
  • WiFi WiFi device
  • the medical device 104 may be configured as a catheter 104 as aforementioned and as shown in FIGS. 1-4, and may move based on any of the aforementioned algorithms, including, but not limited to, the FTL algorithm, the RFTL algorithm, any other algorithm known to those skilled in the art, etc.
  • the middle section and the proximal section (following sections) of the catheter 104 may move at a first position in the same way as the distal section moved at the first position or a second position near the first position.
  • the display controller too, the controller 102, a processor (such as, but not limited to, the processor 1200, any other processor discussed herein, etc.), etc. may operate to cause the catheter 104 to be placed in a pathway of a an object, target, or sample (e.g., a lung, an organ, a patient, tissue, etc.).
  • the display controller too, the controller 102, a processor (such as, but not limited to, the processor 1200, any other processor discussed herein, etc.), etc. may be configured to control a continuum robot or steerable catheter having an atraumatic tip, such as the catheter tip 320, which may include or be an atraumatic tip.
  • One or more embodiments of an apparatus of the present disclosure may include or have: a continuum robot including one or more bending sections, wherein the one or more bending sections is/ are bent by one or more driving wires; and one or more support structures or sleeves that operate to support one or more portions of the one or more driving w ires.
  • the one or more support sleeves operate to one or more of the following: (i) support the one or more portions of the one or more driving wires to prevent wire buckling; (ii) provide smooth and continuous support for the one or more driving wires to prevent buckling at a transition point where the one or more driving wires slide through one body and into a second body (e.g., where no unsupported gaps or ledges may exist for the one or more driving wires to catch on); and/ or (iii) reduce friction of the one or more driving wires.
  • the one or more driving wires may terminate at a distal end of the one or more bending sections.
  • the one or more support sleeves may be made of or include stainless steel, and the one or more support sleeves may each have an inside diameter between o.oi inches and 0.02 inches or between 0.01 inches and 0.015 inches. In one or more embodiments, an inside diameter of each of the one or more support sleeves may be larger than an outside diameter of the one or more driving wires (e.cj., the one or more support sleeves may have corresponding one or more driving wires passing through a respective support sleeve of the one or more support sleeves).
  • the one or more bending sections may include a first section proximal to the transition area and a second section attached to the first section and distal to the transition area.
  • the first section of the one or more bending sections and second section of the one or more bending sections may be constructed of thermoplastic material, and/or the first section of the one or more bending sections and the second section of the one or more bending sections may be constructed of or include thermoplastic material having different durometer scales or values.
  • the one or more driving wires may be bent or operate to bend at least in the transition area.
  • the transition area may include or comprise of a first section proximal to the one or more bending sections and a second section attached to the first section and distal to the one or more bending sections.
  • the first section of the transition area and the section of the transition area may include or be constructed of thermoplastic material having different durometer scales or values.
  • the continuum robot may include or comprise a steerable catheter, and the hub may be connected to the catheter body.
  • the hub may be bonded to the catheter or continuum robot at one or more attachment points.
  • multiple support sleeves of the one or more support sleeves and a tube of a tool channel may all be bonded to the hub and the steerable catheter or continuum robot.
  • One or more embodiments may use or include an outer jacket that operates to add robustness to the catheter/continuum robot-hub connection.
  • One or more embodiments may have multiple push/pull drive/driving wires for the one or more driving wires, and the multiple push/pull driving w ires may be protected at or in a transition area, via the one or more support sleeves, from a body of the hub to a shaft of the catheter or continuum robot.
  • An inner diameter (ID) of the one or more support sleeves may be sized and shaped to support the one or more drive wires and prevent the one or more drive w ires from buckling.
  • the drive wires may terminate in a distal bending end of the shaft of the catheter or continuum robot.
  • a proximal end of each of the one or more support sleeves may be bonded to the hub body.
  • the one or more support sleeves may be bonded into sub-lumens of the catheter body or continuum robot, and/or the one or more support structures or sleeves operate to be made of or include stainless steel.
  • the one or more support sleeves may have a specific orientation or alignment with respect to the hub and catheter/continuum robot.
  • a shaft of the catheter body or continuum robot may have multiple sub-lumens with two different diameters.
  • a diameter of the proximal sub-lumen(s) may be larger at a proximal end of the catheter shaft or continuum robot to fit the one or more support sleeves, and larger sublumens may terminate at a set distance into the catheter shaft or continuum robot.
  • a diameter of the distal sub-lumen(s) may be smaller to support the one or more driving wires, and may start after the end (e.g., a distal end) of the one or more support sleeves and may extend up to the distal end of the catheter shaft or continuum body.
  • the lumens or sub-lumens that operate to support the one or more driving wires may extend through and from an inside of the hub body and may extend into and through one or more portions of the catheter shaft (e.g., from the proximal end of the catheter shaft to the distal end of the catheter shaft, from the proximal end of the catheter shaft and extend along the catheter shaft for a set or predetermined distance, from the proximal end of the catheter shaft and extend along and through a portion of the catheter shaft, etc.).
  • One or more embodiments may operate to provide or have ease of assembly. For example, using the one or more support sleeves to connect the hub to the catheter automatically aligns the lumen/drive wire positions/states/poses/etc. as well as a distance between the hub and the catheter or continuum robot in one or more embodiments. Multiple assembly steps may be combined into a single reflow step in a case where using thermoplastic materials.
  • FIG. 6 is a system block diagram of at least one embodiment example of a bendable medical and/or imaging device system 1000 discussed herein.
  • the system 1000 may be set up as shown in one or more of FIGS. 1-4.
  • the bendable medical device system 1000 may comprise: the actuator or driving unit 103 (also referred to herein as a “driver”) for driving the wires 160; the base platform 108, the linear translation stage 122, and/or the rail 110; the bendable imaging or medical device 104 (also referred to herein as a “catheter 104” or a “continuum robot 104”), a positioning cart (e.g., having the display controller too, the controller 102, etc.), and an operation console or computer 1200 (also referred to herein as a “controller 1200” or a “processor 1200”), which may further include or comprise, in one or more embodiments, push-button, thumbstick, and/or joystick operational console or controller features and navigation software (e.g., software on a display 101, 101-1, 101-2, 1209, any other display discussed herein, etc.).
  • the bendable medical and/or imaging device system 1000 may operate to interact with external system component(s) and clinical user(s) to facilitate use in
  • the continuum robot 104 may comprise or include drive wires i6od, t6oe, i6of, which may be connected to connection portions 123, 124, and 125, respectively, found on an end disc 164b, for controlling the middle bending section 154. Additional drive wires (3 for each of the other bendable sections 156 and 152) 160a, 160g, 160b, i6oh, 160c, i6oi, may be attached at distal ends of each bendable section 156 and 152, to the respective end disc 164a and 164c.
  • each bending section may be operated similarly in one or more embodiments, this discussion will focus on at least one bending section, here the middle bending section 154, to explain the mechanism.
  • the posture, pose, state, position, etc. of the bending section 154 is controlled by pushing and pulling the wires t6od, i6oe, and io6f by using the actuators 103.
  • the continuum robot or steerable catheter 104 may attach to a catheter shaft 148, which may be disposed on a base platform 108, linear translation stage 122, and/or the rail 110 in one or more embodiments (see e.g., FIG. 1 and FIG. 6) and may be moved by the base platform 108, linear translation stage 122, and/or the rail 110 in the longitudinal direction.
  • a catheter shaft 148 which may be disposed on a base platform 108, linear translation stage 122, and/or the rail 110 in one or more embodiments (see e.g., FIG. 1 and FIG. 6) and may be moved by the base platform 108, linear translation stage 122, and/or the rail 110 in the longitudinal direction.
  • it is possible to advance and retract the robot or catheter 104 into a target, object, or sample structure by advancing and retracting the base platform 108, the linear translation stage 122, and/or the rail 110.
  • An operational console or computer 1200 may operate to indicate a driving amount to the base platform 108, the linear translation stage 122, and/or the rail 110 and, independently, to the actuation unit or actuator 103.
  • the operational console or computer 1200 may also be described or eluded to as a control system, processor, or controller.
  • the operational console or computer 1200 may include dedicated hardware including a field-programmable gate array (“FPGA”) and the like; may be a computer including a storage unit, a work memory, and a central processing unit (“CPU”); and/ or may include one or more features discussed herein for any computer, processor, or controller, including, but not limited to, the one or more features shown in FIGS.
  • FPGA field-programmable gate array
  • CPU central processing unit
  • the storage unit may store a software program corresponding to an algorithm of any of the control systems described herein and the central processing unit may operate to expand the program in the work memory and execute the program line by line such that, in one or more embodiments, the computer may operate as the operational console 1200.
  • the operational console 1200 is communicably connected to the base platform 108, the linear translation stage 122, and/ or the rail 110 and to the actuation unit or actuator 103, and the operational console 1200 may operate to send signals representing the driving amount and configuration to these control targets, which may be obtained from an end user through push buttons, joystick, or the like, or may be obtained from another source (e.p., a memory, another computer, automatically determined by the one or more processors discussed herein, etc.).
  • a source e.p., a memory, another computer, automatically determined by the one or more processors discussed herein, etc.
  • the continuum robot or steerable catheter 104 in one or more embodiments may include at least one distal bending section 156 with robotic insertion and removal of the continuum robot or catheter 104 from the target, object, or sample.
  • the distal bending section 156 is attached to the catheter shaft
  • the catheter shaft 148 may retain the overall diameter of the bending section 156 in one or more embodiments.
  • the catheter shaft 148 may be attached to the hub 130, which operates to attach to the actuation unit or actuator 103.
  • the actuation unit or actuator 103 houses one or multiple actuators for pushing and/or pulling each of the drive wires 160 (for example, in the embodiment example of FIG. 7, there are a plurality of drive wires i6oa-t6oi).
  • FIG. 8 and FIG. 9A show the transition area from a hub 130 with a larger diameter to a catheter shaft 148.
  • the structure shown provides continuous support to drive the wires 160 throughout the transition area.
  • the drive wires 160 operate to slide through one or more respective support sleeves 60 in the hub body 130 into respective lumens 166 in the catheter shaft 148.
  • At least two push/pull drive wires 160 slide through the hub body 130 into the catheter shaft 148. Both drive wires 160 are protected by the respective support sleeves 60 throughout the transition.
  • the support sleeves 60 may be partially inserted into the proximal lumens 166 of the catheter shaft 148, up to a specific or predetermined distance.
  • the support sleeves 60 may be bonded to the hub 130 at the other end or at the opposing end(s), may be press fit, may be bonded to the hub 130 along a set or predetermined length of the support sleeves 60, may be disposed in the hub 130 and/or the catheter shaft 148, and/or may be inserted into the hub 130 and/or the catheter shaft 148 during manufacture and/or use.
  • the inner diameter of the support sleeve(s) 60 may be sized to provide nonbuckling support (c.y ., to avoid/ reduce buckling) and/or to avoid/ reduce friction for the drive wires 160.
  • the catheter shaft 148 may be a solid body with two different lumen diameters.
  • the sub-lumens may be sized to fit the support sleeves 60 (and the drive wires 160 extending or passing through the support sleeves 60), and at the distal end, the sub-lumens may be sized to support the drive wires 160 (e.g., the drive wires 160 may extend through the sub-lumens at the distal end without the support sleeves 60 in one or more embodiments).
  • the hub 130 may house or enclose two support sleeves 60, two drive wires 160, with one part catheter body/shaft 104/148- In one or more of such embodiments, no tool channel may be used; in one or more other embodiments, one or more tool channels maybe used.
  • the diameter of the drive w ires 160 may be 0.0095” diameter nitinol wires.
  • the support sleeves 60 may be stainless steel hypotubes, with an inner diameter of 0.0115” and an outer diameter of 0.016”.
  • a hypotube of the support sleeves 60 may have an inner diameter of 0.012” and an outer diameter of 0.018”.
  • the catheter shaft 148 may be any material with an outer diameter of 0.1461”.
  • the catheter shaft 148 may have two proximal lumens sized and shaped for the support sleeves 60 (which may be, but are not limited to, 0.0165” diameter), w ith the 0.0165” lumens continuing to a depth of set or predetermined length, e.g., 5mm. Beyond the set or predetermined (e.cy ., 5mm) depth for the support sleeves 60, the lumen size may decrease to 0.012” diameter to accommodate the size and shape of the respective drive wares 160.
  • the at least two drive wires 160 may be evenly spaced from a center of the catheter shaft 148 at a distance of 0.1225” apart.
  • a benefit of the one or more support structure embodiments of the present disclosure is that the drive wires 160 are fully supported from buckling throughout the transition between the hub 130 and the catheter 104 and/or the catheter shaft 148.
  • the wires 160 are protected from buckling where the wires 160 exit the respective support sleeve 60 hypotube (e.p., while exiting the respective lumen 64 for the support sleeves 60) and enter the small sub-lumen 166 of the catheter shaft 148.
  • one or more features of the present disclosure prevents the issues that may occur where hypotubes could not be inserted into catheter lumens.
  • gaps with exposed wire between a catheter shaft and the support sleeves may be created by any small differences in hypotube lengths.
  • inserting the hypotubes of the support sleeves 60 into the catheter lumens 64 as discussed herein operates to prevent such gaps w ith unsupported wire from forming or existing.
  • a further benefit of the one or more support structure embodiments of the present disclosure is that the positions of the drive wires 160 may be automatically aligned to the corresponding hub 130 locations. Without such support structure, an alignment step of locating specific drive wires and loading the drive wires one at a time under a microscope into the corresponding channels on the hub 130 may be employed. This is a time-consuming manual step and also created many opportunities for error, such as attaching a drive wire 160 to the wrong pusher/puller or actuator 103. Additionally, because the hypotubes could not be inserted into the catheter 104 and/or the catheter shaft 148, the distance between the catheter 104 or catheter shaft 148 and the hub 130 is prone to being inconsistent.
  • the catheter 104 and/or the catheter shaft 148 was also free to twist with respect to the hub 130 during assembly, allow ing for even more wire angle errors. As such, it is preferable that one or more embodiments of the present disclosure employ the one or more support structure features and/ or techniques discussed herein.
  • each drive wire 160 and the overall position of the catheter 104 and/or the catheter shaft 148 relative to the hub 130 can be assembled with better accuracy and repeatability, while also requiring less time to assemble.
  • a tool channel tube 69 may be inserted through a center of, or through about the center of, the catheter shaft 148 and the hub body 130 to create a tool channel 168 for one or more tools (e.q., such as, but not limited to, biopsy tools) which may be inserted and removed during operation or use e.g., as shown in at least FIGS. 10-11).
  • the catheter shaft 148 may have an additional central lumen, which may be sized as a tool channel tube 69.
  • the tool channel tube 69 may be bonded to one or both of the hub 130 and the catheter shaft 148 at the outer diameter of the tool channel tube 69.
  • the tool channel tube 69 inner diameter may be sized and shaped to allow standard tools (such as, but not limited to, biopsy tools) to pass through the tool channel tube 69.
  • a hub 130 may have three drive wires 160, three support sleeves 60, and a tool channel 168 and/or a tool channel tube 69 extending through the hub 130 and into the catheter 104 and/or the catheter shaft 148, and there may be a one-part catheter body.
  • the catheter shaft 148, the three drive wires 160, and the support sleeve hypotubes 60 may be the same materials and have the same dimensions as described above.
  • the catheter shaft 148 may include a tool channel tube 69, which may have (but is not limited to) a 0.101” diameter.
  • the tool channel tube 69 may be a stainless steel hypotube with an outer diameter of 0.097” and an inner diameter of 0.087” in one or more embodiments.
  • the inner diameter of the tool channel 168 may be sized and shaped to fit standard (e.g., 2.0 mm) medical (e.g., biopsy) tools and/ or imaging tools.
  • the tool channel 168 may be centered in the catheter shaft 148 in one or more embodiments with a minimum w all thickness for manufacturability between the tool channel lumen 168’ and the drive wire support lumens 166.
  • the tool channel tube 69 may be bonded with adhesive to the catheter shaft 148 and to the hub body 130 in one or more embodiments. In one or more embodiments w here a tool channel tube 69 may not be used, the tool channel may be the size and shape of the tool channel 168 and/or the tool channel lumen 168’.
  • Using the tool channel tube 69 in one or more embodiments boasts the following advantages: (i) such use adds functionality to the catheter 104 to use medical (e.g., biopsy) tools and endoscopes and/or to use imaging tools; (ii) such use adds an attachment method from the catheter 104 and/or the catheter shaft 148 to the hub 130 (for example, the hub 130 and the catheter 104 and/or the catheter shaft 148 maybe positioned and bonded at a specific or set distance in relation to each other); and (iii) such use provides more rigid support to an inner wall of the catheter shaft 148 at the transition from the hub 130 to the catheter 104 and/ or the catheter shaft 148, as the transition area may be weak due to the difference between the rigid hub body 130 and a much more flexible catheter shaft 148.
  • medical e.g., biopsy
  • endoscopes e.g., endoscopes and/or to use imaging tools
  • an attachment method from the catheter 104 and/or the catheter shaft 148 to the hub 130 for example,
  • a hub 130 may have three (3) drive wires 160, three support sleeves 60, a tool channel 168 through the hub 130 where the catheter body may be a two-part catheter body of thermoplastic extrusions reflowed together (see e.g., FIG. 14 details). While not limited thereto, the materials and dimensions of such support structure may be the same as described above. There are three (3) drive wires 160 with the support sleeves 60 and the tool channel tube 69 connecting the hub 130 to the catheter shaft 148 (as shown in FIGS. 12, 13A, and 13B).
  • the catheter body is made of two parts, a proximal catheter shaft 70 and a distal catheter shaft 72 as shown in FIG. 12.
  • Both catheter shaft components (70 and 72) are specifically made of a thermoplastic material extruded with the cross section shown in FIGS. 13A and 13B, respectively.
  • both the proximal and distal catheter shaft extrusions (70 and 72) are Pebax 72D material.
  • the catheter shaft extrusions (70 and 72) are reflowed together, with the support sleeves 60 also reflowed into the lumens 64 at a specific position.
  • the tool channel 168 may also be a tube made of thermoplastic material, such as, but not limited to, a Pebax lined braided tube w ith 0.090” inner diameter and 0.104” outer diameter.
  • the support sleeves 60 and the two multi-lumen extrusions (70 and 72) may be reflowed together at one or more of the following: -180° Celsius or at 180 0 Celsius; at about 160 0 Celsius to about 200 0 Celsius; and/or at about 160 0 Celsius to about 200 0 Celsius for 20 seconds to 120 seconds.
  • the inner diameter of the two catheter shaft extrusions (70 and 72) is supported by a 0.101” PTFE coated mandrel (best shown by the PTFE mandrel 82 in FIG. 14).
  • the thermoplastic-lined tool channel 168 may also be reflowed in this step using a 0.090” PTFE coated mandrel to support the inner diameter or any other mandrel or structure known to those skilled in the art.
  • the support sleeve hypotubes 60 and extrusion lumens 166 may be filled with gx 0.0113” PTFE coated mandrels (which may be removed post-reflow) to maintain the correct set or predetermined lumen diameter.
  • the tool channel tube 69 may also be reflowed to the catheter extrusions (70 and 72) in the same step.
  • FIG. 15A shows at least one embodiment example of a reflowed two-part catheter shaft and support sleeves.
  • FIG. 15B shows at least one embodiment example of a reflowed catheter shaft including a tool channel.
  • Advantages of at least one embodiment of a reflowed two-part catheter extrusion method of the present disclosure include at least the following: (i) drive wire 160 buckling support to reduce/ avoid buckling: The lumen 166 for the drive wires 160 preferably is smooth and continuous throughout the entire transition area.
  • the catheter shaft extrusions (70 and 72) form to the shape of the hypotubes 60 and the mandrels 82 to create a seamless inner wall at a specific diameter that operates to support the drive wires 160;
  • Attachment point between the hub 130 and the catheter 104 and/or the catheter shaft 148 The hypotubes 60 are held in place after reflowing, so a fixture can be used to ensure that the hypotubes 60 are reflowed at a specific location and/or with even spacing, which allows the hypotubes 60 to be consistently assembled at the same set distance;
  • Reduced assembly time Multiple components of the catheter 104 may be reflowed together in a single assembly step; and
  • a w ire path and/or the drive wire lumens 166 may be completely sealed from an outside of the catheter 104 (as discussed below).
  • the assembly may bond the catheter 104 to the tool channel tube 69 and/or the tool channel 168, and may bond an outer jacket 78 (shown in FIGS. 16, 17A, and 17B) over the catheter 104 and/ or the catheter shaft 148, etc.
  • all components may be reflowed together all at once in a more repeatable process, which leads to the drive wire lumens 166 being completely sealed from the outside (Ease of assembly / robustness).
  • the overall cleanliness of the device/assembly process (blocks out dirt and debris) is advantageous, as is the fact that adhesives and other process aids cannot enter the lumens 166, 168’, and/ or 168 and, therefore, cannot contact the drive wires 160 during the assembly process.
  • the reflowed assembly with all hypotubes 60 and drive wires 160, may be manipulated/assembled in a variety of ways without concern for twisting, damaging, or unintentionally gluing the drive wires 160.
  • the catheter 104 may be bonded with adhesive to the tool channel tube 69 and/ or to the hub body 130, because there is no risk of the adhesive reaching the drive wires 160.
  • a hub 130 may have any set or predetermined number (e.g., nine (9)) drive wires 160 and support sleeves 60 (as shown in FIGS. 16-17B).
  • a two-part catheter body of reflowed thermoplastic extrusions, with different durometers may be used to create an angled/curved entry path for hypotubes 60 (and may also use a reflowed tool channel 168 and reflowed outer jacket 78 reinforcement).
  • the structure, materials, and reflowed assembly method may the same as described above.
  • the proximal catheter shaft extrusion 70 may be made of a lower durometer thermoplastic material than the distal catheter shaft extrusion 72.
  • FIG. 17A shows a crosssection view of the proximal catheter shaft 70 of FIG. 16, and FIG. 17B shows a cross-section view of the distal catheter shaft 72 of FIG. 16.
  • proximal and distal catheter shafts may be Pebax 55D and 72D extrusion, respectively. While not limited thereto, the proximal catheter shaft 70 may be made of Pebax 55D multi-lumen extrusion with 0.0165” lumen diameter, and the distal catheter shaft 72 may be made of Pebax 72D extrusion with 0.0120” lumen diameter.
  • the nine (9) lumens 166 for the support sleeves 60 in the catheter 104 extrusion may be evenly spaced around a central tool channel lumen 168 (or around a longitudinal axis extending along a length of the catheter 104), corresponding to nine (9) evenly spaced connection points on the hub body 130.
  • the lower durometer proximal catheter shaft 70 material is able to flex more than the higher durometer distal catheter shaft 72 material, allowing for easier transition and assembly.
  • FIG. 18A shows at least one embodiment example having an angled hypotube path using lower durometer proximal extrusion.
  • the outer jacket tube 78 may be added to surround the catheter body 104 and cover the seam between the tw o catheter shaft extrusions (70 and 72) to reinforce the outer wall.
  • An outer jacket 78 of thermoplastic material may be added to the reflowed assembly process.
  • FIGS. 18B and 18C show an outer jacket reflowed over two-part catheter shaft extrusions pre- reflow and post-reflow r , respectively.
  • the outer jacket 78 also helps to reinforce the outer wall thickness of reflowed proximal catheter shaft (or shaft extrusion) 70 and the connection with the distal catheter shaft (or shaft extrusion) 72; and (iii) additional drive wires 160 may be desirable for increased control points on the catheter 104 and may provide the ability to bend the catheter 104 to multiple bends and bend planes.
  • the outer wall of the catheter 104 and/or the catheter shaft 148 may need reinforcement to prevent the support sleeves 60 from tearing or from exhibiting deformation, which may be provided by the outer jacket 78.
  • additional drive wires 160 may be desirable for increased control points on the catheter 140 and/or may provide an ability to bend the catheter 104 to multiple bends and bend planes.
  • the catheter shaft 148 may include or comprise of two parts, an angled proximal catheter shaft 70 and a cylindrical distal catheter shaft 72.
  • the materials and dimensions of the drive wires 160, the support sleeve hypotubes 60, and the cylindrical distal catheter shaft 72 may be the same as described above.
  • the angled proximal catheter shaft 70 may have an angled or curved sub-lumens 64 for the hypotubes 60, and may be of any material w 7 hich is stiffer than the distal catheter extrusion 72.
  • FIG. 20A show 7 s a cross-sectional view of the proximal catheter shaft 70 of FIG. 19, and FIG. 20B show 7 s a cross-sectional view of the distal catheter shaft 72 of FIG. 19.
  • FIGS. 20A show 7 s a cross-sectional view of the proximal catheter shaft 70 of FIG. 19, and FIG. 20B show 7 s a cross-sectional view of the distal catheter shaft 72 of FIG. 19.
  • utilizing an angled proximal catheter body include at least: (i) Further improved catheter 104 (and/or catheter shaft 148) to hub 130 connection from a large hub 130 diameter to a very’ small catheter 104 diameter; (ii) Stiffer material of the sub-lumens 64 may provide a more defined bending path and better bending support for the hypotubes 60 and drive wires 160 than the reflowed thermoplastic extrusion; and (iii) The outer wall of the angled proximal catheter shaft 70 may also be stiffer and more durable and/or protective for the delicate hypotubes 60 and wires 160 at this transition point.
  • Any reflowable polymer material compatible with the catheter or continuum robot, may be used for one or more features of the present disclosure.
  • the polymer material may be Pebax (or any other block copolymer variation of PEBA (polyether block amide)) .
  • One or more components of the catheter 104 may be made of higher durometer than one or more other components of the catheter 104 so that the higher durometer may provide useful structure for the catheter during reflow processes.
  • the method may further include disposing or incorporating a sensor (e.tj., an electromagnetic (EM) sensor) or a camera within the geometry and/or structure of the distal tip and/or the atraumatic tip piece.
  • a sensor e.tj., an electromagnetic (EM) sensor
  • the sensor or camera may be used for tracking, and the sensor or camera may be seated (or fully seated) within the distal tip or atraumatic tip piece and be protected by or within the distal tip piece.
  • the method may include fully integrating the sensor or camera into the tip to better maintain or to achieve a smooth outer diameter on the catheter.
  • the aforementioned geometrical and/or structural achievements of the manufacturing processes may be included in and possessed by the geometry 7 and/or structure of any catheter of the present disclosure.
  • a storage medium stores instructions or a program for causing one or more processors of an apparatus or system to perform a method of manufacturing a robotic catheter or imaging apparatus using one or more reflow technique(s) as discussed herein.
  • an apparatus for performing navigation control and/or for controlling, manufacturing, or using a catheter and/or catheter tip may include a flexible medical device or tool; and one or more processors that operate to: bend a distal portion of the flexible medical device or tool; and advance the flexible medical device or tool through a pathway, wherein the flexible medical device or tool may be advanced through the pathway in a substantially centered manner.
  • the flexible medical device or tool may have multiple bending sections, and the one or more processors may further operate to control or command the multiple bending sections of the flexible medical device or tool using one or more of the following modes: a Follow the Leader (FTL) mode, a Reverse Follow the Leader (RFTL) mode, etc.
  • the flexible medical device or tool may include a catheter or scope and the catheter or scope may be part of, include, or be attached to an imaging apparatus, such as, but not limited to, an endoscope, a catheter, a probe, a bronchoscope, or any other imaging device discussed herein or known to those skilled in the art.
  • a method for controlling an apparatus including a flexible medical device or tool that operates to perform navigation control and/or for controlling, manufacturing, or using a catheter and/or catheter tip may include: bending a distal portion of the flexible medical device or tool; and advancing the flexible medical device or tool through a pathway, wherein the flexible medical device or tool may be advanced through the pathway in a substantially centered manner.
  • the flexible medical device or tool may have multiple bending sections, and the method may further include controlling or commanding the multiple bending sections of the flexible medical device or tool using one or more of the following modes: a Follow the Leader (FTL) process or mode, a Reserve Follow the Leader (RFTL) process or mode, etc.
  • a non-transitory computer-readable storage medium storing at least one program for causing a computer to execute a method for controlling an apparatus including a flexible medical device or tool that operates to perform navigation control and/or for controlling, manufacturing, or using a catheter and/or catheter tip, where the method may include: bending a distal portion of the flexible medical device or tool; and advancing the flexible medical device or tool through a pathway, wherein the flexible medical device or tool may be advanced through the pathway in a substantially centered manner.
  • the method may include any other feature discussed herein.
  • the scope may comprise, for example, an anoscope, an arthroscope, a bronchoscope, a colonoscope, a colposcope, a cystoscope, an esophagoscope, a gastroscope, a laparoscope, a laryngoscope, a neuroendoscope, a proctoscope, a sigmoidoscope, a thoracoscope, an ureteroscope, or another device.
  • the scope preferably includes or comprises a bronchoscope.
  • any units described throughout the present disclosure are merely for illustrative purposes and may operate as modules for implementing processes in one or more embodiments described in the present disclosure. However, one or more embodiments of the present disclosure are not limited thereto.
  • the term “unit”, as used herein, may generally refer to firmware, software, hardware, or other component, such as circuitry, etc., or any combination thereof, that is used to effectuate a purpose.
  • the modules may be hardware units (such as circuitry, firmw are, a field programmable gate array, a digital signal processor, an application specific integrated circuit, any other hardware discussed herein or known to those
  • the medical apparatus or system 1000 of FIG. 1 may be configured as a continuum robot or steerable catheter arrangement with a multi-sectional catheter or probe configuration and follow the leader technology (or other control or movement, or tip, feature(s) and/or technique(s) discussed herein) to allow for precise catheter tip movement.
  • Additional features or aspects of present disclosure may also advantageously implement one or more Al (artificial intelligence) or machine learning algorithms, processes, techniques, or the like, to implement a method comprising: advancing the medical tool or catheter through a pathway and/or controlling a medical tool or catheter as discussed herein.
  • Al techniques may use a neural network, a random forest algorithm, a cognitive computing system, a rules-based engine, other Al network structure discussed herein or known to those skilled in the art, etc., and are trained based on a set of data to assess types of data and generate output.
  • a training algorithm may be configured to implement a method comprising: advancing the medical tool or catheter through a pathway, wherein the medical tool or catheter is advanced through the pathway in a substantially centered manner.
  • An Al -implemented algorithm may also be used to train a model to perform the manufacture and/or use of the continuum robot(s) and/or steerable catheter(s) having a tip piece and/or having support structure (e.g., to avoid/reduce wire kink, to avoid/reduce drive wire friction, to avoid/reduce wire buckling, etc.) as discussed herein.
  • support structure e.g., to avoid/reduce wire kink, to avoid/reduce drive wire friction, to avoid/reduce wire buckling, etc.
  • FIG. 21 is a flowchart showing steps of at least one procedure for performing correction, adjustment, and/or smoothing of a continuum robot/ catheter device (e.g., such as continuum robot/ catheter device 104).
  • One or more of the processors discussed herein may execute the steps shown in FIG. 21, and these steps may be performed by executing a softw are program read from a storage medium, including, but not limited to, the ROM 1202 or HDD 150, by CPU 1201 or by any other processor discussed herein.
  • One or more methods of performing correction, adjustment, and/or smoothing for a catheter or probe of a continuum robot device or system may include one or more of the following steps: (i) in step S1300, instructing a distal bending section or portion of a catheter or a probe of a continuum robot such that the distal bending section or portion achieves, or is disposed at, a bending pose or position; (ii) in step S1301, storing or obtaining the bending pose or position of the distal bending section or portion and storing or obtaining a position of a motorized linear stage that operates to move the catheter or probe of the continuum robot in a case where a forward motion, or a motion in a set or predetermined direction or directions, of the motorized linear stage is instructed or commanded; (iii) in step S1302, generating a goal or target bending pose or position (or other state) for each corresponding section or portion of the catheter or probe from, or based on, the
  • a user may provide an operation input through an input element or device, and the continuum robot apparatus or system 104/1000 may receive information of the input element and one or more input/output devices, which may include, but are not limited to, a receiver, a transmitter, a speaker, a display, an imaging sensor, a user input device, which may include a keyboard, a keypad, a mouse, a position tracked stylus, a position tracked probe, a foot switch, a microphone, etc.
  • input/output devices which may include, but are not limited to, a receiver, a transmitter, a speaker, a display, an imaging sensor, a user input device, which may include a keyboard, a keypad, a mouse, a position tracked stylus, a position tracked probe, a foot switch, a microphone, etc.
  • a guide device, component, or unit may include one or more buttons, knobs, switches, etc., that a user may use to adjust various parameters of the continuum robot 104/1000, such as the speed (e.p., rotational speed, translational speed, etc.), angle or plane, or other parameters.
  • the continuum robot 104/1000 may be interconnected with medical instruments or a variety of other devices, and may be controlled independently, externally, or remotely a communication interface, such as, but not limited to the communication interface 1205.
  • the communication interface 1205 may be configured as a circuit or other device for communicating with components included in the apparatus or system 1000, and with various external apparatuses connected to the apparatus via a network.
  • the communication interface 1205 may store information to be output in a transfer packet and may output the transfer packet to an external apparatus via the network by communication technology such as Transmission Control Protocol/Internet Protocol (TCP/IP).
  • TCP/IP Transmission Control Protocol/Internet Protocol
  • the apparatus may include a plurality of communication circuits according to a desired communication form.
  • the CPU 1202, the communication interface 1205, and other components of the computer 1200 may interface with other elements including, for example, one or more of an external storage, a display, a keyboard, a mouse, a sensor, a microphone, a speaker, a projector, a scanner, a display, an illumination device, etc.
  • One or more control, adjustment, correction, and/or smoothing features of the present disclosure may be used with one or more image correction or adjustment features in one or more embodiments.
  • One or more adjustments, corrections, or smoothing functions for a catheter or probe device and/or a continuum robot may adjust a path of one or more sections or portions of the catheter or probe device and/or the continuum robot (e.g., the continuum robot 104, the continuum robot 1000, etc.), and one or more embodiments may make a corresponding adjustment or correction to an image view.
  • the medical tool may be an endoscope, a bronchoscope, any other medical tool discussed herein, any other medical tool known to those skilled in the art, etc.
  • a computer such as the console or computer 1200, may perform any of the steps, processes, and/or techniques discussed herein for any apparatus and/or system being manufactured or used, any of the embodiments shown in FIGS. 1-27, any other apparatus or system discussed herein, etc.
  • a continuum robot There are many ways to control a continuum robot, correct or adjust an image or a path (or one or more sections or portions of) a continuum robot (or other probe or catheter device or system), manufacture or use a catheter having a tip and/or support structure, or perform any other measurement or process discussed herein, to perform continuum robot method(s) or algorithm(s), and/or to control at least one continuum robot device/apparatus, system and/or storage medium, digital as well as analog.
  • a computer such as the console or computer 1200, may be dedicated to control and/or use continuum robot devices, systems, methods, and/or storage mediums for use therewith described herein.
  • the one or more detectors, sensors, cameras, or other components of the apparatus or system embodiments may transmit the digital or analog signals to a processor or a computer such as, but not limited to, an image processor or display controller too, a controller 102, a CPU 1201, a processor or computer 1200 (see e.g., at least FIGS. 1-4, 6-20B, and 22-27), a combination thereof, etc.
  • the image processor may be a dedicated image processor or a general purpose processor that is configured to process images.
  • the computer 1200 may be used in place of, or in addition to, the image processor or display controller too and/ or the controller 102 (or any other processor or controller discussed herein, such as, but not limited to, the computer 1200, etc.).
  • the image processor may include an ADC and receive analog signals from the one or more detectors or sensors of the system 1000 (or any other system discussed herein).
  • the image processor may include one or more of a CPU, DSP, FPGA, ASIC, or some other processing circuitry.
  • the image processor may include memory for storing image, data, and instructions.
  • the image processor may generate one or more images based on the information provided by the one or more detectors, sensors, or cameras.
  • a computer or processor discussed herein such as, but not limited to, a processor of the devices, apparatuses or systems of FIGS. 1-4, 6-20B, and 22-27, the computer 1200, the image processor, etc. may also include one or more components further discussed herein below.
  • Electrical analog signals obtained from the output of the system 1000 or the components thereof, and/or from the devices, apparatuses, or systems of FIGS. 1-4, 6-20B, and 22-27, may be converted to digital signals to be analyzed with a computer, such as, but not limited to, the computers or controllers too, 102 of FIG. 1, the computer 1200, etc.
  • a continuum robot As aforementioned, there are many ways to control a continuum robot, correct or adjust an image, correct, adjust, or smooth a path (or section or portion) of a continuum robot, manufacture or use a continuum robot or steerable catheter having a tip and/or support structure, or perform any other measurement or process discussed herein, to perform continuum robot method(s) or algorithm(s), and/or to control at least one continuum robot device/apparatus, system and/or storage medium, digital as well as analog.
  • a computer such as the computer or controllers too, 102 of FIG. 1, the console or computer 1200, etc., may be dedicated to the control and the monitoring of the continuum robot devices, systems, methods and/or storage mediums described herein.
  • the electric signals used for imaging may be sent to one or more processors, such as, but not limited to, the processors or controllers too, 102 of FIGS. 1-4, a computer 1200 (see e.g., FIG. 6, FIG. 22, etc. as discussed further below, via cable(s) or wire(s), such as, but not limited to, the cable(s) or vx ire(s) 113 (see FIG. 22). Additionally or alternatively, the computers or processors discussed herein are interchangeable, and may operate to perform any of the feature(s) and method(s) discussed herein.
  • a computer system 1200 may include a central processing unit (“CPU”) 1201, a ROM 1202, a RAM 1203, a communication interface 1205 (also referred as an Input/Output or I/O interface), a hard disk (and/or other storage device, such as, but not limited to, an SSD) 1204, a screen (or monitor interface) 1209, a keyboard (or input interface; may also include a mouse or other input device in addition to the keyboard) 1210 and a BUS (or “Bus”) or other connection lines (e.g., connection line 1213) between one or more of the aforementioned components (e.g., as shown in FIG.
  • a computer system 1200 may comprise one or more of the aforementioned components.
  • a computer system 1200 may include a CPU 1201, a RAM 1203, an input/output (I/O) interface (such as the communication interface 1205) and a bus (which may include one or more lines 1213 as a communication system between components of the computer system 1200; in one or more embodiments, the computer system 1200 and at least the CPU 1201 thereof may communicate with the one or more aforementioned components of a continuum robot device or system using same, such as, but not limited to, the system 1000, the devices/systems of FIGS. 1-4, 6-20B, etc., and/or the systems/apparatuses or other components of FIG.
  • the CPU 1201 is configured to read and perform computer-executable instructions stored in a storage medium.
  • the computer-executable instructions may include those for the performance of the methods and/or calculations described herein.
  • the computer system 1200 may include one or more additional processors in addition to CPU 1201, and such processors, including the CPU 1201, may be used for controlling and/or manufacturing a device, system, or storage medium for use w ith same or for use with any continuum robot technique(s), and/ or use or manufacture catheter tip and/or support structure feature(s) and/or technique(s) discussed herein.
  • the system 1200 may further include one or more processors connected via a network connection (e.g., v ia network 1206).
  • the CPU 1201 and any additional processor being used by the system 1200 may be located in the same telecom network or in different telecom networks (e.g., performing, manufacturing, controlling, calculation, and/or using technique(s) may be controlled remotely).
  • the I/O or communication interface 1205 provides communication interfaces to input and output devices, which may include the one or more of the aforementioned components of any of the systems discussed herein (e.g., the controller too, the controller 102, the displays 101-1, 101-2, the actuator 103, the continuum device or catheter 104, the operating portion or controller 105, the tracking sensor 106, the position detector 107, the rail 110, etc.), a microphone, a communication cable and a network (either wired or wireless), a keyboard 1210, a mouse, a touch screen or screen 1209, a light pen and so on.
  • the communication interface of the computer 1200 may connect to other components discussed herein via line 113 (as diagrammatically shown in FIG. 22).
  • the Monitor interface or screen 1209 provides communication interfaces thereto.
  • Any methods and/ or data of the present disclosure such as, but not limited to, the methods for using and/or controlling a continuum robot or catheter device, system, or storage medium for use with same and/or method(s) for imaging, performing tissue or sample characterization or analysis, performing diagnosis, planning and/or examination, for performing control or adjustment techniques (e.g., to a path of, to a pose or position of, or to one or more sections or portions of, a continuum robot, a catheter or a probe), for using or manufacturing catheter, catheter tip, and/or support structure feature(s) and/ or technique(s), and/or for performing image correction or adjustment or other technique(s), as discussed herein, maybe stored on a computer-readable storage medium.
  • control or adjustment techniques e.g., to a path of, to a pose or position of, or to one or more sections or portions of, a continuum robot, a catheter or a probe
  • control or adjustment techniques e.g., to a path of, to a pose or position of, or
  • a computer-readable and/or writable storage medium used commonly such as, but not limited to, one or more of a hard disk (e.g., the harddisk 1204, a magnetic disk, etc.), a flash memory, a CD, an optical disc (e.g., a compact disc (“CD”) a digital versatile disc (“DVD”), a Blu-rayTM disc, etc.), a magneto-optical disk, a random-access memory (“RAM”) (such as the RAM 1203), a DRAM, a read only memory (“ROM”), a storage of distributed computing systems, a memory card, or the like (e.g., other semiconductor memory’, such as, but not limited to, a non-volatile memory’ card, a solid state drive (SSD) (see storage 1204 may be an SSD instead of a hard disk in one or more embodiments; see also, storage 150 in FIG.
  • a hard disk e.g., the harddisk 1204, a magnetic disk, etc.
  • a flash memory
  • the computer-readable storage medium may be a non-transitory computer-readable medium, and/or the computer-readable medium may comprise all computer-readable media, with the sole exception being a transitory 7 , propagating signal in one or more embodiments.
  • the computer-readable storage medium may include media that store information for predetermined, limited, or short period(s) of time and/or only in the presence of power, such as, but not limited to Random Access Memory (RAM), register memory 7 , processor cache(s), etc.
  • Embodiment(s) of the present disclosure may also be realized by a computer of a system or apparatus that reads out and executes computer executable instructions (e.g., one or more programs) recorded on a storage medium (yvhich may also be referred to more fully as a “non- transitory computer-readable storage medium”) to perform the functions of one or more of the above-described embodiment(s) and/or that includes one or more circuits (e.g., application specific integrated circuit (ASIC)) for performing the functions of one or more of the above-described embodiment(s), and by a method performed by the computer of the system or apparatus by, for example, reading out and executing the computer executable instructions from the storage medium to perform the functions of one or more of the aboy e- described embodiment(s) and/or controlling the one or more circuits to perform the functions of one or more of the above-described embodiment(s).
  • computer executable instructions e.g., one or more programs
  • a storage medium yvhich may also
  • the methods, devices, systems, and computer-readable storage mediums related to the processors may be achieved utilizing suitable hardware, such as that illustrated in the figures.
  • suitable hardware such as that illustrated in the figures.
  • Functionality of one or more aspects of the present disclosure may be achieved utilizing suitable hardyvare, such as that illustrated in FIG. 22.
  • Such hardyvare may be implemented utilizing any of the known technologies, such as standard digital circuitry, any of the known processors that are operable to execute software and/or firmware programs, one or more programmable digital devices or systems, such as programmable read only memories (PROMs), programmable array logic devices (PALs), etc.
  • the CPU 1200, 1201 (as shown in FIG. 22, and/or which may be included in the computer, processor, controller and/or CPU too, 102, 1201, etc. of FIGS. 1-4, FIGS. 6-20B, and FIGS. 22-27), etc.
  • microprocessors may also include and/or be made of one or more microprocessors, nanoprocessors, one or more graphics processing units (“GPUs”; also called a visual processing unit (“VPU”)), one or more Field Programmable Gate Arrays (“FPGAs”), or other types of processing components (e.g., application specific integrated circuit(s) (ASIC)).
  • GPUs graphics processing units
  • FPGAs Field Programmable Gate Arrays
  • ASIC application specific integrated circuit
  • the various aspects of the present disclosure may be implemented by way of software and/or firmware program(s) that may be stored on suitable storage medium (e.g., computer-readable storage medium, hard drive, etc.) or media (such as floppy disk(s), memory chip(s), etc.) for transportability and/or distribution.
  • the computer may include a network of separate computers or separate processors to read out and execute the computer executable instructions.
  • the computer executable instructions may be provided to the computer, for example, from a network or the storage medium.
  • the computers or processors e.p., too, 102, 1201, 1200, etc.
  • the computers or processors may include the aforementioned CPU structure, or may be connected to such CPU structure for communication therewith.
  • a computer or processor may include an image/display processor or communicate w ith an image/display processor.
  • the computer 1200 includes a central processing unit (CPU) 1201, and may also include a graphical processing unit (GPU) 1215.
  • the CPU 1201 or the GPU 1215 may be replaced by the field-programmable gate array (FPGA), the application-specific integrated circuit (ASIC) or other processing unit depending on the design of a computer, such as the computer 1200, controller or processor too, controller or processor 102, any other computer, CPU, or processor discussed herein, etc.
  • FPGA field-programmable gate array
  • ASIC application-specific integrated circuit
  • At least one computer program is stored in the HDD/SSD 1204, the data storage 150, or any other storage device or drive discussed herein, and the CPU 1201 loads the at least one program onto the RAM 1203, and executes the instructions in the at least one program to perform one or more processes described herein, as well as the basic input, output, calculation, memory writing, and memory reading processes.
  • the computer such as the computer 1200, the computer, processors, and/or controllers of FIGS. 1-4, FIGS. 6-20B, FIG. 22, FIGS. 23-27, etc., communicates with the one or more components of the apparatuses/systems of FIGS. 1-4, of FIGS. 6-20B, of FIG. 22, and/or of any other apparatus(es) or system(s) discussed herein, to perform any of the methods, techniques, or features discussed herein, including, but not limited to, imaging, and may reconstruct an image from the acquired intensity data.
  • the monitor or display 1209 displays the reconstructed image, and the monitor or display 1209 may display other information about the imaging condition or about an object to be imaged.
  • the monitor 1209 also provides a graphical user interface for a user to operate a system, for example when performing CT, MRI, or other imaging modalities or other imaging technique(s), including, but not limited to, controlling continuum robot devices/systems, and/or manufacturing or using catheter tip feature(s) and/or technique(s).
  • An operation signal is input from the operation unit (e.g., such as, but not limited to, a mouse device 1211, a keyboard 1210, a touch panel device, etc.) into the communication interface 1205 in the computer 1200, and corresponding to the operation signal the computer 1200 instructs the system (e.g., the system 1000, the systems/ apparatuses of FIGS.
  • the camera or imaging device as aforementioned may have interfaces to communicate with the computer 1200 to send and receive the status information and the control signals.
  • one or more processors or computers 1200, 1200’ may be part of a system in which the one or more processors or computers 1200, 1200’ (or any other processor discussed herein) communicate with other devices (e.g., a database 1603, a memory 1602 (which may be used w ith or replaced by any other type of memory discussed herein or known to those skilled in the art), an input device 1600, an output device 1601, etc.).
  • other devices e.g., a database 1603, a memory 1602 (which may be used w ith or replaced by any other type of memory discussed herein or known to those skilled in the art), an input device 1600, an output device 1601, etc.
  • one or more models may have been trained previously and stored in one or more locations, such as, but not limited to, the memory 1602, the database 1603, etc.
  • one or more models and/or data discussed herein may be input or loaded via a device, such as the input device 1600.
  • a user may employ an input device 1600 (which may be a separate computer or processor, a keyboard such as the keyboard 1210, a mouse such as the mouse 1211, a microphone, a screen or display 1209 e.g., a touch screen or display), or any other input device known to those skilled in the art).
  • an input device 1600 may not be used (e.g., where user interaction is eliminated by one or more artificial intelligence features discussed herein).
  • the output device 1601 may receive one or more outputs discussed herein to perform coregistration, autonomous navigation, movement detection, control, and/or any other process discussed herein.
  • the database 1603 and/or the memory 1602 may have outputted information (e.g., trained model(s), detected marker information, image data, test data, validation data, training data, coregistration result(s), segmentation model information, object detection/ regression model information, combination model information, etc.) stored therein. That said, one or more embodiments may include several types of data stores, memory , storage media, etc. as discussed above, and such storage media, memory, data stores, etc. may be stored locally or remotely.
  • the term “subset” of a corresponding set does not necessarily represent a proper subset and may be equal to the corresponding set.
  • any other model architecture, machine learning algorithm, or optimization approach may be employed.
  • One or more embodiments may utilize hyper-parameter combination(s).
  • One or more embodiments may employ data capture, selection, annotation as well as model evaluation (e.g., computation of loss and validation metrics) since data may be domain and application specific.
  • the model architecture may be modified and optimized to address a variety of computer visions issues (discussed below).
  • One or more embodiments of the present disclosure may automatically detect (predict a spatial location of) a catheter tip (e.g., in or near an airway, pathway, a lung or other organ, in a patient, etc.) in a time series of X-ray images to co-register the X-ray images with the corresponding OCT images (at least one example of a reference point of two different coordinate systems).
  • a catheter tip e.g., in or near an airway, pathway, a lung or other organ, in a patient, etc.
  • One or more embodiments may use deep (recurrent) convolutional neural network(s), which may improve catheter tip detection, tissue detection, tissue characterization, robotic control, and image co-registration significantly.
  • One or more embodiments may employ segmentation and/or object/keypoint detection architectures to solve one or more computer vision issues in other domain areas in one or more applications.
  • One or more embodiments employ several novel materials and methods to solve one or more computer vision or other issues (e.g., lesion detection in time series of X-ray images, for instance; tissue detection; tissue characterization; robotic control; support structure manufacture and/or use; etc.).
  • lesion detection in time series of X-ray images for instance; tissue detection; tissue characterization; robotic control; support structure manufacture and/or use; etc.
  • images may include a radiodense marker, a sensor (e.g., an EM sensor), or some other identifier that is specifically used in one or more procedures (e.g., used in catheters/probes with a similar marker, sensor, or identifier to that of an OCT marker, used in catheters/probes with a similar or same marker, sensor, or identifier even compared to another imaging modality, etc.) to facilitate computational detection of a marker, sensor, catheter, and/ or tissue detection, characterization, validation, etc.
  • a radiodense marker e.g., a sensor, or some other identifier that is specifically used in one or more procedures (e.g., used in catheters/probes with a similar marker, sensor, or identifier to that of an OCT marker, used in catheters/probes with a similar or same marker, sensor, or identifier even compared to another imaging modality, etc.) to facilitate computational detection of a marker, sensor, catheter, and/ or tissue detection,
  • One or more embodiments may couple a software device or features (model) to hardware (e.g., a robotic catheter or probe, a steerable probe/catheter using one or more sensors (or other identifier or tracking components), etc.).
  • a software device or features model
  • hardware e.g., a robotic catheter or probe, a steerable probe/catheter using one or more sensors (or other identifier or tracking components), etc.
  • One or more embodiments may utilize animal data in addition to patient data. Training deep learning may use a large amount of data, which may be difficult to obtain from clinical studies. Inclusion of image data from pre-clinical studies in animals into a training set may improve model performance.
  • Training and evaluation of a model may be highly data dependent (e.g., a way in which frames are selected (e.g., during steerable catheter control, frames obtained via a robotic catheter, etc.), split into training/validation/test sets, and grouped into batches as well as the order in which the frames, sets, and/or batches are presented to the model, any other data discussed herein, etc.).
  • such parameters may be more important or significant than some of the model hyper-parameters (e.g., batch size, number of convolution lay ers, any other hyper-parameter discussed herein, etc.).
  • One or more embodiments may use a collection or collections of user annotations after introduction of a device/apparatus, system, and/or method(s) into a market, and may use post market surveillance, retraining of a model or models with new data collected (e.g., in clinical use), and/or a continuously adaptive algorithm/method(s) .
  • One or more embodiments may employ data annotation. For example, one or more embodiments may label pixel(s) representing a marker, a sensor, a camera, or an identifier detection or a tissue and/or catheter detection, characterization, and/or validation as well as pixels representing a blood vessel(s) or portions of a pathway (e.g., a vessel, a lumen, an airway, a bronchial pathway, another type of pathway, etc.) at different phase(s) of a procedure/method (e.g., different levels of contrast due to intravascular contrast agent) of acquired frame(s).
  • a pathway e.g., a vessel, a lumen, an airway, a bronchial pathway, another type of pathway, etc.
  • a marker, sensor, or other portion of a robotic catheter/probe location may be known inside a vessel, pathway, airway, or bronchial pathway (or other type of pathway) and/or inside a catheter or probe; a tissue and/or catheter or catheter tip location may be known inside a vessel, an airway, a lung, a bronchial pathway, or other type of target, object, or specimen; etc.
  • a marker, sensor, or other portion of a robotic catheter/probe location may be known inside a vessel, pathway, airway, or bronchial pathway (or other type of pathway) and/or inside a catheter or probe;
  • tissue and/or catheter or catheter tip location may be known inside a vessel, an airway, a lung, a bronchial pathway, or other type of target, object, or specimen; etc.
  • sensor(s)/marker(s)/identifier(s) may be used to improve sensor/marker detection and/or tissue and/or catheter detection, localization, characterization, and/or validation. For example, in a case where it is confirmed that the sensor of the probe or catheter, or the catheter or probe, is by or near a target area for tissue detection and characterization, the integrity of the tissue identification/detection and/or characterization for that target area is improved or maximized (as compared to a false positive where a tissue may be detected in an area where the probe or catheter (or sensor thereof) is not located).
  • a sensor or other portion of a catheter/probe may move inside a target, object, or specimen (e.g., a pathway, an airway, a bronchial pathway, a lung or another organ, in a patient, in tissue, etc.), and such prior knowledge may be incorporated into the machine learning algorithm or the loss function.
  • a target, object, or specimen e.g., a pathway, an airway, a bronchial pathway, a lung or another organ, in a patient, in tissue, etc.
  • One or more embodiments may employ loss (cost) and evaluation function(s)/metric(s). For example, use of temporal information for model training and evaluation may be used in one or more embodiments.
  • One or more embodiments may evaluate a distance between prediction and ground truth per frame as well as consider a trajectory 7 of predictions across multiple frames of a time series.
  • Machine learning may be used in one or more embodiment(s), as discussed in PCT/US2020/051615, filed on September 18, 2020 and published as WO 2021/055837 A9 on March 25, 2021, and as discussed in U.S. Pat. App. No. 17/761,561, filed on March 17, 2022, the applications and publications of which are incorporated by reference herein in their entireties.
  • At least one embodiment of an overall process of machine learning is shown below: i.Create a dataset that contains both images and corresponding ground truth labels; ii.Split the dataset into a training set and a testing set; iii.Select a model architecture and other hyper-parameters; iv.Train the model with the training set; v.Evaluate the trained model with the validation set; and vi.Repeat iv and v with new dataset(s).
  • steps i and iii may be revisited in one or more embodiments.
  • One or more models may be used in one or more embodiment(s) to detect and/or characterize a tissue or tissues and/or lesion(s), such as, but not limited to, the one or more models as discussed in PCT/US2020/051615, filed on September 18, 2020 and published as WO 2021/055837 A9 on March 25, 2021, and as discussed in U.S. Pat. App. No. 17/761,561, filed on March 17, 2022, the applications and publications of which are incorporated by reference herein in their entireties.
  • one or more embodiments may use a segmentation model, a regression model, a combination thereof, etc.
  • the input may be the entire image frame or frames
  • the output may be the centroid coordinates of sensors/markers (target sensor and stationary sensor or marker, if necessary/desired) and/or coordinates of a portion of a catheter or probe to be used in determining the localization and lesion and/or tissue detection and/or characterization.
  • FIGS. 24-26 an example of an input image on the left side of FIGS. 24-26 and a corresponding output image on the right side of FIGS. 24-26 are illustrated for regression model(s).
  • At least one architecture of a regression model is shown in FIG. 24. In at least the embodiment of FIG.
  • the regression model may use a combination of one or more convolution layers 900, one or more max-pooling layers 901, and one or more fully connected dense layers 902. While not limited to the Kernel size, Width/Number of filters (output size), and Stride sizes shown for each layer (e.g., in the left convolution layer of FIG. 12, the Kernel size is “3x3”, the Width/ # of filters (output size) is “64”, and the Stride size is “2”). In one or more embodiments, another hyper-parameter search with a fixed optimizer and with a different width may be performed, and at least one embodiment example of a model architecture for a convolutional neural network for this scenario is shown in FIG. 25.
  • One or more embodiments may use one or more features for a regression model as discussed in “Deep Residual Learning for Image Recognition” to Kaiming He, et al., Microsoft Research, December to, 2015 (https://arxiv.org/pdf/1512.03385.pdf), which is incorporated by reference herein in its entirety.
  • FIG. 26 shows at least a further embodiment example of a created architecture of or for a regression model(s).
  • the output from a segmentation model is a “probability” of each pixel that may be categorized as a tissue or lesion characterization or tissue or lesion identification/determination
  • post-processing after prediction via the trained segmentation model may be developed to better define, determine, or locate the final coordinate of a tissue location, a catheter tip location, support structure or catheter location, and/ or a lesion location (or a sensor/ marker location where the sensor/ marker is a part of the catheter) and/or determine the type and/or characteristics of the tissue(s) and/or lesion(s).
  • One or more embodiments of a semantic segmentation model may be performed using the One-Hundred Layers Tiramisu method discussed in “The One Hundred Layers Tiramisu: Fully Convolutional DenseNets for Semantic Segmentation” to Simon Jegou, et al., Montreal Institute for Learning Algorithms, published October 31, 2017 (https://arxiv.org/pdf/1611.09326.pdf), which is incorporated by reference herein in its entirety.
  • a segmentation model may be used in one or more embodiment, for example, as shown in FIG. 27. At least one embodiment may utilize an input 600 as shown to obtain an output 605 of at least one embodiment of a segmentation model method.
  • a slicing size may be one or more of the following: 100 x 100, 224 x 224, 512 x 512, and, in one or more of the experiments performed, a slicing size of 224 x 224 performed the best.
  • a batch size (of images in a batch) may be one or more of the following: 2, 4, 8, 16, and, from the one or more experiments performed, a bigger batch size typically performs better (e.g., with greater accuracy).
  • 16 images/batch may be used. The optimization of all of these hyper-parameters depends on the size of the available data set as well as the available computer/computing resources; thus, once more data is available, different hyperparameter values may be chosen. Additionally, in one or more embodiments, steps/epoch may be too, and the epochs may be greater than (>) 1000. In one or more embodiments, a convolutional autoencoder (CAE) may be used.
  • CAE convolutional autoencoder
  • hyper-parameters may include, but are not limited to, one or more of the following: Depth (i.e., # of layers), Width (i.e., # of filters), Batch size (i.e., # of training images/step): May be >4 in one or more embodiments, Learning rate (i.e., a hyper-parameter that controls how fast the weights of a neural network (the coefficients of regression model) are adjusted with respect the loss gradient), Dropout (i.e., % of neurons (filters) that are dropped at each layer), and/or Optimizer: for example, Adam optimizer or Stochastic gradient descent (SGD) optimizer.
  • Depth i.e., # of layers
  • Width i.e., # of filters
  • Batch size i.e., # of training images/step
  • Learning rate i.e., a hyper-parameter that controls how fast the weights of a neural network (the coefficients of regression model) are adjusted with respect the loss gradient
  • Dropout
  • other hyperparameters may be fixed or constant values, such as, but not limited to, for example, one or more of the following: Input size (e.g., 1024 pixel x 1024 pixel, 512 pixel x 512 pixel, another preset or predetermined number or value set, etc.), Epochs: too, 200, 300, 400, 500, another preset or predetermined number, etc. (for additional training, iteration may be set as 3000 or higher), and/or Number of models trained with different hyper-parameter configurations (e.p., 10, 20, another preset or predetermined number, etc.).
  • Input size e.g., 1024 pixel x 1024 pixel, 512 pixel x 512 pixel, another preset or predetermined number or value set, etc.
  • Epochs too, 200, 300, 400, 500, another preset or predetermined number, etc. (for additional training, iteration may be set as 3000 or higher)
  • Number of models trained with different hyper-parameter configurations e
  • the one or more trained models or AI- networks is or uses one or a combination of the following: a neural net model or neural network model, a deep convolutional neural network model, a recurrent neural network model with long short-term memory that can take temporal relationships across images or frames into account, a generative adversarial network (GAN) model, a consistent generative adversarial network (cGAN) model, a three cycle-consistent generative adversarial network (3CGAN) model, a model that can take temporal relationships across images or frames into account, a model that can take temporal relationships into account including tissue location(s) during pullback in a vessel and/or including tissue characterization data during pullback in a vessel, a model that can use prior knowledge about a procedure and incorporate the prior knowledge into the machine learning algorithm or a loss function, a model using feature pyramid(s) that can take different image resolutions into account, and/or a model using residual learning technique(
  • One or more features discussed herein may be determined using a convolutional auto-encoder, Gaussian filters, Haralick features, and/or thickness or shape of the sample or object (e.g., the tissue or tissues, the lesion or lesions, the catheter or catheter tip, the support structure of a catheter, a lung or other organ, an airway, a bronchial pathway, another type of pathway, a specimen, a patient, a target in the patient, etc.).
  • a convolutional auto-encoder e.g., the tissue or tissues, the lesion or lesions, the catheter or catheter tip, the support structure of a catheter, a lung or other organ, an airway, a bronchial pathway, another type of pathway, a specimen, a patient, a target in the patient, etc.
  • One or more embodiments of the present disclosure may use machine learning to determine sensor, tissue, catheter or catheter tip, support structure, and/or lesion location; to determine, detect, or evaluate tissue and/or lesion type(s) and/or characteristic(s); and/or to perform any other feature discussed herein.
  • Machine learning is a field of computer science that gives processors the ability to learn, via artificial intelligence.
  • Machine learning may involve one or more algorithms that allow processors or computers to learn from examples and to make predictions for new unseen data points.
  • such one or more algorithms may be stored as software or one or more programs in at least one memory or storage medium, and the software or one or more programs allow a processor or computer to carry out operation(s) of the processes described in the present disclosure.
  • machine learning may be used to train one or more models to efficiently to control a catheter or catheter tip, to manufacture or use the catheter or catheter tip, to perform imaging, etc.
  • the present disclosure and/or one or more components of devices, systems, and storage mediums, and/or methods, thereof also may be used in conjunction with continuum robot devices, systems, methods, and/or storage mediums and/or with endoscope devices, systems, methods, and/or storage mediums.
  • continuum robot devices, systems, methods, and/or storage mediums are disclosed in at least: U.S. Pat. App. No. 17/673,606, filed February 16, 2022, and U.S. Pat. No. 11,882,365, issued on January 23, 2024, the disclosures of which are incorporated by reference herein in their entireties.
  • Such endoscope devices, systems, methods, and/or storage mediums are disclosed in at least: U.S. Pat. App. No. 17/565,319, filed on December 29, 2021, the disclosure of which is incorporated by

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • Medical Informatics (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Pathology (AREA)
  • Physics & Mathematics (AREA)
  • Biophysics (AREA)
  • Optics & Photonics (AREA)
  • Robotics (AREA)
  • Radiology & Medical Imaging (AREA)
  • Mechanical Engineering (AREA)
  • Signal Processing (AREA)
  • Artificial Intelligence (AREA)
  • Evolutionary Computation (AREA)
  • Endoscopes (AREA)

Abstract

L'invention concerne un ou plusieurs dispositifs, systèmes, procédés et supports de stockage pour effectuer une commande robotique et/ou utiliser une structure de support pour éviter/réduire la déformation. Des exemples comprennent, sans s'y limiter, une utilisation avec un robot ou un cathéter continuum ayant au moins une section pliable pouvant être manipulée indépendamment pour avancer à travers un passage, sans entrer en contact avec un ou plusieurs éléments fragiles à l'intérieur du passage, un ou plusieurs fils d'entraînement du robot ou du cathéter ayant une structure de support pour permettre la prévention de la formation de coude/déformation de fil. La structure de support peut s'étendre d'un raccord à la ou aux sections pliables. Des exemples d'applications comprennent l'imagerie, l'évaluation et le diagnostic d'objets biologiques, par exemple, mais sans s'y limiter, des applications médicales bronchiques ou autres. Les techniques proposées dans l'invention améliorent également le traitement, l'imagerie et l'efficacité de commande cathéter tout en obtenant des images qui sont plus précises, et permettent également d'obtenir des dispositifs, des systèmes, des procédés et des supports de stockage qui réduisent la charge mentale et physique et améliorent la facilité d'utilisation.
PCT/US2024/046243 2023-09-11 2024-09-11 Appareil médical doté d'une structure de support et son procédé d'utilisation Pending WO2025059207A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363581862P 2023-09-11 2023-09-11
US63/581,862 2023-09-11

Publications (1)

Publication Number Publication Date
WO2025059207A1 true WO2025059207A1 (fr) 2025-03-20

Family

ID=95022549

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2024/046243 Pending WO2025059207A1 (fr) 2023-09-11 2024-09-11 Appareil médical doté d'une structure de support et son procédé d'utilisation

Country Status (1)

Country Link
WO (1) WO2025059207A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090099420A1 (en) * 2007-10-11 2009-04-16 Neoguide Systems, Inc. System for managing bowden cables in articulating instruments
US10583271B2 (en) * 2012-11-28 2020-03-10 Auris Health, Inc. Method of anchoring pullwire directly articulatable region in catheter
US20210121051A1 (en) * 2019-10-25 2021-04-29 Canon U.S.A., Inc. Steerable medical device with bending sections and improved connector therefor
US20210369355A1 (en) * 2020-05-26 2021-12-02 Canon U.S.A., Inc. Robotic endoscope probe having orientation reference markers
US20220202277A1 (en) * 2020-12-30 2022-06-30 Canon U.S.A., Inc. Steerable medical device
US20230225758A1 (en) * 2020-06-30 2023-07-20 Beijing Surgerii Technology Co., Ltd. Continuum instrument and surgical robot

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20090099420A1 (en) * 2007-10-11 2009-04-16 Neoguide Systems, Inc. System for managing bowden cables in articulating instruments
US10583271B2 (en) * 2012-11-28 2020-03-10 Auris Health, Inc. Method of anchoring pullwire directly articulatable region in catheter
US20210121051A1 (en) * 2019-10-25 2021-04-29 Canon U.S.A., Inc. Steerable medical device with bending sections and improved connector therefor
US20210369355A1 (en) * 2020-05-26 2021-12-02 Canon U.S.A., Inc. Robotic endoscope probe having orientation reference markers
US20230225758A1 (en) * 2020-06-30 2023-07-20 Beijing Surgerii Technology Co., Ltd. Continuum instrument and surgical robot
US20220202277A1 (en) * 2020-12-30 2022-06-30 Canon U.S.A., Inc. Steerable medical device

Similar Documents

Publication Publication Date Title
US12295672B2 (en) Robotic systems for determining a roll of a medical device in luminal networks
JP7677973B2 (ja) ロボット内視鏡装置及びロボット内視鏡システム
CN110891469B (zh) 用于定位传感器的配准的系统和方法
CN114601559B (zh) 用于基于定位传感器的分支预测的系统和介质
KR102683476B1 (ko) 항행(navigation)을 위한 이미지 기반 분지(branch) 감지 및 매핑
JP7314136B2 (ja) 医療器具のナビゲーションおよびターゲット用のシステムおよび方法
US20220125527A1 (en) Methods and systems for mapping and navigation
US12156704B2 (en) Intraluminal navigation using ghost instrument information
WO2022208253A1 (fr) Estimation de pose de caméra 6dof basée sur la vision en bronchoscopie
WO2025019378A1 (fr) Détection de mouvement de dispositif et planification de navigation et/ou navigation autonome pour un robot continuum ou un dispositif ou système endoscopique
WO2025117336A1 (fr) Cathéters orientables et différences de force de fil
US20250143812A1 (en) Robotic catheter system and method of replaying targeting trajectory
US20250170363A1 (en) Robotic catheter tip and methods and storage mediums for controlling and/or manufacturing a catheter having a tip
WO2025059207A1 (fr) Appareil médical doté d'une structure de support et son procédé d'utilisation
WO2024081745A2 (fr) Localisation et ciblage de petites lésions pulmonaires
US20250169684A1 (en) Distributed Bending and Mode Control for Bendable Medical Devices
US20250275810A1 (en) Systems and methods for navigating hidden anatomic passageways
US20240112407A1 (en) System, methods, and storage mediums for reliable ureteroscopes and/or for imaging
WO2025072201A1 (fr) Commande robotique pour robot continuum
WO2025117590A1 (fr) Appareil médical pliable comportant des actionneurs modulaires
WO2025188850A1 (fr) Dispositif de lithotripsie autonome et procédés d'affichage d'actions correctives associées
JP2024527624A (ja) 経皮医療処置の段階セグメント化
WO2024249691A2 (fr) Lissage de suivi de leader (ftl) pour robot continu

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24866243

Country of ref document: EP

Kind code of ref document: A1