WO2025207554A1 - Inter-instrument collision avoidance for computer-assisted systems - Google Patents
Inter-instrument collision avoidance for computer-assisted systemsInfo
- Publication number
- WO2025207554A1 WO2025207554A1 PCT/US2025/021225 US2025021225W WO2025207554A1 WO 2025207554 A1 WO2025207554 A1 WO 2025207554A1 US 2025021225 W US2025021225 W US 2025021225W WO 2025207554 A1 WO2025207554 A1 WO 2025207554A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- instrument
- motion
- line segments
- commanded motion
- commanded
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Pending
Links
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
- A61B34/37—Leader-follower robots
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/70—Manipulators specially adapted for use in surgery
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/361—Image-producing devices, e.g. surgical cameras
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1656—Programme controls characterised by programming, planning systems for manipulators
- B25J9/1664—Programme controls characterised by programming, planning systems for manipulators characterised by motion, path, trajectory planning
- B25J9/1666—Avoiding collision or forbidden zones
-
- B—PERFORMING OPERATIONS; TRANSPORTING
- B25—HAND TOOLS; PORTABLE POWER-DRIVEN TOOLS; MANIPULATORS
- B25J—MANIPULATORS; CHAMBERS PROVIDED WITH MANIPULATION DEVICES
- B25J9/00—Programme-controlled manipulators
- B25J9/16—Programme controls
- B25J9/1674—Programme controls characterised by safety, monitoring, diagnostic
- B25J9/1676—Avoiding collision or forbidden zones
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/30—Surgical robots
- A61B2034/301—Surgical robots for introducing or steering flexible instruments inserted into the body, e.g. catheters or endoscopes
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/70—Manipulators specially adapted for use in surgery
- A61B34/74—Manipulators with manual electric input means
- A61B2034/742—Joysticks
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/70—Manipulators specially adapted for use in surgery
- A61B34/74—Manipulators with manual electric input means
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/70—Manipulators specially adapted for use in surgery
- A61B34/76—Manipulators having means for providing feel, e.g. force or tactile feedback
Definitions
- the present invention generally provides improved computer-assisted devices, systems, and methods.
- Computer-assisted systems can be used to perform a task at a worksite.
- Example computer-assisted systems include industrial and recreational robotic systems.
- Example computer-assisted systems also include medical robotic systems used in procedures for diagnosis, non-surgical treatment, surgical treatment, etc.
- the computer-assisted system may include one or more robotic manipulators to manipulate instruments.
- An instrument may be an instrument for performing the task.
- An instrument may also be an instrument for viewing the performing of the task.
- the computer-assisted system may be equipped with any number of instruments of any type.
- one or more of the instruments may be articulated.
- one or more instruments may be articulated to manipulate or otherwise interact with the worksite.
- one or more instruments may be articulated to update the field of view available for viewing the task.
- one or more embodiments relate to a computer- assisted system comprising: a repositionable assembly configured to support a first instrument and a second instrument, the first instrument comprising a first plurality of links and the second instrument comprising a second plurality of links; and a control system comprising at least one processor, the control system configured to detect a collision condition between the first instrument and the second instrument by: determining a plurality of first line segments representing a current kinematic configuration the first plurality of links; determining a plurality of second line segments representing a current kinematic configuration of the second plurality of links; determining respective distances between select pairs of line segments, each pair of the select pairs comprising a first line segment of the plurality of first line segments and a second line segment of the plurality of second line segments; and determining, based on the respective distances, whether the collision condition exists.
- one or more embodiments relate to a method for inter-instrument collision avoidance in a computer-assisted system, the computer-assisted system comprising: a repositionable assembly configured to support a first instrument and a second instrument, the first instrument comprising a first plurality of links and the second instrument comprising a second plurality of links; and wherein the method comprises detecting a collision condition between the first instrument and the second instrument by: determining a plurality of first line segments representing a current kinematic configuration the first plurality of links; determining a plurality of second line segments representing a current kinematic configuration of the second plurality of links; determining respective distances between select pairs of line segments, each pair of the select pairs comprising a first line segment of the plurality of first line segments and a second line segment of the plurality of second line segments; and determining, based on the respective distances, whether the collision condition exists.
- one or more embodiments relate to a non-transitory machine-readable medium comprising a plurality of machine-readable instructions executed by one or more processors associated with a computer-assisted system comprising: a repositionable assembly configured to support a first instrument and a second instrument, the first instrument comprising a first plurality of links and the second instrument comprising a second plurality of links, and wherein the plurality of machine-readable instructions cause the one or more processors to perform a method comprising: detecting a collision condition between the first instrument and the second instrument by: determining a plurality of first line segments representing a current kinematic configuration the first plurality of links; determining a plurality of second line segments representing a current kinematic configuration of the second plurality of links; determining respective distances between select pairs of line segments, each pair of the select pairs comprising a first line segment of the plurality of first line segments and a second line segment of the plurality of second line segments; and determining, based
- FIG. 1 shows an example repositionable assembly in accordance with one or more embodiments.
- FIG. 2A shows an example computer-assisted system in accordance with one or more embodiments.
- FIG. 3 shows an example repositionable assembly in accordance with one or more embodiments.
- FIGs. 4A and 4B show example instrument in accordance with one or more embodiments.
- FIGs. 5E and 5F show examples of a show examples of a repositionable assembly with a set of instruments supported by a manipulator- supporting link that is rotatable about an insertion axis
- FIG. 7 shows an example configuration of pluralities of links associated with instruments disposed on a repositionable assembly.
- aspects of this disclosure are described in reference to computer-assisted systems, which can include devices that are teleoperated, externally manipulated, autonomous, semiautonomous, and/or the like. Further, aspects of this disclosure are described in terms of an implementation using a teleoperated surgical system, such as the da Vinci® Surgical System commercialized by Intuitive Surgical, Inc. of Sunnyvale, California. Knowledgeable persons will understand, however, that inventive aspects disclosed herein may be embodied and implemented in various ways, including teleoperated and nonteleoperated, and medical and non-medical embodiments and implementations.
- Implementations on da Vinci® Surgical Systems are merely exemplary and are not to be considered as limiting the scope of the inventive aspects disclosed herein.
- the instruments, systems, and methods described herein may be used for humans, animals, portions of human or animal anatomy, industrial systems, general robotic, or teleoperated systems.
- the instruments, systems, and methods described herein may be used for non-medical purposes including industrial uses, general robotic uses, sensing or manipulating non-tissue work pieces, cosmetic improvements, imaging of human or animal anatomy, gathering data from human or animal anatomy, setting up or taking down systems, training medical or non-medical personnel, and/or the like.
- Additional example applications include use for procedures on tissue removed from human or animal anatomies (with or without return to a human or animal anatomy) and for procedures on human or animal cadavers. Further, these techniques can also be used for medical treatment or diagnosis procedures that include, or do not include, surgical aspects.
- FIG. 1 shows an example repositionable assembly 100, in accordance with one or more embodiments.
- FIG. 1 illustrates an example repositionable assembly 100 of a computer- assisted system.
- the repositionable assembly 100 comprises a proximal repositionable structure 106 and one or more distal repositionable structures, according to various embodiments.
- a repositionable assembly 100 includes, without limitation, a proximal repositionable structure 106 that physically supports one or more distal repositionable structures (e.g., a first manipulator 102 and a second manipulator 104).
- the first manipulator 102 is configured to support a first instrument 122
- the second manipulator 104 is configured to support a second instrument 124.
- each of the proximal repositionable structure 106, the first and second manipulators 102, 104, and the first and second instruments 122, 124 can include any number of joints 144 of any type, and any number of links 142 of any geometry. While FIG. 1 shows a proximal repositionable structure 106 supporting two distal repositionable structures 102, 104 configured to support instruments 122 and 124, respectively, the proximal repositionable structure 106 can support any number of distal repositionable structures, and the distal repositionable structures 102, 104 can each support any number of instruments. Furthermore, the first instrument 122 and/or the second instrument 124 may include any number of links 142 and any number of joints 144. While FIG. 1 illustrates instrument 122 as an instrument with jawed end effectors and instrument 122 as an imaging instrument, such illustration is merely illustrative. In particular, the manipulators 102 and 104 may support any instrument type.
- the combination of the first instrument 122 and the first manipulator 102 may have the same number of joints as the combination of the second instrument 124 and the second manipulator 104. Alternatively, as shown in FIG. 1, the combination of the first instrument 122 and the first manipulator 102 may have fewer joints than the combination of the second instrument 124 and the second manipulator 104.
- the motion of each of the proximal repositionable structure 106, first manipulator 102, first instrument 122 with jawed end effectors (not labeled), second manipulator 104, and second instrument 124 with an imaging device (not labeled) is relative to a corresponding reference.
- the corresponding references can be the same reference, or be different references.
- a reference can be, for example and without limitation, a reference point, a reference line or other geometric feature, a reference frame, etc.
- the reference can be fixed to the environment (e.g., a point, geometric feature, or frame of reference fixed to the environment or the earth; such a reference frame can be called a “world frame”), to a subject of a procedure such as a workpiece or part of a patient (e.g., a point, geometric feature, or frame of reference fixed to a patient feature and/or the like; such a reference frame can be called a “subject frame,”), to a base of the robotic system (e.g., a base point, a base plane, frame of reference of the repositionable assembly or some other part of the robotic system), and/or the like.
- a base of the robotic system e.g., a base point, a base plane, frame of reference of the repositionable assembly or some other part of the robotic system
- the motions of the proximal repositionable structure 106, first manipulator 102, first instrument 122, second manipulator 104, and/or second instrument 124 are determined, controlled, or sensed relative to a same reference, or to different references.
- the motion of the proximal repositionable structure 106 could be relative to a world frame while the motion of the first manipulator 102 and first instrument 122 could be relative to a reference point or reference frame fixed to the proximal repositionable structure 106. In the example shown in FIG.
- a caused motion (e.g., of the first manipulator 102 and first instrument 122 or of the second manipulator 104 and second instrument 124) is determined, responded to, or sensed relative to the same or a different reference as used for the motion of the proximal repositionable structure 106.
- FIG 2A illustrates an example computer-assisted system 200, according to various embodiments.
- the computer-assisted system 200 includes, without limitation, a repositionable assembly 210 and a user input system 250.
- an operator uses the user input system 250 to operate the repositionable assembly 210, such as in a leader-follower configuration (also often called teleoperation configuration or master- slave configuration in industry) of the computer- assisted system 200.
- the leader-follower configuration the user input system 250 is the leader
- the repositionable assembly 210 is the follower of the leader-follower configuration.
- the repositionable assembly 210 can be used to introduce a set of instruments (not shown here, discussed below with reference to FIGs. 4A and 4B) to a work site through a port or entry guide 230 (a cannula is shown) inserted in an aperture.
- a port or entry guide 230 a cannula is shown
- the work site can be on or within a body cavity of a patient, and the aperture can be a minimally invasive incision or a natural body orifice.
- the port or entry guide 230 can be free-floating, held in place by a fixture separate from the repositionable assembly 210, or held by a linkage 222 or other part of the repositionable assembly 210.
- the linkage 222 is coupled to additional joints and links 214, 220 of the repositionable assembly 210, and these additional joints and links 214, 220 are mounted on a base 212.
- the drivable structure 222 terminates in a manipulator- supporting link 224.
- a set of manipulators 226 are coupled to the manipulator-supporting link 224.
- Each of the manipulators 226 include a carriage (or other instrument-coupling link) configured to couple to an instrument, and each of the manipulators 226 include one or more joint(s) that can be driven to move the carriage.
- a manipulator 226 can include a prismatic joint that, when driven, linearly moves the carriage and any instrument(s) coupled to the carriage.
- this linear motion is along an insertion axis, as further described below with reference to FIGs. 4A and 4B.
- the elements of the repositionable assembly 210 that are proximal to the manipulators 226 may be understood as the proximal repositionable structure as previously described in reference to FIG. 1.
- the additional joints and additional links 214 and 220 are used to position the port 230 at the aperture or another location.
- FIG. 2A illustrates a prismatic joint for vertical adjustment (as indicated by arrow “A”) and a rotary joints for horizontal adjustment (as indicated by arrow “B”).
- the drivable structure 222 is used to robotically pivot the port 230 (and the instruments disposed within it at the time) in yaw, pitch and roll angular rotations about the remote center of motion as indicated by arrows C, D and E, respectively.
- These joints may be considered part of the proximal repositionable structure as previously described in reference to FIG. 1.
- actuation of the degrees of freedom provided by joint(s) of the instrument(s) is provided by actuators disposed in, or whose motive force (e.g., linear force or rotary torque) is transmitted to, the instrument(s).
- actuators include rotary motors, linear motors, solenoids, and/or the like.
- the actuators drive transmission elements in the manipulators and/or in the instruments to control the degrees of freedom of the instrument(s).
- the actuators can drive rotary discs of the manipulator that couple with rotary discs of the instrument(s), where driving the rotary discs of the instruments drives transmission elements in the instrument that couple to move the joint(s) of the instrument, or to move the end effector(s) of the instrument, as further discussed below with reference to FIGs. 4A and 4B.
- the degrees of freedom of the instrument(s) are controlled by actuators that drive the instrument(s) in accordance with control signals determined based on inputs from the associated input devices (e.g., input devices 252 of the user input system 250).
- the control signals are determined in order to cause instrument motion or other actuation as indicated by movement of the input control devices or any other control signal.
- appropriately positioned sensors e.g., encoders, potentiometers, and/or the like, are provided to enable measurement of indications of the joint positions, or other data that can be used to derive joint position, such as joint velocity.
- the actuators and sensors are disposed in, transmit to, and/or receive signals from the manipulator(s) 226.
- a repositionable assembly 210 can have any number and any types of degrees of freedom, may or may not be configured to couple to a port, use a port other than a cannula, and/or other configuration different from what is shown in FIG 2A.
- the user input system 250 includes one or more input devices 252 operated by the operator (not shown).
- the one or more input devices 252 are contacted and manipulated by the hands of the operator, with one input device for each hand.
- hand-input-devices include any type of device manually operable by human user, e.g., joysticks, trackballs, button clusters, and/or other types of haptic devices typically equipped with multiple degrees of freedom.
- position, force, and/or tactile feedback devices are employed to transmit position, force, and/or tactile sensations from the instruments back to the operator's hands through the input devices 252.
- the input devices 252 are supported by the user input system 250 and are shown as mechanically grounded, and in other implementations may be mechanically ungrounded.
- An ergonomic support 256 is provided in some implementations.
- FIG. 2A shows an ergonomic support 256 including forearm rests on which the operator may rest his or her forearms while manipulating the input devices 252.
- the operator performs tasks at a work site near the repositionable assembly 210 during a medical procedure by controlling the repositionable assembly 210 using the input devices 252.
- a display unit 254 is included in the user input system 250.
- the display unit 254 displays images for viewing by the operator.
- the display unit 254 provides the operator with a view of the worksite with which the repositionable assembly 210 interacts.
- the view can include, for example, stereoscopic images or three-dimensional images to provide a depth perception of the worksite and the instrument(s) of the repositionable assembly 210 in the worksite.
- the display unit 254 can be moved in various degrees of freedom to accommodate the operator’s viewing position and/or to provide control functions.
- the display unit also includes an input device (e.g., another input device 252).
- the operator can sit in a chair or other support in front of the user input system 250, position his or her eyes to see images displayed by the display unit 254, grasp and manipulate the input devices 252, and rest his or her forearms on the ergonomic support 256 as desired.
- the operator can stand at the workstation or assume other poses, and the display unit 254 and input devices 252 may differ in construction, be adjusted in position (height, depth, etc.), etc.
- FIG. 2B illustrates an example system 270, according to various embodiments.
- the system 270 corresponds to the computer-assisted system 200 and includes one or more computing systems 272.
- a computing system 272 includes a processing system and is used to process input provided by the user input system 250, e.g., from the input device(s) 252 manipulated by an operator.
- a computing system 272 is further used to provide an output, e.g., a video image to the display unit 274.
- Examples of display unit 274 include LCDs, LEDs, organic LED displays, projectors, etc.
- one or more computing systems 272 are used to control the repositionable assembly 210.
- the computing system(s) 272 executes one or more control methods.
- the control methods include instructions for controlling one or more components of the repositionable assembly 210.
- joint movements of the repositionable assembly 210 are controlled by one or more control methods driving one or more joints using actuators of the repositionable assembly 210, the joint movements being calculated by a processor of a processing system of the computing system(s) 272.
- the control methods process control signals from the user input system 250 or elsewhere, and/or sensor signals (e.g., positional encoder data from joint position sensors, image data from image instruments such as ultrasonic probes or cameras or endoscopes, and/or the like), to calculate commands for the joint actuators.
- the control methods perform at least some of the calculations of the joint commands using vectors and/or matrices, some of which have elements corresponding to positions, velocities, and/or forces/torques of the joints.
- the range of alternative joint configurations available to the control methods can be conceptualized as a joint space.
- the joint space has as many dimensions as the repositionable assembly has degrees of freedom, and a particular configuration of the repositionable assembly represents a particular point in the joint space, with each coordinate corresponding to a joint state of an associated joint of the repositionable assembly.
- the term “state” of a joint or multiple joints refers to the control variables associated with the joint or the multiple joints, respectively.
- the state of an angular joint refers to the angle defined by that joint within its range of motion, and/or to the angular velocity (or speed or direction) of the joint.
- the state of an axial or prismatic joint refers to the joint's axial or linear position, and/or to its axial or linear velocity (or speed or direction). While one or more of the control methods described herein include position controllers, they often also have velocity control aspects. Alternative embodiments can rely primarily or entirely on velocity controllers, force controllers, acceleration controllers, and/or the like without departing from the disclosure.
- control modes include one or more other types of control modes. For example, during a robotic task being performed under the control of input devices 252 operated by a user, various joints of the repositionable assembly can be commanded to a same position and controlled to maintain static positions.
- one or more of the joints can be commanded to be “floating”, and facilitate motion of that joint due to externally applied force.
- a joint held in place by a brake can be floated by partially or entirely releasing the brake.
- An example of such a joint includes a passive joint held in place by an electromagnetic brake.
- a joint driven by actuator(s) can be held in place by commanding the actuator(s) to hold the joint position, and be floating by updating the command to the actuator(s) to the then-current position, velocity, and/or acceleration of the joint.
- a floating joint is readily reconfigured by an externally applied force or torque, without a control algorithm and/or a braking force seeking to counteract the reconfiguration caused by sufficient externally applied force or torque.
- a floating joint is further controlled to exhibit other characteristics or provide additional responses, such as to provide a certain type or level of damping response.
- a floating joint can still be braked, actuated, or otherwise managed for friction or gravity compensation. Such compensation can be provided by, for example and without limitation, passive springs, actively driven actuators, and/or the like.
- joints that are not moved by actuators can still be gravity compensated, friction compensated, dampened, and/or the like by actuators.
- multiple different control modes are combined during operation of the repositionable assembly. For example, some joints could be controlled to maintain position and resist or rebound from attempted external articulation of those joints, while other joints could be controlled to be floating and facilitate external articulation of those other joints. Parameters such as joint position, velocity, or acceleration of the joints are detected by joint sensors. The sensor signals are used to provide kinematic information of the repositionable assembly.
- the architecture of the control methods used for controlling the repositionable assembly can be of any appropriate form.
- the control architecture can be hierarchical, and could include a high-level controller and multiple joint controllers.
- a commanded movement is received by the high-level controller in, for example, a Cartesiancoordinate space (referred to herein as Cartesian-space).
- the commanded movement could be, for example, based on a movement command (e.g., in the form of a position and/or velocity) received from the user input system 250, or any other system that provides a movement command.
- the commanded movement is converted into commanded joint positions or joint velocities (e.g., linear or angular joint positions, linear or angular joint velocities).
- the conversion is performed using an inverse kinematics algorithm.
- the joint controllers convert the received commanded joint positions or velocities into commanded currents to drive the actuators producing joint movements.
- the joint movements together produce a repositionable assembly movement that reflects the commanded movement.
- a joint controller controls a joint position.
- the joint controller controls other variables such as joint velocity and/or joint force (linear force or angular torque).
- a joint controller receives a feedback signal in the form of a sensed joint state from an associated joint sensor, which it can use for closed- loop control.
- the sensed joint state includes, for example and without limitation, a joint position, a joint velocity (or component of velocity such as speed or direction), a joint acceleration (or component of acceleration), and/or the like, representing the joint movement.
- the sensed joint state is derived from the signals obtained from the joint sensor.
- a joint sensor can be, for example, an encoder, a potentiometer, an accelerometer, a hall effect sensor, and/or the like.
- a state observer or estimator (not shown) is used.
- Each joint controller can implement any appropriate control scheme, such as a proportional integral derivative (PID), proportional derivative (PD), full state feedback, sliding mode, and/or various other control schemes, without departing from the disclosure.
- PID proportional integral derivative
- PD proportional derivative
- full state feedback sliding mode
- sliding mode sliding mode
- various other control schemes without departing from the disclosure.
- the control methods further perform at least one of the steps described in FIG. 6 below.
- a computing system 272 may include, without limitation, one or more computer processors, non-persistent storage (e.g., volatile memory, such as random access memory (RAM), cache memory), persistent storage (e.g., a hard disk, an optical drive such as a compact disk (CD) drive or digital versatile disk (DVD) drive, a flash memory, etc.), a communication interface (e.g., Bluetooth interface, infrared interface, network interface, optical interface, etc.), and/or numerous other elements and functionalities.
- non-persistent storage e.g., volatile memory, such as random access memory (RAM), cache memory
- persistent storage e.g., a hard disk, an optical drive such as a compact disk (CD) drive or digital versatile disk (DVD) drive, a flash memory, etc.
- a communication interface e.g., Bluetooth interface, infrared interface, network interface, optical interface, etc.
- a computer processor of a computing system 272 is an integrated circuit for processing instructions.
- the computer processor can
- a communication interface of a computing system 272 includes an integrated circuit for connecting the computing system 272 to a network (not shown) and/or to another device, such as another computing system 272. Further, in some embodiments, the computing system 272 includes one or more output devices, such as a display unit 274, a printer, a speaker, external storage, or any other output device.
- Software instructions in the form of computer readable program code to perform embodiments of the disclosure are stored, in whole or in part, temporarily or permanently, on non-transitory computer readable medium. Specifically, the software instructions correspond to computer readable program code that, when executed by a processor(s), is configured to perform one or more embodiments of the disclosure.
- a computing system 272 is connected to or be a part of a network. The network may include multiple nodes. Each node corresponds to a different computing system, group of computing systems, group of nodes, and/or the like.
- the repositionable assembly 210 couples to an instrument when used to perform a procedure.
- the instrument can include an imaging device, e.g., an endoscope or an ultrasonic probe, usable to capture images of the worksite and output the captured images to an auxiliary system 280.
- the auxiliary system 280 processes the captured images using one or more image processing techniques prior to any subsequent display.
- the auxiliary system 280 can overlay the captured images with a virtual control interface prior to displaying the combined images to the operator via the user input system 250.
- one or more separate display units 274 are coupled with a computing system 272 and/or the auxiliary system 280 for local and/or remote display of images, such as images of the procedure site or other related images.
- FIG. 3 illustrates another example repositionable assembly 300 of a computer- assisted system such as a robotic system, according to various embodiments.
- a repositionable assembly 300 includes, without limitation, a repositionable assembly 306, physically supporting a first manipulator 302 and a second manipulator 304.
- the first manipulator 302 supports a first instrument 322, and the second manipulator 304 supports a second instrument 324.
- Each of the first manipulator 302, second manipulator 304, and repositionable assembly 306, the first instrument 322, and second instrument 324 includes any number of joints (e.g., joints 344 and 346) of any type and/or any number of links (e.g., links 342) of any geometry.
- the first instrument 122 and/or the second instrument 124 when equipped with links and joints, may be directly disposed on the proximal repositionable structure 106, without a first manipulator 102 and/or a second manipulator 104.
- the repositionable assembly 306 provides a movable support for the first manipulator 302 and second manipulator 304.
- Prismatic joints 346 enable translational movement of the first manipulator 302 and the second manipulator 304 relative to the repositionable assembly 306. While two manipulators (the first manipulator 302 and the second manipulator 304) are shown in FIG. 3, any number of manipulators can be supported by the repositionable assembly 306.
- the repositionable assembly 300 is part of a medical robotic system.
- the repositionable assembly 300 can be configured as a tableside-installed medical robotic system.
- the repositionable assembly 306 could be attached to a base of a surgical or examination table.
- the medical robotic system could include one or more additional drivable structure assemblies with the same or a different design.
- the repositionable assembly 300 can be installed on one side of the table, and a same or different repositionable assembly can be installed on the same side, or another side, of the table.
- FIG. 4A illustrates an example instrument 400 (also referred to herein as an instrument 400), according to various embodiments.
- the instrument 400 in Figure 4A includes, without limitation, a shaft 410, and an end effector located at a first end of the instrument 400.
- a housing 430 arranged to releasably couple the instrument 400 to a manipulator (shown, for example, in FIG 2A), is located at a second end of the instrument 400.
- the shaft 410 is rotatably coupled to the housing 430 to enable angular displacement of the shaft 410 relative to the housing 430, as indicated by arrows 448.
- Various types of end effectors 440 can be used.
- the end effector 440 can include one finger, two fingers (e.g., jaws 442 that may open and close), or three or more fingers.
- end effectors include, but are not limited to, scissors, forceps, staplers, cutting and cautery instruments, and/or the like.
- an end effector can further include an imaging device, e.g., an endoscope or an ultrasonic probe, to capture images of the worksite.
- an end effector 440 is actuated by transmission elements (e.g., cables, metal bands, screws, tubes, push rods, etc.) that connect parts of the instrument to drive elements (e.g., pulleys, capstans, spools, nuts, linear slides, or the like) (not shown) in the housing 430. Movement (e.g., translation or rotation) of the drive elements thereby controls the position of the end effector, or other degrees of freedom such as jaw opening, such that the end effector may translate or rotate, the jaws may open and close, and/or the like.
- transmission elements e.g., cables, metal bands, screws, tubes, push rods, etc.
- drive elements e.g., pulleys, capstans, spools, nuts, linear slides, or the like
- the drive elements upon coupling of an instrument 400 on a manipulator, engage with actuators of the manipulator, such as by engaging with transmission elements coupled to the actuators.
- actuators of the manipulator such as by engaging with transmission elements coupled to the actuators.
- the joints of the instrument 400 include, without limitation, a wrist 420 proximal to the end effector 440 and two shaft offset joints 422, 424 proximal to the wrist 420.
- the wrist 420 may enable rotation of the end effector 440 in one or more directions.
- the shaft offset joints 422, 424 can enable, for example, a translational offset 426 of the end effector 440 relative to the insertion axis 412 using the additional link serially coupled between the instrument shaft 410 and the end effector 440, in addition to the rotating provided by the wrist 420.
- the shaft offset joints 422, 424 may, thus, increase the workspace reachable by the end effector 440 of the instrument 400.
- the wrist 420 and the shaft offset joints 422, 424 may be actuated by control cables.
- the instrument 450 shown in FIG. 4B includes, without limitation, various elements of the instrument 400 shown in FIG. 4A and operates in a substantially similar manner to the instrument 400 shown in FIG. 4A.
- the instrument 450 includes a shaft 460 with an insertion axis 462 for insertion/retraction, and a wrist 470 proximal to an end effector 490.
- the wrist 470 may enable a pitch and a yaw movement of the end effector 490.
- the instrument 450 is equipped with a shaft offset joint 474 that, together with wrist 470 enables laterally offsetting the end effector 490 from the insertion axis 462 in any direction.
- the instrument 450 does not include a degree of freedom for rotation about the insertion axis 462 relative to the housing 480. Accordingly, the instrument 450 may move in only five degrees of freedom, with there being no instrument roll degree of freedom.
- the end effector is an imaging device such as an endoscopic camera.
- Instruments 400 and 450 can be used for robotic procedures such as robotic medical procedures (e.g., surgeries), in accordance with one or more embodiments.
- FIGs. 4A and 4B show particular configurations of instruments, designed to engage with a particular type of manipulator
- other configurations of instruments are within the scope of the disclosure.
- embodiments of instruments 400 and 450 could have multi-degree-of-freedom wrists (e.g., pitch and yaw degrees of freedom), single-degree-of-freedom wrists (e.g., pitch or jaw), or no wrists.
- various embodiments of instruments could have any suitable type of end effector including, for example, scissors, forceps, staplers, irrigation nozzles, hooks, scissors, blunt dissection instruments, needle drivers, imaging devices, and/or the like.
- different housings can also be used to interface with different types of manipulators.
- FIGs. 5A and 5B show examples of possible inter-instrument collisions based on the configuration previously introduced in reference to FIG. 1.
- the repositionable assembly 100 includes a proximal repositionable structure 106 and one or more distal repositionable structures (e.g., two manipulators 102, 104).
- the first manipulator 102 supports an instrument 122 that is a two-fingered instrument.
- the second manipulator 104 supports an instrument 124 that is an imaging instrument. While particular instruments are shown, the disclosure is not limited to these types of instruments.
- the kinematic configuration of the second manipulator 104 and the second instrument 124 is changed from the original configuration (dashed lines) to an updated configuration (solid lines).
- the change may be the result of a command provided by a user or internally generated by an algorithm.
- the change may be commanded, for example, to update the field of view of the imaging instrument.
- the kinematic configuration of the proximal repositionable structure 106 and the kinematic configuration of the first manipulator 102 and the first instrument 122 remain unchanged.
- the updating of the kinematic configuration the second manipulator 104 and the second instrument 124 results in a collision of the second instrument 122 with the first manipulator 102.
- FIGs. 5A and 5B illustrate a collision of particular elements (the second instrument 124 colliding with one particular link of the first manipulator 102 in FIG. 5A, and the first instrument 122 colliding with one particular link of the second manipulator 104 in FIG. 5B), other types of collisions may occur without departing from the disclosure. Collisions may occur between any pairs of links that are kinematically capable of colliding. These pairs of links may be formed by links belonging to instruments 122, 124, manipulators 102, 104, and/or the proximal repositionable structure 106. Further, a collision may be caused by any type of movement, including movement of one of more of the manipulators 102, 104, and/or movement of one or more of the instruments 122, 124.
- FIGs. 5C and 5D show examples of possible inter-instrument collisions based on the configuration previously introduced in reference to FIG. 2A.
- a repositioning of an instrument 510 using a movement of a manipulator-supporting link proximal to two instruments is shown.
- FIG. 5C illustrates a scenario 500C before the repositioning
- FIG. 5D shows a scenario 500D after the repositioning.
- the end effector 512 of the instrument 510 is repositioned.
- other portions of instrument 510, or other instruments can be repositioned instead or in addition to the end effector 512.
- a repositionable assembly includes a proximal repositionable structure, which is or includes a manipulator- supporting link 502 that supports multiple manipulators 550 and 552 coupled to multiple instruments 504 and 510.
- the manipulator-supporting link 502 forms a common mechanical base for the manipulators 550, and 552 that support the instruments 504 and 510.
- the manipulator-supporting link 502 corresponds to the manipulator-supporting link of a proximal repositionable structure as previously introduced with reference to FIG 2A. Movement of one or more joints of the proximal repositionable structure, for example, results in movement of the manipulator-supporting link 502.
- the portions of the manipulators 550 and 552 attached to the manipulatorsupporting link 502 are also moved.
- Such caused motion of the manipulators 550 and 552 can cause motion of the instruments 504 and 510.
- the manipulators 550 and 552 are held fixed in configuration, then the movement of the manipulator- supporting link 502 also moves instruments 504 and 510.
- repositioning of an instrument e.g., instrument 504 or 510) through movement of a common mechanical base such as the manipulator-supporting link 502 is used to increase the degrees of freedom and/or the range of motion of an instrument (e.g., instrument 504 or 510).
- an operator remotely or teleoperationally controls the end effector 512 of instrument 510 by providing a commanded motion via the input devices 252 in FIG. 2A or that a commanded motion is determined by an autonomous algorithm.
- the operator or the autonomous algorithm
- the operator provides commands intended to cause the end effector 512 to interact with the target 596.
- the end effector 512 is unable to reach the target 596 if the manipulator- supporting link 502 does not move relative to the target 596.
- one factor contributing to the inability of the end effector 512 to reach the target 596 could be that the instrument 510 has fewer joints as compared to some other instruments.
- instrument 510 lacks shaft offset joints while instruments 504 includes shaft offset joints 508. If instruments 504 and 510 are identical except that instrument 510 lacks shaft offset joints, then instrument 504 would generally have a greater number of degrees of freedom and/or an increased range of motion in comparison to instrument 510. However, even if instrument 510 includes additional joints such as shaft offset joints or other joints, end effector 512 may still be unable to reach target 596 without motion of the manipulator-supporting link 502 if the additional joints do not provide sufficient additional range of motion and/or do not support movement the applicable degree(s) of freedom. Accordingly, motion of the manipulator- supporting link 502 can be used to produce movement of the end effector 512.
- Various instruments may comprise or lack joints for various degrees of freedom.
- any of instruments 504 and 510 could lack joints to support movement about a roll degree of freedom about a respective roll axis.
- an instrument may not include an articulated wrist.
- the target 596 can be any kind of object
- the instrument 510 can be any kind of instrument equipped with an end effector suitable to interact with the target 596.
- the end effector could include forceps
- the target could include tissue.
- the repositionable assembly has been positioned, and/or is controlled, such that the remote center of motion 540 is located approximately centrally in the aperture 592.
- movement of the manipulator-supporting link 502 relative to the target 596 such as the pivoting motion 532 or some other movement (e.g., translational movement, combined translational and rotational movement, etc.), help enable the end effector 512 to execute a commanded motion indicated by the operator via the input devices 252 that the end effector 512 may not have been able to otherwise execute.
- the movement moves the manipulator- supporting link 502 supporting the manipulators 550 and 552.
- Such motion of the manipulators 550 and 552 can also move the instruments 504 and 510. If the manipulators 550 and 552 are held static relative to the manipulator- supporting link 502, then the instruments 504 and 510 jointly move in a common motion in response to the movement of the manipulator-supporting link 502 (all pivoting about the remote center of motion 540 with the pivoting motion 532).
- the pivoting motion 532 if the manipulators are held static relative to the manipulator-supporting link 502, would cause motion of the end effector 512 of the instrument 510 such that it reaches the position shown in FIG. 5D.
- the pivoting motion 532 would also cause motion of the end effector 506 of the instrument 504, such that the instrument 504 reaches the position shown by dashed lines in FIG. 5D, if no motion occurs other than these caused motions.
- the movement of instrument 510 is commanded and thus is desired.
- the movement of instrument 504 that would result from performing only the pivoting motion 532 may not be commanded, expected, or desired. For example, if there was no caused motion, instruments 504 could be expected to remain stationary, in absence of other movement commands. As another example, instrument 504 could be expected to follow motion commands provided for these instruments, independent from the movement of instrument 510.
- the manipulator 550 (shown supporting the instrument 504), is driven to move in a manner that compensates partially or entirely for the motion caused by the movement of the manipulator- supporting link 502 (e.g., such as by pivoting motion 532).
- compensating motion of joints of the instrument 504 are commanded in addition, or instead of, compensating motion of the manipulator 550.
- the compensation can involve movement of one or more joints of the manipulator 550 supporting the instrument 504, and/or one or more joints of the instrument 504.
- the compensation can involve moving the shaft offset joints (e.g., 508) or other joints of the instrument (e.g., 504), and/or moving the manipulator (e.g., 550) to perform an insertion or retraction movement along the insertion axes of the instrument 504.
- the compensation can be performed such that the end effector (e.g., end effector 506) remains substantially stationary within the work site, while the end effector 512 of the instrument 510 is repositioned relative to the work site.
- methods for coordinating movements of multiple instruments of a repositionable assembly are described in further detail in International Patent Publication No. WO2022/046787A1, entitled “Method and System for Coordinated Multiple-Tool Movement Using a Drivable Assembly,” which is incorporated herein by reference.
- a first commanded motion of instrument 504 is determined such that a first relative motion of the end effector 506 is effected relative to the manipulator- supporting link 502.
- a second commanded motion of the manipulatorsupporting link 502 is determined such that the first commanded motions performed in conjunction with the second commanded motion cause the end effector 506 to remain stationary relative to the workspace.
- the second commanded motion when performed, causes motion of the second instrument 510, e.g., as illustrated in FIG. 5D, where the end effector 512 of the second instrument 510 is repositioned/reoriented to reach for the target 596.
- the commanded motion of the end effector 512 involving movement of the manipulator- supporting link 502, the manipulator 552, and the instrument 510 with the end effector 512, in combination with the compensatory movement of the manipulator 550 and instrument 504 that keeps the end effector 506 stationary results in a collision of the end effectors 506 and 512.
- FIGs. 5C and 5D illustrate a collision of particular elements (the end effectors 506 and 512), resulting from a particular combination of a movement and a compensatory movement based on a commanded motion
- other types of collisions may occur without departing from the disclosure. Collisions may occur between any pairs of links that are kinematically capable of colliding. These pairs of links may be formed by links belonging to instruments 504, 510 (including end effectors 506, 512), manipulators 550, 552, and/or the manipulator-supporting link 502. Further, a collision may be caused by any type of movement that is not limited to the movement in combination with a compensatory movement as described.
- FIGs. 5C and 5D illustrate a pivoting motion 532 in conjunction with the worksite being accessible through an aperture
- the described methods apply to any type of movement and are not limited to a pivoting motion.
- the movement of the manipulator-supporting link 502 could alternatively be, or include, one or more other rotational movements, linear or nonlinear translational movements, combinations of translational and rotational movements, etc.
- the movement of the manipulator- supporting link 502 is a rotation about the insertion axis 590, unlike the rotation of the insertion axis 590 about the remote center 540 which is illustrated in FIGs. 5C and 5D.
- the rotation may be the rotation identified by arrow “E”.
- the rotation about the insertion axis 590 may be performed, for example, to enable rotation of end effector 512 of instrument 510 about an axis parallel to the insertion axis 590, in absence of a degree of freedom supporting such a rotation by the instrument 510 itself.
- the end effector 512 includes an imaging instrument, and the instrument 510 does not provide a degree of freedom for a rolling of the imaging instrument about an axis aligned with the field of view of the imaging instrument (e.g., to roll the field of view of the imaging instrument).
- the instrument 510 may, however, have a wrist that enables rotation in pitch and/or yaw direction, e.g., analogous to the instrument 450 of FIG. 4B.
- the use of the rotation about the insertion axis 590 along with other adjustments of the joints of the instrument 510 enable a rolling of the field of view of the imaging instrument, in addition to pitch and yaw, even in absence of a dedicated degree of freedom to perform this roll operation.
- An example is provided below in reference to FIGs. 5E and 5F.
- FIGs. 5E and 5F show examples of a repositionable assembly with a manipulator- supporting link that may be rotated about an insertion axis, as previously discussed.
- Three instruments 560, 570, and 580 are supported by a manipulator-supporting link 502.
- the manipulator- supporting link 502 may be rotated about the insertion axis 590.
- the instruments 560 and 570 in the example, are similar to the instrument 400 as described in reference to FIG. 4A, and include shaft offset joints 566, 576 distal to the instrument shaft 562, 572.
- the instruments 560, 570 are equipped with manipulation-type end effectors 564, 574.
- the instrument 580 in the example, is similar to the instrument 450 as described in reference to FIG. 4B, also includes shaft offset joints distal to the instrument shaft 582, but does not have the ability to roll about the instrument shaft.
- a roll movement 588 of the manipulator- supporting link 502 about the insertion axis 590 is introduced. The roll movement changes the position and the orientation of the end effector 584 as illustrated in FIG. 5F.
- the shaft offset joints 566, 576 of instruments 560, 570 may be used to compensate for the roll motion 588 of the manipulator- supporting link, such that the end effectors 564, 574 of instruments 560, 570 remain stationary (e.g., stationary relative to the workspace).
- the roll movement 588 does not result in a collision.
- other roll movements could result in collisions.
- a collision would occur between the instrument shaft 562 of instrument 560 and the link between shaft offset joints 576 of instrument 570, because the instrument shaft 562 protrudes sufficiently to interfere with this link of instrument 570.
- collisions could occur between links of instruments 560 and 580 and/or instruments 570 and 580.
- embodiments of the disclosure may be used to check for collision conditions between pairs of links such as an instrument shaft of one instrument and another link of another instrument. With the instrument shafts 562, 572, and 582 being parallel, it is not necessary to test for collision conditions between the instrument shafts.
- FIGs. 1, 2A, 2B, 3, 4A, 4B, and 5A-5F show various configurations of components, other configurations may be used without departing from the scope of the disclosure.
- various components may be combined to create a single component.
- the functionality performed by a single component may be performed by two or more components.
- FIG. 3 the disclosure generalizes to any type of instrument with any type and number of degrees of freedom.
- the instrument is described as being supported by a robotic arm with an instrument holder, the instrument may be supported by any type of repositionable structure, without departing from the disclosure.
- components are often described in context of medical scenarios such as surgical scenarios, embodiments of the disclosure are equally applicable to other domains that involve robotic manipulation, e.g., non-surgical scenarios or systems, nonmedical scenarios or systems, and/or the like.
- FIG. 6 a flowchart in accordance with one or more embodiments is shown.
- the flowchart of FIG. 6 depicts a method 600 for computer-assisted systems.
- the method 600 may be used to detect collision conditions between links of a robotic or teleoperated computer-assisted system.
- One or more of the steps in FIG. 6 may be performed by various components of systems, previously described with reference to FIGs. 1, 2A, 2B, 3, 4A, 4B, and 5A-5F. While these figures illustrate particular configurations of computer assisted systems, the method is equally applicable to other configurations.
- the method may be executed on one or more processors, e.g., of the control system of the computer-assisted system.
- the various steps in the flowchart are presented and described sequentially, one of ordinary skill will appreciate that some or all of the steps may be executed in different orders, may be combined or omitted, and some or all of the steps may be executed in parallel. Additional steps may further be performed. Furthermore, the steps may be performed actively or passively. For example, some steps may be performed using polling or be interrupt driven in accordance with one or more embodiments of the invention. [0081] The subsequently described steps may be performed for repositionable assemblies as previously described or any other repositionable assemblies. In some embodiments, the repositionable assembly supports multiple instruments (e.g., two, three, or more instruments), each having multiple links.
- instruments e.g., two, three, or more instruments
- a repositionable assembly may comprise a proximal repositionable structure and one or more distal repositionable structures.
- the one or more distal repositionable structures may each support or couple to one or more instruments. Examples of repositionable assemblies and instruments disposed on the repositionable assemblies include but are not limited to those previously discussed in reference to FIGS. 1, 2A, 2B, 3, 4A, and 4B. Collisions between the various links of the instruments and/or between the links of the instruments and the repositionable assembly may occur, for example, as previously discussed in reference to FIGs. 5A-5F.
- the method 600 relies on line representations of corresponding structures such as links.
- a kinematic chain of a first instrument that includes a first set of links may be represented by a set of first line segments.
- Each of the first line segments may represent a corresponding link of the first instrument.
- a kinematic chain of a second instrument that includes a second set of links may be represented by a set of second line segments.
- the line segment representations are regularly updated to reflect the current kinematic configuration of the corresponding structure, which enables the use of these line segment representations to detect collision conditions, e.g., by determining a closest distance between the line segment representations followed by determining whether a collision condition is present or absent, based on the closest distance. Detecting collision conditions using line segmentation representations as opposed to using volume models enables execution of the updating of the line segment representations followed by the collision detection in an optimized manner that reduces computational requirements and makes the method particularly suitable for execution in real-time at high sampling rates. [0083] Turning to the flowchart 600, in Step 602, the set of first line segments that represent the current kinematic configuration of the first set of links associated with the first instrument are determined.
- the first line segments and the arrangement of the first line segments to form a kinematic chain may be specified in a database, where the lengths of the line segments and their ordered arrangement in the kinematic chain of the first instrument may be established.
- the types of joints e.g., revolute, prismatic joint including degree(s) of freedom
- This may enable an updating of the line representation by specifying joint angles and/or joint positions.
- the line representation may be updated by specifying the position and orientation of the first line segments in space, either in absolute coordinates, or relative to one another.
- the line representation of the first instrument may be updated to reflect the current kinematic configuration of the corresponding links of the first instrument using joint sensor data, commanded joint configurations, and/or joint configurations determined by algorithms such as inverse kinematics algorithms. Examples of line representations may be found in, for example, FIGs. 1, 3, and 5A-5F, with FIGs. 5A-5F illustrating an updating of line representations to reflect a current kinematic configuration of the corresponding structures. Although these line representations, for the sake of clear illustration, are in a 2D plane, line representations determined for the first line segments in Step 602 reflect the current kinematic configuration of the first instrument in 3D space. [0084] In Step 604, the set of second line segments that represent the current kinematic configuration of the second set of links associated with the second instrument are determined. Step 604 may be performed analogous to Step 602.
- Step 606 select pairs of line segments are generated. Each select pair includes one first line segment of the set of first line segments, and one second line segment of the set of second line segments.
- the selection of line segments may be performed in various different manners, as subsequently discussed in reference to FIG. 7.
- FIG. 7 shows a sample configuration 700 of two instruments 504, 510 with corresponding end effectors 506, 512.
- the instruments 504 and 510 are supported by a repositionable assembly such as ones described with respect to the prior figures.
- the instruments 504 and 510 are coupled to or disposed on respective distal repositionable structures (e.g., a first manipulator and a second manipulator) which are in turn coupled to a manipulator-supporting link 502 of a proximal repositionable structure.
- distal repositionable structures e.g., a first manipulator and a second manipulator
- manipulator-supporting link 502 of a proximal repositionable structure e.g., a first manipulator and a second manipulator
- the line segments associated and the first instrument 504 are labeled 1A (representing a first instrument shaft of the first instrument), IB (representing a shaft offset joint), optionally 1C (representing one of the two jaws of the end effector 506), and optionally ID (representing one of the two jaws of the end effector 506).
- the line segments associated with the second manipulator 552 and the second instrument 510 are labeled 2A (representing the second manipulator 552), 2B (representing one of the two jaws of the end effector 512), and 2C (representing one of the two jaws of the end effector 512).
- pairs of line segments are generated by selecting combinations of one line segment associated with the kinematic chain of the first instrument and one line segment associated with the kinematic chain of the second instrument.
- one line segment may be selected from line segments 1A, IB, 1C, and ID, and one line segment may be selected from line segments 2A, 2B, and 2C.
- Resulting pairs may include 1A/2A, 1A/2B, 1A/2C, 1B/2B, 1B/2C, etc.
- select pairs of line segments are generated by selecting combinations of one line segment associated with the kinematic chain of the first instrument and one line segment associated with the kinematic chain of the second instrument.
- the select pairs of line segments exclude disregarded pairs of line segments.
- Disregarded pairs of line segments may be line segments for which it is known that the corresponding links are unable to collide.
- the inability to collide may be a result of the mechanical design and/or current kinematic configuration(s) of the first instrument, the second instrument, and/or the manipulators supporting the first and second instruments, respectively.
- the mechanical design of these components may be such that the instrument shafts and/or manipulators are maintained in parallel to one another by design.
- the line segments that form a disregarded pair of line segments correspond to the most proximal links of the corresponding structures (i.e., instrument shafts and/or manipulators).
- links of kinematic chains represented by line segments 1A and 2A e.g., instrument shafts, portions of distal repositionable structures, portions of manipulators, etc.
- the pair 1A/2A is a disregarded pair of line segments.
- the instrument shafts 410, 460 when the instruments 400, 450 are installed on corresponding manipulators 226, may be maintained in parallel.
- the select pairs of line segments may include, for example, a first shaft line segment (e.g., 1A) paired with a second additional line segment of the set of second line segments representing a link of the at least one additional links of the second instrument (e.g., 2B, 2C).
- the select pairs of line segments may further include a second shaft line segment (e.g., 2A) paired with a first additional line segment of the set of first line segments representing a link of the at least one additional links of the first instrument (e.g., IB, 1C, ID).
- the select pairs of line segments may also include a second additional line segment paired with a first additional line segment (e.g., 1B/2B, 1B/2C, 1C/2B, 1C/2C, 1D/2B, 1D/2C).
- the select pairs exclude the disregarded pair 1A/2A.
- the inability to collide may further be a result of the commanded motion(s) (e.g., commanded by operator using the operator input system or commanded by an autonomous algorithm executed by the computer-assisted system) of the repositionable assembly and/or the instruments.
- the select pairs of line segments may be selected, generated, or determined based on one or more of: a type or model of the first instrument or a type or model of the second instrument, a current kinematic configuration of the repositionable assembly, a current kinematic configuration of the first instrument, a current kinematic configuration of the second instrument, a commanded motion of the repositionable assembly, a commanded motion of the first instrument, or a commanded motion of the second instrument.
- a disregarded pair of line segments comprising a first identified line segment representing a first link on the first instrument and a second identified line segment representing a second link on the second instrument may be identified to be disregarded for collision detection based on a position of the first link on the first instrument and/or a position of the second link on the second instrument.
- select pairs of line segments are generated by selecting combinations of one line segment associated with the kinematic chain of the first instrument and one line segment associated with the kinematic chain of the second instrument.
- the select pairs of line segments exclude disregarded pairs of line segments.
- Disregarded pairs of line segments may be line segments for which it is known that, in the current kinematic configuration of the first and second instruments, the corresponding links are mechanically unable to collide. Referring to FIG. 7, in the current kinematic configuration as shown, the links corresponding to line segments 2B and 2C are unable to collide with the link corresponding to line segment 1A, based on their spatial separation. Accordingly, 1A/2B and 1A/2C may be considered disregarded pairs.
- the links corresponding to line segments 2B and 2C may be able to collide with the link corresponding to line segment 1A. Accordingly, in the updated kinematic configuration, 1A/2B and 1A/2C may no longer be considered disregarded pairs.
- the determination as described may be made based on, for example, distance considerations in 3D space, which may be updated dynamically in real-time or near-real-time.
- combinations of different methods may be used to generate the pairs of line segments. For example, some combinations of line segments may always be considered (or disregarded) for the purpose of forming pairs of line segments, some combinations of line segments may be eliminated from consideration (by making them disregarded pairs of line segments) based on the mechanical design of the corresponding structure, and/or some combinations of line segments may be dynamically considered (or disregarded) for the purpose of forming pairs of line segments depending on the current kinematic configuration of the corresponding structure.
- the result of these operations is a set (at least one) of select pairs of line segments, each representing links of a corresponding structure.
- respective distances are determined between the select pairs of line segments. More specifically, for each of the select pairs of line segments, a minimum distance between the first line segment and the second line segments is determined. Standard methods such as the shortest distance between two skew lines in 3D space may be used to 1 determine the minimum distance between the first line segment and the second line segment of a select pair of line segments.
- Step 608 it is determined, based on the respective distances, whether a collision condition exists.
- the determination of a collision condition is intended to enable an anticipatory detection of a collision to enable or trigger performance of amelioration actions intended to prevent the collision from occurring and/or to mitigate the effects of the collision.
- the detection of an existence of a collision condition does not necessarily indicate that a collision has already occurred, but can also indicate that a collision is likely to occur.
- Determining the existence (or absence) of the collision condition may involve examination of the distances determined in Step 606 to identify a closest distance.
- the select pair of line segments associated with the closest distance represents those links that are closest to colliding, based on their distance.
- the closest distance is identified in the three-dimensional space.
- a collision condition is determined to exist if the closest distance is below a distance threshold.
- a collision condition is determined not to exist, if the closest distance is at or above the distance threshold.
- the distance threshold may be previously specified or may be dynamically determined.
- the distance thresholds for determining collision conditions may vary between pairs of links.
- the distance threshold may be set to accommodate a cross-sectional dimension (e.g., a diameter) of links. For example, a first link may have a first cross-sectional diameter, and a second link may have a second cross- sectional diameter.
- the distance threshold may be set to the radius of the first link plus the radius of the second link, plus an additional tolerance that may serve as a margin of error, to ensure a minimum gap between the links, etc.
- the widest cross-sectional dimensions of such links may be used for the purpose of determining a collision condition.
- a single distance threshold may be used.
- the largest diameters present among the links are used to set the distance threshold to ensure reliable detection of a collision condition.
- the distance threshold may be set in a link dependent manner.
- a database may store link diameter data. Depending on the pair of links being examined for the presence of a collision condition, the corresponding link diameter entries are retrieved from the database to calculate the applicable distance threshold.
- This approach may reduce the likeliness of false positive detections of collision conditions when smaller diameter links are examined for the presence of a collision, in Step 608.
- the link may be split into segments for which different diameters are used for the purpose of determining a collision condition.
- a distance threshold for determining a collision condition between a pair of links may be determined based on one or more of: information associated with the first instrument (e.g., instrument type, instrument model, etc.), information associated with the second instrument (e.g., instrument type, instrument model, etc.), information associated with the first link (e.g., link identifier of the first link, information identifying which link the first link is on the first instrument, etc.), or information associated with the second link (e.g., link identifier of the second link, information identifying which link the first link is on the second instrument).
- information associated with the first instrument e.g., instrument type, instrument model, etc.
- information associated with the first link e.g., link identifier of the first link, information identifying which link the first link is on the first instrument, etc.
- information associated with the second link e.g., link identifier of the second link, information identifying which link the first link is on the second instrument.
- Step 610 based on determining that a collision condition is present, the method may proceed with the execution of Step 612. Alternatively, based on determining that a collision condition is not present, the method may proceed with the execution of Step 602.
- Step 612 in response to determining the existence of the collision condition, an amelioration action is performed.
- One or more of a variety of amelioration actions may be performed.
- the user of the computer-assisted system may be notified.
- the notification may be an auditory or visual alarm, a status message, a visual illustration of the collision condition, etc.
- a collision-causing motion may be inhibited. Assume, for example, that the collision condition is detected as a result of a first instrument moving while a second instrument remains stationary. In this case, the continuation of the collision-causing motion of the first instrument may be inhibited or suppressed. The inhibition or suppression of further movement may be direction specific.
- a continuation of the movement towards a collision may be prevented, while movement in other directions may still be allowed.
- the control system in the case that a collision condition is determined during execution of a commanded motion (e.g., of the repositionable assembly, of the first instrument, and/or of the second instrument) in a first direction, the control system is configured to cease executing the commanded motion.
- the control system while the collision condition exists, the control system is configured to reject additional commanded motions in the first direction but execute additional commanded motions in directions other than the first direction.
- the control system is configured to reject additional commanded motions in the first direction but execute additional commanded motions in a second direction that is opposite of the first direction.
- the control system is configured to reject additional commanded motions that would result in a reduction of the minimum distance between the first plurality of line segments representing the first instrument and the second plurality of line segments representing the second instrument.
- haptic feedback may be provided to the user at the input device of the computer-assisted system. The haptic feedback may be selected to indicate the inhibiting of the continuation of the collision-causing motion to the user, i.e., to deter the user input that commands the collision-causing motion.
- a virtual wall may be implemented by the haptic feedback, thereby preventing or at least inhibiting the user from continuing commanding the collision-causing motion.
Landscapes
- Health & Medical Sciences (AREA)
- Engineering & Computer Science (AREA)
- Surgery (AREA)
- Life Sciences & Earth Sciences (AREA)
- Robotics (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- General Health & Medical Sciences (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Mechanical Engineering (AREA)
- Pathology (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Manipulator (AREA)
Abstract
A computer-assisted system includes a repositionable assembly that supports a first instrument and a second instrument. The first instrument includes a first set of links and the second instrument comprising a second set of links. The computer system further includes a control system with at least one processor. The control system detects a collision condition between the first instrument and the second instrument by: determining a set of first line segments representing a current kinematic configuration the first set of links, determining a set of second line segments representing a current kinematic configuration of the second set of links, determining respective distances between select pairs of line segments, each pair of the select pairs including a first line segment of the set of first line segments and a second line segment of the set of second line segments, and determining, based on the respective distances, whether the collision condition exists.
Description
INTER-INSTRUMENT COLLISION AVOIDANCE FOR COMPUTER- ASSISTED SYSTEMS
CROSS-REFERENCE TO RELATED APPLICATIONS
[0001] This application claims priority to and the benefit of the filing date of U.S. Provisional Patent Application 63/569,418, filed on March 25, 2024, which is hereby incorporated by reference herein in its entirety.
BACKGROUND
Field of Invention
[0002] The present invention generally provides improved computer-assisted devices, systems, and methods.
Overview
[0003] Computer-assisted systems can be used to perform a task at a worksite. Example computer-assisted systems include industrial and recreational robotic systems. Example computer-assisted systems also include medical robotic systems used in procedures for diagnosis, non-surgical treatment, surgical treatment, etc. The computer-assisted system may include one or more robotic manipulators to manipulate instruments. An instrument may be an instrument for performing the task. An instrument may also be an instrument for viewing the performing of the task. The computer-assisted system may be equipped with any number of instruments of any type.
[0004] During use of the computer-assisted system, one or more of the instruments may be articulated. For example, one or more instruments may be articulated to manipulate or otherwise interact with the worksite. Similarly, one or more instruments may be articulated to update the field of view available for viewing the task.
[0005] Collisions between the instruments may occur as a result of the articulation. Improved techniques for operating computer-assisted systems that enable detection of a collision or a risk of a collision are desirable.
SUMMARY
[0006] In general, in one aspect, one or more embodiments relate to a computer- assisted system comprising: a repositionable assembly configured to support a first instrument and a second instrument, the first instrument comprising a first plurality of links and the second instrument comprising a second plurality of links; and a control system comprising at least one processor, the control system configured to detect a collision
condition between the first instrument and the second instrument by: determining a plurality of first line segments representing a current kinematic configuration the first plurality of links; determining a plurality of second line segments representing a current kinematic configuration of the second plurality of links; determining respective distances between select pairs of line segments, each pair of the select pairs comprising a first line segment of the plurality of first line segments and a second line segment of the plurality of second line segments; and determining, based on the respective distances, whether the collision condition exists.
[0007] In general, in one aspect, one or more embodiments relate to a method for inter-instrument collision avoidance in a computer-assisted system, the computer-assisted system comprising: a repositionable assembly configured to support a first instrument and a second instrument, the first instrument comprising a first plurality of links and the second instrument comprising a second plurality of links; and wherein the method comprises detecting a collision condition between the first instrument and the second instrument by: determining a plurality of first line segments representing a current kinematic configuration the first plurality of links; determining a plurality of second line segments representing a current kinematic configuration of the second plurality of links; determining respective distances between select pairs of line segments, each pair of the select pairs comprising a first line segment of the plurality of first line segments and a second line segment of the plurality of second line segments; and determining, based on the respective distances, whether the collision condition exists.
[0008] In general, in one aspect, one or more embodiments relate to a non-transitory machine-readable medium comprising a plurality of machine-readable instructions executed by one or more processors associated with a computer-assisted system comprising: a repositionable assembly configured to support a first instrument and a second instrument, the first instrument comprising a first plurality of links and the second instrument comprising a second plurality of links, and wherein the plurality of machine-readable instructions cause the one or more processors to perform a method comprising: detecting a collision condition between the first instrument and the second instrument by: determining a plurality of first line segments representing a current kinematic configuration the first plurality of links; determining a plurality of second line segments representing a current kinematic configuration of the second plurality of links; determining respective distances between select pairs of line segments, each pair of the select pairs comprising a first line segment of the plurality of first line segments and a second line segment of the plurality of second line
segments; and determining, based on the respective distances, whether the collision condition exists.
[0009] Other aspects of the invention will be apparent from the following description and the appended claims.
BRIEF DESCRIPTION OF DRAWINGS
[0010] The accompanying drawings are not intended to be drawn to scale. In the drawings, each identical or nearly identical component that is illustrated in various figures may be represented by a like numeral. For purposes of clarity, not every component may be labeled in every drawing. In the drawings:
[0011] FIG. 1 shows an example repositionable assembly in accordance with one or more embodiments.
[0012] FIG. 2A shows an example computer-assisted system in accordance with one or more embodiments.
[0013] FIG. 2B shows an example system in accordance with one or more embodiments.
[0014] FIG. 3 shows an example repositionable assembly in accordance with one or more embodiments.
[0015] FIGs. 4A and 4B show example instrument in accordance with one or more embodiments.
[0016] FIGs. 5A-5D show examples of possible inter-instrument collisions when no actions for collision avoidance are taken.
[0017] FIGs. 5E and 5F show examples of a show examples of a repositionable assembly with a set of instruments supported by a manipulator- supporting link that is rotatable about an insertion axis
[0018] FIG. 6 shows a flowchart describing an example method for inter-instrument collision avoidance, in accordance with one or more embodiments.
[0019] FIG. 7 shows an example configuration of pluralities of links associated with instruments disposed on a repositionable assembly.
DETAIEED DESCRIPTION
[0020] Specific embodiments of the disclosure will now be described in detail with reference to the accompanying figures. Eike elements in the various figures are denoted by like reference numerals for consistency.
[0021] In the following detailed description of embodiments of the disclosure, numerous specific details are set forth in order to provide a more thorough understanding of the invention. However, it will be apparent to one of ordinary skill in the art that the invention may be practiced without these specific details. In other instances, well-known features have not been described in detail to avoid unnecessarily complicating the description.
[0022] Throughout the application, ordinal numbers (e.g., first, second, third, etc.) may be used as an adjective for an element (i.e., any noun in the application). The use of ordinal numbers is not to imply or create any particular ordering of the elements, and is not to limit any element to being only a single element unless expressly disclosed, such as by the use of the terms “before”, “after”, “single”, and other such terminology. Rather, the use of ordinal numbers is to distinguish between the elements. By way of an example, a first element is distinct from a second element, and the first element may encompass more than one element and succeed (or precede) the second element in an ordering of elements.
[0023] This disclosure describes various devices, elements, and portions of computer- assisted systems and elements in terms of their state in three-dimensional space. As used herein, the term “position” refers to the location of an element or a portion of an element (e.g., three degrees of translational freedom in a three-dimensional space, such as along Cartesian x-, y-, and z-coordinates). As used herein, the term “orientation” refers to the rotational placement of an element or a portion of an element (e.g., three degrees of rotational freedom in three-dimensional space, such as about roll, pitch, and yaw axes, represented in angle-axis, rotation matrix, quaternion representation, and/or the like). As used herein, and for a device with a kinematic series, such as with a repositionable structure with a plurality of links coupled by one or more joints, the term “proximal” refers to a direction toward a base of the kinematic series, and “distal” refers to a direction away from the base along the kinematic series.
[0024] As used herein, the term “pose” refers to the multi-degree of freedom (DOF) spatial position and orientation of a coordinate system of interest attached to a rigid body. In general, a pose includes a pose variable for each of the DOFs in the pose. For example, a full 6-DOF pose for a rigid body in three-dimensional space would include 6 pose variables corresponding to the 3 positional DOFs (e.g., x, y, and z) and the 3 orientational DOFs (e.g., roll, pitch, and yaw). A 3-DOF position only pose would include only pose variables for the 3 positional DOFs. Similarly, a 3-DOF orientation only pose would include only pose variables for the 3 rotational DOFs. Further, a velocity of the pose captures the change in
pose over time (e.g., a first derivative of the pose). For a full 6-DOF pose of a rigid body in three-dimensional space, the velocity would include 3 translational velocities and 3 rotational velocities. Poses with other numbers of DOFs would have a corresponding number of velocities translational and/or rotational velocities.
[0025] Aspects of this disclosure are described in reference to computer-assisted systems, which can include devices that are teleoperated, externally manipulated, autonomous, semiautonomous, and/or the like. Further, aspects of this disclosure are described in terms of an implementation using a teleoperated surgical system, such as the da Vinci® Surgical System commercialized by Intuitive Surgical, Inc. of Sunnyvale, California. Knowledgeable persons will understand, however, that inventive aspects disclosed herein may be embodied and implemented in various ways, including teleoperated and nonteleoperated, and medical and non-medical embodiments and implementations.
Implementations on da Vinci® Surgical Systems are merely exemplary and are not to be considered as limiting the scope of the inventive aspects disclosed herein. For example, techniques described with reference to surgical instruments and surgical methods may be used in other contexts. Thus, the instruments, systems, and methods described herein may be used for humans, animals, portions of human or animal anatomy, industrial systems, general robotic, or teleoperated systems. As further examples, the instruments, systems, and methods described herein may be used for non-medical purposes including industrial uses, general robotic uses, sensing or manipulating non-tissue work pieces, cosmetic improvements, imaging of human or animal anatomy, gathering data from human or animal anatomy, setting up or taking down systems, training medical or non-medical personnel, and/or the like.
Additional example applications include use for procedures on tissue removed from human or animal anatomies (with or without return to a human or animal anatomy) and for procedures on human or animal cadavers. Further, these techniques can also be used for medical treatment or diagnosis procedures that include, or do not include, surgical aspects.
[0026] Referring now to the drawings, in which like reference numerals represent like parts throughout the several views, FIG. 1 shows an example repositionable assembly 100, in accordance with one or more embodiments.
[0027] FIG. 1 illustrates an example repositionable assembly 100 of a computer- assisted system. The repositionable assembly 100 comprises a proximal repositionable structure 106 and one or more distal repositionable structures, according to various embodiments. As shown in FIG. 1, a repositionable assembly 100 includes, without limitation, a proximal repositionable structure 106 that physically supports one or more distal
repositionable structures (e.g., a first manipulator 102 and a second manipulator 104). The first manipulator 102 is configured to support a first instrument 122, and the second manipulator 104 is configured to support a second instrument 124. In various embodiments, each of the proximal repositionable structure 106, the first and second manipulators 102, 104, and the first and second instruments 122, 124 can include any number of joints 144 of any type, and any number of links 142 of any geometry. While FIG. 1 shows a proximal repositionable structure 106 supporting two distal repositionable structures 102, 104 configured to support instruments 122 and 124, respectively, the proximal repositionable structure 106 can support any number of distal repositionable structures, and the distal repositionable structures 102, 104 can each support any number of instruments. Furthermore, the first instrument 122 and/or the second instrument 124 may include any number of links 142 and any number of joints 144. While FIG. 1 illustrates instrument 122 as an instrument with jawed end effectors and instrument 122 as an imaging instrument, such illustration is merely illustrative. In particular, the manipulators 102 and 104 may support any instrument type.
[0028] The combination of the first instrument 122 and the first manipulator 102 may have the same number of joints as the combination of the second instrument 124 and the second manipulator 104. Alternatively, as shown in FIG. 1, the combination of the first instrument 122 and the first manipulator 102 may have fewer joints than the combination of the second instrument 124 and the second manipulator 104.
[0029] In some embodiments, the motion of each of the proximal repositionable structure 106, first manipulator 102, first instrument 122 with jawed end effectors (not labeled), second manipulator 104, and second instrument 124 with an imaging device (not labeled) is relative to a corresponding reference. The corresponding references can be the same reference, or be different references. A reference can be, for example and without limitation, a reference point, a reference line or other geometric feature, a reference frame, etc. The reference can be fixed to the environment (e.g., a point, geometric feature, or frame of reference fixed to the environment or the earth; such a reference frame can be called a “world frame”), to a subject of a procedure such as a workpiece or part of a patient (e.g., a point, geometric feature, or frame of reference fixed to a patient feature and/or the like; such a reference frame can be called a “subject frame,”), to a base of the robotic system (e.g., a base point, a base plane, frame of reference of the repositionable assembly or some other part of the robotic system), and/or the like. In some embodiments, the motions of the proximal repositionable structure 106, first manipulator 102, first instrument 122, second manipulator
104, and/or second instrument 124 are determined, controlled, or sensed relative to a same reference, or to different references. For example, the motion of the proximal repositionable structure 106 could be relative to a world frame while the motion of the first manipulator 102 and first instrument 122 could be relative to a reference point or reference frame fixed to the proximal repositionable structure 106. In the example shown in FIG. 1, the bases of first manipulator 102 and second manipulator 104 are attached to a distal portion of the proximal repositionable structure 106, and motion of the proximal repositionable structure 106 moves the bases of first manipulator 102 and second manipulator 104. This “caused motion” of the bases of first manipulator 102 and second manipulator 104 can move distal portions of first manipulator 102 and/or second manipulator 104, and of any instruments or other elements attached to first manipulator 102 and second manipulator 104. In some embodiments, a caused motion (e.g., of the first manipulator 102 and first instrument 122 or of the second manipulator 104 and second instrument 124) is determined, responded to, or sensed relative to the same or a different reference as used for the motion of the proximal repositionable structure 106.
[0030] FIG 2A illustrates an example computer-assisted system 200, according to various embodiments. As shown in Figure 2A, the computer-assisted system 200 includes, without limitation, a repositionable assembly 210 and a user input system 250. In a teleoperation scenario, an operator (not shown) uses the user input system 250 to operate the repositionable assembly 210, such as in a leader-follower configuration (also often called teleoperation configuration or master- slave configuration in industry) of the computer- assisted system 200. In the leader-follower configuration, the user input system 250 is the leader, and the repositionable assembly 210 is the follower of the leader-follower configuration.
[0031] The repositionable assembly 210 can be used to introduce a set of instruments (not shown here, discussed below with reference to FIGs. 4A and 4B) to a work site through a port or entry guide 230 (a cannula is shown) inserted in an aperture. In a medical scenario, the work site can be on or within a body cavity of a patient, and the aperture can be a minimally invasive incision or a natural body orifice. When used, the port or entry guide 230 can be free-floating, held in place by a fixture separate from the repositionable assembly 210, or held by a linkage 222 or other part of the repositionable assembly 210. The linkage 222 is coupled to additional joints and links 214, 220 of the repositionable assembly 210, and these additional joints and links 214, 220 are mounted on a base 212. In some embodiments, the drivable structure 222 terminates in a manipulator- supporting link 224. A set of manipulators
226 are coupled to the manipulator-supporting link 224. Each of the manipulators 226 include a carriage (or other instrument-coupling link) configured to couple to an instrument, and each of the manipulators 226 include one or more joint(s) that can be driven to move the carriage. For example, a manipulator 226 can include a prismatic joint that, when driven, linearly moves the carriage and any instrument(s) coupled to the carriage. In some embodiments, this linear motion is along an insertion axis, as further described below with reference to FIGs. 4A and 4B. In this configuration the elements of the repositionable assembly 210 that are proximal to the manipulators 226 may be understood as the proximal repositionable structure as previously described in reference to FIG. 1.
[0032] In some embodiments, the additional joints and additional links 214 and 220 are used to position the port 230 at the aperture or another location. FIG. 2A illustrates a prismatic joint for vertical adjustment (as indicated by arrow “A”) and a rotary joints for horizontal adjustment (as indicated by arrow “B”). The drivable structure 222 is used to robotically pivot the port 230 (and the instruments disposed within it at the time) in yaw, pitch and roll angular rotations about the remote center of motion as indicated by arrows C, D and E, respectively. These joints may be considered part of the proximal repositionable structure as previously described in reference to FIG. 1.
[0033] In some embodiments, actuation of the degrees of freedom provided by joint(s) of the instrument(s) is provided by actuators disposed in, or whose motive force (e.g., linear force or rotary torque) is transmitted to, the instrument(s). Examples of actuators include rotary motors, linear motors, solenoids, and/or the like. The actuators drive transmission elements in the manipulators and/or in the instruments to control the degrees of freedom of the instrument(s). For example, the actuators can drive rotary discs of the manipulator that couple with rotary discs of the instrument(s), where driving the rotary discs of the instruments drives transmission elements in the instrument that couple to move the joint(s) of the instrument, or to move the end effector(s) of the instrument, as further discussed below with reference to FIGs. 4A and 4B. Accordingly, the degrees of freedom of the instrument(s) are controlled by actuators that drive the instrument(s) in accordance with control signals determined based on inputs from the associated input devices (e.g., input devices 252 of the user input system 250). The control signals are determined in order to cause instrument motion or other actuation as indicated by movement of the input control devices or any other control signal. Furthermore, in some embodiments, appropriately positioned sensors, e.g., encoders, potentiometers, and/or the like, are provided to enable measurement of indications of the joint positions, or other data that can be used to derive
joint position, such as joint velocity. The actuators and sensors are disposed in, transmit to, and/or receive signals from the manipulator(s) 226.
[0034] While a particular configuration of the repositionable assembly 210 is shown in FIG. 2A, those skilled in the art will appreciate that embodiments of the disclosure can be used with any design of repositionable assembly. For example, a repositionable assembly can have any number and any types of degrees of freedom, may or may not be configured to couple to a port, use a port other than a cannula, and/or other configuration different from what is shown in FIG 2A.
[0035] In the embodiments shown in FIG 2A, the user input system 250 includes one or more input devices 252 operated by the operator (not shown). The one or more input devices 252 are contacted and manipulated by the hands of the operator, with one input device for each hand. Examples of such hand-input-devices include any type of device manually operable by human user, e.g., joysticks, trackballs, button clusters, and/or other types of haptic devices typically equipped with multiple degrees of freedom. Additionally, in some embodiments, position, force, and/or tactile feedback devices (not shown) are employed to transmit position, force, and/or tactile sensations from the instruments back to the operator's hands through the input devices 252.
[0036] The input devices 252 are supported by the user input system 250 and are shown as mechanically grounded, and in other implementations may be mechanically ungrounded. An ergonomic support 256 is provided in some implementations. For example, FIG. 2A shows an ergonomic support 256 including forearm rests on which the operator may rest his or her forearms while manipulating the input devices 252. In some examples, the operator performs tasks at a work site near the repositionable assembly 210 during a medical procedure by controlling the repositionable assembly 210 using the input devices 252.
[0037] A display unit 254 is included in the user input system 250. The display unit 254 displays images for viewing by the operator. The display unit 254 provides the operator with a view of the worksite with which the repositionable assembly 210 interacts. The view can include, for example, stereoscopic images or three-dimensional images to provide a depth perception of the worksite and the instrument(s) of the repositionable assembly 210 in the worksite. The display unit 254 can be moved in various degrees of freedom to accommodate the operator’s viewing position and/or to provide control functions. Where a display unit (such as the display unit 254 is also used to provide control functions, such as to command the repositionable assembly, the display unit also includes an input device (e.g., another input device 252).
[0038] When using the user input system 250, the operator can sit in a chair or other support in front of the user input system 250, position his or her eyes to see images displayed by the display unit 254, grasp and manipulate the input devices 252, and rest his or her forearms on the ergonomic support 256 as desired. In some implementations, the operator can stand at the workstation or assume other poses, and the display unit 254 and input devices 252 may differ in construction, be adjusted in position (height, depth, etc.), etc.
[0039] FIG. 2B illustrates an example system 270, according to various embodiments. The system 270 corresponds to the computer-assisted system 200 and includes one or more computing systems 272. A computing system 272 includes a processing system and is used to process input provided by the user input system 250, e.g., from the input device(s) 252 manipulated by an operator. In some embodiments, a computing system 272 is further used to provide an output, e.g., a video image to the display unit 274. Examples of display unit 274 include LCDs, LEDs, organic LED displays, projectors, etc. In some embodiments, one or more computing systems 272 are used to control the repositionable assembly 210.
[0040] In one or more embodiments, the computing system(s) 272 executes one or more control methods. The control methods include instructions for controlling one or more components of the repositionable assembly 210. In one or more embodiments, joint movements of the repositionable assembly 210 are controlled by one or more control methods driving one or more joints using actuators of the repositionable assembly 210, the joint movements being calculated by a processor of a processing system of the computing system(s) 272. The control methods process control signals from the user input system 250 or elsewhere, and/or sensor signals (e.g., positional encoder data from joint position sensors, image data from image instruments such as ultrasonic probes or cameras or endoscopes, and/or the like), to calculate commands for the joint actuators.
[0041] In some embodiments, the control methods perform at least some of the calculations of the joint commands using vectors and/or matrices, some of which have elements corresponding to positions, velocities, and/or forces/torques of the joints. The range of alternative joint configurations available to the control methods can be conceptualized as a joint space. Lor example, in some embodiments, the joint space has as many dimensions as the repositionable assembly has degrees of freedom, and a particular configuration of the repositionable assembly represents a particular point in the joint space, with each coordinate corresponding to a joint state of an associated joint of the repositionable assembly.
[0042] As used herein, the term “state” of a joint or multiple joints refers to the control variables associated with the joint or the multiple joints, respectively. Lor example,
the state of an angular joint refers to the angle defined by that joint within its range of motion, and/or to the angular velocity (or speed or direction) of the joint. Similarly, the state of an axial or prismatic joint refers to the joint's axial or linear position, and/or to its axial or linear velocity (or speed or direction). While one or more of the control methods described herein include position controllers, they often also have velocity control aspects. Alternative embodiments can rely primarily or entirely on velocity controllers, force controllers, acceleration controllers, and/or the like without departing from the disclosure. Various aspects of control systems that can be used in such devices are described in U.S. Pat. No. 6,699,177, which is incorporated herein by reference. In general, as long as the movements described are based on the associated calculations, the calculations of movements of the joints and movements of an end effector described herein are performed using a position control technique, a velocity control technique, an acceleration control technique, a force or torque control technique, a combination of some or all of the foregoing, and/or the like. [0043] In some embodiments, the control modes include one or more other types of control modes. For example, during a robotic task being performed under the control of input devices 252 operated by a user, various joints of the repositionable assembly can be commanded to a same position and controlled to maintain static positions. However, in another control mode, one or more of the joints can be commanded to be “floating”, and facilitate motion of that joint due to externally applied force. For example, a joint held in place by a brake can be floated by partially or entirely releasing the brake. An example of such a joint includes a passive joint held in place by an electromagnetic brake. As another example, a joint driven by actuator(s) can be held in place by commanding the actuator(s) to hold the joint position, and be floating by updating the command to the actuator(s) to the then-current position, velocity, and/or acceleration of the joint. As a result, a floating joint is readily reconfigured by an externally applied force or torque, without a control algorithm and/or a braking force seeking to counteract the reconfiguration caused by sufficient externally applied force or torque. In some embodiments, a floating joint is further controlled to exhibit other characteristics or provide additional responses, such as to provide a certain type or level of damping response. A floating joint can still be braked, actuated, or otherwise managed for friction or gravity compensation. Such compensation can be provided by, for example and without limitation, passive springs, actively driven actuators, and/or the like. Further, in some embodiments, joints that are not moved by actuators can still be gravity compensated, friction compensated, dampened, and/or the like by actuators.
[0044] In various embodiments, multiple different control modes are combined during operation of the repositionable assembly. For example, some joints could be controlled to maintain position and resist or rebound from attempted external articulation of those joints, while other joints could be controlled to be floating and facilitate external articulation of those other joints. Parameters such as joint position, velocity, or acceleration of the joints are detected by joint sensors. The sensor signals are used to provide kinematic information of the repositionable assembly.
[0045] The architecture of the control methods used for controlling the repositionable assembly can be of any appropriate form. As a specific example, the control architecture can be hierarchical, and could include a high-level controller and multiple joint controllers. A commanded movement is received by the high-level controller in, for example, a Cartesiancoordinate space (referred to herein as Cartesian-space). The commanded movement could be, for example, based on a movement command (e.g., in the form of a position and/or velocity) received from the user input system 250, or any other system that provides a movement command. The commanded movement is converted into commanded joint positions or joint velocities (e.g., linear or angular joint positions, linear or angular joint velocities). In some embodiments, the conversion is performed using an inverse kinematics algorithm. Subsequently, the joint controllers convert the received commanded joint positions or velocities into commanded currents to drive the actuators producing joint movements. The joint movements together produce a repositionable assembly movement that reflects the commanded movement.
[0046] In some embodiments, a joint controller controls a joint position. In some embodiments, the joint controller controls other variables such as joint velocity and/or joint force (linear force or angular torque). A joint controller receives a feedback signal in the form of a sensed joint state from an associated joint sensor, which it can use for closed- loop control. The sensed joint state includes, for example and without limitation, a joint position, a joint velocity (or component of velocity such as speed or direction), a joint acceleration (or component of acceleration), and/or the like, representing the joint movement. The sensed joint state is derived from the signals obtained from the joint sensor. A joint sensor can be, for example, an encoder, a potentiometer, an accelerometer, a hall effect sensor, and/or the like. In some embodiments, a state observer or estimator (not shown) is used. Each joint controller can implement any appropriate control scheme, such as a proportional integral derivative (PID), proportional derivative (PD), full state feedback, sliding mode, and/or various other control schemes, without departing from the disclosure.
[0047] In one or more embodiments, the control methods further perform at least one of the steps described in FIG. 6 below.
[0048] A computing system 272 may include, without limitation, one or more computer processors, non-persistent storage (e.g., volatile memory, such as random access memory (RAM), cache memory), persistent storage (e.g., a hard disk, an optical drive such as a compact disk (CD) drive or digital versatile disk (DVD) drive, a flash memory, etc.), a communication interface (e.g., Bluetooth interface, infrared interface, network interface, optical interface, etc.), and/or numerous other elements and functionalities. In some embodiments, a computer processor of a computing system 272 is an integrated circuit for processing instructions. For example, the computer processor can be one or more cores or micro-cores of a processor.
[0049] In some embodiments, a communication interface of a computing system 272 includes an integrated circuit for connecting the computing system 272 to a network (not shown) and/or to another device, such as another computing system 272. Further, in some embodiments, the computing system 272 includes one or more output devices, such as a display unit 274, a printer, a speaker, external storage, or any other output device. Software instructions in the form of computer readable program code to perform embodiments of the disclosure are stored, in whole or in part, temporarily or permanently, on non-transitory computer readable medium. Specifically, the software instructions correspond to computer readable program code that, when executed by a processor(s), is configured to perform one or more embodiments of the disclosure. In some embodiments, a computing system 272 is connected to or be a part of a network. The network may include multiple nodes. Each node corresponds to a different computing system, group of computing systems, group of nodes, and/or the like.
[0050] In some embodiments, the repositionable assembly 210 couples to an instrument when used to perform a procedure. The instrument can include an imaging device, e.g., an endoscope or an ultrasonic probe, usable to capture images of the worksite and output the captured images to an auxiliary system 280. In some embodiments, the auxiliary system 280 processes the captured images using one or more image processing techniques prior to any subsequent display. For example, the auxiliary system 280 can overlay the captured images with a virtual control interface prior to displaying the combined images to the operator via the user input system 250. In some embodiments, one or more separate display units 274 are coupled with a computing system 272 and/or the auxiliary
system 280 for local and/or remote display of images, such as images of the procedure site or other related images.
[0051] FIG. 3 illustrates another example repositionable assembly 300 of a computer- assisted system such as a robotic system, according to various embodiments. As shown in FIG. 3, a repositionable assembly 300 includes, without limitation, a repositionable assembly 306, physically supporting a first manipulator 302 and a second manipulator 304. The first manipulator 302 supports a first instrument 322, and the second manipulator 304 supports a second instrument 324. Each of the first manipulator 302, second manipulator 304, and repositionable assembly 306, the first instrument 322, and second instrument 324 includes any number of joints (e.g., joints 344 and 346) of any type and/or any number of links (e.g., links 342) of any geometry. The first instrument 122 and/or the second instrument 124, when equipped with links and joints, may be directly disposed on the proximal repositionable structure 106, without a first manipulator 102 and/or a second manipulator 104. As shown in FIG. 3, the repositionable assembly 306 provides a movable support for the first manipulator 302 and second manipulator 304. Prismatic joints 346 enable translational movement of the first manipulator 302 and the second manipulator 304 relative to the repositionable assembly 306. While two manipulators (the first manipulator 302 and the second manipulator 304) are shown in FIG. 3, any number of manipulators can be supported by the repositionable assembly 306.
[0052] In some embodiments, the repositionable assembly 300 is part of a medical robotic system. For example, the repositionable assembly 300 can be configured as a tableside-installed medical robotic system. The repositionable assembly 306 could be attached to a base of a surgical or examination table. Further, the medical robotic system could include one or more additional drivable structure assemblies with the same or a different design. For example, the repositionable assembly 300 can be installed on one side of the table, and a same or different repositionable assembly can be installed on the same side, or another side, of the table.
[0053] FIG. 4A illustrates an example instrument 400 (also referred to herein as an instrument 400), according to various embodiments. The instrument 400 in Figure 4A includes, without limitation, a shaft 410, and an end effector located at a first end of the instrument 400. A housing 430, arranged to releasably couple the instrument 400 to a manipulator (shown, for example, in FIG 2A), is located at a second end of the instrument 400. In some embodiments, the shaft 410 is rotatably coupled to the housing 430 to enable angular displacement of the shaft 410 relative to the housing 430, as indicated by arrows 448.
[0054] Various types of end effectors 440 can be used. For example, the end effector 440 can include one finger, two fingers (e.g., jaws 442 that may open and close), or three or more fingers. Examples of end effectors include, but are not limited to, scissors, forceps, staplers, cutting and cautery instruments, and/or the like. As another example, an end effector can further include an imaging device, e.g., an endoscope or an ultrasonic probe, to capture images of the worksite.
[0055] In some embodiments, an end effector 440 is actuated by transmission elements (e.g., cables, metal bands, screws, tubes, push rods, etc.) that connect parts of the instrument to drive elements (e.g., pulleys, capstans, spools, nuts, linear slides, or the like) (not shown) in the housing 430. Movement (e.g., translation or rotation) of the drive elements thereby controls the position of the end effector, or other degrees of freedom such as jaw opening, such that the end effector may translate or rotate, the jaws may open and close, and/or the like. In some embodiments, upon coupling of an instrument 400 on a manipulator, the drive elements engage with actuators of the manipulator, such as by engaging with transmission elements coupled to the actuators. As an example, a description of the control of an instrument like the instrument 400 can be found in U.S. Pat. No. 6,394,998, entitled “Surgical Tools for Use in Minimally Invasive Telesurgical Applications,” which is incorporated herein by reference.
[0056] In the example shown in FIG. 4A, the joints of the instrument 400 include, without limitation, a wrist 420 proximal to the end effector 440 and two shaft offset joints 422, 424 proximal to the wrist 420. The wrist 420 may enable rotation of the end effector 440 in one or more directions. The shaft offset joints 422, 424 can enable, for example, a translational offset 426 of the end effector 440 relative to the insertion axis 412 using the additional link serially coupled between the instrument shaft 410 and the end effector 440, in addition to the rotating provided by the wrist 420. The shaft offset joints 422, 424 may, thus, increase the workspace reachable by the end effector 440 of the instrument 400. Like the end effector 440, the wrist 420 and the shaft offset joints 422, 424 may be actuated by control cables.
[0057] The instrument 450 shown in FIG. 4B includes, without limitation, various elements of the instrument 400 shown in FIG. 4A and operates in a substantially similar manner to the instrument 400 shown in FIG. 4A. Specifically, the instrument 450 includes a shaft 460 with an insertion axis 462 for insertion/retraction, and a wrist 470 proximal to an end effector 490. The wrist 470 may enable a pitch and a yaw movement of the end effector 490. The instrument 450 is equipped with a shaft offset joint 474 that, together with wrist
470 enables laterally offsetting the end effector 490 from the insertion axis 462 in any direction. Unlike instrument 400, the instrument 450 does not include a degree of freedom for rotation about the insertion axis 462 relative to the housing 480. Accordingly, the instrument 450 may move in only five degrees of freedom, with there being no instrument roll degree of freedom. In one embodiment, the end effector is an imaging device such as an endoscopic camera.
[0058] Instruments 400 and 450 can be used for robotic procedures such as robotic medical procedures (e.g., surgeries), in accordance with one or more embodiments.
[0059] While FIGs. 4A and 4B show particular configurations of instruments, designed to engage with a particular type of manipulator, other configurations of instruments are within the scope of the disclosure. For example, embodiments of instruments 400 and 450 could have multi-degree-of-freedom wrists (e.g., pitch and yaw degrees of freedom), single-degree-of-freedom wrists (e.g., pitch or jaw), or no wrists. Also, various embodiments of instruments could have any suitable type of end effector including, for example, scissors, forceps, staplers, irrigation nozzles, hooks, scissors, blunt dissection instruments, needle drivers, imaging devices, and/or the like. Further, different housings can also be used to interface with different types of manipulators.
[0060] The subsequently discussed figures illustrate the possibility of collisions, as they may occur in absence of a method as described in reference to FIG. 6.
[0061] FIGs. 5A and 5B show examples of possible inter-instrument collisions based on the configuration previously introduced in reference to FIG. 1. Referring to FIG. 5A, the repositionable assembly 100 includes a proximal repositionable structure 106 and one or more distal repositionable structures (e.g., two manipulators 102, 104). The first manipulator 102 supports an instrument 122 that is a two-fingered instrument. The second manipulator 104 supports an instrument 124 that is an imaging instrument. While particular instruments are shown, the disclosure is not limited to these types of instruments.
[0062] In the example of FIG. 5A, the kinematic configuration of the second manipulator 104 and the second instrument 124 is changed from the original configuration (dashed lines) to an updated configuration (solid lines). The change may be the result of a command provided by a user or internally generated by an algorithm. The change may be commanded, for example, to update the field of view of the imaging instrument. The kinematic configuration of the proximal repositionable structure 106 and the kinematic configuration of the first manipulator 102 and the first instrument 122 remain unchanged. As indicated by the lightning bolt symbol, the updating of the kinematic configuration the second
manipulator 104 and the second instrument 124 results in a collision of the second instrument 122 with the first manipulator 102. As FIG. 5B illustrates, a collision may also occur when changing the kinematic configuration of the first manipulator 102 and a first instrument 122. [0063] While FIGs. 5A and 5B illustrate a collision of particular elements (the second instrument 124 colliding with one particular link of the first manipulator 102 in FIG. 5A, and the first instrument 122 colliding with one particular link of the second manipulator 104 in FIG. 5B), other types of collisions may occur without departing from the disclosure. Collisions may occur between any pairs of links that are kinematically capable of colliding. These pairs of links may be formed by links belonging to instruments 122, 124, manipulators 102, 104, and/or the proximal repositionable structure 106. Further, a collision may be caused by any type of movement, including movement of one of more of the manipulators 102, 104, and/or movement of one or more of the instruments 122, 124.
[0064] FIGs. 5C and 5D show examples of possible inter-instrument collisions based on the configuration previously introduced in reference to FIG. 2A. In the example, a repositioning of an instrument 510 using a movement of a manipulator-supporting link proximal to two instruments is shown. FIG. 5C illustrates a scenario 500C before the repositioning, and FIG. 5D shows a scenario 500D after the repositioning. In the scenario illustrated by FIGs. 5C and 5D, the end effector 512 of the instrument 510 is repositioned. In other scenarios, other portions of instrument 510, or other instruments, can be repositioned instead or in addition to the end effector 512.
[0065] As shown in FIGs. 5C and 5D, a repositionable assembly includes a proximal repositionable structure, which is or includes a manipulator- supporting link 502 that supports multiple manipulators 550 and 552 coupled to multiple instruments 504 and 510. In other words, the manipulator-supporting link 502 forms a common mechanical base for the manipulators 550, and 552 that support the instruments 504 and 510. In some embodiments, the manipulator-supporting link 502 corresponds to the manipulator-supporting link of a proximal repositionable structure as previously introduced with reference to FIG 2A. Movement of one or more joints of the proximal repositionable structure, for example, results in movement of the manipulator-supporting link 502. When the manipulator-supporting link 502 is moved, the portions of the manipulators 550 and 552 attached to the manipulatorsupporting link 502 are also moved. Such caused motion of the manipulators 550 and 552 can cause motion of the instruments 504 and 510. For example, if the manipulators 550 and 552 are held fixed in configuration, then the movement of the manipulator- supporting link 502 also moves instruments 504 and 510.
[0066] In various embodiments, repositioning of an instrument (e.g., instrument 504 or 510) through movement of a common mechanical base such as the manipulator-supporting link 502 is used to increase the degrees of freedom and/or the range of motion of an instrument (e.g., instrument 504 or 510). Assume, for example, that an operator remotely or teleoperationally controls the end effector 512 of instrument 510 by providing a commanded motion via the input devices 252 in FIG. 2A or that a commanded motion is determined by an autonomous algorithm. And, further assume that, in the example of FIGs. 5C and 5D, the operator (or the autonomous algorithm) provides commands intended to cause the end effector 512 to interact with the target 596. In the configuration shown in FIG 5C, the end effector 512 is unable to reach the target 596 if the manipulator- supporting link 502 does not move relative to the target 596. In this example, one factor contributing to the inability of the end effector 512 to reach the target 596 could be that the instrument 510 has fewer joints as compared to some other instruments. For example, instrument 510 lacks shaft offset joints while instruments 504 includes shaft offset joints 508. If instruments 504 and 510 are identical except that instrument 510 lacks shaft offset joints, then instrument 504 would generally have a greater number of degrees of freedom and/or an increased range of motion in comparison to instrument 510. However, even if instrument 510 includes additional joints such as shaft offset joints or other joints, end effector 512 may still be unable to reach target 596 without motion of the manipulator-supporting link 502 if the additional joints do not provide sufficient additional range of motion and/or do not support movement the applicable degree(s) of freedom. Accordingly, motion of the manipulator- supporting link 502 can be used to produce movement of the end effector 512.
[0067] Various instruments may comprise or lack joints for various degrees of freedom. For example, any of instruments 504 and 510 could lack joints to support movement about a roll degree of freedom about a respective roll axis. As another example, an instrument may not include an articulated wrist. Also, the target 596 can be any kind of object, and the instrument 510 can be any kind of instrument equipped with an end effector suitable to interact with the target 596. For example, in a surgical scenario, the end effector could include forceps, and the target could include tissue.
[0068] In the example of FIGs. 5C and 5D, the instruments 504 and 510 are inserted through a cannula 530 toward a worksite 598 containing the target 596, and the cannula 530 is inserted through an aperture 592 in the barrier 594. In a medical scenario, the barrier 594 could be a body wall of a patient, and the aperture 592 could be a minimally invasive incision or a natural body orifice of the patient.
[0069] FIG. 5D illustrates a pivoting motion 532 of the manipulator- supporting link 502 about a remote center of motion 540 (also called “remote center” 540). In the example shown in FIG. 5D, the repositionable assembly has been positioned, and/or is controlled, such that the remote center of motion 540 is located approximately centrally in the aperture 592. As shown in the example of FIG 5D, movement of the manipulator-supporting link 502 relative to the target 596, such as the pivoting motion 532 or some other movement (e.g., translational movement, combined translational and rotational movement, etc.), help enable the end effector 512 to execute a commanded motion indicated by the operator via the input devices 252 that the end effector 512 may not have been able to otherwise execute.
[0070] In one or more embodiments, the movement (e.g., pivoting motion 532 that results in a rotation of the insertion axis 590 of the repositionable assembly) moves the manipulator- supporting link 502 supporting the manipulators 550 and 552. Such motion of the manipulators 550 and 552 can also move the instruments 504 and 510. If the manipulators 550 and 552 are held static relative to the manipulator- supporting link 502, then the instruments 504 and 510 jointly move in a common motion in response to the movement of the manipulator-supporting link 502 (all pivoting about the remote center of motion 540 with the pivoting motion 532). Thus, the pivoting motion 532, if the manipulators are held static relative to the manipulator-supporting link 502, would cause motion of the end effector 512 of the instrument 510 such that it reaches the position shown in FIG. 5D. In addition, the pivoting motion 532 would also cause motion of the end effector 506 of the instrument 504, such that the instrument 504 reaches the position shown by dashed lines in FIG. 5D, if no motion occurs other than these caused motions. In the example shown in FIG. 5D, the movement of instrument 510 is commanded and thus is desired. However, the movement of instrument 504 that would result from performing only the pivoting motion 532, may not be commanded, expected, or desired. For example, if there was no caused motion, instruments 504 could be expected to remain stationary, in absence of other movement commands. As another example, instrument 504 could be expected to follow motion commands provided for these instruments, independent from the movement of instrument 510.
In some embodiments, the manipulator 550 (shown supporting the instrument 504), is driven to move in a manner that compensates partially or entirely for the motion caused by the movement of the manipulator- supporting link 502 (e.g., such as by pivoting motion 532). In some instances, compensating motion of joints of the instrument 504 are commanded in addition, or instead of, compensating motion of the manipulator 550. Thus, the compensation can involve movement of one or more joints of the manipulator 550 supporting the
instrument 504, and/or one or more joints of the instrument 504. For example, the compensation can involve moving the shaft offset joints (e.g., 508) or other joints of the instrument (e.g., 504), and/or moving the manipulator (e.g., 550) to perform an insertion or retraction movement along the insertion axes of the instrument 504. The compensation can be performed such that the end effector (e.g., end effector 506) remains substantially stationary within the work site, while the end effector 512 of the instrument 510 is repositioned relative to the work site. As an example, methods for coordinating movements of multiple instruments of a repositionable assembly are described in further detail in International Patent Publication No. WO2022/046787A1, entitled “Method and System for Coordinated Multiple-Tool Movement Using a Drivable Assembly,” which is incorporated herein by reference.
[0071] The operations as described in reference to FIGs. 5C and 5D may be performed as follows. In some embodiments, a first commanded motion of instrument 504 is determined such that a first relative motion of the end effector 506 is effected relative to the manipulator- supporting link 502. Further, a second commanded motion of the manipulatorsupporting link 502 is determined such that the first commanded motions performed in conjunction with the second commanded motion cause the end effector 506 to remain stationary relative to the workspace. The second commanded motion, when performed, causes motion of the second instrument 510, e.g., as illustrated in FIG. 5D, where the end effector 512 of the second instrument 510 is repositioned/reoriented to reach for the target 596.
[0072] As indicated by the lightning bolt symbol, the commanded motion of the end effector 512 involving movement of the manipulator- supporting link 502, the manipulator 552, and the instrument 510 with the end effector 512, in combination with the compensatory movement of the manipulator 550 and instrument 504 that keeps the end effector 506 stationary results in a collision of the end effectors 506 and 512.
[0073] While FIGs. 5C and 5D illustrate a collision of particular elements (the end effectors 506 and 512), resulting from a particular combination of a movement and a compensatory movement based on a commanded motion, other types of collisions may occur without departing from the disclosure. Collisions may occur between any pairs of links that are kinematically capable of colliding. These pairs of links may be formed by links belonging to instruments 504, 510 (including end effectors 506, 512), manipulators 550, 552, and/or the manipulator-supporting link 502. Further, a collision may be caused by any type
of movement that is not limited to the movement in combination with a compensatory movement as described.
[0074] In the above discussion, instrument 510 has fewer joints than instrument 504. However, more generally, instrument 510 can have the same or a greater number of joints, the same or a greater number of degrees of freedom, or the same or a greater range of motion as compared to any instruments supported by the manipulator 550. In all of these cases, movement of the manipulator-supporting link 502 can help increase the degrees of freedom and/or the range of motion of an instrument such as instrument 510 (or of another instrument coupled distally to the manipulator-supporting link (such as instrument 504 or some other instrument (not shown)).
[0075] Further, while FIGs. 5C and 5D illustrate a pivoting motion 532 in conjunction with the worksite being accessible through an aperture, the described methods apply to any type of movement and are not limited to a pivoting motion. For example, the movement of the manipulator-supporting link 502 could alternatively be, or include, one or more other rotational movements, linear or nonlinear translational movements, combinations of translational and rotational movements, etc.
[0076] In one embodiment, the movement of the manipulator- supporting link 502 is a rotation about the insertion axis 590, unlike the rotation of the insertion axis 590 about the remote center 540 which is illustrated in FIGs. 5C and 5D. Referring to FIG. 2A, the rotation may be the rotation identified by arrow “E”. The rotation about the insertion axis 590 may be performed, for example, to enable rotation of end effector 512 of instrument 510 about an axis parallel to the insertion axis 590, in absence of a degree of freedom supporting such a rotation by the instrument 510 itself. In one embodiment, the end effector 512 includes an imaging instrument, and the instrument 510 does not provide a degree of freedom for a rolling of the imaging instrument about an axis aligned with the field of view of the imaging instrument (e.g., to roll the field of view of the imaging instrument). The instrument 510 may, however, have a wrist that enables rotation in pitch and/or yaw direction, e.g., analogous to the instrument 450 of FIG. 4B. In this scenario, the use of the rotation about the insertion axis 590 along with other adjustments of the joints of the instrument 510 enable a rolling of the field of view of the imaging instrument, in addition to pitch and yaw, even in absence of a dedicated degree of freedom to perform this roll operation. An example is provided below in reference to FIGs. 5E and 5F.
[0077] FIGs. 5E and 5F show examples of a repositionable assembly with a manipulator- supporting link that may be rotated about an insertion axis, as previously
discussed. Three instruments 560, 570, and 580 are supported by a manipulator-supporting link 502. In the example as shown, the manipulator- supporting link 502 may be rotated about the insertion axis 590. The instruments 560 and 570, in the example, are similar to the instrument 400 as described in reference to FIG. 4A, and include shaft offset joints 566, 576 distal to the instrument shaft 562, 572. The instruments 560, 570 are equipped with manipulation-type end effectors 564, 574. In contrast, the instrument 580, in the example, is similar to the instrument 450 as described in reference to FIG. 4B, also includes shaft offset joints distal to the instrument shaft 582, but does not have the ability to roll about the instrument shaft. In order to enable instrument 580 to reorient about the roll degree of freedom despite the instrument 580 itself being unable to perform such a roll motion, a roll movement 588 of the manipulator- supporting link 502 about the insertion axis 590 is introduced. The roll movement changes the position and the orientation of the end effector 584 as illustrated in FIG. 5F. The shaft offset joints 566, 576 of instruments 560, 570 may be used to compensate for the roll motion 588 of the manipulator- supporting link, such that the end effectors 564, 574 of instruments 560, 570 remain stationary (e.g., stationary relative to the workspace). In the example as shown, the roll movement 588 does not result in a collision. However, other roll movements could result in collisions. For example, when assuming a roll movement in a direction opposite to the roll movement 588 but with the same amplitude, a collision would occur between the instrument shaft 562 of instrument 560 and the link between shaft offset joints 576 of instrument 570, because the instrument shaft 562 protrudes sufficiently to interfere with this link of instrument 570. Similarly, collisions could occur between links of instruments 560 and 580 and/or instruments 570 and 580.
Accordingly, embodiments of the disclosure may be used to check for collision conditions between pairs of links such as an instrument shaft of one instrument and another link of another instrument. With the instrument shafts 562, 572, and 582 being parallel, it is not necessary to test for collision conditions between the instrument shafts.
[0078] While FIGs. 1, 2A, 2B, 3, 4A, 4B, and 5A-5F show various configurations of components, other configurations may be used without departing from the scope of the disclosure. For example, various components may be combined to create a single component. As another example, the functionality performed by a single component may be performed by two or more components. While a particular instrument with a particular end effector and particular degrees of freedom is shown in FIG. 3, the disclosure generalizes to any type of instrument with any type and number of degrees of freedom. Also, while the instrument is described as being supported by a robotic arm with an instrument holder, the
instrument may be supported by any type of repositionable structure, without departing from the disclosure. Further, while components are often described in context of medical scenarios such as surgical scenarios, embodiments of the disclosure are equally applicable to other domains that involve robotic manipulation, e.g., non-surgical scenarios or systems, nonmedical scenarios or systems, and/or the like.
[0079] Turning to FIG. 6, a flowchart in accordance with one or more embodiments is shown. The flowchart of FIG. 6 depicts a method 600 for computer-assisted systems. The method 600 may be used to detect collision conditions between links of a robotic or teleoperated computer-assisted system. One or more of the steps in FIG. 6 may be performed by various components of systems, previously described with reference to FIGs. 1, 2A, 2B, 3, 4A, 4B, and 5A-5F. While these figures illustrate particular configurations of computer assisted systems, the method is equally applicable to other configurations. The method may be executed on one or more processors, e.g., of the control system of the computer-assisted system.
[0080] While the various steps in the flowchart are presented and described sequentially, one of ordinary skill will appreciate that some or all of the steps may be executed in different orders, may be combined or omitted, and some or all of the steps may be executed in parallel. Additional steps may further be performed. Furthermore, the steps may be performed actively or passively. For example, some steps may be performed using polling or be interrupt driven in accordance with one or more embodiments of the invention. [0081] The subsequently described steps may be performed for repositionable assemblies as previously described or any other repositionable assemblies. In some embodiments, the repositionable assembly supports multiple instruments (e.g., two, three, or more instruments), each having multiple links. In some examples, a repositionable assembly may comprise a proximal repositionable structure and one or more distal repositionable structures. The one or more distal repositionable structures may each support or couple to one or more instruments. Examples of repositionable assemblies and instruments disposed on the repositionable assemblies include but are not limited to those previously discussed in reference to FIGS. 1, 2A, 2B, 3, 4A, and 4B. Collisions between the various links of the instruments and/or between the links of the instruments and the repositionable assembly may occur, for example, as previously discussed in reference to FIGs. 5A-5F. The subsequently described steps may be used to detect the possibility of such collisions between links of the instruments and/or the repositionable assembly, and further to avoid or mitigate these collisions.
[0082] In some embodiments, the method 600 relies on line representations of corresponding structures such as links. For example, a kinematic chain of a first instrument that includes a first set of links may be represented by a set of first line segments. Each of the first line segments may represent a corresponding link of the first instrument. Similarly, a kinematic chain of a second instrument that includes a second set of links may be represented by a set of second line segments. The line segment representations are regularly updated to reflect the current kinematic configuration of the corresponding structure, which enables the use of these line segment representations to detect collision conditions, e.g., by determining a closest distance between the line segment representations followed by determining whether a collision condition is present or absent, based on the closest distance. Detecting collision conditions using line segmentation representations as opposed to using volume models enables execution of the updating of the line segment representations followed by the collision detection in an optimized manner that reduces computational requirements and makes the method particularly suitable for execution in real-time at high sampling rates. [0083] Turning to the flowchart 600, in Step 602, the set of first line segments that represent the current kinematic configuration of the first set of links associated with the first instrument are determined. The first line segments and the arrangement of the first line segments to form a kinematic chain may be specified in a database, where the lengths of the line segments and their ordered arrangement in the kinematic chain of the first instrument may be established. Further, the types of joints (e.g., revolute, prismatic joint including degree(s) of freedom) that link the line segments may be specified. This may enable an updating of the line representation by specifying joint angles and/or joint positions. Alternatively, the line representation may be updated by specifying the position and orientation of the first line segments in space, either in absolute coordinates, or relative to one another. The line representation of the first instrument may be updated to reflect the current kinematic configuration of the corresponding links of the first instrument using joint sensor data, commanded joint configurations, and/or joint configurations determined by algorithms such as inverse kinematics algorithms. Examples of line representations may be found in, for example, FIGs. 1, 3, and 5A-5F, with FIGs. 5A-5F illustrating an updating of line representations to reflect a current kinematic configuration of the corresponding structures. Although these line representations, for the sake of clear illustration, are in a 2D plane, line representations determined for the first line segments in Step 602 reflect the current kinematic configuration of the first instrument in 3D space.
[0084] In Step 604, the set of second line segments that represent the current kinematic configuration of the second set of links associated with the second instrument are determined. Step 604 may be performed analogous to Step 602.
[0085] In Step 606, select pairs of line segments are generated. Each select pair includes one first line segment of the set of first line segments, and one second line segment of the set of second line segments. The selection of line segments may be performed in various different manners, as subsequently discussed in reference to FIG. 7. FIG. 7 shows a sample configuration 700 of two instruments 504, 510 with corresponding end effectors 506, 512. The instruments 504 and 510 are supported by a repositionable assembly such as ones described with respect to the prior figures. In particular, the instruments 504 and 510 are coupled to or disposed on respective distal repositionable structures (e.g., a first manipulator and a second manipulator) which are in turn coupled to a manipulator-supporting link 502 of a proximal repositionable structure. The configuration shown in FIG. 7 corresponds to the configuration previously introduced in FIG. 5C. To facilitate the discussion of the selection of line segments, the line segments in FIG. 7 are labeled. The line segments associated and the first instrument 504 are labeled 1A (representing a first instrument shaft of the first instrument), IB (representing a shaft offset joint), optionally 1C (representing one of the two jaws of the end effector 506), and optionally ID (representing one of the two jaws of the end effector 506). The line segments associated with the second manipulator 552 and the second instrument 510 are labeled 2A (representing the second manipulator 552), 2B (representing one of the two jaws of the end effector 512), and 2C (representing one of the two jaws of the end effector 512).
[0086] In one implementation, pairs of line segments are generated by selecting combinations of one line segment associated with the kinematic chain of the first instrument and one line segment associated with the kinematic chain of the second instrument. In the example of FIG. 7, one line segment may be selected from line segments 1A, IB, 1C, and ID, and one line segment may be selected from line segments 2A, 2B, and 2C. Resulting pairs may include 1A/2A, 1A/2B, 1A/2C, 1B/2B, 1B/2C, etc.
[0087] In one implementation, select pairs of line segments are generated by selecting combinations of one line segment associated with the kinematic chain of the first instrument and one line segment associated with the kinematic chain of the second instrument. The select pairs of line segments exclude disregarded pairs of line segments. Disregarded pairs of line segments may be line segments for which it is known that the corresponding links are unable to collide. The inability to collide may be a result of the mechanical design and/or
current kinematic configuration(s) of the first instrument, the second instrument, and/or the manipulators supporting the first and second instruments, respectively. For example, the mechanical design of these components may be such that the instrument shafts and/or manipulators are maintained in parallel to one another by design. In these examples, the line segments that form a disregarded pair of line segments correspond to the most proximal links of the corresponding structures (i.e., instrument shafts and/or manipulators). Referring to FIG. 7, links of kinematic chains represented by line segments 1A and 2A (e.g., instrument shafts, portions of distal repositionable structures, portions of manipulators, etc.) are designed to be maintained in parallel, thereby eliminating the possibility of a collision. Accordingly, the pair 1A/2A is a disregarded pair of line segments. Referring to FIGs. 4A and 4B, the instrument shafts 410, 460, when the instruments 400, 450 are installed on corresponding manipulators 226, may be maintained in parallel. In this example, line segments that represent the instrument shafts 410, 460 would form a disregarded pair of line segments. Accordingly, the select pairs of line segments may include, for example, a first shaft line segment (e.g., 1A) paired with a second additional line segment of the set of second line segments representing a link of the at least one additional links of the second instrument (e.g., 2B, 2C). The select pairs of line segments may further include a second shaft line segment (e.g., 2A) paired with a first additional line segment of the set of first line segments representing a link of the at least one additional links of the first instrument (e.g., IB, 1C, ID). The select pairs of line segments may also include a second additional line segment paired with a first additional line segment (e.g., 1B/2B, 1B/2C, 1C/2B, 1C/2C, 1D/2B, 1D/2C). However, the select pairs exclude the disregarded pair 1A/2A. The inability to collide may further be a result of the commanded motion(s) (e.g., commanded by operator using the operator input system or commanded by an autonomous algorithm executed by the computer-assisted system) of the repositionable assembly and/or the instruments. The select pairs of line segments may be selected, generated, or determined based on one or more of: a type or model of the first instrument or a type or model of the second instrument, a current kinematic configuration of the repositionable assembly, a current kinematic configuration of the first instrument, a current kinematic configuration of the second instrument, a commanded motion of the repositionable assembly, a commanded motion of the first instrument, or a commanded motion of the second instrument. Furthermore, a disregarded pair of line segments comprising a first identified line segment representing a first link on the first instrument and a second identified line segment representing a second link on the second instrument may be identified to be disregarded for collision detection based on a position of
the first link on the first instrument and/or a position of the second link on the second instrument.
[0088] In one implementation, select pairs of line segments are generated by selecting combinations of one line segment associated with the kinematic chain of the first instrument and one line segment associated with the kinematic chain of the second instrument. The select pairs of line segments exclude disregarded pairs of line segments. Disregarded pairs of line segments may be line segments for which it is known that, in the current kinematic configuration of the first and second instruments, the corresponding links are mechanically unable to collide. Referring to FIG. 7, in the current kinematic configuration as shown, the links corresponding to line segments 2B and 2C are unable to collide with the link corresponding to line segment 1A, based on their spatial separation. Accordingly, 1A/2B and 1A/2C may be considered disregarded pairs. However, in an updated kinematic configuration (not shown) in which instrument 510 is retracted along its insertion axis, the links corresponding to line segments 2B and 2C may be able to collide with the link corresponding to line segment 1A. Accordingly, in the updated kinematic configuration, 1A/2B and 1A/2C may no longer be considered disregarded pairs. The determination as described may be made based on, for example, distance considerations in 3D space, which may be updated dynamically in real-time or near-real-time.
[0089] In some embodiments, combinations of different methods may be used to generate the pairs of line segments. For example, some combinations of line segments may always be considered (or disregarded) for the purpose of forming pairs of line segments, some combinations of line segments may be eliminated from consideration (by making them disregarded pairs of line segments) based on the mechanical design of the corresponding structure, and/or some combinations of line segments may be dynamically considered (or disregarded) for the purpose of forming pairs of line segments depending on the current kinematic configuration of the corresponding structure.
[0090] Regardless of the implementation used for generating select pairs of line segments, the result of these operations is a set (at least one) of select pairs of line segments, each representing links of a corresponding structure.
[0091] Subsequently, respective distances are determined between the select pairs of line segments. More specifically, for each of the select pairs of line segments, a minimum distance between the first line segment and the second line segments is determined. Standard methods such as the shortest distance between two skew lines in 3D space may be used to 1
determine the minimum distance between the first line segment and the second line segment of a select pair of line segments.
[0092] In Step 608, it is determined, based on the respective distances, whether a collision condition exists. The determination of a collision condition is intended to enable an anticipatory detection of a collision to enable or trigger performance of amelioration actions intended to prevent the collision from occurring and/or to mitigate the effects of the collision. In other words, the detection of an existence of a collision condition does not necessarily indicate that a collision has already occurred, but can also indicate that a collision is likely to occur. Determining the existence (or absence) of the collision condition may involve examination of the distances determined in Step 606 to identify a closest distance. The select pair of line segments associated with the closest distance represents those links that are closest to colliding, based on their distance. For a computer-assisted system operating in a three-dimensional space, the closest distance is identified in the three-dimensional space. A collision condition is determined to exist if the closest distance is below a distance threshold. In contrast, a collision condition is determined not to exist, if the closest distance is at or above the distance threshold. The distance threshold may be previously specified or may be dynamically determined. Furthermore, the distance thresholds for determining collision conditions may vary between pairs of links. The distance threshold may be set to accommodate a cross-sectional dimension (e.g., a diameter) of links. For example, a first link may have a first cross-sectional diameter, and a second link may have a second cross- sectional diameter. In this case, the distance threshold may be set to the radius of the first link plus the radius of the second link, plus an additional tolerance that may serve as a margin of error, to ensure a minimum gap between the links, etc. For non-circular cross sections of links, the widest cross-sectional dimensions of such links may be used for the purpose of determining a collision condition. A single distance threshold may be used. In this case, the largest diameters present among the links are used to set the distance threshold to ensure reliable detection of a collision condition. Alternatively, the distance threshold may be set in a link dependent manner. In this case, a database may store link diameter data. Depending on the pair of links being examined for the presence of a collision condition, the corresponding link diameter entries are retrieved from the database to calculate the applicable distance threshold. This approach may reduce the likeliness of false positive detections of collision conditions when smaller diameter links are examined for the presence of a collision, in Step 608. In case of a link with a non-constant cross-sectional diameter (e.g., a non- cylindrical link that is tapered from a proximal end of the link to a distal end of the link), the
link may be split into segments for which different diameters are used for the purpose of determining a collision condition. Thus, in the manner described above, a distance threshold for determining a collision condition between a pair of links (e.g., between a first link of a first instrument and a second link of a second instrument) may be determined based on one or more of: information associated with the first instrument (e.g., instrument type, instrument model, etc.), information associated with the second instrument (e.g., instrument type, instrument model, etc.), information associated with the first link (e.g., link identifier of the first link, information identifying which link the first link is on the first instrument, etc.), or information associated with the second link (e.g., link identifier of the second link, information identifying which link the first link is on the second instrument).
[0093] In Step 610, based on determining that a collision condition is present, the method may proceed with the execution of Step 612. Alternatively, based on determining that a collision condition is not present, the method may proceed with the execution of Step 602.
[0094] In Step 612, in response to determining the existence of the collision condition, an amelioration action is performed. One or more of a variety of amelioration actions may be performed. For example, the user of the computer-assisted system may be notified. The notification may be an auditory or visual alarm, a status message, a visual illustration of the collision condition, etc. In addition or alternatively, a collision-causing motion may be inhibited. Assume, for example, that the collision condition is detected as a result of a first instrument moving while a second instrument remains stationary. In this case, the continuation of the collision-causing motion of the first instrument may be inhibited or suppressed. The inhibition or suppression of further movement may be direction specific. For example, a continuation of the movement towards a collision may be prevented, while movement in other directions may still be allowed. For instance, in the case that a collision condition is determined during execution of a commanded motion (e.g., of the repositionable assembly, of the first instrument, and/or of the second instrument) in a first direction, the control system is configured to cease executing the commanded motion. In an example, while the collision condition exists, the control system is configured to reject additional commanded motions in the first direction but execute additional commanded motions in directions other than the first direction. In another example, while the collision condition exists, the control system is configured to reject additional commanded motions in the first direction but execute additional commanded motions in a second direction that is opposite of the first direction. In yet another example, while the collision condition exists, the control
system is configured to reject additional commanded motions that would result in a reduction of the minimum distance between the first plurality of line segments representing the first instrument and the second plurality of line segments representing the second instrument. Furthermore, haptic feedback may be provided to the user at the input device of the computer-assisted system. The haptic feedback may be selected to indicate the inhibiting of the continuation of the collision-causing motion to the user, i.e., to deter the user input that commands the collision-causing motion. For example, a virtual wall may be implemented by the haptic feedback, thereby preventing or at least inhibiting the user from continuing commanding the collision-causing motion.
[0095] Although only a few example embodiments have been described in detail above, those skilled in the art will readily appreciate that many modifications are possible in the example embodiments without materially departing from this invention. Accordingly, all such modifications are intended to be included within the scope of this disclosure as defined in the following claims.
Claims
1. A computer-assisted system comprising: a repositionable assembly configured to support a first instrument and a second instrument, the first instrument comprising a first plurality of links and the second instrument comprising a second plurality of links; and a control system comprising at least one processor, the control system configured to detect a collision condition between the first instrument and the second instrument by: determining a plurality of first line segments representing a current kinematic configuration the first plurality of links; determining a plurality of second line segments representing a current kinematic configuration of the second plurality of links; determining respective distances between select pairs of line segments, each pair of the select pairs comprising a first line segment of the plurality of first line segments and a second line segment of the plurality of second line segments; and determining, based on the respective distances, whether the collision condition exists.
2. The computer-assisted system of claim 1, wherein the control system is further configured to, in response to determining that the collision condition exists: perform one or more amelioration actions.
3. The computer-assisted system of claim 2, wherein the one or more amelioration actions comprise: inhibiting continuation of a collision-causing motion.
4. The computer-assisted system of claim 3, further comprising an input device configured to receive user input, wherein: the one or more amelioration actions further comprise: providing haptic feedback at the input device, the haptic feedback indicating the inhibiting of the continuation of the collision-causing motion; or inhibiting continuation of a collision-causing motion comprises providing haptic feedback to deter user input that commands the collision-causing motion.
5. The computer-assisted system of claim 1, wherein the control system is further configured to: determine a commanded motion of at least one selected from a group consisting of: (i) the repositionable assembly, (ii) the first instrument, and (iii) the second instrument; determine that the collision conditions between the first instrument and the second instrument exists during execution of the commanded motion; and in response to determining that the collision condition exists, perform one or more amelioration actions comprising ceasing the execution of the commanded motion.
6. The computer-assisted system of claim 5, wherein the commanded motion is in a first direction and wherein the one or more amelioration actions further comprise: while the collision condition exists, rejecting a first additional commanded motion in the first direction and executing a second additional commanded motion in a second direction, the second direction being opposite of the first direction.
7. The computer-assisted system of claim 5, wherein the commanded motion is in a first direction and wherein the one or more amelioration actions further comprise: rejecting any additional commanded motions except those in a second direction opposite of the first direction.
8. The computer-assisted system of claim 1, wherein determining whether the collision condition exists comprises: determining a closest distance based on the respective distances; and comparing the closest distance to a distance threshold.
9. The computer-assisted system of claim 8, wherein the distance threshold is based on a cross-sectional dimension of the first plurality of links or the second plurality of links.
10. The computer-assisted system of claim 8, wherein the distance threshold is based on a first cross-sectional dimension of the first plurality of links, a second cross-sectional dimension of the second plurality of links, and an error margin.
11. The computer-assisted system of claim 1, wherein determining the respective distances between the select pairs of line segments comprises, for each pair of the select pairs: determining a minimum distance between the first line segment and the second line segment.
12. The computer-assisted system of claim 1, wherein the control system is further configured to detect the collision condition by: selecting the select pairs of line segments to exclude a disregarded pair of line segments, the disregarded pair comprising a first identified line segment of the plurality of first line segments and a second identified line segment of the plurality of second line segments.
13. The computer-assisted system of claim 12, wherein the disregarded pair of line segments comprising the first identified line segment and the second identified line segment is identified based on a position of a first link of the first plurality of links on the first instrument or a position of a second link of the second plurality of links on the second instrument.
14. The computer-assisted system of claim 12, wherein the first identified line segment represents a most proximal link of the first plurality of links.
15. The computer-assisted system of claim 12, wherein the second identified line segment represents a most proximal link of the second plurality of links.
16. The computer-assisted system of claim 12, wherein selecting the select pairs of line segments to exclude a disregarded pair of line segments comprises: selecting the disregarded pair based on the current kinematic configurations of the first and second instruments which prevent a collision between a first link of the first plurality of links and a second link of the second plurality of links.
17. The computer-assisted system of claim 1, wherein the control system is further configured to detect the collision condition by selecting the select pairs of line segments based on a type or model of the first instrument or a type or model of the second instrument.
18. The computer-assisted system of claim 1, wherein the control system is further configured to detect the collision condition by selecting the select pairs of the line segments based on one or more selected from a group consisting of: (i) a current kinematic configuration of the repositionable assembly, (ii) a current kinematic configuration of the first instrument, and (iii) a current kinematic configuration of the second instrument.
19. The computer-assisted system of claim 1, wherein the control system is further configured to detect the collision condition by selecting the select pairs of line segments based on one or more of: (i) a commanded motion of the repositionable assembly, (ii) a commanded motion of the first instrument, or (iii) a commanded motion of the second instrument.
20. The computer-assisted system of any of claims 1 to 19, wherein the control system is further configured to: determine a first commanded motion of the repositionable assembly that, when performed, causes motion of the second instrument, determine a second commanded motion of the first instrument for effecting a first relative motion of a first end effector of the first instrument, wherein when the second commanded motion is performed in conjunction with the first commanded motion, the first end effector remains stationary relative to a workspace, and wherein the first plurality of links comprises: a first instrument shaft of the first instrument, wherein the second plurality of links comprises: a second instrument shaft of the second instrument, and wherein the select pairs of line segments exclude a disregarded pair of line segments formed of: a first shaft line segment of the plurality of first line segments representing the first instrument shaft, and a second shaft line segment of the plurality of second line segments representing the second instrument shaft.
21. The computer-assisted system of claim 20, wherein the first commanded motion comprises a rotation about an insertion axis of the repositionable assembly.
22. The computer-assisted system of claim 20, wherein the select pairs of line segments comprise at least one pair of line segments selected from a group consisting of: the first shaft line segment and a second additional line segment of the plurality of second line segments, the second additional line segment representing a second additional link distal to the second instrument shaft; the second shaft line segment and a first additional line segment of the plurality of first line segments, the first additional line segment representing a first additional link distal to the first instrument shaft; and the second additional line segment and the first additional line segment.
23. The computer-assisted system of claim 20, wherein the control system is further configured to, in response to determining that the collision condition exists, perform one or more amelioration actions comprising: ceasing execution of the first commanded motion of the repositionable assembly.
24. The computer-assisted system of claim 23, wherein the one or more amelioration actions further comprise ceasing execution of the second commanded motion of the first instrument.
25. The computer-assisted system of claim 23, wherein the first commanded motion is in a first direction and wherein the one or more amelioration actions further comprise, while the collision condition exists, rejecting a first additional commanded motion of the repositionable assembly in the first direction.
26. The computer-assisted system of claim 25, wherein the one or more amelioration actions further comprise, while the collision condition exists, executing a second additional commanded motion of the repositionable assembly in a second direction, the second direction being opposite of the first direction.
27. The computer-assisted system of claim 23, wherein the first commanded motion is in a first direction and wherein the one or more amelioration actions further comprise, while the collision condition exists, rejecting any additional commanded motions of repositionable assembly except those in a second direction opposite of the first direction.
28. The computer-assisted system of any of claims 1 to 19, wherein the control system is further configured to: determine a first commanded motion of the repositionable assembly that, when performed, causes motion of a third instrument, determine a second commanded motion of the first instrument for effecting a first relative motion of a first end effector of the first instrument, wherein when the second commanded motion is performed in conjunction with the first commanded motion, the first end effector remains stationary relative to a workspace, and determine a third commanded motion of the second instrument for effecting a second relative motion of a second end effector of the second instrument, wherein when the third commanded motion is performed in conjunction with the first commanded motion, the second end effector remains stationary relative to the workspace.
29. The computer-assisted system of any of claims 1 to 19, wherein the repositionable assembly is further configured to support an imaging instrument having a field of view; wherein the control system is further configured to: determine a first commanded motion of the repositionable assembly about an insertion axis to cause a rotation of the field of view of the imaging instrument, determine a second commanded motion of the repositionable assembly or the first instrument, wherein when the second commanded motion is performed in conjunction with the first commanded motion, a first end effector of the first instrument remains stationary relative to a workspace, execute the first commanded motion in conjunction with the second commanded motion, and while executing the first commanded motion in conjunction with the second commanded motion, detect the collision condition.
30. The computer-assisted system of claim 29, wherein the control system is further configured to, in response to determining that the collision condition exists, cease the first commanded motion of the repositionable assembly.
31. The computer-assisted system of claim 29, wherein the first commanded motion of the repositionable assembly is a rotation in a first direction about an insertion axis of the repositionable assembly and wherein the control system is further configured to, while the collision condition exists, reject any additional commanded motion of the repositionable assembly except those commanding a rotation in a second direction about the insertion axis, the second direction being opposite of the first direction.
32. A method for inter-instrument collision avoidance in a computer-assisted system, the computer-assisted system comprising: a repositionable assembly configured to support a first instrument and a second instrument, the first instrument comprising a first plurality of links and the second instrument comprising a second plurality of links; and wherein the method comprises detecting a collision condition between the first instrument and the second instrument by: determining a plurality of first line segments representing a current kinematic configuration the first plurality of links; determining a plurality of second line segments representing a current kinematic configuration of the second plurality of links; determining respective distances between select pairs of line segments, each pair of the select pairs comprising a first line segment of the plurality of first line segments and a second line segment of the plurality of second line segments; and determining, based on the respective distances, whether the collision condition exists.
33. The method of claim 32, further comprising, in response to determining that the collision condition exists: performing one or more amelioration actions.
34. The method of claim 33, wherein the one or more amelioration actions comprise: inhibiting continuation of a collision-causing motion.
35. The method of claim 34, wherein the computer-assisted system further comprises an input device configured to receive user input, and wherein the one or more amelioration actions further comprise: providing haptic feedback at the input device, the haptic feedback indicating the inhibiting of the continuation of the collision-causing motion; or inhibiting continuation of a collision-causing motion comprises providing haptic feedback to deter user input that commands the collision-causing motion.
36. The method of claim 32, further comprising: determining a commanded motion of at least one selected from a group consisting of: (i) the repositionable assembly, (ii) the first instrument, and (iii) the second instrument;
determining that the collision conditions between the first instrument and the second instrument exists during execution of the commanded motion; and in response to determining that the collision condition exists, performing one or more amelioration actions comprising ceasing the execution of the commanded motion.
37. The method of claim 36, wherein the commanded motion is in a first direction and wherein the one or more amelioration actions further comprise: while the collision condition exists, rejecting a first additional commanded motion in the first direction and executing a second additional commanded motion in a second direction, the second direction being opposite of the first direction.
38. The method of claim 36, wherein the commanded motion is in a first direction and wherein the one or more amelioration actions further comprise: rejecting any additional commanded motions except those in a second direction opposite of the first direction.
39. The method of claim 32, wherein determining whether the collision condition exists comprises: determining a closest distance based on the respective distances; and comparing the closest distance to a distance threshold.
40. The method of claim 39, wherein the distance threshold is based on a cross-sectional dimension of the first plurality of links or the second plurality of links.
41. The method of claim 39, wherein the distance threshold is based on a first cross-sectional dimension of the first plurality of links, a second cross-sectional dimension of the second plurality of links, and an error margin.
42. The method of claim 32, wherein determining the respective distances between the select pairs of line segments comprises, for each pair of the select pairs: determining a minimum distance between the first line segment and the second line segment.
43. The method of claim 32, wherein detecting the collision condition further comprises: selecting the select pairs of line segments to exclude a disregarded pair of line segments, the disregarded pair comprising a first identified line segment of the plurality of
first line segments and a second identified line segment of the plurality of second line segments.
44. The method of claim 43, wherein the disregarded pair of line segments comprising the first identified line segment and the second identified line segment is identified based on a position of a first link of the first plurality of links on the first instrument or a position of a second link of the second plurality of links on the second instrument.
45. The method of claim 43, wherein the first identified line segment represents a most proximal link of the first plurality of links.
46. The method of claim 43, wherein the second identified line segment represents a most proximal link of the second plurality of links.
47. The method of claim 43, wherein selecting the select pairs of line segments to exclude a disregarded pair of line segments comprises: selecting the disregarded pair based on the current kinematic configurations of the first and second instruments which prevent a collision between a first link of the first plurality of links and a second link of the second plurality of links.
48. The method of claim 32, wherein detecting the collision condition further comprises selecting the select pairs of line segments based on a type or model of the first instrument or a type or model of the second instrument.
49. The method of claim 32, wherein detecting the collision condition further comprises selecting the select pairs of the line segments based on one or more selected from a group consisting of: (i) a current kinematic configuration of the repositionable assembly, (ii) a current kinematic configuration of the first instrument, and (iii) a current kinematic configuration of the second instrument.
50. The method of claim 32, wherein detecting the collision condition further comprises selecting the select pairs of line segments based on one or more of: (i) a commanded motion of the repositionable assembly, (ii) a commanded motion of the first instrument, or (iii) a commanded motion of the second instrument.
51. The method of any of claims 32-50, further comprising:
determining a first commanded motion of the repositionable assembly that, when performed, causes motion of the second instrument, determining a second commanded motion of the first instrument for effecting a first relative motion of a first end effector of the first instrument, wherein when the second commanded motion is performed in conjunction with the first commanded motion, the first end effector remains stationary relative to a workspace, and wherein the first plurality of links comprises: a first instrument shaft of the first instrument, wherein the second plurality of links comprises: a second instrument shaft of the second instrument, and wherein the select pairs of line segments exclude a disregarded pair of line segments formed of: a first shaft line segment of the plurality of first line segments representing the first instrument shaft, and a second shaft line segment of the plurality of second line segments representing the second instrument shaft.
52. The method of claim 51, wherein the first commanded motion comprises a rotation about an insertion axis of the repositionable assembly.
53. The method of claim 51, wherein the select pairs of line segments comprise at least one pair of line segments selected from a group consisting of: the first shaft line segment and a second additional line segment of the plurality of second line segments, the second additional line segment representing a second additional link distal to the second instrument shaft; the second shaft line segment and a first additional line segment of the plurality of first line segments, the first additional line segment representing a first additional link distal to the first instrument shaft; and the second additional line segment and the first additional line segment.
54. The method of claim 51, further comprising, in response to determining that the collision condition exists, performing one or more amelioration actions comprising: ceasing execution of the first commanded motion of the repositionable assembly.
55. The method of claim 54, wherein the one or more amelioration actions further comprise ceasing execution of the second commanded motion of the first instrument.
56. The method of claim 54, wherein the first commanded motion is in a first direction and wherein the one or more amelioration actions further comprise, while the collision condition exists, rejecting a first additional commanded motion of the repositionable assembly in the first direction.
57. The method of claim 56, wherein the one or more amelioration actions further comprise, while the collision condition exists, executing a second additional commanded motion of the repositionable assembly in a second direction, the second direction being opposite of the first direction.
58. The method of claim 54, wherein the first commanded motion is in a first direction and wherein the one or more amelioration actions further comprise, while the collision condition exists, rejecting any additional commanded motions of repositionable assembly except those in a second direction opposite of the first direction.
59. The method of any of claims 32 to 50, further comprising: determining a first commanded motion of the repositionable assembly that, when performed, causes motion of a third instrument, determining a second commanded motion of the first instrument for effecting a first relative motion of a first end effector of the first instrument, wherein when the second commanded motion is performed in conjunction with the first commanded motion, the first end effector remains stationary relative to a workspace, and determining a third commanded motion of the second instrument for effecting a second relative motion of a second end effector of the second instrument, wherein when the third commanded motion is performed in conjunction with the first commanded motion, the second end effector remains stationary relative to the workspace.
60. The method of any of claims 32 to 50, wherein the repositionable assembly is further configured to support an imaging instrument having a field of view; and wherein the method further comprises: determining a first commanded motion of the repositionable assembly about an insertion axis to cause a rotation of the field of view of the imaging instrument, determine a second commanded motion of the repositionable assembly or the first instrument, wherein when the second commanded motion is performed
in conjunction with the first commanded motion, a first end effector of the first instrument remains stationary relative to a workspace, executing the first commanded motion in conjunction with the second commanded motion, and while executing the first commanded motion in conjunction with the second commanded motion, detecting the collision condition.
61. The method of claim 60, further comprising, in response to determining that the collision condition exists, ceasing the first commanded motion of the repositionable assembly.
62. The method of claim 60, wherein the first commanded motion of the repositionable assembly is a rotation in a first direction about an insertion axis of the repositionable assembly and wherein the method further comprises, while the collision condition exists, rejecting any additional commanded motion of the repositionable assembly except those commanding a rotation in a second direction about the insertion axis, the second direction being opposite of the first direction.
63. A non-transitory machine-readable medium comprising a plurality of machine-readable instructions executed by one or more processors associated with a computer-assisted system, the plurality of machine-readable instructions causing the one or more processors to perform the method of any of claims 32 to 62.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202463569418P | 2024-03-25 | 2024-03-25 | |
| US63/569,418 | 2024-03-25 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2025207554A1 true WO2025207554A1 (en) | 2025-10-02 |
Family
ID=95446479
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/US2025/021225 Pending WO2025207554A1 (en) | 2024-03-25 | 2025-03-25 | Inter-instrument collision avoidance for computer-assisted systems |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2025207554A1 (en) |
Citations (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6394998B1 (en) | 1999-01-22 | 2002-05-28 | Intuitive Surgical, Inc. | Surgical tools for use in minimally invasive telesurgical applications |
| US6699177B1 (en) | 1996-02-20 | 2004-03-02 | Computer Motion, Inc. | Method and apparatus for performing minimally invasive surgical procedures |
| US20170000577A1 (en) * | 2012-08-03 | 2017-01-05 | Stryker Corporation | Manipulator and Method for Controlling the Manipulator Based on Joint Limits |
| US20190262085A1 (en) * | 2014-09-17 | 2019-08-29 | Intuitive Surgical Operations, Inc. | Systems and methods for utilizing augmented jacobian to control manipulator joint movement |
| WO2022046787A1 (en) | 2020-08-28 | 2022-03-03 | Intuitive Surgical Operations, Inc. | Method and system for coordinated multiple-tool movement using a drivable assembly |
-
2025
- 2025-03-25 WO PCT/US2025/021225 patent/WO2025207554A1/en active Pending
Patent Citations (6)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US6699177B1 (en) | 1996-02-20 | 2004-03-02 | Computer Motion, Inc. | Method and apparatus for performing minimally invasive surgical procedures |
| US6394998B1 (en) | 1999-01-22 | 2002-05-28 | Intuitive Surgical, Inc. | Surgical tools for use in minimally invasive telesurgical applications |
| US20170000577A1 (en) * | 2012-08-03 | 2017-01-05 | Stryker Corporation | Manipulator and Method for Controlling the Manipulator Based on Joint Limits |
| US20190262085A1 (en) * | 2014-09-17 | 2019-08-29 | Intuitive Surgical Operations, Inc. | Systems and methods for utilizing augmented jacobian to control manipulator joint movement |
| WO2022046787A1 (en) | 2020-08-28 | 2022-03-03 | Intuitive Surgical Operations, Inc. | Method and system for coordinated multiple-tool movement using a drivable assembly |
| US20230263585A1 (en) * | 2020-08-28 | 2023-08-24 | Intuitive Surgical Operations, Inc. | Method and system for coordinated multiple-tool movement using a drivable assembly |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US20240115335A1 (en) | Computer-assisted medical systems and methods | |
| JP6262216B2 (en) | System and method for avoiding collision between operating arms using null space | |
| CN104840253B (en) | Master control set and its control method for operating robot | |
| CN107595392B (en) | Avoidance of manipulator arm collisions with patient using null space | |
| US12396816B2 (en) | Method and system for coordinated multiple-tool movement using a drivable assembly | |
| CN113286682B (en) | Reducing energy accumulation in servo control | |
| EP3468498A1 (en) | Computer-assisted tele-operated surgery systems and methods | |
| KR20240144439A (en) | System and method for rapid halt and recovery of motion deviations in medical device repositionable arms | |
| JP2023539864A (en) | Control of endoscope by surgical robot | |
| JP2022545684A (en) | Movable display unit on track | |
| KR20250133695A (en) | External force adjustment for remote operation | |
| EP4205688A1 (en) | Surgical robot, graphical control device thereof and graphical display method therefor | |
| US12447618B2 (en) | Techniques for constraining motion of a drivable assembly | |
| Bihlmaier | Endoscope robots and automated camera guidance | |
| WO2023281333A1 (en) | Projection of user interface pose command to reduced degree of freedom space for a surgical robot | |
| WO2025207554A1 (en) | Inter-instrument collision avoidance for computer-assisted systems | |
| US20220331025A1 (en) | Method and system for controlling instrument grip behavior | |
| WO2024226849A1 (en) | Assisted positioning of a repositionable structure | |
| US20240189051A1 (en) | Controlled resistance in backdrivable joints | |
| Shin et al. | Design of a dexterous and compact laparoscopic assistant robot | |
| US20250302558A1 (en) | System and methods for improving teleoperational control | |
| WO2025024508A1 (en) | View-based motion of imaging instruments | |
| WO2025019679A1 (en) | Positioning an imaging device to view a portion of an instrument during insertion of the instrument | |
| WO2024086122A1 (en) | Controlling software remote centers of motion for computer-assisted systems subject to motion limits | |
| CN120981208A (en) | Robotic surgical manipulators that provide enhanced flexibility in the surgical workspace |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 25719502 Country of ref document: EP Kind code of ref document: A1 |