[go: up one dir, main page]

WO2025027493A1 - Système et procédé pour procédé d'apprentissage automatique pour vérifier une distribution cohérente de lignes d'agrafes - Google Patents

Système et procédé pour procédé d'apprentissage automatique pour vérifier une distribution cohérente de lignes d'agrafes Download PDF

Info

Publication number
WO2025027493A1
WO2025027493A1 PCT/IB2024/057305 IB2024057305W WO2025027493A1 WO 2025027493 A1 WO2025027493 A1 WO 2025027493A1 IB 2024057305 W IB2024057305 W IB 2024057305W WO 2025027493 A1 WO2025027493 A1 WO 2025027493A1
Authority
WO
WIPO (PCT)
Prior art keywords
profile
staple line
machine learning
motor
stapling
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/IB2024/057305
Other languages
English (en)
Inventor
Binesh KUMAR
Phani K. Bidarahalli
Andrew M. Miesse
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Covidien LP
Original Assignee
Covidien LP
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Covidien LP filed Critical Covidien LP
Publication of WO2025027493A1 publication Critical patent/WO2025027493A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B17/068—Surgical staplers, e.g. containing multiple staples or clamps
    • A61B17/072—Surgical staplers, e.g. containing multiple staples or clamps for applying a row of staples in a single action, e.g. the staples being applied simultaneously
    • A61B17/07207—Surgical staplers, e.g. containing multiple staples or clamps for applying a row of staples in a single action, e.g. the staples being applied simultaneously the staples being applied sequentially
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B17/11—Surgical instruments, devices or methods for performing anastomosis; Buttons for anastomosis
    • A61B17/115—Staplers for performing anastomosis, e.g. in a single operation
    • A61B17/1155—Circular staplers comprising a plurality of staples
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
    • A61B90/36—Image-producing devices or illumination devices not otherwise provided for
    • A61B90/361—Image-producing devices, e.g. surgical cameras
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H20/00—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance
    • G16H20/40—ICT specially adapted for therapies or health-improving plans, e.g. for handling prescriptions, for steering therapy or for monitoring patient compliance relating to mechanical, radiation or invasive therapies, e.g. surgery, laser therapy, dialysis or acupuncture
    • G—PHYSICS
    • G16—INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR SPECIFIC APPLICATION FIELDS
    • G16H—HEALTHCARE INFORMATICS, i.e. INFORMATION AND COMMUNICATION TECHNOLOGY [ICT] SPECIALLY ADAPTED FOR THE HANDLING OR PROCESSING OF MEDICAL OR HEALTHCARE DATA
    • G16H40/00—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices
    • G16H40/60—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices
    • G16H40/63—ICT specially adapted for the management or administration of healthcare resources or facilities; ICT specially adapted for the management or operation of medical equipment or devices for the operation of medical equipment or devices for local operation
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B2017/00017—Electrical control of surgical instruments
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B2017/00017—Electrical control of surgical instruments
    • A61B2017/00199—Electrical control of surgical instruments with a console, e.g. a control panel with a display
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B2017/00367—Details of actuation of instruments, e.g. relations between pushing buttons, or the like, and activation of the tool, working tip, or the like
    • A61B2017/00398—Details of actuation of instruments, e.g. relations between pushing buttons, or the like, and activation of the tool, working tip, or the like using powered actuators, e.g. stepper motors, solenoids
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B17/00—Surgical instruments, devices or methods
    • A61B2017/00681—Aspects not otherwise provided for
    • A61B2017/00734—Aspects not otherwise provided for battery operated
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
    • A61B34/25—User interfaces for surgical systems

Definitions

  • the present disclosure relates to surgical devices. More specifically, the present disclosure relates to electromechanical surgical systems for performing surgical stapling procedures.
  • Surgical fastener devices for applying fasteners or staples to tissue include surgical staplers, which may be manual or motor-powered.
  • surgical staplers which may be manual or motor-powered.
  • powered surgical staplers such as linear or circular staplers, which are specifically designed to perform certain types of surgical procedures, including laparoscopic procedures that provide a real-time video of a surgical site through a laparoscope or laparoscopic camera.
  • Powered surgical staplers currently are not capable of verifying whether the staple line was properly delivered. This limits the staplers’ ability to provide feedback to the surgical staff to assure that a consistent staple line was delivered. Thus, to verify the staple line, the surgeon visually observes or performs a leak test which adds a burden to the surgical workflow. Thus, there is a need for powered surgical staplers capable of verifying whether the staple line was properly delivered.
  • the present disclosure provides a system and method for verifying staple line delivery using sensors in a powered surgical stapler by leveraging data analytics and machine learning.
  • the system provides built-in self-learning and improvement capabilities with the ability to receive feedback as well as collect and store data.
  • the system provides a distributed learning approach to comply with regional privacy laws based on data collected during stapling procedures.
  • the powered surgical stapler is configured to verify a consistent staple line delivery by matching sensor-provided data to pre-defined staple delivery profiles.
  • the powered surgical stapler runs a lightweight machine learning algorithm that uses a Siamese configured to identify and compare the staple delivery profile.
  • the classifier is based on selected features derived from multi-modal sensors, including motor current and strain gauge readings.
  • the powered surgical stapler is also configured to provide feedback (e.g., on a graphical user interface (GUI)) regarding the staple line delivery, informing the user of the quality of the staple line based on the classifier’s output.
  • GUI graphical user interface
  • This evaluation is used to create new staple delivery profiles and retrain the models locally within the powered stapler.
  • the delivery profile updates are also sent to a cloud-based engine and are averaged with other updates to improve the shared model.
  • the device manufacturer’s internal development efforts are included in shared model updates. Any patient data is removed from the delivery profile sent to the cloud by following privacy principles of data minimization and anonymization.
  • a powered surgical stapler includes a staple cartridge having a plurality of staples.
  • the stapler also includes a motor configured to eject the plurality of staples to form a staple line and a sensor configured to measure a property of the motor during ejection of the plurality of staples.
  • the stapler also includes a memory storing a first model profile and software instructions implementing a machine learning algorithm, and a controller configured to: generate a stapling profile based on the measured property of the motor and time; classify the stapling profile using the machine learning algorithm based on the first model profile to output a similarity score; and output a prompt indicating whether the staple line is properly formed based on the similarity score.
  • the machine learning algorithm may be a Siamese classifier having a pair of neural networks, each of which may be one of a recurrent neural network or a long short-term memory network.
  • the controller may be further configured to receive from a central server a second model profile updating the first model profile in the memory.
  • the prompt may request a visual inspection of the staple line.
  • the powered surgical stapler may include a display screen configured to display the prompt.
  • the display screen may output a graphical user interface configured to receive user input providing feedback regarding the visual inspection of the staple line.
  • the controller may also be configured to retrain the machine learning algorithm based on feedback from the visual inspection of the staple line.
  • the property of the motor may be one of current draw, force, or torque.
  • a surgical stapling system includes a powered surgical stapler having a staple cartridge with a plurality of staples.
  • the powered surgical stapler also includes a motor configured to eject the plurality of staples to form a staple line and a sensor configured to measure a property of the motor during ejection of the plurality of staples.
  • the stapler further includes a memory storing a first model profile and software instructions implementing a machine learning algorithm.
  • the stapler additionally includes a controller configured to: generate a stapling profile based on the measured property of the motor and time; classify the stapling profile using the machine learning algorithm based on the first model profile to output a similarity score; and output a prompt indicating whether the staple line is properly formed based on the similarity score.
  • the system further includes a central server configured to communicate with the powered surgical stapler to receive the first model profile and generate a second model profile based on the first model profile. The central server is also configured to transmit the second model profile to the powered surgical stapler to replace the first model profile in the memory.
  • the machine learning algorithm may be a Siamese classifier having a pair of neural networks, where each neural network may be one of a recurrent neural network or a long short-term memory network.
  • the controller may also be configured to receive from the central server the second model profile replacing the first model profile in the memory.
  • the prompt may request a visual inspection of the staple line.
  • the surgical stapling system may include a display screen configured to display the prompt.
  • the display screen may output a graphical user interface configured to receive user input providing feedback regarding the visual inspection of the staple line.
  • the controller may be additionally configured to retrain the machine learning algorithm based on feedback from the visual inspection of the staple line.
  • the surgical stapling system may also include a laparoscopic camera configured to capture a video of the staple line and a video processing device configured to process the video of the staple line and determine whether the staple line is properly formed.
  • a method of using a powered surgical stapler includes activating a motor to eject a plurality of staples from a staple cartridge to form a staple line and measuring, at a sensor coupled to the motor, a property of the motor.
  • the method also includes retrieving from memory a model profile and software instructions implementing a machine learning algorithm.
  • the method further includes generating a stapling profile based on the measured property of the motor and time and classifying the stapling profile using the machine learning algorithm based on the model profile to output a similarity score.
  • the method further includes outputting a prompt indicating whether the staple line is properly formed based on the similarity score.
  • the machine learning algorithm may be a Siamese classifier having a pair of neural networks, each of which is a recurrent neural network or a long short-term memory network.
  • FIG. 1 is a surgical system for use with powered surgical staplers according to an embodiment of the present disclosure
  • FIG. 2 is a perspective view of a powered linear stapler including a handle assembly, an adapter assembly, and an end effector, according to an embodiment of the present disclosure
  • FIG. 3 is a perspective view of a powered circular stapler including a handle assembly, an adapter assembly, and an end effector, according to an embodiment of the present disclosure
  • FIG. 4 is a schematic diagram of the handle assembly, the adapter assembly, and the end effector of FIG. 3;
  • FIG. 5 is a perspective, exploded view of a loading unit of FIG. 2, according to an embodiment of the present disclosure
  • FIG. 6 is a flow chart of a method for distributed machine learning to verify consistent staple line delivery according to an embodiment of the present disclosure
  • FIG. 7 is a flow chart of a method for training a machine learning algorithm according to an embodiment of the present disclosure.
  • FIG. 8 is a schematic diagram of model training between multiple providers and central servers according to an embodiment of the present disclosure.
  • distal refers to that portion of the surgical instrument, or component thereof, farther from the user
  • proximal refers to that portion of the surgical instrument, or component thereof, closer to the user.
  • FIG. 1 shows a surgical system 10 configured to communicate with one or more surgical devices and to output information pertaining to the devices on one or more displays 13.
  • the display 13 may be any suitable monitor, an augmented or virtual reality headset, a heads-up display, a projector, etc. In embodiments, the display 13 may also include touchscreens.
  • the system 10 also includes an interface device 14, which is configured to communicate with one or more powered surgical staplers, namely, a linear stapler 20 and a circular stapler 30.
  • the interface device 14 is further configured to receive device information from the stapler 20 and/or 30 and to process device information for display on one or more display 13.
  • the system 10 may also include a video processing device 16 configured to couple to one or more cameras, such as a laparoscopic camera 15 configured to couple to a laparoscope 17.
  • a light source 18 is coupled to the camera 15 and may include any suitable light sources, e.g., white light, near infrared, etc., having light emitting diodes, lamps, etc.
  • the video processing device 16 is configured to receive the image data signals, process the raw image data from the camera 15, and may generate blended white light, NIR images for recording and/or real-time display.
  • the video processing device 16 is also configured to blend images using various Al augmentations.
  • the video processing device 16 is configured to process video images from the camera 15 and includes a first processing unit configured to perform operations, calculations, and/or set of instructions described in the disclosure and may be a hardware processor, a field programmable gate array (FPGA), a digital signal processor (DSP), a central processing unit (CPU), a microprocessor, and combinations thereof.
  • a hardware processor e.g., a field programmable gate array (FPGA), a digital signal processor (DSP), a central processing unit (CPU), a microprocessor, and combinations thereof.
  • FPGA field programmable gate array
  • DSP digital signal processor
  • CPU central processing unit
  • microprocessor e.g., central processing unit
  • the video processing device 16 may also include a second processing device, which may be a graphics processing unit (GPU) or an FPGA, which is capable of more parallel executions than a CPU (e.g., first processing unit) due to a larger number of cores, e.g., thousands of compute unified device architecture (CUDA) cores, making it more suitable for processing images.
  • a second processing device which may be a graphics processing unit (GPU) or an FPGA, which is capable of more parallel executions than a CPU (e.g., first processing unit) due to a larger number of cores, e.g., thousands of compute unified device architecture (CUDA) cores, making it more suitable for processing images.
  • GPU graphics processing unit
  • FPGA field-programmable gate array
  • each of the staplers 20 and 30 may share a common power platform, i.e., a handle assembly 12 including one or more motors, a power source, a main controller, storage device, transmitter/receiver, etc.
  • the stapler 20 also includes a linear adapter 22 configured to connect the handle assembly 12 to a loading unit 24 including an end effector 26 having a first jaw having a staple cartridge and a second jaw having an anvil (FIG. 5).
  • the staple cartridge stores a plurality of staples that are ejected during firing of the stapler 20 to staple tissue.
  • the liner adapter 22 includes various mechanical linkages coupling the end effector 26 with the handle assembly 12 enabling actuation of the end effector 26 to perform various functions, e.g., clamp, staple, cut.
  • various functions e.g., clamp, staple, cut.
  • the stapler 30 also includes a circular adapter 32 configured to connect the handle assembly 12 to an end effector 36 having a reload 38 with a staple cartridge 31.
  • the end effector 36 also includes an anvil 34 that is movable relative to the reload 38.
  • the staple cartridge 31 stores a plurality of staples that are ejected during firing of the stapler 30 to staple tissue.
  • the circular adapter 32 includes various mechanical linkages coupling the end effector 36 with the handle assembly 12 enabling actuation of the end effector 36 to perform various functions, e.g., clamp, staple, cut.
  • U.S. Patent No. 11,045,199 filed on May 7, 2018, the entire contents of which being incorporated by reference herein.
  • the handle assembly 102 includes a main controller circuit board 142, a rechargeable battery 144 configured to supply power to any of the electrical components of handle assembly 102, and a plurality of motors, e.g., a first motor 152a, a second motor 152b, and a third motor 152c coupled to the battery 144.
  • the handle assembly 102 also includes a display 146.
  • the motors 152a, 152b, 152c may be coupled to any suitable power source configured to provide electrical energy to the motors 152a, 152b, 152c, such as an AC/DC transformer.
  • Each of the motors 152a, 152b, 152c is coupled to a motor controller 143, which controls the operation of the corresponding motors 152a, 152b, 152c including the flow of electrical energy from the battery 144 to the motors 152a, 152b, 152c.
  • a main controller 147 is configured to execute software instructions embodying algorithms, such as clamping, stapling, and cutting algorithms, which control operation of the handle assembly 102.
  • the motor controller 143 includes a plurality of sensors 160a ... 160n configured to measure operational states of the motors 152a, 152b, 152c and the battery 144.
  • the sensors 160a-n include a strain gauge 160b and may also include voltage sensors, current sensors, temperature sensors, telemetry sensors, optical sensors, and combinations thereof.
  • the strain gauge 160b may be disposed within the linear adapter 22 and/or the circular adapter 32.
  • the sensors 160a-160n may measure voltage, current, and other electrical properties of the electrical energy supplied by the battery 144.
  • the sensors 160a-160n may also measure angular velocity (e.g., rotational speed) as revolutions per minute (RPM), torque, force, angular position, temperature, current draw, and other operational properties of the motors 152a, 152b, 152c.
  • the sensor 160a also includes an encoder configured to count revolutions or other indicators of the motors 152a, 152b, 152c, which is then used by the main controller 147 to calculate linear movement of components movable by the motors 152a, 152b, 152c.
  • Angular velocity may be determined by measuring the rotation of the motors 152a, 152b, 152c or a drive shaft (not shown) coupled thereto and rotatable by the motors 152a, 152b, 152c.
  • the position of various axially movable drive shafts may also be determined by using various linear sensors disposed in or in proximity to the shafts or extrapolated from the RPM measurements.
  • torque may be calculated based on the regulated current draw of the motors 152a, 152b, 152c at a constant RPM.
  • the motor controller 143 and/or the main controller 147 may measure time and process the above-described values as a function of time, including integration and/or differentiation, e.g., to determine the rate of change in the measured values.
  • the main controller 147 is also configured to determine distance traveled of various components of the adapters 22 and/or 32 and/or the end effector by counting revolutions of the motors 152a, 152b, 152c.
  • the motor controller 143 is coupled to the main controller 147, which includes a plurality of inputs and outputs for interfacing with the motor controller 143.
  • the main controller 147 receives measured sensor signals from the motor controller 143 regarding operational status of the motors 152a, 152b, 152c and the battery 144 and, in turn, outputs control signals to the motor controller 143 to control the operation of the motors 152a, 152b, 152c based on the sensor readings and specific algorithm instructions.
  • the main controller 147 is also configured to accept a plurality of user inputs from a user interface (e.g., switches, buttons, touch screen, etc.) coupled to the main controller 147.
  • a user interface e.g., switches, buttons, touch screen, etc.
  • the main controller 147 is also coupled to a memory 141.
  • the memory 141 may include volatile (e.g., RAM) and non-volatile storage configured to store data, including software instructions for operating the handle assembly 102.
  • the main controller 147 is also coupled to the strain gauge 160b using a wired or a wireless connection and is configured to receive strain measurements from the strain gauge 160b which are used during operation of the handle assembly 102.
  • the handle assembly 102 includes a plurality of motors 152a, 152b, 152c each including a respective motor shaft (not explicitly shown) extending therefrom and configured to drive a respective transmission assembly. Rotation of the motor shafts by the respective motors functions to drive shafts and/or gear components of adapters 22 and/or 32 in order to perform the various operations of handle assembly 102.
  • motors 152a, 152b, 152c of handle assembly 102 are configured to drive shafts and/or gear components of adapter assemblies 22 and 32 in order to actuate the end effectors 26 and 36.
  • the handle assembly 102 also includes a communication interface 162 configured to connect to the interface device 14 using a wired (e.g., Firewire®, USB®, Serial RS232®, Serial RS485®, USART®, Ethernet®, etc.) or wireless (e.g., Bluetooth®, ANT3®, KNX®, ZWave®, X10® Wireless USB®, IrDA®, Nanonet®, Tiny OS®, ZigBee®, 802.11 IEEE, and other radio, infrared, UHF, VHF communications and the like) connection.
  • the interface device 14 is configured to store the data transmitted thereto by the staplers 20 and/or 30 as well as process and analyze the data.
  • the interface device 14 is also connected to other devices, such as the display 13.
  • the loading unit 24 includes a drive assembly 360 having a flexible drive shaft 364 having a distal end which is secured to a drive beam 365, and a proximal engagement section 368.
  • Engagement section 368 includes a stepped portion defining a shoulder 370.
  • a proximal end of engagement section 368 includes diametrically opposed inwardly extending fingers 372.
  • Fingers 372 engage a hollow drive member 374 to fixedly secure drive member 374 to the proximal end of shaft 364.
  • Drive member 374 defines a proximal porthole which receives a connection member (not shown) of adapter 22 when loading unit 24 is attached to distal coupling 230 of adapter 22.
  • Proximal body portion 302 of loading unit 24 includes a sheath or outer tube 301 enclosing an upper housing portion 301a and a lower housing portion 301b.
  • the housing portions 301a and 301b enclose an articulation link 366 having a hooked proximal end 366a which extends from a proximal end of loading unit 24.
  • Hooked proximal end 366a of articulation link 366 engages a coupling hook (not shown) of adapter 22 when loading unit 24 is secured to distal housing 232 of adapter 22.
  • articulation link 366 of loading unit 24 is advanced or retracted within loading unit 24 to pivot end effector 26 in relation to a distal end of proximal body portion 302.
  • Cartridge assembly 308 of end effector 26 includes a staple cartridge 305 supportable in carrier 316.
  • Staple cartridge 305 defines a central longitudinal slot 305a, and three linear rows of staple retention slots 305b positioned on each side of longitudinal slot 305a.
  • Each staple retention slot 305b receives a single staple 307 and a portion of a staple pusher 309.
  • drive assembly 360 abuts an actuation sled 350 and pushes actuation sled 350 through cartridge 305.
  • cam wedges of the actuation sled 350 sequentially engage staple pushers 309 to move staple pushers 309 vertically within staple retention slots 305b and sequentially eject a single staple 307 therefrom for formation against anvil plate 312.
  • the loading unit 24 may also include one or more mechanical lockout mechanisms, such as those described in commonly-owned U.S. Patent Nos. 5,071,052, 5,397,046, 5413,267, 5,415,335, 5,715,988, 5,718,359, or 6,109,500, the entire contents of all of which are incorporated by reference herein.
  • the powered surgical stapler 20 and/or 30 is configured to verify a consistent staple line delivery by matching a measured stapling profile based on sensor-provided data to pre-defined staple delivery profiles.
  • a lightweight machine learning algorithm is executed as software instructions by the controller 147. The algorithm is configured to identify and compare the staple delivery profiles using a Siamese classifier that runs on the controller 147.
  • the staple delivery profile is based on specifically selected features derived from multi-modal sensors, such as motor current and strain gauge readings. Feedback is sent to the user informing the quality of the staple line based on the classifier’s output. The user then visually verifies the staple formation and provides an evaluation through a GUI.
  • the video processing device 16 automatically verifies the staple formation based on image processing algorithms of the laparoscopic video capturing the staple formation. This evaluation is used to create new staple delivery profiles and retrain the models locally within the powered stapler 20 and/or 30. The delivery profile updates are sent to a cloud-based engine and are averaged with other updates to improve the shared model.
  • a flow chart of a method for verifying a consistent staple line delivery is shown.
  • the powered surgical stapler 20 and/or 30 is operated to eject staples.
  • a user actuates the motor 152a via a trigger or a button, which advances a staple pusher or a sled to eject staples from the staple cartridge 25 and/or 31 into the tissue.
  • the staples may be ejected into the tissue using any motor control algorithm which may control the staple ejection process based on one or more parameters such as current draw of the motor 152a, torque or force output by the motor 152a or applied to the tissue, position, which may be angular position of the motor 152a or linear displacement of linkages (e.g., stapler pusher or sled) advanced by the motor.
  • the sensors 160a-n record sensor data, e.g., torque, force, angular position, temperature, current draw, and other operational properties of the motor 152a, etc., to build and store a stapling profile in memory 141.
  • the stapling profile may include a plurality of sensor values along with a timestamp for each measured sensor value.
  • the stapling profile may include a time series of sensor values for each of the property being measured and recorded.
  • the controller 147 analyzes the stapling profile to evaluate the formed staple line. The evaluation may be done based on detecting certain thresholds, pattern detection, and/or comparison to a model stapling profile stored in memory 141. In embodiments, one or more model profiles may be stored in memory 141 and the controller 147 selects a specific one from the group of profiles.
  • the controller 147 is configured to execute a lightweight machine learning algorithm that identifies and compares the staple delivery profile with a Siamese classifier also executed by the controller 147.
  • the Siamese classifier is based on features derived from multimodal sensors, including motor current and strain gauge readings.
  • FIG. 7 shows a Siamese classifier, which includes a pair of Siamese (i.e., identical twin) neural networks 250a and 250b which may be recurrent neural networks (RNN) and/or long short-term memory (LSTM) networks.
  • the output of the Siamese neural networks is merged and provided to a feedforward neural network 251, which is configured to derive a similarity score 252 for stapling profile matching.
  • Siamese networks are extendable type of neural networks, which do not learn any classification rules, but distinguish representatives of one class from those of other classes by means of a distance function.
  • these networks deduce the features of two inputs to be compared in two identical, parallel processing lines, allowing the references to be constantly changed and extended if the type of the calculated features remains the same. Accordingly, the units in these networks share their weights.
  • the final derived similarity score is output, which estimates the match between the input vectors, i.e., the measured stapling profile 240a and a model stapling profile 240b.
  • the controller 147 compares the similarity score 252 to a threshold to the classify the stapling profile.
  • a high similarity score 252 is indicative of the match to the model profile and that the resulting staple line was formed properly.
  • a low similarity score 252 is indicative of a mismatch between the model profile and the stapling profile and that the staple line was possibly malformed.
  • Feedback is sent to the user informing the quality of the staple line based on the classifier’s output.
  • the user will visually verify the staple formation and provide an evaluation through a GUI or this can also be gathered from laparoscopic video via a computer vision algorithm.
  • the controller 147 outputs a prompt via the display 13 and/or 146, which is based on the analysis of the stapling profile. If the controller 147 classifies the stapling profile as being indicative of a properly formed staple line, the prompt may indicate that the stapling operation was successful. If the controller 147 classifies the stapling profile as being indicative of an improperly formed staple line, then the prompt indicates this is the case and that visual inspection of the staple line needs to be performed.
  • step 204 visual inspection is performed by observing the staple line captured by the laparoscopic camera 15. This may be done manually by the surgeon or via the video processing device 16 using computer vision software.
  • the video feed capturing the staple line is analyzed to determine whether the staple line is properly formed.
  • the computer vision algorithm may be an artificial intelligence or machine learning (AI/ML) algorithm.
  • the AI/ML algorithm may be based on statistical ML that is configured to a develop a statistical model and draw inferences therefrom. As more training data is provided, the system adjusts the statistical model and improves its ability to analyze or make predictions.
  • Suitable statistical ML models include, but are not limited to, linear regression, logistic regression, decision trees, random forest, Naive Bayes, ensemble methods, support vector machines, K-Nearest Neighbor, and the like.
  • the AL/ML algorithm may be a deep learning algorithm that incorporates neural networks in successive layers.
  • Suitable deep learning models include, but are not limited to, convolutional neural network, recurrent neural network, deep reinforcement network, deep belief network, transformer network, and the like.
  • the input provided to train the models may be previously collected data, including images of properly formed stapling lines.
  • This evaluation is used to create new staple delivery profile(s) and retrain the models locally within the powered stapler.
  • the controller 147 receives input from the user or from the video processing device 16 characterizing the staple line.
  • the input may include one or more scales (e.g., 1-5) describing a parameter of the staple line, e.g., linearity, number of malformed staples, etc.
  • the user may enter input via a GUI which may be shown on the display screen 13 and/or 146.
  • the controller 147 retrains the Siamese classifier based on the feedback regarding the staple line.
  • steps 200-206 are performed at specific providers (e.g., hospitals) 400a-c, which then share their specific classifier algorithms with a central server 402, where further refinement occurs.
  • the central server 402 forms part of the system 10 and may be a private cloud that is operated by a manufacturer of the system 10 and is also used to distribute updated classifier algorithm to the providers 400a-c.
  • the delivery profile updates are also sent to a cloud-based engine and are averaged with other updates to improve the shared model.
  • the cloud-based engine cleans the delivery profile sent to the cloud of any patient data by following privacy principles of data minimization and anonymization that are based on the jurisdiction where the providers 400a-c are located as different jurisdictions have different patient and data privacy laws and regulations.
  • the overall process complies with the total product life cycle approach with privacy requirements as detailed in FDA’s good machine learning practice framework.
  • the stapler 20 and/or 30 and/or the interface device 14 communicates with the central server 402 and provides firing profile updates based on one or more firings of the stapler 20 and/or 30 as well as feedback during visual verification of the stapler line and any localized training of the classifier algorithm performed by the controller 147.
  • the device manufacturer's internal development efforts are included in shared model updates.
  • the central server 402 also receives internal firing development data at step 210.
  • the central server 402 includes the cloud-based engine, which aggregates the data and performs the calculations for obtaining model averages for profile updates.
  • This data may be obtained by performing stapling operations and stapling profile generation in a controlled setting (e.g., laboratory) on known tissues.
  • a controlled setting e.g., laboratory
  • the field and laboratory data are combined to fine tune the model and data aggregation to generate a refined model profile.
  • the steps 208-212 are segregated for each of the model profiles.
  • the cloud-based engine of the central server 402 validates the profile updates and distributes the updated profile to the staplers 20 and/or 30 and/or the interface device 14, which then perform a software or firmware update to update the classifier to include the updated model profile.
  • the described techniques may be implemented in hardware, software, firmware, or any combination thereof. If implemented in software, the functions may be stored as one or more instructions or code on a computer-readable medium and executed by a hardware-based processing unit.
  • Computer-readable media may include non- transitory computer-readable media, which corresponds to a tangible medium such as data storage media (e.g., RAM, ROM, EEPROM, flash memory, or any other medium that can be used to store desired program code in the form of instructions or data structures and that can be accessed by a computer).
  • processors such as one or more digital signal processors (DSPs), general purpose microprocessors, application specific integrated circuits (ASICs), field programmable logic arrays (FPGAs), or other equivalent integrated or discrete logic circuitry.
  • DSPs digital signal processors
  • ASICs application specific integrated circuits
  • FPGAs field programmable logic arrays
  • processors may refer to any of the foregoing structure or any other physical structure suitable for implementation of the described techniques. Also, the techniques could be fully implemented in one or more circuits or logic elements.

Landscapes

  • Health & Medical Sciences (AREA)
  • Engineering & Computer Science (AREA)
  • Surgery (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Public Health (AREA)
  • Biomedical Technology (AREA)
  • General Health & Medical Sciences (AREA)
  • Medical Informatics (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Animal Behavior & Ethology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Molecular Biology (AREA)
  • Veterinary Medicine (AREA)
  • Primary Health Care (AREA)
  • Epidemiology (AREA)
  • General Business, Economics & Management (AREA)
  • Business, Economics & Management (AREA)
  • Oral & Maxillofacial Surgery (AREA)
  • Pathology (AREA)
  • Urology & Nephrology (AREA)
  • Surgical Instruments (AREA)

Abstract

L'invention concerne une agrafeuse chirurgicale motorisée comprenant une cartouche d'agrafes comportant une pluralité d'agrafes. L'agrafeuse comprend également un moteur conçu pour éjecter la pluralité d'agrafes afin de former une ligne d'agrafes et un capteur conçu pour mesurer une propriété du moteur pendant l'éjection de la pluralité d'agrafes. L'agrafeuse comprend également une mémoire stockant un profil de modèle et des instructions logicielles mettant en oeuvre un algorithme d'apprentissage automatique et un dispositif de commande conçu pour générer un profil d'agrafage sur la base de la ou des propriétés mesurées du moteur et du temps, classifier le profil d'agrafage à l'aide de l'algorithme d'apprentissage automatique sur la base du profil de modèle pour délivrer un score de similarité, et délivrer en sortie une invite indiquant si la ligne d'agrafes est correctement formée sur la base du score de similarité.
PCT/IB2024/057305 2023-08-01 2024-07-29 Système et procédé pour procédé d'apprentissage automatique pour vérifier une distribution cohérente de lignes d'agrafes Pending WO2025027493A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
US202363516886P 2023-08-01 2023-08-01
US63/516,886 2023-08-01

Publications (1)

Publication Number Publication Date
WO2025027493A1 true WO2025027493A1 (fr) 2025-02-06

Family

ID=92543380

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/IB2024/057305 Pending WO2025027493A1 (fr) 2023-08-01 2024-07-29 Système et procédé pour procédé d'apprentissage automatique pour vérifier une distribution cohérente de lignes d'agrafes

Country Status (1)

Country Link
WO (1) WO2025027493A1 (fr)

Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5071052A (en) 1988-09-22 1991-12-10 United States Surgical Corporation Surgical fastening apparatus with activation lockout
US5397046A (en) 1991-10-18 1995-03-14 United States Surgical Corporation Lockout mechanism for surgical apparatus
US5413267A (en) 1991-05-14 1995-05-09 United States Surgical Corporation Surgical stapler with spent cartridge sensing and lockout means
US5415335A (en) 1994-04-07 1995-05-16 Ethicon Endo-Surgery Surgical stapler cartridge containing lockout mechanism
US5715988A (en) 1995-08-14 1998-02-10 United States Surgical Corporation Surgical stapler with lockout mechanism
US5718359A (en) 1995-08-14 1998-02-17 United States Of America Surgical Corporation Surgical stapler with lockout mechanism
US6109500A (en) 1996-10-04 2000-08-29 United States Surgical Corporation Lockout mechanism for a surgical stapler
US9839425B2 (en) 2014-06-26 2017-12-12 Covidien Lp Adapter assembly for interconnecting electromechanical surgical devices and surgical loading units, and surgical systems thereof
US20190125455A1 (en) * 2017-10-30 2019-05-02 Ethicon Llc Method of hub communication with surgical instrument systems
US20190206563A1 (en) * 2017-12-28 2019-07-04 Ethicon Llc Method for adaptive control schemes for surgical network control and interaction
US11045199B2 (en) 2017-06-09 2021-06-29 Covidien Lp Handheld electromechanical surgical system
WO2024123647A1 (fr) * 2022-12-04 2024-06-13 Genesis Medtech (USA) Inc. Instrument chirurgical intelligent avec une méthode de combinaison pour la détection et le marquage d'une ligne d'agrafage

Patent Citations (12)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US5071052A (en) 1988-09-22 1991-12-10 United States Surgical Corporation Surgical fastening apparatus with activation lockout
US5413267A (en) 1991-05-14 1995-05-09 United States Surgical Corporation Surgical stapler with spent cartridge sensing and lockout means
US5397046A (en) 1991-10-18 1995-03-14 United States Surgical Corporation Lockout mechanism for surgical apparatus
US5415335A (en) 1994-04-07 1995-05-16 Ethicon Endo-Surgery Surgical stapler cartridge containing lockout mechanism
US5715988A (en) 1995-08-14 1998-02-10 United States Surgical Corporation Surgical stapler with lockout mechanism
US5718359A (en) 1995-08-14 1998-02-17 United States Of America Surgical Corporation Surgical stapler with lockout mechanism
US6109500A (en) 1996-10-04 2000-08-29 United States Surgical Corporation Lockout mechanism for a surgical stapler
US9839425B2 (en) 2014-06-26 2017-12-12 Covidien Lp Adapter assembly for interconnecting electromechanical surgical devices and surgical loading units, and surgical systems thereof
US11045199B2 (en) 2017-06-09 2021-06-29 Covidien Lp Handheld electromechanical surgical system
US20190125455A1 (en) * 2017-10-30 2019-05-02 Ethicon Llc Method of hub communication with surgical instrument systems
US20190206563A1 (en) * 2017-12-28 2019-07-04 Ethicon Llc Method for adaptive control schemes for surgical network control and interaction
WO2024123647A1 (fr) * 2022-12-04 2024-06-13 Genesis Medtech (USA) Inc. Instrument chirurgical intelligent avec une méthode de combinaison pour la détection et le marquage d'une ligne d'agrafage

Non-Patent Citations (1)

* Cited by examiner, † Cited by third party
Title
ZHAO XUECHEN ET AL: "Cooperative Use of Recurrent Neural Network and Siamese Region Proposal Network for Robust Visual Tracking", IEEE ACCESS, IEEE, USA, vol. 9, 12 April 2021 (2021-04-12), pages 57704 - 57715, XP011850654, DOI: 10.1109/ACCESS.2021.3072778 *

Similar Documents

Publication Publication Date Title
US11672534B2 (en) Communication capability of a smart stapler
US11058505B2 (en) Machine-learning-based visual-haptic feedback system for robotic surgical platforms
US11510743B2 (en) Communication control for a surgeon controlled secondary display and primary display
US20220108789A1 (en) Cloud analytics packages
EP3505088B1 (fr) Réglage de la hauteur des agrafes d'au moins une rangée d'agrafes en fonction de l'épaisseur ou de la force de fermeture du tissu détecté
US11877792B2 (en) Smart energy combo control options
EP3505089B1 (fr) Dispositif d'agrafage motorisé configuré pour ajuster la force, la vitesse d'avancement et la course globale d'élément de coupe sur la base d'une détection de paramètre de mise à feu ou de serrage
CN116670777A (zh) 具有可变互连能力的外科集线器
CN120380549A (zh) 对等外科器械监测
WO2024141973A1 (fr) Plage de fonctionnement adaptable pour dispositif chirurgical
WO2025027493A1 (fr) Système et procédé pour procédé d'apprentissage automatique pour vérifier une distribution cohérente de lignes d'agrafes
CN120418887A (zh) 对仿冒或伪造外科装置的检测
US20230404691A1 (en) Autonomous Surgical System Instrument Actuation
CN119547148A (zh) 外科装置控制算法的自主适应
JP2025518524A (ja) 外科的自律性レベルを動的に決定すること
CN119547149A (zh) 检测与自主外科任务相关联的故障减轻
CN119522456A (zh) 自适应自主功能和系统互连
JP2025518528A (ja) 自律的な器具内外科用システム作動
CN119655874A (zh) 用于基于人工智能评估手术的方法及手术机器人系统

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24762028

Country of ref document: EP

Kind code of ref document: A1