RU2841089C1 - Robot-assisted complex of minimally invasive abdominal surgery for automated execution of operations by methods of local destruction of liver growth - Google Patents

Abstract

FIELD: medicine.

SUBSTANCE: invention refers to medical equipment, namely to a robot-assisted system of minimally invasive abdominal surgery for automated execution of operations by methods of local destruction of liver new growths. Complex includes a surgical navigation system, an ultrasound apparatus, an electrosurgical system, a manipulator having six degrees of mobility, with a working member fixed on the mounting surface, having two additional degrees of mobility, including a transducer of the ultrasound apparatus and having the possibility of fixing a local destruction instrument. Complex has a surgical table rigidly connected by a bed to a manipulator, and a control and data processing server connected to a surgical navigation system, to a manipulator by means of a manipulator controller, to a working member and with an ultrasound apparatus and configured to output information to a graphical input/output device with a user interface, configured to perform a preliminary simulation of the flow of the operation with displaying voxel data of the working area, given target positions and trajectories of local destruction tools, and a virtual model of the manipulator, and with solution by means of server of control and data processing of inverse problem of kinematics by position taking into account limits of allowable values of generalized coordinates for all points of specified trajectories, selected with such discrete, in order to simulate the manipulator movement with the specified speed limitation in order to detect errors of the operation flow planning. Management server and data processing and graphic input/output device are made with possibility to select target area from voxel data obtained on the basis of computed, magnetic resonance or positron emission tomography, subject to local destruction, critical structures and the intended area of introducing the local destruction tool, with automatic division of the target area into clusters by the KMeans algorithm, by solving a multidimensional optimization problem for the identified clusters of target positions and trajectories of local destruction tools, which do not cross critical structures and local destruction tools.

EFFECT: higher quality of robot-assisted operations performed by methods of local destruction of hepatic growths, reduced injuries for the patient and loads for operating personnel.

5 cl, 14 dwg

Description

State of the art.

The use of robot-assisted systems is becoming a standard in modern surgery. They are already used to treat a wide range of diseases. In particular, to perform automated operations using local destruction methods on organs with complex anatomical architecture, such as the liver, modern robot-assisted systems require the combined use of such technologies as segmentation of volumetric data obtained using various medical imaging systems, stereotaxy, numerical optimization methods, preliminary simulation in a virtual environment, comparison of images of different modalities and tracking of pattern movements on a video sequence, which in turn will reduce the human factor, especially with multiple punctures on large neoplasms.

The following patents are known from scientific and technical literature and patent documentation: WO 2020034146 A1, RU 2758753 C1, RU 2720830 C1, US 8774901 B2, US 20220142702 A1, RU 2594100 C1, US 20120245914 A1, CN 103970988 B, CN 113456229 A, CN 103371870 A, JP 5933723 B2, US 8267927 B2, CN 110537960 A, US 20200367971 A1, FR 124072 A1. The patent review includes both robot-assisted systems and individual technologies that can be used in this area.

Patent WO 2020034146 A1 from February 2020, owned by Zhangjiagang Institute of Industrial Technologies Soochow University, describes a trans-abdominal minimally invasive surgical system. This system consists of a stand on which a control device and robotic manipulators are mounted. The key feature is the fact that the developers claim to use at least three manipulators simultaneously.

Patent RU 2758753 C1 of April 2019, owned by SHANKHAJ MAJKROPORT MEDBOT (GRUP) KO., LTD, describes a robotic surgical system comprising an actuator terminal that includes two instrument arms. The first manipulator and the surgical instrument mounted thereon have the ability to pull out tissues of internal organs, and the second manipulator and the second surgical instrument have the ability to perform a surgical operation on tissues and organs.

Patent RU 2720830 C1 from March 2020, owned by Assisted Surgical Technologies LTD, describes a robot-assisted surgical complex capable of performing both minimally invasive surgeries and simulating such surgeries in a virtual environment. One of the goals of developing this system was the possibility of laparoscopic surgeries in the copy control mode through a specialized console.

The above patents WO 2020034146 A1, RU 2758753 C1 and RU 2720830 C1 represent devices that may be suitable for operations in the abdominal cavity for performing operations using local destruction methods, however, first of all, it is the presence of copying control that is the main difference from the automated performance of operations. These systems also have a number of significant limitations for the purposes of automated performance of local destruction methods, since they are only general-purpose robotic systems and do not have such capabilities as, for example, tracking the pattern of a neoplasm on an ultrasound image to compensate for movement caused by the patient's breathing, which imposes an additional burden on the surgeon, as well as the lack of control of surgical actions in real time using ultrasound diagnostic technology that is safe for the patient and medical personnel.

Patent US 8774901 B2 of July 2014, owned by the Indian company Perfint Healthcare Private Limited, describes a method for positioning a biopsy needle based on previously obtained CT data. The claimed invention is a method for positioning medical instruments and reducing the number of punctures, which should reduce the time of surgical intervention and discomfort for the patient. The point of entry of the needle can be selected on CT images, and the target point is set similarly. The method is based on the controller calculating the motion vector from the entry point to the target point and transmitting the obtained data to the manipulator. The advantage of this system is the patient's breathing tracking system, however, the use of intraoperative CT is a significant disadvantage, since it exposes the patient to additional intraoperative radiation, which can be avoided by using ultrasound devices.

Patent US 20220142702 A1 from November 2021, owned by the University of Arkansas, describes a radiofrequency ablation system with the ability to compensate for breathing-induced motion. The system is a lightweight complex that is attached directly to the patient. The developers report that, depending on the patient, the median deviation of the needle from the center of the tumor can be from 0.64 to 1.22 mm, which can be considered a fairly low error, since during radiofrequency ablation, part of the healthy tissue is often burned to prevent the reappearance of the tumor.

Patent RU 2594100 C1 from May 2015 describes the creation of a fully automated technology for performing a puncture biopsy. In the proposed robotic complex, sections of a focal lesion obtained using computed tomography are sent to the control system of the complex, which carries out minimally invasive surgical intervention, while the computer system is used to mark the course and model the puncture biopsy, and the obtained CT sections of the focal lesion in a three-dimensional projection are subject to computer analysis for the purpose of automatically establishing optimal parameters. To carry out the corresponding surgical intervention, the tomograph scans it throughout its entire length and projects its course onto the computer system monitor.

The above-described group of patents US 8774901 B2, US 20220142702 A1, RU 2594100 C1 is distinguished by the feature of automated surgical operations and surgical interventions, however, none of them uses a wide range of additional technologies designed to improve the quality of operations and convenience for medical personnel, which can be used in this area. In particular, the use of multimodal medical imaging and real-time tumor displacement monitoring. Another disadvantage is the fact that the patient is placed in an intraoperative computed tomography device and, as a result, is exposed to additional radiation.

Also of interest is a group of patents that do not describe the complexes as a whole, but describe technologies that may be its constituent parts - they are presented below.

Patent US 20120245914 A1 from October 2010, owned by Siemens AG, describes a system for positioning a hollow needle in images of diagnostic tools such as computed tomography, magnetic resonance imaging and ultrasound diagnostics. The method involves positioning a hollow needle relative to bone tissue. The presence of skeletal bones in the image is a mandatory condition for the operation of this system, which is also its main limitation, since, for example, when using radiofrequency ablation of liver tumors, it is not always possible to select a shooting angle in which both the bone and the tumor would be located simultaneously, using a diagnostic method such as ultrasound examination.

Patent CN 113456229 A of March 2020, owned by Beijing Turing Minimally Invasive Medical Technology Co Ltd, describes a robotic system for abdominal surgery. The main idea is the use of voice control. The speech recognition device receives and processes the voice data of the operating personnel, the processing result of which is transmitted in the form of commands to the control controller of the robotic system. The system also assumes the presence of a visual navigation device, the essence of which is based on the recognition of the image obtained during laparoscopic surgery.

Patent CN 103371870 A, issued in July 2013 by Shenzhen Institute of Advanced Technology of CAS, describes a system for registering pre-operative data such as computed tomography or magnetic resonance imaging with real-time data obtained by an ultrasound machine and a video capture board. The superimposed images thus enable the surgeon to obtain a high-quality image of the area of interest during surgery.

Patent JP 5933723 B2 of June 15, 2016, owned by Philips NV, describes a device for planning ablation of liver tumors. The system consists of a computing device, a device with a graphical interface and a device for transmitting and displaying information. The tumor, previously determined by manual segmentation, is used to calculate the safe area by increasing the tumor size. The system allows for modeling the ablation contour for a certain electrode position.

US Patent 8267927 B2 of September 18, 2012, owned by Philips NV, describes a procedure for planning a target ablation volume and the position of ablation zones. A method for planning ablation zones is described, including determining the minimum required number of ablation zones in the form of ellipsoids using a knowledge base. A search for an electrode insertion point on the patient's skin using a beam marching algorithm is also described.

Patent CN 110537960 A, dated December 6, 2019, Shanghai United Imaging Healthcare Co Ltd, describes a method for determining a puncture trajectory for a robotic surgery system. The method includes the following steps: obtaining a medical image of a target structure, reconstructing a three-dimensional model of the target structure from the medical image. Constructing a plurality of virtual puncture paths passing through the target structure and a selected area of the body surface through which the instrument is to be inserted. The result is selecting at least one virtual puncture path satisfying constraints from a plurality of virtual paths, wherein the constraints include that the target puncture path does not pass through other anatomical structures, such as blood vessels and nerves.

Patent CN 103970988 B of April 2014, owned by Chinese PLA General Hospital, describes a method for constructing a path for a medical instrument for ablation. The procedure for planning the trajectory of needle insertion is important because it can reduce trauma to the patient's internal organs and, as a result, reduce the postoperative recovery time. Moreover, this task is complicated by the potential presence of large vessels or bone tissue on the shortest path. The proposed system allows you to build a path to the center of the tumor using virtual limiters, which will allow you to bypass large vessels and bones, removing this task from the medical staff. Thus, this system allows you to optimize preoperative planning, as well as improve the quality of the operation as a whole.

Patent US 20200367971 A1, dated November 26, 2020, Shanghai Jiaotong University, relates to computer technology for preoperative planning and describes a method for preoperative ablation planning based on multimodal data, freezing with liquid nitrogen followed by radiofrequency heating using a needle probe and a device for it, which can automatically provide local ablation planning. The invention includes a method for preoperative planning taking into account the involvement of critical anatomical structures and assessing the achievability of the tumor elimination condition by the ablation procedure.

The above-described patents JP 5933723 B2, US 8267927 B2, CN 110537960 A, US 20200367971 A1 describe various approaches to automating preoperative planning of local tumor destruction, however, a common drawback is the low level of automation of the planning process, since a number of stages are performed manually. Also, metrics that clearly determine the quality of the selected positions of local destruction zones and instrument entry points are not provided.

The closest analogue of the system proposed for patenting is patent FR 3124072 A1 dated December 23, 2022, owned by Quantum Surgical. This patent describes the Epione robotic surgical system designed for radiofrequency ablation of neoplasms not only of abdominal organs, but also of the lungs. The system includes a manipulator with a fixed ultrasound transducer and a guide for manual insertion of a medical instrument - an ablation electrode, a surgical navigation system that allows determining the position of the medical instrument, as well as a control unit that allows determining the pressure exerted by the ultrasound sensor on the patient's body. This system includes visualization of tomograms combined with an ultrasound image in the scanning plane of the transducer, obtained using various diagnostic methods - computed tomography and magnetic resonance imaging.

The conducted patent search showed that at the moment there are no robot-assisted complexes for automated performance of operations by local destruction methods, which would simultaneously include such capabilities as preliminary simulation based on algorithms for automated determination of the optimal position of the working area and optimal trajectories of movement of local destruction instruments taking into account the kinematic limitations of the manipulator, multimodal visualization of data at the stages of preliminary planning, intraoperative preparation and execution of the operation, robotization of the introduction of local destruction instruments using a specialized working element with at least two additional degrees of mobility, under the control of a surgical navigation system and algorithms of technical vision for determining and tracking the target area in ultrasound images.

Disclosure of invention.

The invention proposed for patenting is aimed at implementing the task of automated local destruction of neoplasms of the internal organs of the abdominal cavity. Figure 1 shows the general structure of the elements necessary for the functioning of the robot-assisted complex of minimally invasive abdominal surgery for automated performance of operations using local destruction methods. The robot-assisted operation is performed as a sequence of actions that can be divided into three stages: preoperative planning, intraoperative preparation and execution of the operation. The planning stage includes such steps as determining the optimal position of the medical instruments in the work area, determining the optimal position of the work area relative to the robot. The preparation stage includes determining the current position of the work area relative to the robot and verification of the operation plan. The execution stage includes the movement of the robot along the planned trajectories, the introduction of the medical instrument of local destruction (hereinafter the medical instrument) 105 and performing local destruction of the target area 102.

A robot-assisted minimally invasive abdominal surgery complex for automated performance of operations using local destruction methods includes a surgical navigation system 109, a local destruction apparatus 117, an ultrasound apparatus 114, a manipulator (an articulated robot for joint work with a person) having at least six degrees of mobility, hereinafter referred to as a manipulator 106 with a working element 107 fixed to a mounting surface having at least two additional degrees of mobility, including a transducer 601 of the ultrasound apparatus 114, and having the ability to fix a medical instrument 105, a surgical table 110 rigidly connected by a frame 115 to the manipulator 106, and a control and data processing server 111 connected to the surgical navigation system 109, the manipulator 106 via a manipulator controller 116, the working element 107 and with Ultrasound apparatus 114, and having the ability to input and output information to the graphical input-output device 113 with a user interface. Figure 7 shows a structural diagram of data exchange between devices within the complex. The general algorithm for performing the operation is presented in Figure 8. Individual components of the algorithm for performing the operation are presented in Figures 9, 10, 11, 12 and 13.

Preoperative planning stage

The first step of the preoperative planning stage is the selection of the target area 102 subjected to local destruction and the expected zone of introduction of the medical instrument. The target area 102 and the permissible zone for introduction 112 are selected from three-dimensional voxel data of the working area 104 obtained on the basis of computer, magnetic resonance or positron emission tomography 101. The task of the algorithm is to place the zones of local destruction so that the voxels of the target area 102 are completely covered, and healthy tissues are affected minimally, if possible. For this purpose, the target area 102 is divided into geometrically similar parts - clusters 201, by adjusting the number of zones of local destruction 202, their position and radii. The KMeans algorithm is used to automate the clustering process. Next, the radii of the local destruction zones 202 required to cover the cluster 201 are determined by calculating the distance to each voxel from the center of the cluster 204, and then selecting the voxel with the maximum distance. This distance is considered the minimum required radius 212. The operation is repeated for each cluster to obtain a set of local destruction zones 202 required to completely cover the target area 102, after which they are increased by a specified margin 211, forming an increased local destruction zone 203. Of the obtained radii, the maximum is selected to check whether it belongs to a specified range. If the radius is beyond a certain upper limit, the number of clusters is increased by 1; if it is beyond a certain lower limit, it is decreased by 1 until the number of clusters equals 1 or until the radius belongs to the range.

For the identified clusters, a position of the medical instrument 105 is found so that it does not touch critical structures 205 (such as bones or vessels). The task of the second step algorithm is to find the best position of the medical instrument 105 that satisfies the constraints defined by the given metrics, thus avoiding the intersection of critical structures 205 and other medical instruments 209.

Figure 2 shows the minimum distance 207 from the nearest voxel 213 of the critical structure 205 to the axis of the medical instrument 105, the angle a (206) between the line formed by the nearest voxel 213 of the critical structures 205 to the medical instrument 105 and the target position of the center of the cluster 204, and the line formed by the target position of the center of the cluster 204 and the entry point 208 on the allowable zone for entry 112. An undesirable entry point 214 on the allowable zone for entry 112 that intersects the critical structure 205 is also shown. To calculate the first metric of the distance from the axis of the trajectory of the medical instrument 105 to the voxel of the critical structure 205, the coordinates of three points that form a triangle in space are used: the coordinates of the point of the voxel of the critical structure 213, the entry point 208 of the medical instrument 105 and the point of the center 204 of the cluster of the target area 201. The height h (207) from the vertex B (213) to the line formed by the target position of the cluster center C (204) and the input point T (208) in the permissible zone for input 112 is considered the distance from the trajectory axis to the nearest voxel of the critical structure and is calculated using Heron's formula for calculating the area of a triangle by three sides, where p is the semiperimeter.

The calculation of the second metric of the cosine of the angle α (206) between the lines is based on the same points B (213), T (208) and C (204) and its value is the result of the scalar product of the normalized vectors CT and CB, which is the value of the cosine of the angle between the lines. The angle between the lines determines the intersection of the trajectory of the medical instrument 105 with the critical structures 205. The value of the metric is determined by the following formula for the scalar product of unit vectors:

To determine how well the position fits, the metric value is calculated for each voxel of the critical structures 205 for a given trajectory of the medical instrument 105. From the obtained values of the first metric, the voxel with the smallest value is selected, and the second metric with the largest value is selected by solving an optimization problem in which the value of the first objective function is maximized, and the value of the second is minimized. In this case, the voxels of the critical structures 205 are determined, which are located as close as possible to the trajectory of the medical instrument 105.

When installing several medical instruments, their intersection is taken into account simultaneously by solving a multidimensional optimization problem:

Let a medical instrument be defined by a segment (points T and C). In this case, any point P on the segment is located:

where α _i ∈ [0;1], i=0,…, k where k is the number of segments.

Thus, two medical instruments are selected and the distance between the points belonging to their segments is found using the following function:

where j=0,…, k, i≠j.

The minimum of the obtained function is determined using the Nelder-Mead simplex method, and the obtained distance is checked for intersection taking into account the diameter of the medical instrument 215 and the specified distance reserve 210. A point is found that belongs to the permissible zone for insertion 112 with the condition that the straight line passing through it and the center of the cluster of the target region 204 does not intersect critical structures 205 and other medical instruments 209.

As shown in figure 4, the obtained positions of the medical instruments are visualized in the virtual space of the user interface on the data management and processing server 111 together with the three-dimensional bill of materials data of the working area 103. This visualization allows displaying the relative position of the working area 103 and the medical instruments from the point of view of various angles and sections. In addition, the surgeon has the opportunity to correct the position of the medical instrument 105 on the visualization in the user interface.

The second step of the pre-operational planning stage is the determination of the optimal position of the working area 103 relative to the manipulator 106. The determination of the optimal position of the working area 103 is performed by searching for such a position of the voxel data of the working area 104 within the specified boundaries 402, at which the generalized coordinates of the virtual model of the manipulator 401 along the entire trajectory are closest to the centers of the permissible ranges of the generalized coordinates 304. The sum of the squares of the differences between the generalized coordinates and the centers of the ranges for all points of the trajectory is used as an estimate of the distance to the center of the permissible range.

To obtain F _∑ - the estimate of the total difference - the following steps are performed: for all points of the trajectory, the inverse kinematics problem is solved for the position taking into account the geometric characteristics of the working element 107 and the current checked position of the working area and the value of the generalized coordinates g _i 301 is determined. For each solution for all generalized coordinates, the discrepancy 302 between the center of the permissible range m _i 304 and the current value of the generalized coordinate g _i,j is estimated. The center of the permissible range m _i 304 is between the upper limit of the permissible range 303 and the lower limit of the permissible range 305.

Figure 3 shows an example of a graph of the change in the generalized coordinate from the trajectory point number t. The point number is on the abscissa axis 306, and the generalized coordinate value is on the ordinate axis 307. The figure shows the difference between the center of the permissible range 304 and the current value of the generalized coordinate 301.

The resulting discrepancy assessment values are used to find the position of the work area within the given boundaries for which the value of this assessment is minimal. This position is used as a recommendation at the next stage.

Stage of intraoperative preparation

After determining the optimal position, the working area 103 is positioned relative to the manipulator 106 within the first step of the preparation stage. The working area 103 is positioned in accordance with the recommended position of the bill of exchange data of the working area 104, but since the positioning is performed manually, its exact position is unknown. In order to determine the position of the working area 103, the surgical navigation system 109 is used. The process of determining the position of the working area 103 consists of two subprocesses: determining the position of the working area 103 relative to the local coordinate system of the surgical navigation system 109 and determining the position of the local coordinate system of the surgical navigation system 190 relative to the base of the manipulator 106.

The first subprocess is performed by aligning N points defined on the voxel data of the work area 104 with points defined relative to the local coordinate system of the surgical navigation system 109 on the work area 103. The first group of points (a) is defined by selecting characteristic elements on the voxel data 104, the second (b) is selected by the pointing tool of the surgical navigation system 108, whereby both groups of points point to the same anatomical markers.

The second subprocess is carried out using the working element 107 fixed on the manipulator 106. This working element 107 is calibrated in the surgical navigation system 109 in such a way that the central point of the working element 606 specified in the control system of the manipulator 106 coincides with the central point of this working element 606 in the surgical navigation system 109. The manipulator 106 moves the working element 107 to several (M) points in space selected in such a way that the points do not lie on one straight line. For these points, their position relative to the base of the manipulator 106 (the position of the central point of the working element 606) (c) and the position relative to the local coordinate system of the surgical navigation system 109 (d) are recorded.

For both subprocesses, matrices R_{a,b} and R_{c,d} and vectors v_{a,b} and v_{c,d} are found such that the values W_{a,b} and W_{c,d} are minimal:

The found matrix-vector pairs (R, v) make it possible to determine the position of the working area 103 relative to the base of the manipulator 106.

The next step of the preparation stage is to perform a preliminary simulation of the movement of the manipulator 106. For this purpose, the planned position of the medical instrument 105, the position of the working area 103 and the geometric characteristics of the working element 107 are loaded into the computer simulation system. In the computer simulation system of the operating room, the virtual model of the manipulator 401 performs the specified movements, taking into account the position of the central point of the working element 107. For this purpose, the inverse kinematics problem is solved for all trajectory points. The trajectory points are selected with such a discreteness as to simulate the movement of the manipulator 106 with a specified limitation on the speed of movement of the working element 107. The preliminary simulation allows identifying planning errors that can lead to a collision between the manipulator 106, the working element 107, the equipment and the working area 103, or reaching the limitations of the permissible values of the generalized coordinates by the manipulator 106.

The process of preliminary simulation is accompanied by multimodal visualization in the user interface of the graphical input-output device 113. Within the framework of this visualization, three-dimensional bill of materials data of the working area 104 and models of solid objects surfaces, such as the model of the manipulator 401, equipment or working element 107, are displayed. Such visualization provides the opportunity to display the course of the operation in the space of one three-dimensional scene, taking into account the mutual arrangement of the equipment, working element 107, manipulator 106, and the position of the working area 103.

If planning errors are detected at the preliminary simulation stage, then planning is performed again, taking into account the current position of the working area 103. If planning with the current position of the working area 103 is impossible, then the position of the working area 103 is changed under the surgeon's control. If no errors are detected, then the stage of performing the operation begins.

Operation execution stage

The first step of the execution stage is the movement of the manipulator 106 with the prepared working element 107 to a preliminary position lying on the approach trajectory of the medical instrument 105. This position is selected on the approach trajectory so that during the movement to it the manipulator 106 does not collide with the working area.

The next step is to move the manipulator 106 to the working position. Depending on the operation protocol, the manipulator 106 either moves itself to the target position or provides the surgeon with the possibility of manual movement.

Figure 6 shows a schematic representation of the working element 107 secured to the manipulator 106. In the case of manual movement, the following procedure is used: external forces and moments applied to the working element 107 held by the manipulator 106 are measured. The moment applied to the axis of the second degree of freedom 604 of the working element 107 is used to calculate the rotation of the working element about this axis. The forces applied to the working element 107 are used to calculate the movement. Based on these calculations, a new position of the working element 107 is determined. A check is made for reachability for this position, and if the position is reachable, movement is performed to it. This procedure is repeated with a given frequency until the condition for stopping the movement is met, for example, the target position is reached.

After reaching the working position, the step of guiding the medical instrument 105 begins. Guidance of the medical instrument 105 is performed in the scanning plane 602 of the ultrasound transducer 601. On the ultrasound image, a local destruction zone 202 is determined, which must be within the working zone of the working element 603. For the selected local destruction zone 202, the inverse kinematics problem is solved for the position of the working element 107. The solution to the inverse kinematics problem is based on the geometric characteristics of the working element 107, which has two degrees of mobility 605 and 604.

The obtained solution of the inverse kinematics problem for the working element 107 is used by the control and data processing server 111 to guide the working element 107 to the local destruction zone 202. The guided working element 107 begins the automated introduction of the medical instrument 105. Due to the design of the working element, the medical instrument 105 is introduced into the scanning plane of the transducer 602. Due to this, the medical instrument 105 is visible on the ultrasound image.

The ultrasound image determines the position of the tip of the medical instrument 501. During the movement of the medical instrument 105, the position of the tip shifts. Based on the data collected at different points in time about the position of the tip, a predicted trajectory of movement of the medical instrument 504 is constructed by extrapolation. As shown in Figure 5, the ultrasound image and the predicted trajectory of movement of the medical instrument 504 are displayed on the visualization in the user interface, together with data about the position of the critical structures 205.

The user interface used during the operation allows displaying various information, such as the position of the bill data of the working area 103 in the general scene 403 together with the virtual model of the manipulator 401, ultrasound data 404, characteristics of the movement of the manipulator 405, as well as such interface elements as visualization of the structure (scene) of the computer modeling system 408, the interface of the settings of the computer modeling system 407, elements of the control interface for performing an action within the framework of stages 409 and for switching stages 406.

After the medical instrument 105 reaches the planned local destruction zone 202, the introduction of the medical instrument 105 is stopped, its position is fixed. The medical instrument is released from the working element fastening, after which the manipulator moves the working element aside, leaving the medical instrument 105 in the local destruction zone 202 for subsequent action on the neoplasm using the local destruction apparatus 117.

The objective of the invention is robot-assisted performance of local destruction in minimally invasive abdominal surgery with automation of the stages of preliminary planning, intraoperative preparation, and also execution of the operation.

The technical result is an increase in the quality of robot-assisted operations performed using methods of local destruction of liver tumors, a decrease in trauma for the patient and the load on the operating personnel.

The stated task is solved, and the declared technical result is achieved through the synergistic application of technologies for preliminary simulation of a robot-assisted complex, automated determination of the optimal position of the patient and trajectories of movement of medical instruments, multimodal intraoperative visualization, robotization of movement of medical instruments, stereotactic navigation and algorithms of technical vision.

The claimed technical solution allows, unlike the closest analogue (prototype), to conduct a preliminary simulation of a robot-assisted complex based on digital algorithms for the automated determination of the optimal position of the working area and optimal trajectories of movement of medical instruments, taking into account the kinematic limitations of the manipulator, multimodal visualization of data at the stages of preliminary planning, intraoperative preparation and execution of the operation, robotization of the introduction of medical instruments using a specialized working element with at least two additional degrees of mobility, under the control of a surgical navigation system and algorithms of technical vision for determining and tracking the target area on the ultrasound image.

Brief description of the drawings.

Figure 1. General structure of the elements necessary for the functioning of the complex.

Figure 2. General diagram of the process of determining the optimal position of medical instruments.

Figure 3. Example of a graph of the change in the generalized coordinate from the trajectory point number.

Figure 4. Visualization of 3D billet data and surface models in a single space in the user interface.

Figure 5. Visualization of augmented ultrasound data in the user interface

Figure 6. Schematic representation of the working element fixed on the manipulator.

Figure 7. Structural diagram of data exchange between devices within the complex.

Figure 8. Algorithm for performing an operation using a robot-assisted minimally invasive abdominal surgery complex.

Figure 9. Algorithm for the procedure for determining the optimal position of the working element.

Figure 10. Algorithm of the procedure for determining the actual position of the working area.

Figure 11. Algorithm of the instrument guidance procedure.

Figure 12. Algorithm of the procedure for performing surgical intervention.

Figure 13. Algorithm for the procedure of manipulator movement along a trajectory in the joint operation mode.

Figure 14. General view of the robot-assisted complex for minimally invasive abdominal surgery for automated performance of operations using local destruction methods.

The positions shown in the images correspond to the following:

101 equipment for computed tomography, magnetic resonance imaging or positron emission tomography

102 target area

103 work area

104 bill of exchange data work area

105 medical instrument, medical instrument for performing surgical operations using local destruction methods

106 manipulator (articulated robot for joint work with a person) having at least six degrees of freedom

107 working body

108 surgical navigation system pointing instrument

109 surgical navigation system

110 surgical table

111 data management and processing server

112 permissible zone of insertion of a medical instrument

113 Graphics Input/Output Device

114 Ultrasound machine

115 frame

116 Manipulator Controller

117 local destruction apparatus

201 cluster of local destruction zone

202 local destruction zone

203 enlarged zone of local destruction

204 cluster center of local destruction zone

205 critical structures

206 angle between the line formed by the critical structure voxel closest to the medical instrument and the target position of the cluster center, and the line formed by the target position of the cluster center and the input point

207 minimum distance from the nearest voxel of the critical structure to the axis of the medical instrument

208 entry point in the permissible zone of insertion of a medical instrument

209 other medical instruments

210 specified distance reserve from medical instrument

211 specified distance reserve to increase the local destruction zone

212 minimum required radius of local destruction zone

213 closest voxel of critical structure to the axis of medical instrument

301 generalized coordinate value

302 discrepancy assessment

303 upper limit of the permissible range of the generalized coordinate value

304 center of the permissible range of the generalized coordinate value

305 lower limit of the permissible range of the generalized coordinate value

306 axis of trajectory point indices

307 axis of values of the i-th generalized coordinate

401 virtual model of manipulator

402 specified boundaries of possible location of the work area

403 general scene of computer simulation system

404 visualization of augmented ultrasound data in the user interface

405 visualization of manipulator motion characteristics

406 control interface elements for switching stages

407 computer simulation system settings interface

408 visualization of the scene structure of the computer modeling system

409 control interface elements for performing actions within stages

501 detected position of the tip of a medical instrument

502 scan plane visualization

503 Target area position detected

504 predicted trajectory of movement of a medical instrument

601 transducer, ultrasound system transducer

602 transducer scanning plane

603 working area of the working element

604 second degree of mobility of the working element

605 first degree of mobility of the working element

606 center point of the tool

Implementation of the invention.

To confirm the possibility of using and working capacity of the invention, it was implemented in the form of a test stand, a photograph of which is shown in Figure 14.

The following equipment was used as system elements:

• Manipulator KUKA MED 14 R820;

• Prototype of the working body;

• KUKA Sunrise manipulator control system;

• Philips Affinity 50 ultrasound machine;

• Stereotactic surgical navigation system Multitrack;

• Data management and processing server based on CPU IntelCore i7-9700K 3.70 GHz GPU Nvidia GTX2080TI;

• Local destruction apparatus - electrosurgical system RITA 1500X;

• Graphical input/output device with user interface - IIYAMA TH4265MIS-B1AG touchscreen monitor.

A specialized silicone phantom simulating the liver, ribs, fat layer and skin was used in the experiment. The phantom used is similar in radiolucency and similar in ultrasound characteristics to the corresponding human tissues. The phantom was equipped with a set of silicone inserts simulating tumors. A computed tomography scan of this phantom was performed on a Definition AS 128 tomograph (Siemens). An umbrella electrode of the RITA 1500X electrosurgical system was used as a medical instrument.

At the first stage of the experiment, in accordance with the steps presented in the section "Disclosure of the invention", the obtained data of computer tomography were marked, the target area was selected. The target area was automatically divided into clusters, after which, using the developed algorithm, the target positions of the instruments were found. For the selected target positions of the instrument, preliminary trajectories of the instrument input were constructed in such a way that the scanning plane of the ultrasound transducer fixed in the prototype of the working element passed between the ribs of the phantom.

The 3D research data and trajectories were loaded into the prepared software. The 3D research data were oriented in the 3D virtual space according to the expected location, after which the optimal position of the work area was searched for in the given range of possible positions.

After determining the optimal position of the work area, the phantom was positioned on the operating table together with the reference frame of the surgical navigation system in accordance with the determined optimal position of the work area. The position of the work area was determined relative to the local coordinate system of the navigation system. The position of the local coordinate system of the navigation system relative to the base of the manipulator was also determined.

Based on the obtained data, preliminary simulations of the manipulator movement were performed for all required trajectories, which did not reveal any obstacles to the execution of the movement. The movement commands were transmitted to the manipulator control system - the KUKA Sunrise controller, after which the manipulator moved the working element to a preliminary position, from which it was moved along the planned trajectory in the joint operation mode until the transducer contacted the phantom and stable ultrasound image quality was obtained. Then the movement of the manipulator in the joint operation mode was stopped, and the manipulator began to hold a fixed position providing a constant force.

The actual position of the target area was determined on the ultrasound image obtained by the video capture board. For certain coordinates of the target area, the inverse kinematics problem was solved for the position of the working element, after which the working element made the corresponding movement to reach the target area with the medical instrument. After reaching the target area, the medical instrument was moved to the open state and released from the working element, the manipulator was given a command to move the working element to the side.

Bibliographic data (list of sources).

Patents:

WO 2020034146 A1,

RU 2758753 C1,

RU 2720830 C1,

US 008774901 B2,

US 20220142702 A1,

RU 2594100 C1,

US 20120245914 A1,

CN 103970988 B,

CN 113456229 A,

CN 103371870 A,

JP 5933723 B2,

US 8267927 B2,

CN 110537960 A,

US 20200367971 A1,

FR 3124072 A1.

GOST R 60.0.0.4 - 2023 / ISO 8373:2021 Robots and Robotic Devices, Terms and Definitions.

Claims (5)

1. A robot-assisted minimally invasive abdominal surgery complex for automated performance of operations using methods of local destruction of liver neoplasms, comprising a surgical navigation system, an ultrasound machine, an electrosurgical system, a manipulator having at least six degrees of mobility with a working element fixed to a mounting surface, having at least two additional degrees of mobility, including a transducer of the ultrasound machine and having the ability to fix a local destruction instrument, a surgical table rigidly connected by a frame to the manipulator, and a control and data processing server connected to the surgical navigation system, to the manipulator via a manipulator controller, to the working element and to the ultrasound machine and configured to output information to a graphical input-output device with a user interface configured to perform a preliminary simulation of the course of the operation with the display of voxel data of the working area, specified target positions and trajectories of movement of local destruction, and a virtual model of the manipulator, and with the solution, by means of the control and data processing server, of the inverse problem of kinematics by position taking into account the restrictions of the admissible values of the generalized coordinates for all points of the specified trajectories selected with such a discreteness in order to simulate the movement of the manipulator with a specified speed limitation in order to identify errors in planning the course of the operation, wherein the control and data processing server and the graphical input-output device are configured to select, at the stage of preoperative planning, from voxel data obtained on the basis of computer, magnetic resonance or positron emission tomography, a target area subjected to local destruction, critical structures and the expected zone of introduction of the local destruction instrument, with automatic division of the target area into clusters by the KMeans algorithm, by solving the problem of multidimensional optimization for the identified clusters of target positions and trajectories of movement of the local destruction instruments that do not intersect the critical structures and local destruction instruments. 2. The robot-assisted complex according to claim 1, in which the data control and processing server and the graphical input-output device are configured to determine and display at the stage of preoperative planning in the user interface the position of the work area relative to the manipulator by searching for such a position of the voxel data of the work area within the specified boundaries, in which the generalized coordinates of the manipulator on all specified trajectories of movement of the instruments for local destruction are closest to the centers of the permissible ranges of the generalized coordinates, wherein the sum of the squares of the differences between the generalized coordinates and the centers of the ranges for all points of the trajectory is used as an estimate of the distance to the center of the permissible range, wherein the obtained values of the discrepancy estimate are used to find such a position of the work area within the specified boundaries for which the value of this estimate is minimal. 3. A robot-assisted complex according to paragraphs 1 and 2, in which the data control and processing server and the manipulator are designed with the possibility of measuring external forces and moments at the stage of executing an operation, applied to a working element fixed to the mounting surface of the manipulator, by means of force sensors built into the links of the manipulator, wherein the measured forces and moment applied to the axis of the second degree of mobility of the working element are designed with the possibility of being used for moving and rotating the working element by determining a new position and moving the working element after checking for reachability at a given frequency until the condition for stopping the movement is met. 4. A robot-assisted complex according to paragraphs 1-3, in which the data control and processing server, the manipulator and the working element are designed with the possibility, at the stage of executing the operation, of automated guidance and movement of the central point of the tool of the working element, coinciding with the position of the tip of the local destruction tool, in the scanning plane of the transducer to the center of the selected cluster of the local destruction zone by determining the target zone of local destruction in the ultrasound image, solving the inverse problem of the kinematics of the working element based on its geometric characteristics, and automated movement of the local destruction tool based on the solution of the inverse problem of the kinematics of the working element. 5. A robot-assisted complex according to paragraphs 1-4, in which the data control and processing server and the graphical input-output device are designed with the possibility of visualizing in the user interface the projection of the target zone of local destruction and the predicted trajectory of movement of the local destruction instrument superimposed in real time on the ultrasound images by extrapolation based on the data collected at moments in time about the position of the tip of the local destruction instrument in the scanning plane of the transducer.