CN105169570A - Imaging guided intelligentialized laser minimally invasive surgery system and control method thereof - Google Patents

Abstract

The invention discloses an image-guided intelligent laser minimally invasive surgery system and a control method thereof. The intelligent laser minimally invasive surgery system of the present invention includes: a mechanical arm, a scanner, a camera, a laser, a light guide arm, a movable bed for a patient, and an information processing and control terminal; the intelligent laser minimally invasive surgery system based on three-dimensional image information of the present invention , can accurately identify the pigmented lesion area, strictly control the laser dose, and irradiate the laser on the pigmented lesion area, making the minimally invasive laser surgery more efficient, accurate, and safe. After obtaining enough patient information, the system automatically completes the operation; it acts on the surface of the human body , can be applied to laser minimally invasive surgery for pigmented lesions, equipped with a visual servo system and a six-degree-of-freedom lightweight robotic arm, which is autonomous and fully automated; in addition, the intelligent minimally invasive surgery system has strong scalability and can Applied to a variety of minimally invasive surgery based on the body surface, such as high-intensity focused ultrasound HIFU skin lift and wrinkle removal surgery.

Description

An image-guided intelligent laser minimally invasive surgery system and its control method

technical field

The invention relates to the field of medical robots, in particular to an image-guided intelligent laser minimally invasive surgery system and a control method thereof.

Background technique

With the development of artificial intelligence, intelligent surgical robots have gradually become an international research hotspot. How to assist doctors to complete operations accurately, quickly and safely has become the core issue of medical robot research. Usually, open surgery is limited by the doctor's experience and skills, and the operation is uncertain. The high-precision operation of the surgical robot can greatly improve the efficiency of the operation, reduce the risk of the operation, avoid wound infection, and make the patient's wound heal faster.

At the beginning of 1996, ComputerMotion of the United States launched the powerful ZEUS robotic surgery system, which was used in minimally invasive surgery. pain. Each robotic arm of the slave operating system of the ZEUS system has 6+1 degrees of freedom, of which 6 are used for pose adjustment and the other is used for position optimization. Through the ZEUS system, doctors can manipulate the movements of the main hand in a comfortable working environment, and monitor the operation process in real time through the monitor; at the operation site, the slave hand faithfully simulates and scales the movements of the doctor's main hand to complete the operation .

In 2000, IntuitiveSurgical of the United States successfully developed the DaVinci system, and in 2001, it was certified by the US FDA and began to implement minimally invasive abdominal surgery. The DaVinci robotic arm-assisted minimally invasive surgery system consists of a main console and a surgical manipulator. The doctor can observe the three-dimensional image of the operation area through the binocular sight hole on the main console. There are three robotic arms on the driven operation vehicle of the system, and the middle arm is equipped with a visualization system composed of a binocular camera. Two independent visual signals are sent to the binocular viewing hole for synthesis to obtain the real surgical area. 3D image of .

The earliest robotic arm system for brain stereotaxic surgery jointly developed by Beijing University of Aeronautics and Astronautics and the Naval General Hospital has successfully performed 40 cases of frameless positioning brain surgery (including 10 cases of remote control operation) in clinical practice, and achieved satisfactory results. In 2010, the "Miaoshou A" system jointly developed by Tianjin University, Nankai University and Tianjin Medical University General Hospital was identified as the first surgical robot arm with independent intellectual property rights successfully developed in China. It is mainly used for abdominal minimally invasive surgery. Breakthroughs have been made in the key technologies of minimally invasive surgical robots such as robotic system mechanical design, master-slave control, stereoscopic images and system integration. However, existing robotic surgery systems are only applied to surgical operations, not to the surface of the human body; moreover, they do not have the ability to act autonomously and must be operated by doctors.

Contents of the invention

In view of the current situation that the existing robot system has not been used on the surface of the human body and has no ability to act autonomously, the present invention provides an intelligent minimally invasive laser surgery that acts on the surface of the human body and can be applied to laser minimally invasive surgery for pigmentation lesions. The system, equipped with a visual servo system and a six-degree-of-freedom lightweight robotic arm, is autonomous and fully automated.

An object of the present invention is to provide an image-guided intelligent laser minimally invasive surgery system.

The image-guided intelligent laser minimally invasive surgery system of the present invention includes: a mechanical arm, a scanner, a camera, a laser, a light guide arm, a movable bed for a patient, and an information processing and control terminal; wherein, the light guide arm is connected to the laser, and the laser The emitted laser light is guided out; the mechanical arm is connected to the information processing and control terminal; the mechanical arm is physically connected to the light guide arm through the end gripper to control the position and direction of the light guide arm, so as to control the output laser to align with the patient's movable bed. Pigment lesion area; the scanner and the camera are respectively fixed on the same rack to form a visual servo system. The terminal obtains the location information of the pigmented lesion area; the camera transmits the real-time image of the pigmented lesion area to the information processing and control terminal to obtain the color features of the pigmented lesion area; the information processing and control terminal automatically analyzes the geometric and color features of the pigmented lesion area , generate the motion path of the robotic arm, and control the movement of the robotic arm according to the motion path, so as to control the light guide arm to guide the laser to act on the pigmented lesion area for completely autonomous laser irradiation.

The mechanical arm adopts a six-degree-of-freedom mechanical arm.

The scanner is a three-dimensional scanner, which scans the pigmented lesion area, and then transmits the three-dimensional point cloud data to the information processing and control terminal through the data interface to obtain the three-dimensional information of the position.

Another object of the present invention is to provide a control method for an image-guided intelligent laser minimally invasive surgery system.

The control method of the intelligent laser minimally invasive surgery system under image guidance of the present invention comprises the following steps:

1) Jointly calibrate the scanner and the robotic arm so that they work in the same coordinate system;

2) Connect the robotic arm to the light guide arm, and correct the end of the robotic arm so that the laser focus is set as the end of the robotic arm;

3) Initialize the calibration camera and perform image calibration;

4) Expose the pigmented lesion area to the field of view of the scanner and camera, the scanner scans the pigmented lesion area, transmits the data to the information processing and control terminal, and obtains the location information of the pigmented lesion area;

5) The camera images the pigmented lesion area, transmits the real-time image of the pigmented lesion area to the information processing and control terminal, and obtains the color characteristics of the pigmented lesion area;

6) The information processing and control terminal automatically analyzes the geometric and color features of the pigmented lesion area, and extracts the three colors of red R, green G, and blue B in the pigmented lesion area by comparing with the data retrieval of the preset expert system. information;

7) The expert system in the information processing and control terminal determines the wavelength, intensity and irradiation time of the laser used for irradiation according to the extracted color feature information;

8) Accurately locate the three-dimensional coordinates (x, y, z) of each point in the pigmented lesion area according to the real-time image provided by the camera, and the information processing and control terminal pre-generates the motion path of the robotic arm according to the collected image;

9) The information processing and control terminal controls the movement of the robotic arm according to the motion path, so as to control the light guide arm to guide the laser irradiation on the pigmented lesion area, and perform completely autonomous laser irradiation until the robotic arm completes all operations according to the pre-generated motion path , the laser irradiation stops automatically.

Among them, in step 8), it further includes: according to the contact distance between the light guide arm and the pigmented lesion area, the position of the gripper and the light guide arm, correcting the movement path of the mechanical arm, adjusting the appropriate posture, so that the clamping The device can be equipped with a light guide arm for laser irradiation.

In step 9), autonomous laser irradiation specifically includes the following steps:

a) Start the irradiation program: turn on the laser, and the laser irradiates the pigmented lesion area through the light guide arm;

b) The scanner performs target position detection in real time, and the information processing and control terminal judges whether the position of the pigmented lesion area is displaced, if not, continue to perform laser irradiation, and if so, enter step c);

c) judging whether the displacement exceeds the limit value, if not, enter step d), if yes, stop the laser irradiation;

d) The information processing and control terminal performs position calibration to determine whether the position calibration is successful, if yes, continue the laser irradiation, and if not, stop the laser irradiation.

Advantages of the present invention:

The intelligent laser minimally invasive surgery system based on three-dimensional image information of the present invention can accurately identify the pigmented lesion area, strictly control the laser dose, and irradiate the laser on the pigmented lesion area, making the laser minimally invasive surgery more efficient, accurate and safe. After sufficient patient information, the system automatically completes the operation; it acts on the surface of the human body and can be applied to laser minimally invasive surgery for pigmented lesions. It is equipped with a visual servo system and a six-degree-of-freedom lightweight robotic arm, which is autonomous and fully automated. ; In addition, the intelligent minimally invasive surgery system is highly expandable and can be applied to a variety of minimally invasive surgery based on the body surface, such as high-intensity focused ultrasound HIFU skin-lifting and wrinkle removal surgery.

Description of drawings

Fig. 1 is the schematic diagram of an embodiment of the intelligentized laser minimally invasive surgery system under image guidance of the present invention;

Fig. 2 is the structural block diagram of the intelligentized laser minimally invasive surgery system under image guidance of the present invention;

3 is a flow chart of the control method of the image-guided intelligent laser minimally invasive surgery system of the present invention;

Fig. 4 is a flowchart of the autonomous laser irradiation of the image-guided intelligent laser minimally invasive surgery system of the present invention.

Detailed ways

The present invention will be further described through the embodiments below in conjunction with the accompanying drawings.

As shown in Figure 1, the intelligent laser minimally invasive surgery system of the present embodiment includes: a mechanical arm 1, a scanner 2, a camera 3, a laser 4, a light guide arm 5, a patient movable bed 6, and an information processing and control terminal 7; Among them, the light guide arm 5 is connected to the laser 4 to guide the laser light emitted by the laser 4; the mechanical arm 1 is connected to the information processing and control terminal 7; the mechanical arm is physically connected to the light guide arm 5 through the end gripper to control the light guide arm 5, so as to control the output of the laser to align with the pigmented lesion area on the patient's movable bed 6; the scanner 2 and the camera 3 are respectively fixed on the same frame to form a visual servo system together, and the field of view is the patient's movable bed 6 on the pigmented lesion area; the scanner 2 and the camera 3 are connected to the information processing and control terminal 7 respectively. The mechanical arm adopts a six-degree-of-freedom mechanical arm, and the scanner adopts a three-dimensional scanner;

Fig. 2 is a structural block diagram of the intelligent laser minimally invasive surgery system of the present invention.

As shown in Figure 3, the control method of the intelligent laser minimally invasive surgery system under the image guidance of the present invention comprises the following steps:

1) Jointly calibrate the scanner and the robotic arm so that they work in the same coordinate system;

2) Connect the robotic arm to the light guide arm, and correct the end of the robotic arm so that the laser focus is set as the end of the robotic arm;

3) Initialize the calibration camera and perform image calibration;

4) Expose the pigmented lesion area to the field of view of the scanner and camera, the scanner scans the pigmented lesion area, transmits the data to the information processing and control terminal, and obtains the location information of the pigmented lesion area;

5) The camera images the pigmented lesion area, transmits the real-time image of the pigmented lesion area to the information processing and control terminal, and obtains the color characteristics of the pigmented lesion area;

6) The information processing and control terminal automatically analyzes the geometric and color features of the pigmented lesion area, and extracts the three colors of red R, green G, and blue B in the pigmented lesion area by comparing with the data retrieval of the preset expert system. information;

7) The expert system in the information processing and control terminal determines the wavelength, intensity and irradiation time of the laser used for irradiation according to the extracted color feature information;

8) Accurately locate the three-dimensional coordinates (x, y, z) of each point in the pigmented lesion area according to the real-time image provided by the camera, and the information processing and control terminal pre-generates the motion path of the robotic arm according to the collected image;

9) The information processing and control terminal controls the movement of the robotic arm according to the motion path, so as to control the light guide arm to guide the laser irradiation on the pigmented lesion area, and perform completely autonomous laser irradiation until the robotic arm completes all operations according to the pre-generated motion path , the laser irradiation stops automatically.

As shown in Figure 4, autonomous laser irradiation specifically includes the following steps:

a) Start the irradiation program: turn on the laser, and the laser irradiates the pigmented lesion area through the light guide arm;

b) The scanner performs target position detection in real time, and the information processing and control terminal judges whether the position of the pigmented lesion area is displaced, if not, continue to perform laser irradiation, and if so, enter step c);

c) judging whether the displacement exceeds the limit value, if not, enter step d), if yes, stop the laser irradiation;

d) The information processing and control terminal performs position calibration to judge whether the position calibration is successful, if yes, continue the laser irradiation, if not, stop the laser irradiation to avoid danger.

Finally, it should be noted that the purpose of publishing the implementation is to help further understand the present invention, but those skilled in the art can understand that various replacements and modifications can be made without departing from the spirit and scope of the present invention and the appended claims. It is possible. Therefore, the present invention should not be limited to the content disclosed in the embodiments, and the protection scope of the present invention is subject to the scope defined in the claims.

Claims (6)

1. the intelligent laser minimally invasive surgery system under image guiding, it is characterized in that, described intelligent laser minimally invasive surgery system comprises: mechanical arm, scanner, photographic head, laser instrument, light-conducting arm, patient activity bed and information processing and controlling terminal; Wherein, described light-conducting arm connecting laser, the laser aiming sent by laser instrument goes out; Described mechanical arm is connected to information processing and controlling terminal; Described mechanical arm, by end clamper and light-conducting arm physical connection, controls position and the direction of light-conducting arm, thus controls the pigment lesion region on the laser alignment patient activity bed of output; Described scanner and photographic head are separately fixed in same frame, jointly form vision servo system, and the visual field is the pigment lesion region on patient activity bed; Described scanner scans pigment lesion region, sends data to information processing and controlling terminal, obtains the positional information of pigment lesion region; The real time imaging of pigment lesion region is transferred to information processing and controlling terminal by described photographic head, obtains the color characteristic of pigment lesion region; The geometric properties of described information processing and controlling terminal automatic analysis pigment lesion region and color characteristic, the motion path of generation machine mechanical arm, according to the movement of motion path controller mechanical arm, thus control light-conducting arm guided laser acts on pigment lesion region, carries out completely autonomous laser irradiation.

2. intelligent laser minimally invasive surgery system as claimed in claim 1, is characterized in that, described mechanical arm adopts sixdegree-of-freedom simulation.

3. intelligent laser minimally invasive surgery system as claimed in claim 1, it is characterized in that, described scanner is spatial digitizer, scans pigment lesion region, then three dimensional point cloud is sent to information processing and controlling terminal by data-interface, obtains the three-dimensional information of position.

4. the control method of the intelligent laser minimally invasive surgery system under image guiding, it is characterized in that, control method comprises the following steps:

1) combined calibrating is carried out to scanner and mechanical arm, make the two be operated in the same coordinate system;

2) mechanical arm is connected with light-conducting arm, mechanical arm tail end is corrected, make laser focusing place be set to mechanical arm tail end;

3) initialize demarcation photographic head, carry out image calibration;

4) pigment lesion region be exposed in the visual field of scanner and photographic head, scanner scans pigment lesion region, sends data to information processing and controlling terminal, obtains the positional information of pigment lesion region;

5) photographic head carries out imaging to pigment lesion region, and the real time imaging of pigment lesion region is transferred to information processing and controlling terminal, obtains the color characteristic of pigment lesion region;

6) geometric properties of information processing and controlling terminal automatic analysis pigment lesion region and color characteristic, by contrasting with the data retrieval of pre-set specialist system, extracts red R, the green G of pigment lesion region and the information of blue B tri-kinds of colors;

7) wavelength of laser, intensity and the exposure time that use according to the color characteristic information determination irradiation extracted of information processing and controlling terminal;

8) according to the real time imaging that photographic head provides, accurately three coordinates (x, y, z) of each point of location pigment lesion region, information processing and controlling terminal is according to the motion path of the image collected generation machine mechanical arm in advance;

9) information processing and controlling terminal is according to the movement of motion path controller mechanical arm, thus control light-conducting arm guided laser irradiation in pigment lesion region, carry out completely autonomous laser irradiation, until mechanical arm completes all operations according to the motion path generated in advance, laser irradiation stops automatically.

5. control method as claimed in claim 4, it is characterized in that, in step 8) in, comprise further: according to the contact distance of light-conducting arm and pigment lesion region, the motion path of position to mechanical arm of clamper and light-conducting arm corrects, adjust suitable pose, make clamper can carry light-conducting arm and carry out laser irradiation.

6. control method as claimed in claim 4, is characterized in that, in step 9) in, autonomous laser irradiation comprises the following steps:

A) start irradiation process: open laser instrument, laser passes through light-conducting arm irradiation in pigment lesion region;

B) scanner carries out target location detection in real time, and whether the position of information processing and controlling terminal judges pigment lesion region is subjected to displacement, and if not, proceeds laser irradiation, if so, enters step c);

C) judge whether displacement exceedes limit value, if not, enter steps d), if so, then laser irradiation stops;

D) information processing and controlling terminal position calibration, judge that whether position correction is successful, if so, laser irradiation proceeds, and if not, then laser irradiation stops.