CN116687328A - Catheter movement control device, catheter movement control method, and storage medium - Google Patents

Abstract

The present application relates to a catheter movement control device, method and storage medium. The movement control device of the catheter includes: a first acquisition module, which is used to acquire point cloud data collected by the catheter, and the point cloud data is used to characterize the position of the catheter in the target object; The current mapping position of the catheter is determined in the centerline model of the object; the control module is used to adjust the direction and bending degree of the catheter according to the current mapping position of the catheter; according to the path planning data of the catheter, control the movement of the catheter after adjusting the direction and bending degree preset distance. The device can adjust the direction and bending degree of the catheter according to the point cloud data before each movement of the catheter, thereby accurately realizing the automatic movement control of the catheter and improving the efficiency of the catheter into the mirror.

Description

Catheter movement control device, method and storage medium

technical field

The present application relates to the technical field of robot control, in particular to a catheter movement control device, method and storage medium.

Background technique

A bronchoscope is a medical device inserted into the lower respiratory tract of a patient through the mouth or nose for observation of lung lobes, segmental and subsegmental bronchial lesions, biopsy sampling, bacteriological and cytological examinations.

In the related art, the way of entering the bronchoscope mainly depends on the manual insertion of the bronchoscope by the doctor. According to the preoperative computed tomography (Computed Tomography, CT) images, the doctor plans the route of the bronchoscope. During the operation, the doctor obtains the real-time position of the catheter of the bronchoscope through the real-time images obtained by the bronchoscope, so as to match the path planning of the bronchoscope, and continuously adjust and move the position of the bronchoscope manually.

However, it is time-consuming and labor-intensive for the doctor to plan and manually move the path of the bronchoscope, resulting in low efficiency of the bronchoscope.

Contents of the invention

Based on this, it is necessary to address the above technical problems and provide a movement control device and method for a catheter and a movement control of a storage medium catheter that can improve the efficiency of bronchoscopy.

In a first aspect, the present application provides a catheter movement control device for catheter movement control. The devices include:

The first acquisition module is used to acquire point cloud data collected by the catheter, and the point cloud data is used to characterize the position of the catheter in the target object;

a determination module, configured to determine the current mapping position of the catheter in the centerline model of the target object according to the point cloud data;

The control module is configured to adjust the direction and bending degree of the catheter according to the current mapped position of the catheter; and control the catheter to move a preset distance after adjusting the direction and bending degree according to the path planning data of the catheter.

In one of the embodiments, the control module includes:

a centerline direction determining unit, configured to determine the direction of the target centerline corresponding to the current mapping position of the catheter;

a moving direction determining unit, configured to determine the moving direction of the catheter according to the point cloud data;

The adjustment unit is configured to adjust the orientation and bending degree of the catheter according to the moving direction of the catheter and the direction of the target centerline.

In one of the embodiments, the centerline direction determining unit is specifically configured to determine a first plane of the catheter, and the first plane is a plane of the head of the catheter; combining the first plane with the The direction of the tangent between the target centerlines is determined as the direction of the target centerlines.

In one of the embodiments, the moving direction determining unit is specifically configured to determine the center curve of the catheter according to the point cloud data; A target tangent; determining the direction of the target tangent as the moving direction of the catheter.

In one of the embodiments, the adjustment unit is specifically configured to adjust the direction of the catheter according to the angle difference between the moving direction of the catheter and the direction of the target centerline on the second plane, and the second The plane is the tail plane of the catheter; according to the included angle between the moving direction of the catheter and the direction of the target centerline, the degree of bending of the catheter is adjusted.

In one of the embodiments, the adjusting unit is specifically configured to: if the angle difference between the moving direction of the catheter and the direction of the target centerline on the second plane is greater than a first threshold, determine the angle difference is the to-be-rotated angle of the catheter; the direction of the catheter is adjusted according to the to-be-rotated angle of the catheter.

In one of the embodiments, the adjustment unit is specifically configured to: if the angle difference between the moving direction of the catheter and the direction of the centerline corresponding to the current mapping position of the catheter on the target plane is less than or equal to the first threshold, then Keep the orientation of the catheter unchanged.

In one of the embodiments, the adjustment unit is specifically configured to determine the included angle as the The angle to be bent of the catheter; according to the angle to be bent of the catheter, the degree of bending of the catheter is adjusted; if the angle between the moving direction of the catheter and the direction of the target centerline is less than or equal to the second threshold value, then keep the The bending degree of the catheter is unchanged.

In a second aspect, the present application provides a method for controlling movement of a catheter. The methods include:

Acquiring point cloud data collected by the catheter, where the point cloud data is used to characterize the position of the catheter in the target object;

determining a current mapped position of the catheter in a centerline model of the target object based on the point cloud data;

adjusting the direction and bending degree of the catheter according to the current mapped position of the catheter;

According to the path planning data of the catheter, the catheter moves a preset distance after adjusting the direction and bending degree.

Catheter Movement Control In the third aspect, the present application also provides a computer device. The computer device includes a memory and a processor, the memory stores a computer program, and when the processor executes the computer program, the method for controlling the movement of the catheter described in the second aspect above is implemented.

In a fourth aspect, the present application also provides a computer-readable storage medium. The computer-readable storage medium has a computer program stored thereon, and when the computer program is executed by a processor, the method for controlling the movement of the catheter described in the first aspect above realizes the movement control of the catheter.

In a fifth aspect, the present application also provides a computer program product. The computer program product includes a computer program, and when the computer program is executed by a processor, the movement control method of the catheter described in the first aspect is implemented to control the movement of the catheter.

The above catheter movement control device, method and storage medium The movement control of the catheter first obtains the point cloud data collected by the catheter, and the point cloud data is used to represent the position of the catheter in the target object. Secondly, according to the point cloud data, determine the current mapping position of the catheter in the centerline model of the target object; thirdly, adjust the direction and bending degree of the catheter according to the current mapping position of the catheter; finally, control the adjustment according to the path planning data of the catheter The catheter moves a preset distance after the direction and degree of bending. Because the direction and bending degree of the catheter are adjusted according to the point cloud data before each movement of the catheter of the bronchoscope, the automatic movement control of the bronchoscope is accurately realized without manual intervention, which improves the efficiency of the bronchoscope into the mirror.

Description of drawings

FIG. 1 is an application environment diagram of a catheter movement control method provided by an embodiment of the present application;

FIG. 2 is an application environment diagram of another catheter movement control method provided by the embodiment of the present application;

FIG. 3 is a schematic flowchart of a catheter movement control method provided by an embodiment of the present application;

FIG. 4 is a schematic diagram of a bronchoscope provided in an embodiment of the present application;

FIG. 5 is a schematic diagram of point cloud data collected at a pre-acquisition position provided by an embodiment of the present application;

FIG. 6 is a schematic diagram of a coordinate system transformation provided by the embodiment of the present application;

FIG. 7 is a schematic diagram of a real-time coordinate registration provided by the embodiment of the present application;

Fig. 8 is a schematic diagram of extracting a bronchial centerline provided in the embodiment of the present application;

Fig. 9 is a diagram of the positional relationship between a bronchoscope and the centerline of the bronchus provided by the embodiment of the present application;

Fig. 10 is a schematic diagram of the angle to be rotated of a bronchoscope provided in the embodiment of the present application;

Fig. 11 is a schematic diagram of a bronchoscope provided in an embodiment of the present application with the rotation angle kept constant;

Fig. 12 is a schematic diagram of the angle to be bent of a bronchoscope provided in the embodiment of the present application;

Fig. 13 is a schematic diagram of a bronchoscope provided in an embodiment of the present application with the bending angle kept constant;

Fig. 14 is a schematic diagram of conversion of an angle to be bent provided by the embodiment of the present application;

Fig. 15 is a schematic diagram of path planning of a bronchoscope provided in the embodiment of the present application;

Fig. 16 is a schematic flow chart of an automatic mirror-introduction method for a bronchoscope provided in an embodiment of the present application;

Fig. 17 is a schematic flowchart of another catheter movement control provided by the embodiment of the present application;

Fig. 18 is a structural block diagram of a catheter movement control device provided by an embodiment of the present application;

FIG. 19 is an internal structural diagram of a computer device provided by an embodiment of the present application.

Detailed ways

In order to make the purpose, technical solution and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described here are only used to explain the present application, and are not intended to limit the present application.

The catheter movement control method provided in the embodiment of the present application can be applied to the application environment shown in FIG. 1 and FIG. 2 .

As shown in FIG. 1 , before a bronchoscope is inserted, a terminal device 101 is connected to a computed tomography (CT) device 102 to acquire a CT image of a human chest scanned by the CT device 102 . Subsequently, the terminal device 101 processes the CT image, and extracts the centerline of the bronchi therein, so as to establish a model of the centerline of the bronchi.

As shown in FIG. 2 , when the bronchoscope 103 enters the mirror, the terminal device 101 is connected to the bronchoscope 103, and the terminal device 101 can configure the position of the bronchoscope 103 and the bronchial centerline model, so that the terminal device 101 can control the bronchoscope 103 to perform automatic mirroring. Subsequently, the terminal device 101 controls the bronchoscope 103 to move a small distance at a time according to the path planning data until the bronchoscope 103 reaches the target entrance position indicated by the path planning data. During each movement, the terminal device 101 will adjust the direction and bending degree of the bronchoscope 103 according to the point cloud data collected by the bronchoscope 103 .

In some embodiments, the above-mentioned terminal device 101 can also display the endoscopic image collected by the bronchoscope 10 when the bronchoscope 103 enters the mirror, so that the doctor can observe the movement of the bronchoscope 103 .

It should be understood that the embodiment of the present application does not limit the type of the terminal device 101, which may be, but not limited to, various personal computers, notebook computers, smart phones, tablet computers, Internet of Things devices, and portable wearable devices.

In some embodiments, the bronchoscope 103 together with the electromagnetic generator and the drive assembly can form an endoscopic device. The driving assembly is used to drive the bronchoscope to move and adjust the direction and bending degree of the bronchoscope 103 under the control of the terminal device 101 . The electromagnetic generator is used to obtain the point cloud data collected during the movement of the bronchoscope to determine the real-time pose of the bronchoscope. The above-mentioned bronchoscope 103 may be a catheter, which may include a monocular endoscope and an electromagnetic patch. Among them, the monocular endoscope is used to collect endoscopic images, and the electromagnetic patch is used to collect coordinate information of the catheter relative to the magnetic field generator to generate the above-mentioned point cloud data.

In one embodiment, as shown in FIG. 3 , a catheter movement control method is provided. The method is applied to the terminal device in FIG. 1 and FIG. 2 as an example, including S201-S202:

S201. Obtain point cloud data collected by the bronchoscope, where the point cloud data is used to characterize the position of the bronchoscope in the bronchus of the target object.

In the present application, when the terminal device controls the bronchoscope to enter the mirror, the terminal device may control the bronchoscope to move multiple times until the bronchoscope reaches the target mirror entry position. During each movement, the terminal device can obtain the point cloud data collected by the bronchoscope, so as to determine the position of the bronchoscope and adjust the direction and bending degree of the bronchoscope according to the point cloud data.

Wherein, the above-mentioned point cloud data includes position data and attitude data of the bronchoscope in the bronchus of the target object. The aforementioned target object may be a patient to be mirrored. The aforementioned pose may include the position and posture of the bronchoscope.

It should be understood that the embodiment of the present application does not limit how to collect the point cloud data collected by the bronchoscope. In some embodiments, the above point cloud data may be collected through an electromagnetic patch inside the bronchoscope. Exemplarily, as shown in FIG. 4 , two electromagnetic patches may be provided at the catheter head and the catheter tail of the bronchoscope, including electromagnetic patch 1 , electromagnetic patch 2 , electromagnetic patch 3 and electromagnetic patch 4 . The electromagnetic patch 1 and the electromagnetic patch 2 arranged at the catheter head can collect the coordinate information of the catheter head of the bronchoscope relative to the head of the magnetic field generator, and the electromagnetic patch 3 and the electromagnetic patch 4 arranged at the catheter tail can The coordinate information of the tail of the catheter of the bronchoscope relative to the magnetic field generator is collected. Subsequently, the terminal device may use the collected head coordinate information and tail coordinate information as point cloud data collected by the bronchoscope at the current position.

S202. According to the point cloud data, determine the current mapping position of the bronchoscope in the bronchus centerline model of the target object.

In the step, after the terminal device obtains the point cloud data collected by the bronchoscope, the current mapping position of the bronchoscope can be determined in the bronchial centerline model of the target object according to the point cloud data.

It should be understood that the embodiment of the present application does not limit how to determine the current mapping position of the bronchoscope in the bronchial centerline model of the target object according to the point cloud data. In some embodiments, if the point cloud data is passed through the bronchoscope For electromagnetic patch collection, the position data of the bronchoscope is in the electromagnetic coordinates. At this time, the terminal device can determine the relative position between the position of the bronchoscope and the pre-acquisition position in the bronchus of the target subject under electromagnetic coordinates, so as to determine the position in the center of the bronchus based on the relative position and the mapping position of the pre-acquisition position in the centerline model. The current mapped position of the bronchoscope in the wire model.

It should be understood that the embodiment of the present application does not limit the above-mentioned pre-acquisition position. Exemplarily, the pre-acquisition position includes the left and right bronchus bifurcation, the left secondary bifurcation point, and the right secondary bifurcation point.

It should be noted that the coordinates of the above-mentioned pre-acquisition position in the electromagnetic coordinates can be collected when the bronchial centerline model is registered with the coordinate system of the point cloud data. The mapping position of the pre-acquisition position in the centerline model can be determined according to the label information or through automatic identification.

In some embodiments, before determining the current mapped position of the bronchoscope in the bronchus centerline model of the target subject, the terminal device acquires point cloud data collected by the bronchoscope at a pre-acquisition position in the bronchus of the target subject. According to the point cloud data collected at the pre-acquisition position and the mapping position of the pre-acquisition position in the center line model, the bronchial center line model is registered with the coordinate system of the point cloud data.

It should be understood that the embodiment of the present application does not limit how to collect the point cloud data of the pre-acquisition position in the bronchi of the target object. In some embodiments, the doctor can manually enter the bronchoscope to the above-mentioned pre-acquisition position to collect point cloud data before the automatic bronchoscope is officially entered.

Exemplarily, as shown in FIG. 5 , the bronchoscope can be moved first to enter the trachea, reach the bifurcation of the left and right bronchus, and collect point cloud data of the bifurcation of the left and right bronchi. Then, control the bronchoscope to enter the left main bronchus to reach the left secondary bifurcation point, and collect point cloud data of the left secondary bifurcation point. Finally, control the bronchoscope to enter the right main bronchus to reach the right secondary bifurcation point, and collect point cloud data of the right secondary bifurcation point.

As shown in Figure 6, since the point cloud data at the pre-acquisition position is collected through the electromagnetic patch of the bronchoscope, correspondingly, the point cloud data is in the electromagnetic coordinate system. Since the bronchus centerline model is established based on chest CT images, correspondingly, the bronchus centerline model is in the CT coordinate system. Therefore, before automatic mirroring, it is necessary to register the bronchial centerline model with the electromagnetic coordinate system of the point cloud data to determine the mapping position of the position of the bronchoscope in the bronchial centerline model.

As shown in Figure 7, after collecting the point cloud data at the pre-acquisition position, the global registration matrix T corresponding to the electromagnetic coordinate system and the CT coordinate system where the point cloud data is located can be calculated based on the collected point cloud data, and then , the real-time registration of the bronchus centerline model and the electromagnetic coordinate system of the point cloud data is performed through the global registration matrix T.

Next, the bronchi centerline model of the target object will be described.

It should be understood that the embodiment of the present application does not limit how to generate the bronchial centerline model of the target object. In some embodiments, the terminal device can obtain the chest CT image of the target object, and then extract the bronchus center of the target object from the chest CT image. Wire. Subsequently, according to the bronchus centerline, a bronchial centerline model of the target object is established.

Wherein, extracting the center line of the bronchus of the target object from the above-mentioned chest CT image can be realized by reconstructing the bronchus model of the target object after image segmentation. In some embodiments, the terminal device may input the computed tomography image of the chest into the image processing model, and acquire the bronchus segmentation image output by the image processing model. Subsequently, the terminal device reconstructs the bronchi model of the target subject according to the segmented image of the bronchi. Finally, the terminal device extracts the centerline of the bronchus of the target subject according to the bronchus model of the target subject.

It should be understood that the embodiment of the present application does not limit the foregoing image processing model, which may be any type of deep learning model. Exemplarily, the historical chest CT model may be marked to form a training set of the image processing model, and the training set of the image processing model is used to train the image processing model. After the training is completed, the above-mentioned chest CT image of the target object is input into the image processing model, and the segmented image of the bronchi of the target object can be obtained.

Exemplarily, the network structure (not shown) of an image processing model provided in the embodiment of the present application. The coding unit in the image processing model can down-sample the input chest CT image to obtain image features with different receptive fields at different scales. Subsequently, the encoding unit transfers image features of different scales containing different receptive fields to the decoding unit in the image processing model to prevent features from disappearing due to too deep a model. The decoding unit can upwardly adopt the obtained image features to obtain image features associated with the bronchi.

Exemplarily, FIG. 8 is a schematic diagram of extracting a bronchial centerline provided in the embodiment of the present application. As shown in FIG. 8 , after the segmented image of the bronchus is obtained, the three-dimensional model reconstruction of the bronchus model can be performed using multiple segmented images of the bronchi at different positions. After the three-dimensional model is reconstructed, a 3D skeleton extraction algorithm (itk.BinaryThinningImageFilter3D) is used to extract the bronchial centerline in the bronchial model.

In some embodiments, when the bronchus centerline is extracted, the terminal device may also display the reconstructed bronchus model and/or the extracted bronchus centerline.

S203. Adjust the direction and bending degree of the bronchoscope according to the current mapping position of the bronchoscope.

In this step, after the terminal device determines the current mapping position of the bronchoscope in the bronchus centerline model of the target object, it can adjust the direction and bending degree of the bronchoscope according to the current mapping position of the bronchoscope and the posture data of the bronchoscope.

It should be understood that the embodiment of the present application does not limit how to adjust the direction and degree of bending of the bronchoscope. In some embodiments, the terminal device can first determine the direction of the centerline of the target bronchus corresponding to the current mapping position of the bronchoscope, and then, the terminal device The device determines the moving direction of the bronchoscope based on the point cloud data. Finally, the terminal device adjusts the orientation and bending degree of the bronchoscope according to the moving direction of the bronchoscope and the direction of the centerline of the target bronchus.

In some embodiments, the terminal device may first determine the center curve of the catheter of the bronchoscope according to the point cloud data. Subsequently, determine the target tangent of the center curve of the bronchoscope's catheter at the centerline point of the head of the bronchoscope's catheter. Finally, the terminal device determines the direction of the target tangent line as the moving direction of the bronchoscope.

Exemplarily, continuing to refer to FIG. 4, two electromagnetic patches can be set at the catheter head and the catheter tail of the bronchoscope, and the above-mentioned point cloud data can be electromagnetic patch 1, electromagnetic patch 2, electromagnetic patch 3, and electromagnetic patch The coordinates of patch 4 in the electromagnetic coordinate system. Since the front end and the rear end of the catheter of the bronchoscope are in the same plane, the perpendicular line of the line segment L ₁₂ and the line segment L ₃₄ is a point intersecting on this plane. Correspondingly, with this point as the center of the circle, the arc between the centerline point A of the head of the catheter of the bronchoscope and the midline point B of the tail of the catheter of the bronchoscope is the center curve of the catheter of the bronchoscope. Subsequently, a target tangent I of the center curve of the bronchoscope catheter at the centerline point of the head of the bronchoscope catheter and a tangent J of the center curve of the bronchoscope catheter at the caudal midline point of the bronchoscope catheter can be determined. At this time, the direction of the target tangent line I can be determined as the moving direction of the bronchoscope, and the tangent line J at the midline point of the tail is the direction vector of the catheter tail.

In some embodiments, the terminal device may determine a first plane of the bronchoscope, where the first plane is a head plane of a catheter of the bronchoscope. Subsequently, the direction of the tangent between the first plane and the centerline of the target bronchi is determined as the direction of the centerline of the target bronchi.

Exemplarily, FIG. 9 is a positional relationship diagram between a bronchoscope and a bronchial central line provided in an embodiment of the present application. As shown in FIG. 9 , the plane of the head of the catheter of the bronchoscope is the first plane, and the plane of the tail of the catheter of the bronchoscope is the second plane. Take the tail direction vector J of the catheter of the bronchoscope as the z-axis, take the line connecting the two electromagnetic patches on the tail plane as the x-axis, and take the line connecting the two electromagnetic patches on the second plane passing through the midline point of the tail The vertical line is the y-axis, and a space rectangular coordinate system xyz is established. In the spatial rectangular coordinate system xyz, the tangent direction of the intersection point of the first plane and the centerline of the target bronchus is the centerline direction P, that is, the direction of the centerline of the target bronchus.

If in the space Cartesian coordinate system xyz shown in Figure 10, the moving direction I of the bronchoscope and the direction P of the centerline of the target bronchus are not on the same plane and the direction is not consistent, continuing to control the movement of the bronchoscope will cause the bronchoscope to deviate further Path planning, at this time, need to adjust the bronchoscope.

It should be understood that the embodiment of the present application does not limit how to adjust the orientation and bending degree of the bronchoscope. In some embodiments, the terminal device may adjust the direction of the bronchoscope according to the angle difference between the moving direction of the bronchoscope and the direction of the centerline of the target bronchus on the second plane. Subsequently, the bending degree of the bronchoscope is adjusted according to the angle between the moving direction of the bronchoscope and the direction of the centerline of the target bronchus.

In some embodiments, if the angle difference between the moving direction of the bronchoscope and the direction of the centerline of the target bronchus on the second plane is greater than the first threshold, the terminal device determines the angle difference as the angle to be rotated for the bronchoscope. The terminal device can adjust the direction of the bronchoscope according to the angle to be rotated of the bronchoscope. If the angle difference between the moving direction of the bronchoscope and the direction of the center line corresponding to the current mapping position of the bronchoscope on the target plane is less than or equal to the first threshold, then keep the direction of the bronchoscope unchanged.

Exemplarily, FIG. 10 is a schematic diagram of a to-be-rotated angle of a bronchoscope provided in an embodiment of the present application. As shown in Figure 10, take the line connecting the two electromagnetic patches on the tail plane as the x-axis, and take the line connecting the two electromagnetic patches on the second plane as the vertical line passing through the midline point of the tail as the y-axis to establish rectangular coordinates Department xoy. In the Cartesian coordinate system xoy, the moving direction I of the bronchoscope is projected onto the second plane xoy to obtain a projection I ₁ , and the angle between I ₁ and the y-axis is α ₀ . The direction P of the centerline of the target bronchi is projected onto the second plane xoy to obtain a projection P ₁ , and the included angle between P ₁ and the y-axis is α ₁ . Correspondingly, the angle difference between the moving direction of the bronchoscope and the direction of the centerline of the target bronchus on the second plane is Δα=α ₀ −α ₁ .

At this time, as shown in FIG. 11 , if Δα is less than or equal to the first threshold, it can be determined that the movement direction of the bronchoscope does not deviate greatly from the centerline of the target bronchus, and the direction of the bronchoscope can be kept unchanged and the mirror can be continued. If Δα is greater than the first threshold, it can be determined that the moving direction of the bronchoscope deviates greatly from the centerline of the target bronchus, and the angle difference Δα can be determined as the angle to be rotated of the bronchoscope, thereby adjusting the direction of the bronchoscope.

It should be noted that, the embodiment of the present application does not limit the above-mentioned first threshold, which can be specifically set according to the control accuracy, for example, it can be set to 5°, 10°, and so on.

It should be understood that when the direction of the bronchoscope is adjusted according to the angle to be rotated, the direction of adjustment can be determined according to the positive or negative value of Δα. If Δα>0, the bronchoscope is controlled to rotate by an angle Δα toward the y-axis direction; if Δα<0, then the bronchoscope is rotated by an angle Δα toward the direction away from the y-axis. The moving direction of the adjusted bronchoscope is on the same plane as the direction of the centerline of the target bronchus.

In some embodiments, if the included angle between the moving direction of the bronchoscope and the direction of the centerline of the target bronchus is greater than a second threshold, the included angle is determined as the angle to be bent of the bronchoscope. Adjust the bending degree of the bronchoscope according to the bending angle of the bronchoscope. If the included angle between the moving direction of the bronchoscope and the direction of the centerline of the target bronchus is smaller than or equal to the second threshold, then keep the bending degree of the bronchoscope unchanged.

Exemplarily, FIG. 12 is a schematic diagram of an angle to be bent of a bronchoscope provided in an embodiment of the present application. As shown in FIG. 12 , a rectangular coordinate system poz is established with the direction vector J of the tail end of the catheter of the bronchoscope as the z-axis and the direction P of the centerline of the target bronchus as the p-axis. After adjusting the direction of the bronchoscope after the waiting rotation angle, the directions of the bronchoscope and the centerline of the target bronchi are on the plane poz. When the moving directions I ₂ and P of the rotated bronchoscope are known, it can be determined that the angle between the moving direction I ₂ of the rotated bronchoscope and the z-axis is θ ₀ , and the target bronchus center direction P and the z-axis The included angle is θ ₁ . The included angle between the head of the bronchoscope and the origin o and the z-axis is β ₀ , and the included angle between the front end of the target bronchus central line and the origin o and the z-axis is β ₁ . Correspondingly, the included angle Δβ=β ₀ −β ₁ between the moving direction of the bronchoscope and the direction of the centerline of the target bronchus.

At this time, as shown in Figure 13, if Δβ is less than or equal to the second threshold, it can be determined that the bending degree of the bronchoscope is not much different from the bending degree of the centerline of the target bronchus, and the bending degree of the bronchoscope can be kept unchanged, and the endoscope can be continued . If Δβ is greater than the second threshold, it can be determined that the bending degree of the bronchoscope is quite different from the bending degree of the centerline of the target bronchus, and the angle difference Δβ can be determined as the bending angle of the bronchoscope of the bronchoscope, thereby adjusting the direction of the bronchoscope .

It should be noted that, the embodiment of the present application does not limit the above-mentioned second threshold, which can be specifically set according to the adjustment accuracy, for example, it can be set to 5°, 10°, and so on.

It should be understood that when adjusting the bending degree of the bronchoscope according to the angle to be bent, the direction of adjustment can be determined according to the positive or negative of Δβ. If Δβ>0, control the bronchoscope to increase the bending degree away from the z-axis; if Δα<0, reduce the bending degree away from the z-axis.

Exemplarily, FIG. 14 is a schematic diagram of conversion of an angle to be bent provided in an embodiment of the present application. As shown in FIG. 14 , a rectangular coordinate system poz is established with the direction vector J of the tail end of the catheter of the bronchoscope as the z-axis and the direction P of the centerline of the target bronchus as the p-axis. It is known that I is a circular arc from point O to point B, and OA and OB are the tangent lines of point O and point B on the circular arc, and the distance from the intersection point to the tangent point of the two tangent lines of the circle is equal, so it can be known that OA=AB. Since OA=AB, we know that △AOB is an isosceles triangle, then ∠AOB=∠ABO=β ₀ . Since the exterior angle of a triangle is equal to the sum of two non-adjacent interior angles, θ ₀ =∠AOB=∠ABO=2β ₀ , that is, β ₀ =θ ₀ /2. therefore, At this time, the terminal device controls the bending Δβ of the bronchoscope so that the moving direction of the bronchoscope is parallel to the direction of the centerline of the target bronchus.

S204. According to the path planning data of the bronchoscope, control the movement of the preset distance of the bronchoscope after adjusting the direction and bending degree.

In this step, after the terminal device adjusts the direction and bending degree of the bronchoscope, it can control the movement of the bronchoscope by a preset distance after adjusting the direction and bending degree according to the path planning data of the bronchoscope.

Wherein, the path planning data of the above-mentioned bronchoscope may be a path for the bronchoscope to reach the target mirror entry position. The above-mentioned target mirror entry position may be a position where puncture needs to be performed, a position to be treated, a lesion position, and the like.

Exemplarily, as shown in FIG. 15 , it is a schematic diagram of path planning of a bronchoscope, and the schematic diagram of the path planning of the bronchoscope shows the bronchi that needs to pass through to reach the target mirror entry position.

It should be understood that the embodiment of the present application does not limit how to determine the path planning data of the bronchoscope. In some embodiments, the terminal device can combine the reconstructed bronchial model and the chest CT of the target subject according to the target mirror entry position input by the doctor. Automatic planning generates path planning data for bronchoscopy. In some other embodiments, the route planning data of the bronchoscope can be manually input by the doctor.

It should be understood that the above path planning data is used to indicate the bifurcation intersection that the bronchoscope should enter when the bronchoscope enters the bifurcation intersection.

It should be understood that the embodiment of the present application does not limit the preset distance, for example, 1 mm, 2 mm, 5 mm, and so on.

In this application, the point cloud data marked with the real-time pose of the bronchoscope obtained during the movement of the bronchoscope is used to calculate the relationship between the position of the bronchoscope and the centerline of the target bronchus, thereby adjusting the direction and degree of bending of the bronchoscope, thereby ensuring The accuracy of the movement control of the catheter is improved.

In the catheter movement control method provided in the embodiment of the present application, the point cloud data collected by the bronchoscope is first obtained, and the point cloud data is used to characterize the position of the bronchoscope in the bronchus of the target subject. Second, based on the point cloud data, the current mapped position of the bronchoscope is determined in the bronchial centerline model of the target subject. Again, according to the current mapping position of the bronchoscope, adjust the direction and bending degree of the bronchoscope. Finally, according to the path planning data of the bronchoscope, the bronchoscope moves a preset distance after adjusting the direction and degree of bending. Because the direction and bending degree of the bronchoscope are adjusted according to the point cloud data before each movement of the bronchoscope, the automatic movement control of the bronchoscope is accurately realized without manual intervention, and the efficiency of the bronchoscope entering the mirror is improved.

The following describes the complete procedure of bronchoscopy. FIG. 16 is a schematic flowchart of a method for automatically advancing a bronchoscope provided in an embodiment of the present application. As shown in Figure 16, the automatic mirror entry method of the bronchoscope includes:

S301. Obtain path planning data of the bronchoscope, where the path planning data includes a target entrance position of the bronchoscope.

S302. Obtain point cloud data collected by the bronchoscope, where the point cloud data is used to characterize the position of the bronchoscope in the bronchus of the target object.

S303. According to the point cloud data, determine the current mapping position of the bronchoscope in the bronchus centerline model of the target object.

S304. Adjust the direction and bending degree of the bronchoscope according to the current mapping position of the bronchoscope.

S305. According to the path planning data of the bronchoscope, control the preset distance of moving the bronchoscope after adjusting the direction and bending degree.

S306. Determine whether the bronchoscope has reached the target endoscope position.

If yes, execute S307; if not, execute S302.

S307. End the bronchoscopy.

In the embodiment of the present application, real-time point cloud data can be used for real-time registration before each control of the movement of the bronchoscope during the automatic mirror entry process, and the direction and bending degree of the bronchoscope can be adjusted, thereby avoiding errors caused by preoperative registration results. Poor performance causes the tube mirror to deviate from the planned path. At the same time, the terminal device automatically controls the bronchoscope to complete the mirror entry, which can realize the movement of the bronchoscope without the control of the doctor, reducing the difficulty of surgery.

The process of adjusting the direction and bending degree of the bronchoscope is described below. Fig. 17 is a schematic flowchart of another catheter movement control provided by the embodiment of the present application. As shown in Figure 17, the movement control of the catheter includes:

S401. Control the bronchoscope to move a preset distance.

S402. Determine whether the bronchoscope has reached the target entrance position.

If yes, execute S411; if not, execute S403.

S403. Obtain point cloud data collected by the bronchoscope.

S404. According to the point cloud data, determine the current mapping position of the bronchoscope in the bronchus centerline model of the target object.

S405. According to the current mapped position of the bronchoscope, determine the angle difference between the movement direction of the bronchoscope and the direction of the center line of the target bronchus on the second plane.

S406. Determine whether the angle difference is greater than a first threshold.

If yes, execute S407; if not, execute S408.

S407. Adjust the direction of the bronchoscope

S408. According to the current mapping position of the bronchoscope, determine the angle between the moving direction of the bronchoscope and the direction of the centerline of the target bronchus.

S409. Determine whether the included angle difference is greater than a second threshold.

If yes, execute S410; if not, execute S401.

S410, adjusting the bending degree of the bronchoscope.

After S410, execute S401.

S411. End the bronchoscopy.

In the catheter movement control method provided in the embodiment of the present application, the point cloud data collected by the bronchoscope is first obtained, and the point cloud data is used to characterize the position of the bronchoscope in the bronchus of the target subject. Second, based on the point cloud data, the current mapped position of the bronchoscope is determined in the bronchial centerline model of the target subject. Again, according to the current mapping position of the bronchoscope, adjust the direction and bending degree of the bronchoscope. Finally, according to the path planning data of the bronchoscope, the bronchoscope moves a preset distance after adjusting the direction and degree of bending. Because the direction and bending degree of the bronchoscope are adjusted according to the point cloud data before each movement of the bronchoscope, the automatic movement control of the bronchoscope is accurately realized without manual intervention, and the efficiency of the bronchoscope entering the mirror is improved.

It should be understood that although the steps in the flow charts involved in the above embodiments are shown sequentially according to the arrows, these steps are not necessarily executed sequentially in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least some of the steps in the flow charts involved in the above embodiments may include multiple steps or stages, and these steps or stages are not necessarily executed at the same time, but may be executed at different times, The execution order of these steps or stages is not necessarily performed sequentially, but may be executed in turn or alternately with other steps or at least a part of steps or stages in other steps.

Based on the same inventive concept, an embodiment of the present application further provides a catheter movement control device for implementing the above-mentioned catheter movement control method. The solution to the problem provided by the device is similar to the implementation described in the above method, so the specific limitations in the embodiment of one or more catheter movement control devices provided below can be referred to above for the movement control of the catheter The limitation of the method will not be repeated here.

In one embodiment, as shown in FIG. 18 , a catheter movement control device 500 is provided, including: a first acquisition module 501, a determination module 502 and a control module 503, wherein:

The first acquisition module 501 is configured to acquire point cloud data collected by the catheter, and the point cloud data is used to characterize the position of the catheter in the target object;

A determining module 502, configured to determine the current mapping position of the catheter in the centerline model of the target object according to the point cloud data;

The control module 503 is configured to adjust the direction and bending degree of the catheter according to the current mapped position of the catheter; and control the catheter to move a preset distance after adjusting the direction and bending degree according to the path planning data of the catheter.

In one of the embodiments, the control module 503 includes:

a centerline direction determining unit, configured to determine the direction of the target centerline corresponding to the current mapping position of the catheter;

a moving direction determining unit, configured to determine the moving direction of the catheter according to the point cloud data;

The adjusting unit is used for adjusting the orientation and bending degree of the catheter according to the moving direction of the catheter and the direction of the target centerline.

In one of the embodiments, the centerline direction determination unit is specifically configured to determine the first plane of the catheter, the first plane is the plane of the head of the catheter; and determine the tangent direction between the first plane and the target centerline as the target The direction of the centerline.

In one of the embodiments, the moving direction determination unit is specifically used to determine the center curve of the catheter according to the point cloud data; determine the target tangent of the center curve of the catheter at the center line point of the catheter head; determine the direction of the target tangent as The direction of movement of the catheter.

In one of the embodiments, the adjustment unit is specifically configured to adjust the direction of the catheter according to the angle difference between the moving direction of the catheter and the direction of the target centerline on the second plane, the second plane being the tail plane of the catheter; The angle between the direction of movement and the direction of the target centerline adjusts the degree of curvature of the catheter.

In one of the embodiments, the adjustment unit is specifically configured to determine the angle difference as the to-be-rotated angle of the catheter if the angle difference between the moving direction of the catheter and the direction of the target centerline on the second plane is greater than a first threshold; The angle to be rotated of the catheter, adjust the direction of the catheter.

In one of the embodiments, the adjustment unit is specifically configured to keep the direction of the catheter unchanged if the angle difference between the moving direction of the catheter and the direction of the center line corresponding to the current mapping position of the catheter on the target plane is less than or equal to the first threshold .

In one of the embodiments, the adjustment unit is specifically configured to determine the included angle as the to-be-bent angle of the catheter if the angle between the moving direction of the catheter and the direction of the target centerline is greater than a second threshold; The bending angle is used to adjust the bending degree of the catheter; if the angle between the moving direction of the catheter and the direction of the target centerline is smaller than or equal to the second threshold, the bending degree of the catheter is kept unchanged.

Each module in the above catheter movement control device can be fully or partially realized by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, and can also be stored in the memory of the computer device in the form of software, so that the processor can invoke and execute the corresponding operations of the above-mentioned modules.

Catheter Movement Control In one embodiment, a computer device is provided, which may be a terminal, and its internal structure may be shown in FIG. 19 . The computer device includes a processor, a memory, an input/output interface, a communication interface, a display unit and an input device. Wherein, the processor, the memory and the input/output interface are connected through the system bus, and the communication interface, the display unit and the input device are connected to the system bus through the input/output interface. Wherein, the processor of the computer device is used to provide calculation and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs in the non-volatile storage medium. The input/output interface of the computer device is used for exchanging information between the processor and external devices. The communication interface of the computer device is used to communicate with an external terminal in a wired or wireless manner, and the wireless manner can be realized through WIFI, mobile cellular network, NFC (Near Field Communication) or other technologies. When the computer program is executed by the processor, the above-mentioned method for controlling the movement of the catheter, or the above-mentioned method for automatically advancing the bronchoscope can be realized.

The display unit of the computer equipment is used to form a visually visible picture, which may be a display screen, a projection device or a virtual reality imaging device. The display screen may be a liquid crystal display screen or an electronic ink display screen, and the input device of the computer device may be a touch layer covered on the display screen, or a button, a trackball or a touch pad set on the casing of the computer device, or a External keyboard, touchpad or mouse etc.

Those skilled in the art can understand that the structure shown in Figure 19 is only a block diagram of a part of the structure related to the solution of this application, and does not constitute a limitation on the computer equipment on which the solution of this application is applied. The specific computer equipment can be More or fewer components than shown in the figures may be included, or some components may be combined, or have a different arrangement of components.

In one embodiment, a computer device is provided, including a memory and a processor, where a computer program is stored in the memory, and the above-mentioned catheter movement control method is realized when the processor executes the computer program.

In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, the above method for controlling the movement of the catheter is implemented.

In one embodiment, a computer program product is provided, including a computer program, and when the computer program is executed by a processor, the above method for controlling movement of the catheter catheter is implemented.

It should be noted that the user information (including but not limited to user equipment information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all It is information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data need to comply with relevant laws, regulations and standards of relevant countries and regions.

Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above-mentioned embodiments can be completed by instructing related hardware through computer programs, and the computer programs can be stored in a non-volatile computer-readable memory In the medium, when the computer program is executed, it may include the processes of the embodiments of the above-mentioned methods. Wherein, any reference to storage, database or other media used in the various embodiments provided in the present application may include at least one of non-volatile and volatile storage. Non-volatile memory can include read-only memory (Read-Only Memory, ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive variable memory (ReRAM), magnetic variable memory (Magnetoresistive Random Access Memory, MRAM), Ferroelectric Random Access Memory (FRAM), Phase Change Memory (Phase Change Memory, PCM), graphene memory, etc. The volatile memory may include random access memory (Random Access Memory, RAM) or external cache memory. As an illustration and not a limitation, RAM can be in various forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (Dynamic Random Access Memory, DRAM). The databases involved in the various embodiments provided in this application may include at least one of a relational database and a non-relational database. The non-relational database may include a blockchain-based distributed database, etc., but is not limited thereto. The processors involved in the various embodiments provided by this application can be general-purpose processors, central processing units, graphics processors, digital signal processors, programmable logic devices, data processing logic devices based on quantum computing, etc., and are not limited to this.

The technical features of the above embodiments can be combined arbitrarily. To make the description concise, all possible combinations of the technical features in the above embodiments are not described. However, as long as there is no contradiction in the combination of these technical features, they should be It is considered to be within the range described in this specification.

The above-mentioned embodiments only express several implementation modes of the present application, and the description thereof is relatively specific and detailed, but should not be construed as limiting the patent scope of the present application. It should be noted that those skilled in the art can make several modifications and improvements without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application should be determined by the appended claims.

Claims (10)

1. A movement control device for a catheter, characterized in that the device comprises: The first acquisition module is used to acquire point cloud data collected by the catheter, and the point cloud data is used to characterize the position of the catheter in the target object; a determination module, configured to determine the current mapping position of the catheter in the centerline model of the target object according to the point cloud data; The control module is configured to adjust the direction and bending degree of the catheter according to the current mapped position of the catheter; and control the catheter to move a preset distance after adjusting the direction and bending degree according to the path planning data of the catheter. 2. The device according to claim 1, wherein the control module comprises: a centerline direction determining unit, configured to determine the direction of the target centerline corresponding to the current mapping position of the catheter; a moving direction determining unit, configured to determine the moving direction of the catheter according to the point cloud data; The adjustment unit is configured to adjust the orientation and bending degree of the catheter according to the moving direction of the catheter and the direction of the target centerline. 3. The device according to claim 2, wherein the centerline direction determining unit is specifically configured to determine a first plane of the catheter, the first plane being the plane of the head of the catheter; A tangent direction between the first plane and the target centerline is determined as the direction of the target centerline. 4. The device according to claim 2, wherein the moving direction determining unit is specifically configured to determine the center curve of the catheter according to the point cloud data; determine the center curve of the catheter in the A target tangent to the heartline point of the head of the catheter; the direction of the target tangent is determined as the moving direction of the catheter. 5. The device according to claim 2, wherein the adjusting unit is specifically configured to adjust the angle difference between the moving direction of the catheter and the direction of the target centerline on the second plane. The direction of the catheter, the second plane is the tail plane of the catheter; according to the included angle between the moving direction of the catheter and the direction of the target centerline, the degree of bending of the catheter is adjusted. 6. The device according to claim 5, wherein the adjusting unit is specifically configured to: if the angle difference between the moving direction of the catheter and the direction of the target centerline on the second plane is greater than a first threshold , the angle difference is determined as the to-be-rotated angle of the catheter; and the direction of the catheter is adjusted according to the to-be-rotated angle of the catheter. 7. The device according to claim 5, wherein the adjustment unit is specifically configured to determine the angle between the moving direction of the catheter and the direction of the centerline corresponding to the current mapping position of the catheter on the target plane If the difference is less than or equal to a first threshold, the direction of the catheter is kept unchanged. 8. The device according to claim 5, wherein the adjustment unit is specifically configured to: if the included angle between the moving direction of the catheter and the direction of the target centerline is greater than a second threshold, then set The angle is determined as the angle to be bent of the catheter; according to the angle to be bent of the catheter, the degree of bending of the catheter is adjusted; if the angle between the moving direction of the catheter and the direction of the target centerline is less than equal to the second threshold, then keep the bending degree of the catheter unchanged. 9. A method for controlling movement of a catheter, characterized in that the method comprises: Acquiring point cloud data collected by the catheter, where the point cloud data is used to characterize the position of the catheter in the target object; determining a current mapped position of the catheter in a centerline model of the target object based on the point cloud data; adjusting the direction and bending degree of the catheter according to the current mapped position of the catheter; According to the path planning data of the catheter, the catheter moves a preset distance after adjusting the direction and bending degree. 10. A computer-readable storage medium, on which a computer program is stored, characterized in that, when the computer program is executed by a processor, the steps of the method according to claim 1 are realized.