CN114399503A - Medical image processing method, device, terminal and storage medium - Google Patents

Abstract

The application provides a medical image processing method, a device, a terminal and a storage medium, wherein the method comprises the steps of obtaining a three-dimensional bile duct tree model corresponding to a bile duct tree and a three-dimensional bone structure marking model corresponding to a first bone structure marking in magnetic resonance pancreaticobiliary duct imaging information; acquiring second marker bony structure data corresponding to a second marker bony structure in the X-ray image; performing cross-dimension image matching on the second marker bony structure data and the three-dimensional marker bony structure model to obtain cross-dimension image matching information; and projecting the three-dimensional biliary tree model onto the X-ray image based on the cross-dimension image matching information to obtain target projection information. The embodiment of the application improves the matching precision of the cross-latitude images.

Description

Medical image processing method, device, terminal and storage medium

technical field

The present application relates to the field of auxiliary medical technology, and in particular, to a medical image processing method, device, terminal and storage medium.

Background technique

Medical image registration technology refers to the image data obtained in different ways (such as CT and MRI images) through a certain spatial transformation or adjustment, so that the target values between different data can be spatially matched, so as to achieve high-dimensional, multi-modality. Diagnosis and treatment data are conducive to correct medical diagnosis.

Percutaneous transhepatic cholangio drainage (PTCD) is mainly under the guidance of X-ray, using a puncture needle to penetrate the bile duct percutaneously through the liver, which can be used for the palliative treatment of malignant biliary tumors and acute suppurative cholangitis. biliary decompression, etc. However, due to the limitation of the principle of X-ray imaging, doctors cannot obtain bile duct images under X-ray fluoroscopy as a reference, and cannot judge whether the position of the puncture needle and the puncture direction are correct. Doctors can only perform empirical puncture based on the three-dimensional magnetic resonance imaging of the bile duct in preoperative Magnetic Resonance Cholangiopancreatography (MRCP) and the two-dimensional imaging of tissue density under intraoperative X-ray fluoroscopy, which is difficult to operate. Inexperienced operators have high learning costs; and have a high risk of complications such as pancreatitis, cholangitis, bleeding, and perforation.

Efficient image registration technology can realize multi-temporal and multi-modal image registration before and during PTCD operation, and effectively promote the development of medical imaging technology. However, the existing cross-latitude images are inaccurate and cannot provide effective guidance for subsequent surgery, which increases the risk of patients during surgery.

Therefore, how to improve the matching accuracy of cross-latitude images is a technical problem that needs to be solved urgently in the field of current auxiliary medical technology.

SUMMARY OF THE INVENTION

The present application provides a medical image processing method, device, terminal and storage medium, aiming at solving the technical problem of improving the matching accuracy of cross-latitude images.

In one aspect, the present application provides a medical image processing method, the method comprising:

Obtaining the three-dimensional bile duct tree model corresponding to the bile duct tree and the three-dimensional landmark bone structure model corresponding to the first marked bone structure in the magnetic resonance cholangiopancreatography information, wherein the magnetic resonance cholangiopancreatography information is that the patient underwent abdominal magnetic resonance before the operation. Magnetic resonance cholangiopancreatography information obtained during the resonance examination;

Acquiring second marker bony structure data corresponding to the second marker bony structure in the X-ray image, the first marker bony structure is the same as the bony structure corresponding to the second marker bony structure, and the X-ray image is the X-ray image taken by the patient in real time during the operation;

performing cross-dimensional image matching on the second marker bone structure data and the three-dimensional marker bone structure model to obtain cross-dimensional image matching information;

Based on the cross-dimensional image matching information, the three-dimensional bile duct tree model is projected onto the X-ray image to obtain target projection information.

In a possible implementation manner of the present application, the acquisition of the three-dimensional bile duct tree model corresponding to the bile duct tree in the magnetic resonance cholangiopancreatography information and the three-dimensional landmark bone structure model corresponding to the first marked bone structure include:

Obtain magnetic resonance cholangiopancreatography information;

constructing a three-dimensional bile duct tree model based on the bile duct tree data information in the magnetic resonance cholangiopancreatography information;

Based on the first marker bone structure information in the magnetic resonance cholangiopancreatography information, a three-dimensional marker bone structure model is constructed.

In a possible implementation manner of the present application, the construction of a three-dimensional marker bone structure model based on the first marker bone structure information in the magnetic resonance cholangiopancreatography information includes:

identifying the first landmark bony structure in the magnetic resonance cholangiopancreatography information;

segmenting the first marker bone structure in the magnetic resonance cholangiopancreatography information to obtain first marker bone structure information;

Based on the first marker bone structure information, a three-dimensional marker bone structure model is constructed.

In a possible implementation manner of the present application, the obtaining of the second marker bony structure data corresponding to the second marker bony structure in the X-ray image includes:

performing image preprocessing on the X-ray image;

Perform image enhancement processing on the X-ray image after image preprocessing;

Identify the second landmark bony structure in the X-ray image after image enhancement processing;

segmenting the second marker bony structure to obtain a preliminary second marker bony structure;

The preliminary second landmark bone structure after edge sampling is segmented to obtain second landmark bone structure data.

In a possible implementation manner of the present application, performing cross-dimensional image matching on the second marker bone structure data and the three-dimensional marker bone structure model to obtain cross-dimensional image matching information, including:

performing projection processing on the three-dimensional marker bony structure model to obtain a first two-dimensional image corresponding to the first marker bony structure;

performing histogram equalization processing on the first two-dimensional image to obtain a second two-dimensional image;

performing edge enhancement processing on the structural edges in the second two-dimensional image to obtain a third two-dimensional image;

calculating the similarity between the third two-dimensional image and the X-ray image;

Adjust the six-degree-of-freedom motion parameter corresponding to the projection process to reach the target six-degree-of-freedom motion parameter, so as to maximize the similarity value corresponding to the similarity, and obtain cross-dimensional image matching information.

In a possible implementation manner of the present application, after projecting the three-dimensional bile duct tree model onto the X-ray image based on the cross-dimensional image matching information to obtain target projection information, the method further includes:

Obtaining the positioning information of the puncture needle in the X-ray image during the operation;

generating puncture direction adjustment information and position adjustment information for the puncture needle based on the positioning information and the target projection information;

Based on the puncture direction adjustment information and the position adjustment information, puncture navigation is performed on the puncture needle during the operation.

In a possible implementation manner of the present application, performing image preprocessing on the X-ray image includes:

performing edge cropping processing on the X-ray image;

Perform image downsampling processing on the X-ray image after edge cropping processing;

The grayscale range specification process is performed on the X-ray image after the image downsampling process.

In another aspect, the present application provides a medical image processing device, the device comprising:

A first acquisition unit, configured to acquire a three-dimensional bile duct tree model corresponding to the bile duct tree in the magnetic resonance cholangiopancreatography information, and a three-dimensional marked bone structure model corresponding to the first marked bone structure, wherein the magnetic resonance cholangiopancreatography information is: Magnetic resonance cholangiopancreatography information obtained during the patient's abdominal magnetic resonance examination before surgery;

a second acquiring unit, configured to acquire second marker bony structure data corresponding to the second marker bony structure in the X-ray image, the first marker bony structure and the bony structure corresponding to the second marker bony structure The same, the X-ray image is the X-ray image taken by the patient in real time during the operation;

a first cross-dimensional image matching unit, configured to perform cross-dimensional image matching on the second marker bone structure data and the three-dimensional marker bone structure model to obtain cross-dimensional image matching information;

The first projection unit is configured to project the three-dimensional bile duct tree model onto the X-ray image based on the cross-dimensional image matching information to obtain target projection information.

In a possible implementation manner of the present application, the first obtaining unit specifically includes:

a third acquiring unit, configured to acquire magnetic resonance cholangiopancreatography information;

a first construction unit, configured to construct a three-dimensional bile duct tree model based on the bile duct tree data information in the magnetic resonance cholangiopancreatography information;

The second construction unit is configured to construct a three-dimensional marked bone structure model based on the first marked bone structure information in the magnetic resonance cholangiopancreatography information.

In a possible implementation manner of the present application, the second construction unit is specifically used for:

identifying the first landmark bony structure in the magnetic resonance cholangiopancreatography information;

segmenting the first marker bone structure in the magnetic resonance cholangiopancreatography information to obtain first marker bone structure information;

Based on the first marker bone structure information, a three-dimensional marker bone structure model is constructed.

In a possible implementation manner of the present application, the second obtaining unit specifically includes:

a first image preprocessing unit, configured to perform image preprocessing on the X-ray image;

a first image enhancement processing unit, configured to perform image enhancement processing on the X-ray image after image preprocessing;

a first identification unit for identifying the second landmark bone structure in the X-ray image after image enhancement processing;

a first segmentation unit, configured to segment the second marker bony structure to obtain a preliminary second marker bony structure;

The preliminary second landmark bone structure after edge sampling is segmented to obtain second landmark bone structure data.

In a possible implementation manner of the present application, the first cross-dimensional image matching unit is specifically used for:

performing projection processing on the three-dimensional marker bony structure model to obtain a first two-dimensional image corresponding to the first marker bony structure;

performing histogram equalization processing on the first two-dimensional image to obtain a second two-dimensional image;

performing edge enhancement processing on the structural edges in the second two-dimensional image to obtain a third two-dimensional image;

calculating the similarity between the third two-dimensional image and the X-ray image;

Adjust the six-degree-of-freedom motion parameter corresponding to the projection process to reach the target six-degree-of-freedom motion parameter, so as to maximize the similarity value corresponding to the similarity, and obtain cross-dimensional image matching information.

In a possible implementation manner of the present application, the device is further used for:

Obtaining the positioning information of the puncture needle in the X-ray image during the operation;

generating puncture direction adjustment information and position adjustment information for the puncture needle based on the positioning information and the target projection information;

Based on the puncture direction adjustment information and the position adjustment information, puncture navigation is performed on the puncture needle during the operation.

In a possible implementation manner of the present application, the first image preprocessing unit is specifically used for:

performing edge cropping processing on the X-ray image;

Perform image downsampling processing on the X-ray image after edge cropping processing;

The grayscale range specification process is performed on the X-ray image after the image downsampling process.

On the other hand, the present application also provides a terminal, and the terminal includes:

one or more processors;

memory; and

One or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the medical image processing method.

On the other hand, the present application also provides a computer-readable storage medium on which a computer program is stored, and the computer program is loaded by a processor to execute the steps in the medical image processing method.

The medical image processing method provided by the present application includes acquiring a three-dimensional bile duct tree model corresponding to the bile duct tree in the magnetic resonance cholangiopancreatography information, and a three-dimensional marked bony structure model corresponding to the first marked bony structure, wherein the magnetic resonance cholangiopancreatography information is the magnetic resonance cholangiopancreatography information obtained by the patient during the abdominal magnetic resonance examination before the operation; then, the second marker bone structure data corresponding to the second marker bone structure in the X-ray image is obtained, and the first marker bone structure is the same as The bone structure corresponding to the second marker bone structure is the same, and the X-ray image is the X-ray image taken by the patient in real time during the operation; the second marker bone structure data and the three-dimensional marker bone structure model are then cross-dimensional image matching. Cross-dimensional image matching information is obtained; and based on the cross-dimensional image matching information, the three-dimensional bile duct tree model is projected onto the X-ray image to obtain target projection information. Compared with the traditional method, under the background that the existing cross-latitude images are inaccurate, cannot provide effective guidance for subsequent surgery, and increase the risk of patients during surgery, the present application creatively proposes to use the second landmark bony structure data. Perform cross-dimensional image matching with the three-dimensional landmark bone structure model to obtain cross-dimensional image matching information, and based on the cross-dimensional image matching information, the three-dimensional bile duct tree model is projected onto the X-ray image, which overcomes the obstacle of cross-latitude image matching and improves the matching accuracy across latitude images.

Description of drawings

In order to illustrate the technical solutions in the embodiments of the present application more clearly, the following briefly introduces the drawings that are used in the description of the embodiments. Obviously, the drawings in the following description are only some embodiments of the present application. For those skilled in the art, other drawings can also be obtained from these drawings without creative effort.

1 is a schematic diagram of a scene of a medical image processing system provided by an embodiment of the present application;

FIG. 2 is a schematic flowchart of an embodiment of the medical image processing method provided in the embodiment of the present application;

FIG. 3 is a schematic flowchart of an embodiment of step 201 provided in the embodiment of the present application;

FIG. 4 is a schematic flowchart of an embodiment of step 303 provided in the embodiment of the present application;

FIG. 5 is a schematic flowchart of an embodiment of step 202 provided in the embodiment of the present application;

FIG. 6 is a schematic flowchart of an embodiment of step 203 provided in the embodiment of the present application;

7 is a schematic flowchart of an embodiment of performing puncture navigation on a puncture needle in an operation based on the puncture direction adjustment information and the position adjustment information provided in the embodiment of the present application;

8 is a schematic structural diagram of an embodiment of a medical image processing apparatus provided in an embodiment of the present application;

FIG. 9 is a schematic structural diagram of an embodiment of a terminal provided in an embodiment of the present application.

Detailed ways

The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only a part of the embodiments of the present application, but not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative work fall within the protection scope of the present application.

In the description of this application, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", " The orientation or positional relationship indicated by "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", etc. is based on the orientation shown in the drawings Or the positional relationship is only for the convenience of describing the present application and simplifying the description, rather than indicating or implying that the indicated device or element must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation on the present application. In addition, the terms "first" and "second" are only used for descriptive purposes, and should not be construed as indicating or implying relative importance or implying the number of indicated technical features. Thus, a feature defined as "first", "second" may expressly or implicitly include one or more features. In the description of the present application, "plurality" means two or more, unless otherwise expressly and specifically defined.

In this application, the word "exemplary" is used to mean "serving as an example, illustration, or illustration." Any embodiment described in this application as "exemplary" is not necessarily to be construed as preferred or advantageous over other embodiments. The following description is presented to enable any person skilled in the art to make and use the present application. In the following description, details are set forth for the purpose of explanation. It is to be understood that one of ordinary skill in the art can realize that the present application may be practiced without the use of these specific details. In other instances, well-known structures and procedures have not been described in detail so as not to obscure the description of the present application with unnecessary detail. Therefore, this application is not intended to be limited to the embodiments shown but is to be accorded the widest scope consistent with the principles and features disclosed herein.

Embodiments of the present application provide a medical image processing method, device, terminal, and storage medium, which will be described in detail below.

As shown in FIG. 1, FIG. 1 is a schematic diagram of a scene of a medical image processing system provided by an embodiment of the present application. The medical image processing system may include multiple terminals 100 and a server 200. The terminals 100 and the server 200 are connected to the network, and the server 200 integrates There is a medical image processing apparatus, such as the server in FIG. 1 , and the terminal 100 can access the server 200 .

In this embodiment of the present application, the server 200 is mainly configured to obtain a three-dimensional bile duct tree model corresponding to the bile duct tree in the magnetic resonance cholangiopancreatography information, and a three-dimensional marked bone structure model corresponding to the first marked bone structure, wherein the magnetic resonance cholangiopancreatography information is the magnetic resonance cholangiopancreatography information obtained by the patient during the abdominal magnetic resonance examination before the operation; the second landmark bony structure data corresponding to the second landmark bony structure in the X-ray image is obtained, the first landmark bony structure and the second landmark bony structure are obtained. The bone structure corresponding to the marked bone structure is the same, and the X-ray image is the X-ray image taken by the patient in real time during the operation; the cross-dimensional image matching is performed between the second marked bone structure data and the three-dimensional marked bone structure model, and the cross-dimensional image is obtained. Image matching information; based on the cross-dimensional image matching information, the three-dimensional bile duct tree model is projected onto the X-ray image to obtain target projection information.

In this embodiment of the present application, the server 200 may be an independent server, or may be a server network or server cluster composed of servers. For example, the server 200 described in the embodiment of the present application includes but is not limited to a computer, a network terminal, a single server A network server, a set of multiple network servers, or a cloud server composed of multiple servers. The cloud server is composed of a large number of computers or network servers based on cloud computing. In the embodiments of the present application, communication between the server and the terminal may be implemented in any communication manner, including but not limited to, based on the 3rd Generation Partnership Project (3GPP), Long Term Evolution (Long Term Evolution, LTE) , Worldwide Interoperability for Microwave Access (WiMAX) mobile communications, or computers based on TCP/IP Protocol Suite (TCP/IP), User Datagram Protocol (UDP) network communication, etc.

It can be understood that the terminal 100 used in the embodiments of the present application may be a device including both receiving and transmitting hardware, and a device having both receiving and transmitting hardware capable of performing bidirectional communication on a bidirectional communication link. Such terminals may include cellular or other communication devices with a single-line display or a multi-line display or a cellular or other communication device without a multi-line display. The specific terminal 100 may specifically be a desktop terminal or a mobile terminal, and the terminal 100 may specifically be one of a mobile phone, a tablet computer, a notebook computer, a medical auxiliary instrument, and the like.

Those skilled in the art can understand that the application environment shown in FIG. 1 is only an application scenario of the solution of the present application, and does not constitute a limitation on the application scenario of the solution of the present application. More or less terminals are shown, or the server network connection relationship, for example, only 1 server and 2 terminals are shown in FIG. 1 . It can be understood that the medical image processing system may also include one or more other servers, or/and one or more terminals connected to the server network, which is not specifically limited here.

In addition, as shown in FIG. 1 , the medical image processing system may further include a memory 300 for storing data, such as storing user magnetic resonance cholangiopancreatography data and percutaneous bile duct puncture navigation data, for example, transdermal Skin bile duct puncture navigation data.

It should be noted that the schematic diagram of the scene of the medical image processing system shown in FIG. 1 is only an example, and the medical image processing system and the scene described in the embodiments of the present application are for the purpose of illustrating the technical solutions of the embodiments of the present application more clearly, not This constitutes a limitation on the technical solutions provided by the embodiments of the present application. Those of ordinary skill in the art know that, with the evolution of medical image processing systems and the emergence of new business scenarios, the technical solutions provided by the embodiments of the present application are also subject to similar technical problems. Be applicable.

Next, the medical image processing method provided by the embodiment of the present application is introduced.

In the embodiment of the medical image processing method in this embodiment of the present application, a medical image processing apparatus is used as the execution body. For simplicity and convenience of description, the execution body will be omitted in subsequent method embodiments. The medical image processing apparatus is applied to a terminal, and the method includes: : Obtain the three-dimensional bile duct tree model corresponding to the bile duct tree and the three-dimensional landmark bone structure model corresponding to the first landmark bone structure in the magnetic resonance cholangiopancreatography information, wherein the magnetic resonance cholangiopancreatography information is the patient's abdominal magnetic resonance before surgery Magnetic resonance cholangiopancreatography information obtained during the examination; obtain the second marker bony structure data corresponding to the second marker bony structure in the X-ray image, the first marker bony structure and the bony structure corresponding to the second marker bony structure In the same way, the X-ray image is the X-ray image taken by the patient in real time during the operation; cross-dimensional image matching is performed between the second landmark bone structure data and the three-dimensional landmark bone structure model, and the cross-dimensional image matching information is obtained; based on the cross-dimensional image matching information, the three-dimensional bile duct tree model is projected onto the X-ray image, and the target projection information is obtained.

Please refer to FIG. 2 to FIG. 9. FIG. 2 is a schematic flowchart of an embodiment of the medical image processing method provided in the embodiment of the present application. The medical image processing method includes steps 201 to 204:

201. Acquire a three-dimensional biliary tree model corresponding to the bile duct tree in the magnetic resonance cholangiopancreatography information and a three-dimensional marked bone structure model corresponding to the first marked bone structure.

The magnetic resonance cholangiopancreatography information is the magnetic resonance cholangiopancreatography information obtained by the patient during abdominal magnetic resonance examination before surgery.

The magnetic resonance cholangiopancreatography information refers to MRCP scan data, and the MRCP scan data includes data such as 3D-MRCP sequence, normal phase/reverse phase/water sequence, T1 sequence, T2 sequence of LAVA-FLEX volume-enhanced scan.

202. Acquire second marker bone structure data corresponding to the second marker bone structure in the X-ray image.

Wherein, the first marked bony structure is the same as the bony structure corresponding to the second marked bony structure, and the X-ray image is an X-ray image taken by the patient in real time during the operation. Specifically, its bony structure can be understood as the type of bone, the first marked bony structure is the same as the bony structure corresponding to the second marked bony structure, and the type of bone is the same, the marked bony structure refers to the To a certain marked bone structure, the first marked bone structure and the second marked bone structure in the present application can select structures such as ribs, spine and the like as the marked bone structure.

It should be noted that the above-mentioned before and during the operation specifically refers to the state information when the relevant scan information of the patient is obtained. Specifically, the magnetic resonance cholangiopancreatography information is the magnetic resonance cholangiopancreatography information obtained by the patient during the abdominal magnetic resonance examination before the operation, and the X-ray image is the X-ray image taken by the patient in real time during the operation.

203. Perform cross-dimensional image matching between the second marker bone structure data and the three-dimensional marker bone structure model to obtain cross-dimensional image matching information.

Wherein, the cross-latitude image matching information may include target six-degree-of-freedom motion parameters.

204. Based on the cross-dimensional image matching information, project the three-dimensional bile duct tree model onto the X-ray image to obtain target projection information.

According to the target six-degree-of-freedom motion parameters in step 203, the current input X-ray image is used as the projection plane, and the three-dimensional bile duct tree model is projected onto the X-ray image, thereby obtaining target projection information.

Compared with the traditional method, the embodiment of the present application creatively proposes that the existing cross-latitude image is inaccurate, cannot provide effective guidance for subsequent operations, and increases the risk of the patient's operation. Cross-dimensional image matching is performed between the bone structure data and the three-dimensional landmark bone structure model to obtain cross-dimensional image matching information, and based on the cross-dimensional image matching information, the three-dimensional bile duct tree model is projected onto the X-ray image, which overcomes the cross-dimensional image matching. , which improves the matching accuracy of cross-latitude images.

Further, it breaks through the multi-dimensional image registration technology based on the same principle imaging method (ray density imaging), such as traditional CT and X-ray registration, and realizes different modalities (MRCP and X-ray), that is, between water proton imaging and ray density imaging. image matching problem.

In some embodiments of the present application, as shown in FIG. 3 , in step 201 , acquiring a three-dimensional bile duct tree model corresponding to the bile duct tree in the magnetic resonance cholangiopancreatography information, and a three-dimensional marker bony structure model corresponding to the first marked bony structure, Including steps 301 to 303:

301. Obtain magnetic resonance cholangiopancreatography information.

Specifically, magnetic resonance cholangiopancreatography imaging may be performed on the patient before the patient undergoes surgery to obtain magnetic resonance cholangiopancreatography information of the patient.

302. Build a three-dimensional bile duct tree model based on the bile duct tree data information in the magnetic resonance cholangiopancreatography information.

Among them, according to the above, the MRCP scan data includes 3D-MRCP sequence, normal phase/reverse phase/water sequence of LAVA-FLEX volume-enhanced scan, T1 sequence, T2 sequence, etc. Specifically, the 3D-MRCP sequence corresponding to the bile duct tree can be The MRCP sequence is imported into the preset volume rendering unit, and then the volume rendering is performed using the default preset of the MRI data, and the volume transparency is adjusted until the bile duct structure is clearly displayed, thereby constructing a three-dimensional bile duct tree model.

303. Build a three-dimensional marker bone structure model based on the first marker bone structure information in the magnetic resonance cholangiopancreatography information.

Among them, the selection of the first landmark bony structure is determined according to the specificity of the operation. During the percutaneous bile duct puncture operation, it is necessary to obtain relevant images of the upper abdomen of the patient. Due to its correspondence, the inventor of the present application has proved through experiments. , the spine and ribs are selected as the landmark bony structures, and the reference effect is the best.

In some embodiments of the present application, as shown in FIG. 4 , step 303 , constructing a three-dimensional marker bone structure model based on the first marker bone structure information in the magnetic resonance cholangiopancreatography information, includes steps 401 to 403 :

401. Identify a first landmark bone structure in the magnetic resonance cholangiopancreatography information.

402. Segment the first marker bony structure in the magnetic resonance cholangiopancreatography information to obtain first marker bony structure information.

Specifically, the OUTPHASE reversed-phase sequence data of the LAVA-FLEX volume-enhanced scan in the MRCP scan data is selected and imported into the volume rendering unit; a target localization algorithm can be selected to segment the first landmark bony structure in the magnetic resonance cholangiopancreatography information to obtain the first Sign bone structure information.

Take the segmentation of the first landmark bony structures such as the spine and ribs as an example: Use the Magnetic Resonance Imaging (MRI) spinal image intervertebral disc localization algorithm to segment the vertebral body and the intervertebral disc, so as to extract the contour of the spine, and carry out the body according to the extracted contour. Draw and obtain the vertebral structure; take the sternum as the anterior boundary and the anterior edge of the vertebral body obtained by the above segmentation as the posterior boundary, and use the automatic segmentation algorithm to obtain the region of interest (ROI, region of interest) for the voxels with the MRI value, and obtain the first landmark bony structure.

403. Based on the first marker bone structure information, construct a three-dimensional marker bone structure model.

Based on the first marker bone structure information obtained in step 402, volume rendering is performed in the ROI to construct a three-dimensional marker bone structure model.

In some embodiments of the present application, as shown in FIG. 5 , step 202 , acquiring second marker bony structure data corresponding to the second marker bony structure in the X-ray image, includes steps 501 to 505 :

501. Perform image preprocessing on the X-ray image.

In some embodiments of the present application, performing image preprocessing on the X-ray image includes: performing edge cropping processing on the X-ray image; performing image downsampling processing on the edge cropped X-ray image; downsampling the image The processed X-ray image is subjected to a grayscale range specification process.

502. Perform image enhancement processing on the X-ray image after image preprocessing.

503. Identify the second landmark bone structure in the X-ray image after image enhancement processing.

504. Segment the second marker bony structure to obtain a preliminary second marker bony structure.

Specifically, a Gaussian passivation mask threshold algorithm may be used to segment the second marker bony structure to obtain a preliminary second marker bony structure. For example, the ribs and vertebrae are preliminarily segmented with a Gaussian passivation mask thresholding algorithm.

505. Segment the preliminary second landmark bone structure after edge sampling to obtain second landmark bone structure data.

Specifically, a parabola fitting method may be used to segment the preliminary second landmark bone structure after edge sampling to obtain the second landmark bone structure data. For example, fine segmentation with parabolic fit after edge sampling of segmented ribs and vertebrae.

In some embodiments of the present application, as shown in FIG. 6 , in step 203, cross-dimensional image matching is performed between the second marker bone structure data and the three-dimensional marker bone structure model to obtain cross-dimensional image matching information, including steps 601 to 601 to Step 605:

601. Perform projection processing on the three-dimensional marker bony structure model to obtain a first two-dimensional image corresponding to the first marker bony structure.

Specifically, the first two-dimensional image of the marked bone structure is obtained by performing digitally reconstructed radiographs (DRR) projection using the three-dimensional marked bony structure model. Among them, DRR is a technique for reconstructing a plane of interest radiological image using tomographic data obtained from a CT or MRI scan by simulating the X-ray imaging process. The main method is the ray projection algorithm. By simulating the X-ray focus as the projection center point, the light passing through the 3D data field is converted into the grayscale of the X-ray image, and then the simulated X-ray photo is obtained. By controlling the parameters of generating DRR images, such as degrees of freedom, relative position of focus, etc., multi-angle reconstructed images of interest can be obtained from a set of tomographic data at a fixed level. In the embodiment of the present application, in the process of multi-dimensional image registration involving two-dimensional and three-dimensional data, DRR technology helps to reduce high-dimensional and multi-level grouped data into a single low-dimensional data, and at the same time obtain spatial freedom and arbitrariness.

602. Perform histogram equalization processing on the first two-dimensional image to obtain a second two-dimensional image.

Wherein, by performing histogram equalization processing on the first two-dimensional image, the contrast of the first two-dimensional image can be effectively improved to obtain the second two-dimensional image.

603. Perform edge enhancement processing on the structural edges in the second two-dimensional image to obtain a third two-dimensional image.

Specifically, a Sobel edge detector filter can be used to enhance the structural edges in the second two-dimensional image.

604. Calculate the similarity between the third two-dimensional image and the X-ray image.

Specifically, the normalized correlation function can be used to calculate the similarity between the third two-dimensional image and the X-ray image.

605. Adjust the six-degree-of-freedom motion parameter corresponding to the projection process to reach the target six-degree-of-freedom motion parameter, so as to maximize the similarity value corresponding to the similarity, and obtain cross-dimensional image matching information.

Wherein, the cross-latitude image matching information may include target six-degree-of-freedom motion parameters.

In some embodiments of the present application, as shown in FIG. 7 , in step 204, based on the cross-dimensional image matching information, the three-dimensional bile duct tree model is projected onto the X-ray image, and after the target projection information is obtained, the method further includes:

701. Acquire the positioning information of the puncture needle in the X-ray image during the operation.

702. Based on the positioning information and the target projection information, generate puncture direction adjustment information and position adjustment information for the puncture needle.

703. Perform puncture navigation for the puncture needle in the operation based on the puncture direction adjustment information and the position adjustment information.

Through the target projection information, the puncture needle under X-ray fluoroscopy during the operation can be positioned in the three-dimensional structure of the hepatic vein, portal vein and bile duct tree in real time, and the puncture direction and position of the puncture needle can be adjusted in real time, so as to puncture the puncture needle during the operation. navigation.

The embodiment of the present application breaks the limitation that traditional auxiliary tools are only improved for puncture needles and other instruments, and projects the three-dimensional bile duct tree model on the X-ray image based on the cross-dimensional image matching information, which overcomes the obstacle of cross-latitude image matching and improves the The matching accuracy of cross-latitude images directly provides visualized three-dimensional bile duct information for percutaneous bile duct puncture under X-ray, which reduces the difficulty of intraoperative puncturing of the puncture needle and reduces the intraoperative risk of patients.

In order to better implement the medical image processing method in the embodiment of the present application, on the basis of the medical image processing method, the embodiment of the present application further provides a medical image processing apparatus. As shown in FIG. 8 , the medical image processing apparatus 800 includes a first An acquisition unit 801, a second acquisition unit 802, a first cross-dimensional image matching unit 803, and a first projection unit 804:

The first obtaining unit 801 is configured to obtain a three-dimensional bile duct tree model corresponding to the bile duct tree in the magnetic resonance cholangiopancreatography information, and a three-dimensional marked bony structure model corresponding to the first marked bony structure, wherein the magnetic resonance cholangiopancreatography information is a patient Magnetic resonance cholangiopancreatography information acquired during an abdominal MRI before surgery.

The second obtaining unit 802 is configured to obtain second marker bony structure data corresponding to the second marker bony structure in the X-ray image, where the first marker bony structure is the same as the bony structure corresponding to the second marker bony structure, X A light image is an X-ray image of the patient taken in real time during surgery.

The first cross-dimensional image matching unit 803 is configured to perform cross-dimensional image matching between the second marked bone structure data and the three-dimensional marked bone structure model to obtain cross-dimensional image matching information.

The first projection unit 804 is configured to project the three-dimensional bile duct tree model onto the X-ray image based on the cross-dimensional image matching information to obtain target projection information.

In some embodiments of the present application, the first obtaining unit 801 specifically includes:

The third acquiring unit is configured to acquire magnetic resonance cholangiopancreatography information.

The first construction unit is configured to construct a three-dimensional bile duct tree model based on the bile duct tree data information in the magnetic resonance cholangiopancreatography information.

The second construction unit is configured to construct a three-dimensional marked bone structure model based on the first marked bone structure information in the magnetic resonance cholangiopancreatography information.

In some embodiments of the present application, the second building unit is specifically used for:

Identify the first landmark bony structures in magnetic resonance cholangiopancreatography information.

The first landmark bone structure in the magnetic resonance cholangiopancreatography information is segmented to obtain the first landmark bone structure information.

Based on the first landmark bone structure information, a three-dimensional landmark bone structure model is constructed.

In some embodiments of the present application, the second obtaining unit 802 specifically includes:

The first image preprocessing unit is used to perform image preprocessing on the X-ray image.

The first image enhancement processing unit is configured to perform image enhancement processing on the X-ray image after image preprocessing.

The first identification unit is used to identify the second landmark bone structure in the X-ray image after image enhancement processing.

The first segmentation unit is used for segmenting the second marked bone structure to obtain a preliminary second marked bone structure.

The preliminary second landmark bone structure after edge sampling is segmented to obtain second landmark bone structure data.

In some embodiments of the present application, the first cross-dimensional image matching unit 803 is specifically configured to:

The three-dimensional marker bone structure model is subjected to projection processing to obtain a first two-dimensional image corresponding to the first marker bone structure.

A histogram equalization process is performed on the first two-dimensional image to obtain a second two-dimensional image.

Perform edge enhancement processing on the structure edges in the second two-dimensional image to obtain a third two-dimensional image.

The similarity of the third two-dimensional image to the X-ray image is calculated.

Adjust the six-degree-of-freedom motion parameter corresponding to the projection processing to reach the target six-degree-of-freedom motion parameter, so as to maximize the similarity value corresponding to the similarity, and obtain cross-dimensional image matching information.

In some embodiments of the present application, the device is also used for:

Obtain the positioning information of the puncture needle in the X-ray image during the operation.

Based on the positioning information and the target projection information, puncture direction adjustment information and position adjustment information for the puncture needle are generated.

Based on the puncture direction adjustment information and the position adjustment information, puncture navigation is performed on the puncture needle during the operation.

In some embodiments of the present application, the first image preprocessing unit is specifically configured to:

Edge cropping of X-ray images.

Image downsampling is performed on the X-ray image after edge cropping.

The grayscale range specification process is performed on the X-ray image after the image downsampling process.

In the present application, the first acquisition unit 801 is used to acquire the three-dimensional bile duct tree model corresponding to the bile duct tree in the magnetic resonance cholangiopancreatography information and the three-dimensional marked bone structure model corresponding to the first marked bone structure, wherein the magnetic resonance cholangiopancreatography information is the magnetic resonance cholangiopancreatography information obtained when the patient performs abdominal magnetic resonance examination before surgery; then, the second obtaining unit 802 is configured to obtain the second marker bony structure data corresponding to the second marker bony structure in the X-ray image , the first marked bony structure is the same as the bony structure corresponding to the second marked bony structure, and the X-ray image is the X-ray image taken by the patient in real time during the operation; thirdly, the first cross-dimensional image matching unit 803 matches the second Cross-dimensional image matching is performed between the marked bone structure data and the three-dimensional marked bone structure model to obtain cross-dimensional image matching information; and, the first projection unit 804, based on the cross-dimensional image matching information, projects the three-dimensional bile duct tree model to the X-ray image , get the target projection information. Compared with traditional devices, under the background that the existing cross-latitude images are inaccurate, cannot provide effective guidance for subsequent operations, and increase the risk of patients during surgery, the present application creatively proposes to use the second landmark bony structure data. Perform cross-dimensional image matching with the three-dimensional landmark bone structure model to obtain cross-dimensional image matching information, and based on the cross-dimensional image matching information, the three-dimensional bile duct tree model is projected onto the X-ray image, which overcomes the obstacle of cross-latitude image matching and improves the matching accuracy across latitude images.

In addition to the above-mentioned methods and devices for medical image processing, the embodiments of the present application also provide a terminal that integrates any medical image processing device provided by the embodiments of the present application, and the terminal includes:

one or more processors;

memory; and

One or more application programs, wherein the one or more application programs are stored in the memory and configured to be performed by the processor to perform operations of any of the above-described medical image processing method embodiments.

The embodiments of the present application further provide a terminal, which integrates any medical image processing apparatus provided by the embodiments of the present application. Referring to FIG. 9, FIG. 9 is a schematic structural diagram of an embodiment of a terminal provided by an embodiment of the present application.

As shown in FIG. 9 , it shows a schematic structural diagram of a medical image processing apparatus designed in an embodiment of the present application, specifically:

The medical image processing apparatus may include a processor 901 of one or more processing cores, a storage unit 902 of one or more computer-readable storage media, a power supply 903 and an input unit 904 and other components. Those skilled in the art can understand that the structure of the medical image processing apparatus shown in FIG. 9 does not constitute a limitation on the medical image processing apparatus, and may include more or less components than the one shown, or combine some components, or different component layout. in:

The processor 901 is the control center of the medical image processing device, and uses various interfaces and lines to connect various parts of the entire medical image processing device, by running or executing the software programs and/or modules stored in the storage unit 902, and calling the storage unit 902. In the data storage unit 902, various functions of the medical image processing apparatus are executed and data is processed, so as to perform overall monitoring of the medical image processing apparatus. Optionally, the processor 901 may include one or more processing cores; preferably, the processor 901 may integrate an application processor and a modem processor, wherein the application processor mainly processes the operating system, user interface, and application programs, etc. , the modem processor mainly deals with wireless communication. It can be understood that, the above-mentioned modulation and demodulation processor may not be integrated into the processor 901.

The storage unit 902 can be used to store software programs and modules, and the processor 901 executes various functional applications and data processing by running the software programs and modules stored in the storage unit 902 . The storage unit 902 may mainly include a storage program area and a storage data area, wherein the storage program area may store an operating system, an application program required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the storage data area may store Data created according to the use of the medical image processing apparatus, etc. Additionally, storage unit 902 may include high-speed random access memory, and may also include non-volatile memory, such as at least one magnetic disk storage device, flash memory device, or other volatile solid state storage device. Accordingly, the storage unit 902 may also include a memory controller to provide the processor 901 with access to the storage unit 902 .

The medical image processing apparatus also includes a power supply 903 for supplying power to various components. Preferably, the power supply 903 can be logically connected to the processor 901 through a power management system, so as to manage charging, discharging, and power consumption management functions through the power management system. Power supply 903 may also include one or more DC or AC power sources, recharging systems, power failure detection circuits, power converters or inverters, power status indicators, and any other components.

The medical image processing apparatus may further include an input unit 904, which may be used to receive input numerical or character information and generate keyboard, mouse, joystick, optical or trackball signal input related to user settings and function control.

Although not shown, the medical image processing apparatus may further include a display unit and the like, which will not be described herein again. Specifically, in the embodiment of the present application, the processor 901 in the medical image processing apparatus loads the executable files corresponding to the processes of one or more application programs into the storage unit 902 according to the following instructions, and the processor 901 to run the application program stored in the storage unit 902, thereby realizing various functions, as follows:

Obtaining the three-dimensional bile duct tree model corresponding to the bile duct tree and the three-dimensional landmark bone structure model corresponding to the first landmark bone structure in the magnetic resonance cholangiopancreatography information, wherein the magnetic resonance cholangiopancreatography information is the abdominal magnetic resonance examination performed by the patient before the operation The magnetic resonance cholangiopancreatography information obtained at the time of acquisition; the second marker bony structure data corresponding to the second marker bony structure in the X-ray image is obtained, the first marker bony structure is the same as the bony structure corresponding to the second marker bony structure , the X-ray image is the X-ray image taken by the patient in real time during the operation; cross-dimensional image matching is performed between the second landmark bone structure data and the three-dimensional landmark bone structure model, and the cross-dimensional image matching information is obtained; based on the cross-dimensional image matching information , the three-dimensional bile duct tree model is projected onto the X-ray image to obtain the target projection information.

Compared with the traditional method, the embodiment of the present application creatively proposes that the existing cross-latitude image is inaccurate, cannot provide effective guidance for subsequent operations, and increases the risk of the patient's operation. Cross-dimensional image matching is performed between the bone structure data and the three-dimensional landmark bone structure model to obtain cross-dimensional image matching information, and based on the cross-dimensional image matching information, the three-dimensional bile duct tree model is projected onto the X-ray image, which overcomes the cross-dimensional image matching. , which improves the matching accuracy of cross-latitude images.

To this end, an embodiment of the present application provides a computer-readable storage medium, where the computer-readable storage medium may include: read only memory (ROM, Read Only Memory), random access memory (RAM, Random Access Memory), magnetic disk or CD etc. A plurality of instructions are stored in the computer-readable storage medium, and the instructions can be loaded by the processor to execute the steps in any of the medical image processing methods provided in the embodiments of the present application. For example, the instruction can perform the following steps:

Obtaining the three-dimensional bile duct tree model corresponding to the bile duct tree and the three-dimensional landmark bone structure model corresponding to the first landmark bone structure in the magnetic resonance cholangiopancreatography information, wherein the magnetic resonance cholangiopancreatography information is the abdominal magnetic resonance examination performed by the patient before the operation The magnetic resonance cholangiopancreatography information obtained at the time of acquisition; the second marker bony structure data corresponding to the second marker bony structure in the X-ray image is obtained, the first marker bony structure is the same as the bony structure corresponding to the second marker bony structure , the X-ray image is the X-ray image taken by the patient in real time during the operation; cross-dimensional image matching is performed between the second landmark bone structure data and the three-dimensional landmark bone structure model, and the cross-dimensional image matching information is obtained; based on the cross-dimensional image matching information , the three-dimensional bile duct tree model is projected onto the X-ray image to obtain the target projection information.

In the above-mentioned embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

The medical image processing method, device, terminal, and storage medium provided by the embodiments of the present application have been described in detail above. The principles and implementations of the present application are described with specific examples. The descriptions of the above embodiments are only It is used to help understand the method and the core idea of the present application; meanwhile, for those skilled in the art, according to the idea of the present application, there will be changes in the specific embodiments and application scope. It should be understood as a limitation of this application.

Claims (10)

1. A medical image processing method, wherein the method comprises: Obtaining the three-dimensional bile duct tree model corresponding to the bile duct tree and the three-dimensional landmark bone structure model corresponding to the first marked bone structure in the magnetic resonance cholangiopancreatography information, wherein the magnetic resonance cholangiopancreatography information is that the patient underwent abdominal magnetic resonance before the operation. Magnetic resonance cholangiopancreatography information obtained during the resonance examination; Acquiring second marker bony structure data corresponding to the second marker bony structure in the X-ray image, the first marker bony structure is the same as the bony structure corresponding to the second marker bony structure, and the X-ray image is the X-ray image taken by the patient in real time during the operation; performing cross-dimensional image matching on the second marker bone structure data and the three-dimensional marker bone structure model to obtain cross-dimensional image matching information; Based on the cross-dimensional image matching information, the three-dimensional bile duct tree model is projected onto the X-ray image to obtain target projection information. 2 . The medical image processing method according to claim 1 , wherein the acquiring the second marker bony structure data corresponding to the second marker bony structure in the X-ray image comprises: 2 . performing image preprocessing on the X-ray image; Perform image enhancement processing on the X-ray image after image preprocessing; Identify the second landmark bony structure in the X-ray image after image enhancement processing; segmenting the second marker bony structure to obtain a preliminary second marker bony structure; The preliminary second landmark bone structure after edge sampling is segmented to obtain second landmark bone structure data. 3. The medical image processing method according to claim 2, wherein the performing image preprocessing on the X-ray image comprises: performing edge cropping processing on the X-ray image; Perform image downsampling processing on the X-ray image after edge cropping processing; The grayscale range specification process is performed on the X-ray image after the image downsampling process. 4 . The medical image processing method according to claim 1 , wherein in the acquisition of the magnetic resonance cholangiopancreatography information, a three-dimensional bile duct tree model corresponding to the bile duct tree and a three-dimensional marker skeletal structure corresponding to the first marker skeletal structure are obtained. 5 . models, including: Obtain magnetic resonance cholangiopancreatography information; constructing a three-dimensional bile duct tree model based on the bile duct tree data information in the magnetic resonance cholangiopancreatography information; Based on the first marker bone structure information in the magnetic resonance cholangiopancreatography information, a three-dimensional marker bone structure model is constructed. 5. The medical image processing method according to claim 4, wherein the building a three-dimensional marker bone structure model based on the first marker bone structure information in the magnetic resonance cholangiopancreatography information, comprising: identifying the first landmark bony structure in the magnetic resonance cholangiopancreatography information; segmenting the first marker bone structure in the magnetic resonance cholangiopancreatography information to obtain first marker bone structure information; Based on the first marker bone structure information, a three-dimensional marker bone structure model is constructed. 6 . The medical image processing method according to claim 1 , wherein, performing cross-dimensional image matching on the second marker bony structure data and the three-dimensional marker bony structure model to obtain cross-dimensional image matching. 7 . information, including: subjecting the three-dimensional marker bony structure model to projection processing to obtain a first two-dimensional image corresponding to the first marker bony structure; performing histogram equalization processing on the first two-dimensional image to obtain a second two-dimensional image; performing edge enhancement processing on the structural edges in the second two-dimensional image to obtain a third two-dimensional image; calculating the similarity between the third two-dimensional image and the X-ray image; Adjust the six-degree-of-freedom motion parameter corresponding to the projection process to reach the target six-degree-of-freedom motion parameter, so as to maximize the similarity value corresponding to the similarity, and obtain cross-dimensional image matching information. 7. The medical image processing method according to claim 1, wherein, after projecting the three-dimensional bile duct tree model on the X-ray image based on the cross-dimensional image matching information, and obtaining target projection information, The method also includes: Obtaining the positioning information of the puncture needle in the X-ray image during the operation; generating puncture direction adjustment information and position adjustment information for the puncture needle based on the positioning information and the target projection information; Based on the puncture direction adjustment information and the position adjustment information, puncture navigation is performed on the puncture needle during the operation. 8. A medical image processing device, wherein the device comprises: A first acquisition unit, configured to acquire a three-dimensional bile duct tree model corresponding to the bile duct tree in the magnetic resonance cholangiopancreatography information, and a three-dimensional marked bone structure model corresponding to the first marked bone structure, wherein the magnetic resonance cholangiopancreatography information is: Magnetic resonance cholangiopancreatography information obtained during the patient's abdominal magnetic resonance examination before surgery; a second acquiring unit, configured to acquire second marker bony structure data corresponding to the second marker bony structure in the X-ray image, the first marker bony structure and the bony structure corresponding to the second marker bony structure The same, the X-ray image is the X-ray image taken by the patient in real time during the operation; a first cross-dimensional image matching unit, configured to perform cross-dimensional image matching on the second marker bone structure data and the three-dimensional marker bone structure model to obtain cross-dimensional image matching information; The first projection unit is configured to project the three-dimensional bile duct tree model onto the X-ray image based on the cross-dimensional image matching information to obtain target projection information. 9. A terminal, wherein the terminal comprises: one or more processors; memory; and one or more application programs, wherein the one or more application programs are stored in the memory and configured to be executed by the processor to implement the medical image processing of any one of claims 1 to 7 method. 10. A computer-readable storage medium, characterized in that a computer program is stored thereon, and the computer program is loaded by a processor to execute the method in the medical image processing method according to any one of claims 1 to 7. step.

Images