CN116172605A - An image registration method and an ultrasonic imaging system - Google Patents

Abstract

An image registration method and an ultrasonic imaging system register first image data and second image data of an inspected object in real time according to real-time pose information of an ultrasonic probe, real-time pose information of the inspected object and pose change information of the inspected object caused by breathing, so that an excellent registration effect can be achieved.

Description

An image registration method and an ultrasonic imaging system

technical field

The invention relates to an image registration method and an ultrasonic imaging system.

Background technique

One or more imaging systems can be used in clinical imaging of the object to be examined, so that medical personnel can obtain medical images of one or more modalities. The medical images of multiple modalities can be, for example, CT (Computed Tomography , computed tomography) images, MR (Magnetic Resonance, magnetic resonance) images, ultrasound images, etc. The principle of ultrasound fusion imaging navigation is to establish real-time ultrasound images and other data (such as three-dimensional ultrasound images, CT images or MR images) obtained in advance through spatial positioning devices (usually such as magnetic positioning sensors installed on the probe). The corresponding relationship achieves the complete fusion of the two images, realizes the joint guidance of the two images for the diagnosis and treatment process, and greatly improves the diagnostic confidence of clinicians and the effect of surgery.

During the diagnosis process, due to the possible displacement of the positioning system and the patient, etc., this will lead to incorrect medical image registration, resulting in poor image fusion.

Contents of the invention

In view of the above problems, the present invention proposes an image registration method and an ultrasonic imaging system, which will be described in detail below.

According to the first aspect, an image registration method is provided in an embodiment, including:

controlling the ultrasonic probe to transmit ultrasonic waves to the object under inspection and receiving ultrasonic echoes returned from the object under inspection to obtain ultrasonic echo signals, and obtain real-time ultrasonic image data of the object under inspection according to the ultrasonic echo signals;

Acquiring non-real-time detection image data of the inspected object;

Obtain the real-time pose information of the ultrasonic probe through the probe locator, and obtain the real-time pose information of the inspected object and the pose change information of the inspected object caused by breathing through the body surface locator; wherein, the The probe positioner is set on the ultrasonic probe, and the body surface positioner is used to set on the object to be examined;

Acquiring the real-time ultrasonic image data of the inspected object and the initial registration mapping relationship between the detected image data according to the real-time pose information of the ultrasonic probe;

Remove the pose change information caused by breathing from the real-time pose information of the detected object; according to the pose information of the detected object after removing the pose change information caused by respiration, the registration mapping relationship make corrections;

According to the corrected registration mapping relationship, registration is performed on the real-time ultrasonic image data and detection image data of the inspected object.

In one embodiment, obtaining information on pose changes of the subject under inspection caused by breathing through the body surface locator includes:

Acquiring information on pose changes of the inspected object caused by respiration within at least one respiration cycle.

In one embodiment, the acquisition of information on pose changes caused by respiration within at least one respiration cycle of the subject under inspection includes:

Determine the respiratory phase corresponding to the initial registration mapping relationship, and its corresponding pose information caused by breathing;

Determine each breathing phase in the breathing cycle and its corresponding pose information caused by breathing;

According to the respiratory phase corresponding to the initial registration mapping relationship and the corresponding pose information caused by breathing, each breathing phase in the breathing cycle and its corresponding pose information caused by breathing, A pose change matrix corresponding to each breath in the breathing cycle caused by breathing is calculated as the pose change information caused by breathing.

In one embodiment, the removing the pose change information caused by breathing from the real-time pose information of the detected object includes:

determining the real-time respiratory phase of the detected object;

According to the real-time breathing phase of the detected object, and the pose change information caused by breathing of the detected object within at least one breathing cycle, the real-time pose information of the detected object contained in the breathing The resulting pose change information is removed from the real-time pose information of the detected object.

In one embodiment:

According to the real-time breathing phase of the detected object and the pose change information caused by breathing of the detected object within at least one breathing cycle, the real-time pose information of the detected object The pose change information caused by breathing is removed from the real-time pose information of the detected object, including:

According to the real-time breathing phase of the detected object, and the pose change matrix corresponding to each breathing time in the breathing cycle caused by the breathing, the corresponding real-time breathing time of the detected object caused by breathing is obtained. Pose variation matrix;

The real-time pose information of the detected object includes the real-time pose matrix of the detected object, and the real-time pose matrix of the detected object is multiplied by the corresponding pose change when the detected object breathes in real time The inverse matrix of the quantity matrix is used to remove the pose change information caused by breathing from the real-time pose information of the detected object.

In an embodiment, the correcting the registration mapping relationship further includes: according to the pose change information of the inspected object caused by breathing, the ultrasonic image data changes caused by the inspected object caused by breathing Correction for changes in the registration mapping relationship.

In an embodiment, the image registration method further includes: controlling to perform fusion display or comparative display of the registered real-time ultrasound image data and the detection image data.

In an embodiment, the detection image data is three-dimensional ultrasound image data or image data of a different modality from the ultrasound image data.

According to the second aspect, an embodiment provides an image registration method, including:

Obtain real-time first image data of the inspected object through the ultrasonic probe;

acquiring second image data of the inspected object;

Obtaining real-time pose information of the ultrasonic probe;

Obtaining real-time pose information of the inspected object;

Obtaining information on pose changes of the inspected object caused by respiration;

According to the real-time pose information of the ultrasonic probe, the real-time pose information of the inspected object, and the pose change information of the inspected object caused by breathing, the real-time first image data and The second image data is registered.

In one embodiment, the acquisition of information on the pose change of the inspected object caused by breathing includes:

Acquiring information on pose changes of the inspected object caused by respiration within at least one respiration cycle.

In one embodiment, the acquisition of information on pose changes caused by respiration within at least one respiration cycle of the subject under inspection includes:

Determine the respiratory phase corresponding to the initial registration mapping relationship, and its corresponding pose information caused by breathing;

Determine each breathing phase in the breathing cycle and its corresponding pose information caused by breathing;

According to the respiratory phase corresponding to the initial registration mapping relationship and the corresponding pose information caused by breathing, each breathing phase in the breathing cycle and its corresponding pose information caused by breathing, A pose change matrix corresponding to each breath in the breathing cycle caused by breathing is calculated as the pose change information caused by breathing.

In an embodiment, the registering the real-time first image data and second image data of the inspected object includes:

Acquiring the real-time first image data of the detected object and the initial registration mapping relationship between the second image data according to the real-time pose information of the ultrasonic probe;

Correcting the registration mapping relationship according to the pose change information of the inspected object caused by breathing and the real-time pose information of the inspected object;

According to the corrected registration mapping relationship, the real-time first image data and second image data of the inspected object are registered.

In one embodiment, correcting the registration mapping relationship according to the pose change information of the inspected object caused by breathing and the real-time pose information of the inspected object includes:

correcting the real-time pose information of the inspected object according to the pose change information of the inspected object caused by breathing;

The registration mapping relationship is corrected according to the corrected real-time pose information of the inspected object.

In one embodiment, the correcting the real-time pose information of the inspected object according to the pose change information of the inspected object caused by breathing includes:

From the real-time pose information of the detected object, the pose change information caused by respiration is identified and eliminated.

In one embodiment, the identifying and removing the pose change information caused by breathing from the real-time pose information of the detected object includes:

determining the real-time respiratory phase of the detected object;

According to the real-time breathing phase of the detected object, and the pose change information caused by breathing of the detected object within at least one breathing cycle, the real-time pose information of the detected object contained in the breathing The resulting pose change information is removed from the real-time pose information of the detected object.

In one embodiment, the real-time pose of the detected object is calculated according to the real-time breathing phase of the detected object and the pose change information of the detected object caused by breathing within at least one breathing cycle. The pose change information caused by breathing contained in the information is removed from the real-time pose information of the detected object, including:

According to the real-time breathing phase of the detected object, and the pose change matrix corresponding to each breathing time in the breathing cycle caused by the breathing, the corresponding real-time breathing time of the detected object caused by breathing is obtained. Pose variation matrix;

The real-time pose information of the detected object includes the real-time pose matrix of the detected object, and the real-time pose matrix of the detected object is multiplied by the corresponding pose change when the detected object breathes in real time The inverse matrix of the quantity matrix is used to remove the pose change information caused by breathing from the real-time pose information of the detected object.

In one embodiment, the correcting the registration mapping relationship according to the pose change information of the inspected object caused by breathing and the real-time pose information of the inspected object further includes:

Correcting the change of the registration mapping relationship caused by the change of the first image data of the inspected object caused by the breathing according to the pose change information of the inspected object caused by the breathing.

In an embodiment, the image registration method further includes: controlling the fusion display or comparative display of the registered real-time first image data and the second image data, wherein the first image data includes Two-dimensional ultrasound image data, the second image data includes three-dimensional ultrasound image data or image data of a different modality from the ultrasound image data.

According to a third aspect, an embodiment provides an ultrasound imaging system, comprising:

The ultrasonic probe is used to transmit ultrasonic waves to the detected object and receive corresponding ultrasonic echoes to obtain ultrasonic echo signals;

The transmitting and receiving control circuit is used to control the ultrasonic probe to perform ultrasonic emission and ultrasonic echo signal reception;

A probe locator, arranged on the ultrasonic probe, for obtaining real-time pose information of the ultrasonic probe;

A body surface locator, configured to be installed on the inspected object to obtain real-time pose information of the inspected object;

A processor, configured to execute the image registration method described in any embodiment herein.

According to a fourth aspect, an embodiment provides a computer-readable storage medium, the computer-readable storage medium stores a program, and the program can be executed by a processor to implement the method described in any embodiment herein .

According to the image registration method, ultrasonic imaging system and computer-readable storage medium of the above-mentioned embodiments, according to the real-time pose information of the ultrasonic probe, the real-time pose information of the inspected object, and the pose of the inspected object caused by breathing The change information is used to register the real-time first image data and the second image data of the inspected object, so that an excellent registration effect can be achieved.

Description of drawings

FIG. 1 is a schematic diagram of a registration principle of an embodiment;

Fig. 2 is a schematic diagram of the registration principle of an embodiment;

Fig. 3 is a schematic structural diagram of an ultrasonic imaging system of an embodiment;

Fig. 4 is a flowchart of an image registration method of an embodiment;

FIG. 5 is a flowchart of an image registration method of an embodiment;

6 is an embodiment of the real-time pose information of the ultrasonic probe, the real-time pose information of the inspected object, and the pose change information of the inspected object caused by breathing. A further schematic diagram of the real-time registration of the first image data and the second image data of the object;

Fig. 7 is a further schematic diagram of an embodiment of identifying and removing the pose change information caused by breathing contained in the pose information of the detected object at the current moment from the pose information of the detected object at the current moment;

FIG. 8 is a flowchart of an image registration method in an embodiment;

FIG. 9 is a flow chart of an image registration method in an embodiment;

Fig. 10 is a further schematic diagram of an embodiment of registering the real-time ultrasound image data and detection image data of the object under inspection;

Fig. 11 is a further schematic diagram of an embodiment of performing a second type of correction on the registration mapping relationship according to the pose change information caused by breathing.

Detailed ways

The present invention will be further described in detail below through specific embodiments in conjunction with the accompanying drawings. Wherein, similar elements in different implementations adopt associated similar element numbers. In the following implementation manners, many details are described for better understanding of the present application. However, those skilled in the art can readily recognize that some of the features can be omitted in different situations, or can be replaced by other elements, materials, and methods. In some cases, some operations related to the application are not shown or described in the description, this is to avoid the core part of the application being overwhelmed by too many descriptions, and for those skilled in the art, it is necessary to describe these operations in detail Relevant operations are not necessary, and they can fully understand the relevant operations according to the description in the specification and general technical knowledge in the field.

In addition, the characteristics, operations or characteristics described in the specification can be combined in any appropriate manner to form various embodiments. At the same time, the steps or actions in the method description can also be exchanged or adjusted in a manner obvious to those skilled in the art. Therefore, the various sequences in the specification and drawings are only for clearly describing a certain embodiment, and do not mean a necessary sequence, unless otherwise stated that a certain sequence must be followed.

The serial numbers assigned to components in this document, such as "first", "second", etc., are only used to distinguish the described objects, and do not have any sequence or technical meaning. The "connection" and "connection" mentioned in this application all include direct and indirect connection (connection) unless otherwise specified.

Image fusion functions such as multimodal image fusion functions can be realized based on the magnetic navigation positioning system, specifically, by installing a magnetic positioning sensor Sensor (hereinafter referred to as the probe Sensor) on the ultrasonic probe to track and locate the spatial position of the ultrasonic probe in real time, using The face-to-face registration algorithm completes the real-time two-dimensional ultrasound image, and other three-dimensional volume data (such as three-dimensional ultrasound image data or image data of a different modality from the ultrasound image data, wherein the image data of a different modality from the ultrasound image data can be a CT image data or MR image data, etc.) Referring to FIG. 1 , it may be illustrated by the registration principle between ultrasound images and CT/MR images. Understandably, the registration principle between real-time 2D ultrasound images and 3D ultrasound image data is also similar. The formula for registration of ultrasound images and CT/MR images is:

T=P×R×A;

The above formula may be named as formula (1). The parameters in formula (1) have the following physical meanings: T represents the pose matrix from the ultrasound image coordinate system to the CT/MR image coordinate system; P represents the magnetic field generation in the magnetic navigation positioning system The pose matrix from the coordinate system of the sensor to the CT/MR image coordinate system; R represents the pose matrix from the sensor coordinate system of the probe to the magnetic field generator coordinate system, and the R matrix can be obtained in real time through the magnetic navigation positioning system; A represents the ultrasound image The pose matrix from the coordinate system to the sensor coordinate system of the probe, the A matrix can be obtained through the mechanical size parameters of the probe, and is known. When the ultrasound parameters (such as imaging depth, ROI frame position, three-dimensional scanning range, etc.) are constant, the A matrix is fixed. During the diagnosis process, the operator generally moves the ultrasound probe to perform real-time ultrasound imaging. During this process, the A matrix and P matrix generally remain unchanged, and the R matrix will change with the movement of the ultrasound probe, but as above As mentioned above, the R matrix can be obtained through the magnetic navigation and positioning system.

Therefore, a registration process is as follows:

First perform initial registration to calculate the P matrix. Specifically, the operator selects an ultrasound image, then finds the corresponding section in the CT/MR image, and then superimposes the two, and makes the ultrasound image and the corresponding section through translation or rotation The same anatomical structure in the CT/MR image is superimposed to obtain the T matrix at this time, and then the P matrix can be calculated by the formula P=T×A ⁻¹ ×R ⁻¹ . After obtaining the P matrix, since the A matrix is known, the R matrix can be obtained in real time through the magnetic navigation and positioning system, so at this time, the real-time registration and fusion display of the ultrasound image and the CT/MR image can be realized through the formula (1) .

During the real-time image registration process, if the patient’s body position changes greatly after the initial registration is completed, or someone on site (doctor or nurse, etc.) accidentally touches the magnetic field generator, the position of the magnetic field generator This will lead to the fact that the P matrix calculated by the initial registration cannot faithfully reflect the new pose relationship between the current magnetic field generator and the CT/MR image, so it is necessary to perform a new registration to obtain a new Matrix P. For example: after the initial registration, the spatial position of the ultrasound probe remains unchanged, but the patient moves toward the sole of the foot. At this time, the ultrasound probe may move from the patient’s abdomen to the patient’s intercostal position, and the ultrasound image changes. In the formula T=P×R×A used for system real-time registration, since the P, R, A and other matrices do not change from the perspective of the system, the corresponding CT/MR image slices always remain unchanged, which is displayed as patient translation For the previous section images, the registration result is invalid; therefore, if the doctor wants to restore the correct fusion effect, he has to re-register, which will take up valuable operation time for the doctor and the patient, and bring a poor user experience for the doctor.

For the fusion failure problem caused by the change of the patient's body position or the position of the magnetic field generator after registration, the body position tracking function can be introduced to detect the change of the patient's body position and the position of the magnetic field generator in real time and correct them. Referring to FIG. 2 , it may still be illustrated by taking the registration and fusion between ultrasound images and CT/MR images based on the magnetic navigation positioning system as an example. A new sensor is introduced into the magnetic navigation positioning system, which may be called the body surface sensor. The body surface sensor is fixed on the skin surface of the patient's sternum or belly, so that it is firmly connected with the patient. Its function is to obtain body position change signals in real time. The signal represents the relative position change between the patient and the magnetic field generator, including the position change of the patient itself and the position change of the magnetic field generator. After solving the matrix P through the initial registration, the doctor can click the button to start the body position tracking function, and the system records the pose matrix R of the body surface Sensor coordinate system relative to the magnetic field generator coordinate system at this time according to the data of the magnetic navigation system _p0 , using the matrix P and R _p0 , the pose matrix M of the body surface sensor coordinate system relative to the CT/MR image coordinate system can be calculated:

M=P×R _p0 ;

Since the body surface Sensor is fixedly connected to the patient, the spatial relationship between the body surface Sensor coordinate system and the CT/MR image coordinate system will not change with the patient's position or the position of the magnetic field generator. Therefore, when the patient's position or the magnetic field generator When the position changes, the matrix M always remains unchanged. At this time, the calculation formula of the new pose matrix T' of the ultrasound image coordinate system relative to the CT/MR image coordinate system is:

You may wish to name the above formula as formula (2). In formula (2), R _p0 represents the moment when the doctor turns on the body position tracking function or the initial registration moment, and the position of the body surface Sensor coordinate system recorded by the system relative to the magnetic field generator coordinate system It is understandable that this will not change with the position of the patient or the position of the magnetic field generator, and is a fixed value; while R _p represents the real-time pose matrix of the body surface Sensor coordinate system relative to the magnetic field generator coordinate system, It will change with the patient's body position or the position of the magnetic field generator, so the body position change signal can be fed back in real time. Formula (2) is a new registration process for real-time two-dimensional ultrasound images and three-dimensional volume data such as CT/MR images under body position tracking, which can effectively solve the problems caused by changes in the patient's body position or the position of the magnetic field generator after face-to-face registration. Fusion failure problem.

The prerequisite for the body position tracking function to take effect is that the body surface Sensor is required to be firmly connected to the patient. The purpose is to ensure that there is no relative movement between the body surface Sensor and the patient during the body position tracking process. The applicant found that in clinical practice, in order to ensure that the sticking position is firm and close to the site of liver surgery, the body surface Sensor is usually pasted on the patient's breastbone surface skin or belly, and the body surface Sensor will follow the patient's breathing when the position tracking function takes effect. Movement and movement, which will cause redundant breathing motion signals in the body position change signal detected by the body surface Sensor during body position tracking, which introduces new registration errors for real-time registration and fusion functions under body position tracking. For example: after registration and fusion, the doctor turns on the body position tracking function, and the ultrasound probe is fixed to collect two-dimensional ultrasound images. At this time, the patient's body position and the position of the magnetic field generator do not change. The CT/MR section image should remain unchanged, that is, the fusion effect remains unchanged, but since the body surface Sensor will move up and down with the patient's breathing, that is, the R _p matrix in formula (2) moves with the patient's breathing space The position changes in real time, causing the new pose matrix T' to change in real time, which is finally reflected in the real-time change of the CT/MR section image corresponding to the ultrasound image. The fusion effect is good and bad, causing unnecessary interference to the doctor's diagnosis and treatment.

Based on the above understanding, the applicant proposes an image registration method for the problem of poor fusion effect caused by redundant breathing motion signals in the body position change signals detected by the body surface Sensor:

(1) Fix the Sensor on the patient's body surface and the probe respectively, and track the spatial position of the Sensor;

(2) Obtain 3D volume data and complete the initial registration with real-time 2D ultrasound; the 3D volume data here can be 3D ultrasound image data, or image data of a different modality from ultrasound image data, such as CT image data or MR image data, etc.;

(3) Collect body surface Sensor positioning information for a period of time to extract respiratory cycle information, such as the pose information of the body surface Sensor at all phases of the patient's entire respiratory cycle under static conditions;

(4) Remove the influence of the breathing signal from the real-time positioning signal of the body surface Sensor according to the above breathing cycle information, so as to obtain body position change information after removing the breathing signal;

(5) Based on the above body position change information, the registration mapping relationship between the ultrasound and the three-dimensional volume data is corrected in real time.

The principle of each step is further analyzed and explained below.

(1) Fix the Sensor on the patient's body surface and the probe respectively, and track the spatial position of the Sensor.

Sensors are respectively fixed on the ultrasound probe and on the patient's body surface. The sensor fixed on the ultrasound probe may be called a probe sensor, and the sensor fixed on the patient's body surface may be called a body surface Sensor. The probe sensor is used to track the spatial position of the ultrasound probe in real time. The body surface sensor is firmly pasted on the patient's breastbone surface skin or belly to detect body position changes in real time. The two sensor sensors must always be in the position of the positioning system during the entire image fusion process. within the precise measurement range.

A positioning system such as a magnetic navigation system, in which the magnetic field generator can track the spatial position of each magnetic positioning Sensor in the magnetic field in real time, that is, the pose relationship of each sensor Sensor coordinate system relative to the magnetic field generator coordinate system, using a set of parameters (q ₀ ,q ₁ ,q ₂ ,q ₃ ,x,y,z) to represent, where (q ₀ ,q ₁ ,q ₂ ,q ₃ ) represents the attitude of the Sensor coordinate system relative to the magnetic field generator coordinate system Information, which is a unit quaternion, (x, y, z) indicates the position information of the Sensor coordinate system relative to the magnetic field generator coordinate system. Based on the parameters (q ₀ ,q ₁ ,q ₂ ,q ₃ ,x,y,z), the pose matrix R of the Sensor coordinate system relative to the magnetic field generator coordinate system can be calculated, which is a 4*4 homogeneous matrix , the calculation formula (maybe denoted as formula (3)) is:

At any point in the Sensor coordinate system, assuming that its coordinates are (x ₀ , y ₀ , z ₀ ) ^T , it is transformed into a homogeneous coordinate (x ₀ , y ₀ , z ₀ ,1) ^T and then multiplied by the R matrix to the left. The homogeneous coordinates of the point in the coordinate system of the magnetic field generator can be obtained.

(2) Acquire 3D volume data and complete initial registration with real-time 2D ultrasound.

Import the patient's preoperative three-dimensional volume data such as three-dimensional ultrasound, CT, MR or PET images into the ultrasound system, and the doctor browses the images of each slice to find slices with unique structures such as vascular bifurcations and tumors as the volume data to be registered layer image. The doctor uses the ultrasound probe installed with the magnetic positioning Sensor to collect and browse the real-time two-dimensional ultrasound image, find the two-dimensional ultrasound image consistent with the anatomical structure of the volume data slice image to be registered and freeze it as the two-dimensional ultrasound image to be registered .

Since the probe Sensor is fixedly connected to the ultrasound probe, the pose relationship between the two-dimensional ultrasound image coordinate system and the probe Sensor coordinate system remains unchanged, so the pose matrix A from the ultrasound image coordinate system to the probe Sensor coordinate system can be obtained by The mechanical size parameters of the probe are obtained, which is also a 4*4 homogeneous matrix. For any pixel point (x _U , y _U , z _U , 1) ^T in the ultrasound image coordinate system, the coordinates of the point in the sensor coordinate system of the probe can be obtained by multiplying the A matrix to the left. Based on the positioning information of the probe Sensor corresponding to the frozen two-dimensional ultrasound image to be registered, the R matrix of the probe Sensor at the time of freezing can be calculated by using the above formula (3), and the ultrasound image to be registered can be obtained according to matrix A and matrix R The coordinates (x _M ,y M ,z _M ,1) ^T of any pixel point (x _U ,y _U ,z _{U ,} 1) ^T in the image coordinate system in the magnetic field generator coordinate system, the calculation formula (may as well remember For formula (4)) is:

The volume data layer image to be registered and the 2D ultrasound image are superimposed and displayed with different transparency. The doctor manually translates and rotates the volume data layer image to be registered or the 2D ultrasound image to be registered, and aligns the two images. For the same anatomical structure such as blood vessels or tumors in two images, the pose relationship of the 2D ultrasound image to be registered relative to the 3D volume data can be obtained at this time, that is, the T matrix mentioned above, based on the above formula (1) T=P×R×A, then the calculation formula of the pose matrix P from the magnetic field generator coordinate system to the three-dimensional volume data coordinate system (may be denoted as formula (5)) is:

P＝T×A ^-1 ×R ^-1

After the P matrix is calculated, it can be seen from the schematic diagram in Figure 1 that when the doctor moves the ultrasound probe in real time to collect two-dimensional ultrasound images, the ultrasound system calculates the volume data corresponding to the current two-dimensional ultrasound image in real time according to the formula T=P×R×A Slice image, any pixel point (x _U , y _U , z _U , 1) ^T in the current ultrasound image coordinate system corresponds to the coordinates of the same point in the three-dimensional volume data coordinate system (x _V , y _V , z _V , 1) The formula for calculating ^T (maybe denoted as formula (6)) is:

Adjust the transparency of the current ultrasound image and the corresponding three-dimensional volume data slice image to superimpose and display them, or compare and display them side by side. At this time, the registration and fusion of real-time two-dimensional ultrasound images and three-dimensional volume data is completed, which can provide doctors with, for example, Multimodal image fusion navigation facilitates doctors to complete minimally invasive operations such as percutaneous ablation.

(3) Collect body surface Sensor positioning information for a period of time to extract respiratory cycle information.

其中1≤n≤N。Assuming that the patient’s respiratory phase is ψ ₀ when the initial registration is completed, the patient is guaranteed to keep breathing calmly and the body position remains unchanged. When the patient’s respiratory phase reaches ψ ₀ , the doctor informs the system to start collecting the body surface Sensor by means such as pressing a button. The doctor can customize the acquisition time, as long as the period includes at least one complete breathing cycle, preferably, it can be multiple (for example, greater than or equal to 3) breathing cycles of the patient. In an example, take N complete breathing cycles, where N is greater than or equal to 3; then according to the positioning information of the body surface Sensor provided by the magnetic navigation system, for each breathing phase ψ _t , N sets of parameters will be collected

where 1≤n≤N.

The physical meaning of each unit quaternion q=(q ₀ ,q ₁ ,q ₂ ,q ₃ ) represents a number of angles rotated around a certain rotation axis, which can be expressed as an ordered pair of scalars and vectors:

表示该四元数的旋转轴，为单位向量，另外，/>

而θ表示旋转的角度，其计算公式(不妨记为公式(8))为：The above formula may be recorded as formula (7), in formula (7),

Represents the rotation axis of the quaternion, which is a unit vector, and, in addition, />

And θ represents the angle of rotation, and its calculation formula (maybe denoted as formula (8)) is:

θ = 2 arccosq ₀

的计算公式(不妨记为公式(9))为：where the axis of rotation

The calculation formula of (maybe denoted as formula (9)) is:

参数，根据其中的N组四元数/>

可得到N个旋转角度θ⁽ⁿ⁾和旋转轴/>

由于患者平静呼吸，可认为N组数据的旋转轴和旋转角度之间差别很小，可通过计算它们的参数均值获取对于呼吸时相ψ_t的精确旋转角度θ_t和旋转轴/>

θ_t的计算公式(不妨记为公式(10))为：Therefore, the N groups collected for each respiratory phase ψ _t

Parameters, according to the N groups of quaternions />

N rotation angles θ ⁽ⁿ⁾ and rotation axes can be obtained />

Since the patient breathes calmly, it can be considered that the difference between the rotation axis and the rotation angle of the N sets of data is very small, and the precise rotation angle θ _t and the rotation axis for the respiratory phase ψ _t can be obtained by calculating the mean value of their parameters />

The calculation formula of θ _t (may be denoted as formula (10)) is:

的计算公式(不妨记为公式(11))为：

The calculation formula of (maybe denoted as formula (11)) is:

即可得到针对每个呼吸时相ψ_t的精确单位四元数q_t＝(q_0t,q_1t,q_2t,q_3t)，其计算公式(不妨记为公式(12))为：Among them, Norm represents the normalization function, which can normalize the vector into a unit vector with a modulus length of 1. Based on precise rotation angle θ _t and rotation axis

The precise unit quaternion q _t =(q _0t ,q _1t ,q _2t ,q _3t ) for each breathing phase ψ _t can be obtained, and its calculation formula (may be recorded as formula (12)) is:

According to the N sets of translation parameters (x ⁽ⁿ⁾ , y ⁽ⁿ⁾ , z ⁽ⁿ⁾ ) collected for each respiratory phase ψ _t , the precise displacement vector d _t for the respiratory phase ψ _t can be obtained by calculating the mean value of the parameters =(x _t , y _t , z _t ), the calculation formula (maybe denoted as formula (13)) is:

Based on the exact unit quaternion q _t =(q _0t ,q _1t ,q _2t ,q _3t ) and the exact displacement vector d _t =(x _t ,y _t ,z _t ) for each respiratory phase ψ _t , using The above formula (3) can calculate the precise pose matrix R _t of the body surface Sensor coordinate system relative to the magnetic field generator coordinate system at the moment of respiratory phase ψ _t , and the calculation formula (may be denoted as formula (14)) is:

Assuming that when the initial registration is completed, the respiratory phase of the patient is ψ ₀ , and the pose matrix of the body surface Sensor coordinate system relative to the magnetic field generator coordinate system is R ₀ , then the body surface Sensor is _relatively The formula for calculating the pose change ΔR _t at the moment of respiratory phase ψ ₀ (may be denoted as formula (15)) is:

Therefore, for each respiratory phase ψ _t in the entire respiratory cycle, there is a unique body surface Sensor pose change ΔR _t corresponding to it, which means that the body surface Sensor at the moment of respiratory phase ψ _t is due to the patient’s respiratory movement The changes in the spatial position and posture, and the changes in the posture and posture of the body surface Sensor in all phases during the entire respiratory cycle are the respiratory cycle information of the patient.

(4) Eliminate the influence of the respiratory signal from the real-time positioning signal of the body surface Sensor according to the above respiratory cycle information, so as to obtain body position change information after removing the respiratory signal.

As mentioned above, the real-time positioning signal of the body surface sensor can be represented by the matrix _Rp , which represents the real-time pose matrix of the body surface sensor coordinate system relative to the magnetic field generator coordinate system, which will change with the patient's body position or the position of the magnetic field generator And change. In the process of body position tracking, the detected real-time positioning signal of the body surface Sensor includes not only body position change signals (patient body position changes and magnetic field generator position changes), but also redundant respiratory motion signals. The time phase is ψ _t , and the pose matrix of the body surface Sensor coordinate system relative to the magnetic field generator coordinate system is R _mt . Then the real-time positioning pose information R _p of the body surface Sensor can be considered as body position change and The compounding of respiratory movement is the following formula (may be recorded as formula (16)):

R _p =R _mt ×ΔR _t

Therefore, if you want to obtain the body position change information at this time, you only need to remove the respiratory cycle signal from the real-time positioning signal of the body surface Sensor, then the position of the body surface Sensor coordinate system representing the body position change information at this time relative to the magnetic field generator coordinate system The calculation formula of the attitude matrix R _mt (may be denoted as formula (17)) is:

R _mt =R _p ×(ΔR _t ) ^-1

Formula (17) can be understood as the body position change information after removing the respiratory cycle signal from the real-time positioning signal of the body surface Sensor at any time of the respiratory phase _ψ t is a matrix R _mt . Since the pose variation ΔR of all time phases in the entire respiratory cycle is known, the respiratory signal ΔR can be removed from the real-time positioning signal R _p of the body surface Sensor according to the patient's respiratory phase during the body tracking process, and only the Body position change information R _m .

(5) Based on the above body position change information, the registration mapping relationship between the ultrasound and the three-dimensional volume data is corrected in real time.

As in the body position tracking registration process described above, the formula (2) indicates that the real-time pose matrix R _p of the body surface Sensor coordinate system relative to the magnetic field generator coordinate system is mixed with respiratory cycle information, so it is the body position tracking process The registration relationship (or registration mapping relationship) between real-time two-dimensional ultrasound images and three-dimensional volume data such as CT/MR images introduces errors. In order to eliminate the influence of the patient's respiratory movement on body position tracking, the R _p matrix in formula (2) is replaced by the matrix R _m that only represents body position change information, so that the respiratory signal can be removed in real time during the body position tracking process, ensuring real-time Good registration relationship between 2D ultrasound images and 3D volume data such as CT/MR images. The registration formula (may be denoted as formula (18)) under body position tracking after removing the influence of breathing motion is:

In the body position tracking process, based on the current respiratory phase of the patient, the body position change signal after removing the respiratory signal can be obtained according to formula (17), and then the real-time two-dimensional ultrasound image and CT/MR image can be realized according to the registration process of formula (18) The registration of three-dimensional volume data can ensure that after the registration fusion, even if the patient's body position changes or the position of the magnetic field generator moves, a good registration effect can still be maintained.

It can be seen that, compared with the prior art, the present invention optimizes the pose information of the body surface Sensor during the registration process, and can detect the position change signal detected by the body surface Sensor during the body position tracking process. Excess respiratory motion signals are eliminated in real time, thereby effectively removing the influence of patient respiratory motion on body position change information (patient body position changes and magnetic field generator position changes), ensuring real-time two-dimensional ultrasound images under body position tracking and three-dimensional ultrasound, CT, MR or PET Good registration and fusion effect of 3D volume data.

The above is an example and description based on the registration method of the magnetic navigation and positioning system. It is understandable that the magnetic navigation system and the positioning Sensor can be replaced by the optical positioning system and the positioning Marker (marker point), and the Marker on the patient's body surface can also be Replace it by drawing markers on the patient's body surface, the specific scheme is as follows:

(1) Fix or mark the Marker on the patient's body surface and the probe, and use the optical camera to locate the spatial position of the probe and the patient;

(2) Acquire 3D volume data and complete the initial registration with real-time 2D ultrasound;

(3) Analyze the video of the patient's body surface Marker movement collected by the optical camera for a period of time to extract respiratory cycle information, such as the pose information of the patient's body surface Sensor at all phases during the entire respiratory cycle under static conditions;

(4) Remove the influence of the breathing signal from the real-time positioning signal of the body surface Marker by the optical camera according to the above breathing cycle information, so as to obtain the body position change information after removing the breathing signal;

(5) Based on the above body position change information, the registration mapping relationship between the ultrasound and the three-dimensional volume data is corrected in real time.

Please refer to FIG. 3 , an ultrasonic imaging system is disclosed in some embodiments of the present invention, which includes an ultrasonic probe 10 , a transmitting/receiving control circuit 20 , an echo processing module 30 , a processor 40 and a display module 50 . parts are described.

The ultrasonic probe 10 can be a matrix probe or a four-dimensional probe with a mechanical device, which is not limited in the present invention, as long as the ultrasonic probe used can obtain ultrasonic echo signals or data of the target area of the examinee. In some embodiments, the ultrasonic probe acquires a set of four-dimensional image data (ie dynamic three-dimensional ultrasonic images) or acquires a volume of three-dimensional ultrasonic image data, and the obtained ultrasonic image data needs to have relatively clear structural information such as blood vessels. In some specific embodiments, the ultrasonic probe 10 includes a plurality of array elements, which are used to realize mutual conversion between electrical pulse signals and ultrasonic waves, so as to transmit ultrasonic waves to the detected biological tissue 60 (such as biological tissue in a human body or an animal body) and receive ultrasonic waves. Ultrasonic echoes reflected back by tissues to obtain ultrasonic echo signals. The multiple array elements included in the ultrasonic probe 10 can be arranged in a row to form a linear array, or arranged in a two-dimensional matrix to form a planar array, and these multiple array elements can also form a convex array. The array element can emit ultrasonic waves according to the excitation electrical signal, or convert the received ultrasonic waves into electrical signals. Therefore, each array element can be used to transmit ultrasonic waves to biological tissues in the region of interest, and can also be used to receive ultrasonic echoes returned through the tissues. When performing ultrasonic testing, it is possible to control which array elements are used to transmit ultrasonic waves and which array elements are used to receive ultrasonic waves through the transmitting sequence and receiving sequence, or control the array elements to transmit ultrasonic waves or receive ultrasonic echoes in time slots. All array elements participating in ultrasonic emission can be excited by electrical signals at the same time, so as to emit ultrasonic waves at the same time; or the array elements participating in ultrasonic emission can also be excited by several electrical signals with a certain time interval, so as to continuously emit ultrasonic waves with a certain time interval.

The transmission/reception control circuit 20 is used to control the ultrasonic probe 10 to perform ultrasonic transmission and ultrasonic echo signal reception. Specifically, the transmission/reception control circuit 20 is used to control the ultrasonic probe 10 to transmit ultrasonic beams to the biological tissue 60 on the one hand, and the other On the one hand, it is used to control the ultrasonic probe 10 to receive the ultrasonic echo reflected by the ultrasonic beam through the tissue. In a specific embodiment, the transmission/reception control circuit 20 is used to generate a transmission sequence and a reception sequence, and output them to the ultrasound probe. The transmission sequence is used to control some or all of the plurality of array elements in the ultrasonic probe 10 to transmit ultrasonic waves to the target of interest in the biological tissue 60. The parameters of the transmission sequence include the number of array elements used for transmission and ultrasonic transmission parameters (such as amplitude, frequency, wave times, launch intervals, launch angles, wave patterns and/or focus positions, etc.). The receiving sequence is used to control some or all of the plurality of array elements to receive the echoes of the organized ultrasonic waves. The parameters of the receiving sequence include the number of receiving array elements and the receiving parameters of the echo (such as receiving angle, depth, etc.). The ultrasonic parameters in the transmitting sequence and the echo parameters in the receiving sequence are also different depending on the application of the ultrasonic echo or the image generated based on the ultrasonic echo.

The echo processing module 30 is used for processing the ultrasonic echo signals received by the ultrasonic probe 10 , for example, filtering, amplifying, and beamforming the ultrasonic echo signals to obtain ultrasonic echo data. In a specific embodiment, the echo processing module 30 can output the ultrasonic echo data to the processor 40, or store the ultrasonic echo data in a memory first, and when it is necessary to perform operations based on the ultrasonic echo data, the processor 40 Read the ultrasound echo data from the memory. Those skilled in the art should understand that, in some embodiments, when the ultrasonic echo signal does not need to be processed by filtering, amplifying, beamforming, etc., the echo processing module 30 can also be omitted.

The processor 40 is used to obtain ultrasonic echo data or signals, and use correlation algorithms to obtain required parameters or images.

The display module 50 can be used for displaying information, such as displaying parameters and images calculated by the processor 40 . Those skilled in the art should understand that, in some embodiments, the ultrasound imaging system itself may not integrate a display module, but is connected to a computer device (such as a computer) to display information through a display module (such as a display screen) of the computer device.

The ultrasonic imaging system in some embodiments of the present invention also introduces sensors for acquiring pose information. These sensors use position sensors based on electromagnetic induction, but in other embodiments, they can also be position sensors based on optical principles, and can also be Other types of position sensors based on acoustic principles, etc., can even be image positioning markers (markers). In some embodiments, the Sensor for acquiring pose information may include a probe locator 11 and a body surface locator 13, which will be described in detail below.

In some embodiments, the above-mentioned probe positioner 11 is set on the ultrasonic probe 10, in other words, the probe positioner 11 is set on the ultrasonic probe 10; specifically, the probe positioner 11 can be built into the ultrasonic probe 10, or can be are fixed to the casing of the ultrasonic probe 10 and the like. The probe positioner 11 is used to obtain real-time pose information of the ultrasonic probe . During the ultrasonic image data acquisition process, the probe locator 11 generates real-time attitude information of the ultrasonic probe 10 such as a pose matrix R according to the sensed movement of the ultrasonic probe 10, so that each frame of ultrasonic image data corresponds to a corresponding real-time attitude information.

In some embodiments, the body surface locator 13 is used to be arranged on the object to be inspected, for example, to be arranged on the body surface of the object to be inspected by wearing, pasting, and the like. The body surface locator 13 is used to obtain the real-time pose information of the above-mentioned inspected object, such as pose matrix R _p .

A job can look like this:

The ultrasonic imaging system excites the ultrasonic probe 10 through the transmitting/receiving control circuit 20 to transmit ultrasonic waves to the parts of the human body to be examined to obtain ultrasonic echo signals; the processor 40 processes the obtained ultrasonic echo signals to obtain ultrasonic image data of target tissues and organs; The processor 40 can also acquire other detection image data, which can be three-dimensional ultrasound image data of the same modality as the ultrasound image data, or another modality image data of a different modality from the ultrasound image data, such as CT or MR image data, for example, before the processor 40 performs registration, import the above-mentioned another modality image data through a modality import interface; the acquisition method of another modality image data can refer to the existing related technology, which will not be described here; In addition, another imported modality image data may be three-dimensional data, or multiple three-dimensional data at different times. The probe positioner 11 arranged on the ultrasonic probe 10, along with the movement of the ultrasonic probe 10, continuously provides the pose information of the ultrasonic probe, for example including six degrees of freedom (that is, vertical, lateral, longitudinal, pitch, roll and sway) ), the body surface locator set on the object to be inspected also obtains the real-time pose information of the object to be inspected, such as space including six degrees of freedom (that is, vertical, lateral, longitudinal, pitch, roll, and sway) orientation information. Processor 40 uses image data and real-time posture information to register and fuse the ultrasound image and another modality image, and then sends the fusion result to display unit 50, which displays the fusion result; or, processor 40 utilizes Image data and real-time posture information, register the ultrasonic image and another modality image, and then send the registration result to the display part 50, and the real-time ultrasonic image data and detection image data after the registration are performed by the display part 50 contrast display. A more specific description of the registration process is given below.

In some embodiments, the processor 40 acquires real-time first image data of the object under inspection, such as ultrasound image data, through an ultrasound probe; state image data such as CT or MR image data; the processor 40 obtains the real-time pose information of the ultrasonic probe 10—for example, through the probe positioner 11; the processor 40 obtains the real-time pose information of the inspected object—for example, through body surface locator 13 to acquire; processor 40 also acquires the pose change information caused by respiration of the inspected object, for example, the processor acquires the pose change information caused by respiration in at least one breathing cycle of the inspected object, specifically, Referring to the method mentioned in step (3) above, the breathing cycle information can be extracted by collecting the pose information output by the body surface locator 13 for a period of time. In some specific embodiments, the processor 40 obtains the pose change information caused by respiration within at least one respiration cycle of the subject under inspection, which is further implemented as follows: the processor 40 determines the respiration phase corresponding to the initial registration mapping relationship, and Its corresponding pose information caused by breathing; processor 40 determines each breathing phase in the breathing cycle, and its corresponding pose information caused by breathing; processor 40 according to the above-mentioned initial registration mapping relationship corresponding The breathing phase and its corresponding pose information caused by breathing, each breathing phase in the above breathing cycle and its corresponding pose information caused by breathing, calculate the breathing phase of each breathing phase in the breathing cycle caused by breathing The corresponding pose change matrix is used as the above-mentioned pose change information caused by breathing; an example of the pose change information caused by breathing calculated in this way is the pose change ΔR _t mentioned in the above formula (15) .

The processor 40 performs real-time first image data and second image data of the inspected object according to the real-time pose information of the ultrasonic probe 10, the real-time pose information of the inspected object, and the pose change information of the inspected object caused by breathing. for registration. In some specific embodiments, the processor 40 calculates the real-time first image data of the inspected object according to the real-time pose information of the ultrasonic probe 10, the real-time pose information of the inspected object, and the pose change information of the inspected object caused by breathing. Registration with the second image data can be done as follows:

The processor 40 acquires an initial registration mapping relationship (or registration relationship) between the first image data and the second image data. The acquisition of the initial registration mapping relationship can refer to the description of the initial registration above. For example, the operator selects a first image, then finds the corresponding slice in the second image, and then superimposes the two, through translation or rotation Make the first image coincide with the same anatomical structure in the second image of the corresponding section to obtain the T matrix at this time, and then the P matrix can be calculated by the formula P=T×A ^-1 ×R ^-1 , the P matrix That is, the initial registration mapping relationship between the first image data and the second image data. After obtaining the initial registration mapping relationship, the processor 40 obtains the first image data to be registered during the real-time dynamic registration process; the processor 40 obtains the real-time pose information of the inspected object; Correct the above-mentioned registration mapping relationship by the pose change information of the inspected object caused by breathing and the real-time pose information of the inspected object; the processor 40 then performs the real-time first image of the inspected object according to the corrected registration mapping relationship. The data and the second image data are registered.

The processor 40 corrects the registration mapping relationship, some embodiments may be:

The processor 40 corrects the real-time pose information of the inspected object according to the posture change information of the inspected object caused by respiration, for example, identifying and removing the respiration-induced pose information contained in the real-time pose information of the inspected object. pose change information; in some specific embodiments, the processor 40 identifies and removes the pose change information contained in it from the real-time pose information of the detected object. Detect the real-time respiratory phase of the object; the processor 40 converts the real-time position of the detected object to The pose change information caused by breathing included in the pose information is removed from the real-time pose information of the detected object; specifically, the processor 40 according to the real-time breathing phase of the detected object and The pose change matrix corresponding to each breath in the breathing cycle is obtained to obtain the pose change matrix corresponding to the real-time breathing of the detected object caused by breathing; the real-time pose information of the detected object includes the detected For the real-time pose matrix of the object, the processor 40 multiplies the real-time pose matrix of the detected object by the inverse matrix of the pose change matrix corresponding to the real-time breathing of the detected object to obtain the real-time pose matrix of the detected object The pose change information caused by breathing is removed from the pose information.

During real-time registration, especially for some patients with abdominal breathing, when the ultrasound images of abdominal organs and other images (such as three-dimensional ultrasound, CT, MR or PET and other three-dimensional volume data) are registered, the organs caused by respiratory movement Displacement, rotation and deformation will affect the change of ultrasonic image data, thus making the registration result unsatisfactory. In order to eliminate or weaken this effect, in some embodiments, the correction of the registration mapping relationship by the processor 40 also includes: the processor 40 according to The pose change information of the inspected object caused by respiration is used to correct the change of the registration mapping relationship caused by the change of the ultrasound image data caused by the respiration of the inspected object.

So some embodiments could be like this:

The processor 40 controls the ultrasonic probe 10 to transmit ultrasonic waves to the object under inspection and receive the ultrasonic echo returned from the object under inspection to obtain ultrasonic echo signals, and obtain real-time ultrasonic image data of the object under inspection according to the above ultrasonic echo signals;

The processor 40 acquires the non-real-time detection image data of the object to be inspected;

The processor 40 obtains the real-time pose information of the ultrasonic probe 10 through the probe locator 11, and obtains the real-time pose information of the inspected object and the pose change information of the inspected object caused by breathing through the body surface locator 13; The processor 40 acquires the pose change information of the inspected object caused by respiration through the body surface locator, including: acquiring the pose change information of the inspected object caused by respiration within at least one breathing cycle. In some specific embodiments, the processor 40 acquires the pose change information caused by respiration within at least one respiration cycle of the subject to be examined, which is further implemented as follows: the processor 40 determines the respiration phase corresponding to the initial registration mapping relationship, and Its corresponding pose information caused by breathing; processor 40 determines each breathing phase in the breathing cycle, and its corresponding pose information caused by breathing; processor 40 according to the above-mentioned initial registration mapping relationship corresponding The breathing phase and its corresponding pose information caused by breathing, each breathing phase in the above breathing cycle and its corresponding pose information caused by breathing, calculate the breathing phase of each breathing phase in the breathing cycle caused by breathing The corresponding pose change matrix is used as the above pose change information caused by breathing; the pose change information calculated in this way is caused by breathing, an example is the pose change ΔR _t mentioned in the above formula (15) ;

The processor 40 acquires the initial registration mapping relationship between the real-time ultrasonic image data of the inspected object and the detected image data according to the real-time pose information of the ultrasonic probe 10 . The acquisition of the initial registration mapping relationship can refer to the description of the initial registration above. For example, the operator selects an ultrasound image, then finds the corresponding slice in the detection image, and then superimposes the two, and makes the ultrasound The image is superimposed with the same anatomical structure in the detection image of the corresponding section to obtain the T matrix at this time, and then the P matrix can be calculated by the formula P=T×A ^-1 ×R ^-1 , and the P matrix is the ultrasound image The initial registration mapping relationship between the data and the detected image data;

The processor 40 removes the pose change information caused by breathing from the real-time pose information of the detected object; and the processor 40 removes the pose information of the detected object after removing the pose change information caused by breathing, Correct the above-mentioned registration mapping relationship; in some embodiments, the processor 40 corrects the real-time pose information of the inspected object according to the pose change information of the inspected object caused by breathing, for example, from the real-time pose information of the inspected object In some specific embodiments, the processor 40 identifies and removes the pose change information contained in the detected object from the real-time pose information of the detected object and removes the pose information caused by breathing. The change information can be carried out in the following way: the processor 40 determines the real-time respiratory phase of the detected object; the processor 40 determines the real-time respiratory phase of the detected object according to the real-time respiratory phase of the detected object, and the breath caused by the detected object in at least one respiratory cycle. pose change information of the detected object, and remove the pose change information caused by breathing contained in the real-time pose information of the detected object from the real-time pose information of the detected object; specifically, the processor 40 according to the above-mentioned The real-time breathing phase of the detected object, and the corresponding pose change matrix for each breathing cycle caused by the above breathing, and obtain the pose change matrix corresponding to the real-time breathing of the above detected object caused by breathing The real-time pose information of the detected object includes the real-time pose matrix of the detected object, and the processor 40 multiplies the real-time pose matrix of the detected object by the corresponding pose change during the real-time breathing of the detected object The inverse matrix of the quantity matrix is used to remove the pose change information caused by breathing from the real-time pose information of the detected object; when performing real-time registration, especially for some patients with abdominal breathing, abdominal viscera When registering an ultrasound image of an organ with other images (such as three-dimensional ultrasound, CT, MR or PET and other three-dimensional volume data), the organ displacement, rotation, and deformation caused by breathing motion will affect the changes in the ultrasound image data, thus making the registration result inconsistent. Ideally, in order to eliminate or weaken this effect, in some embodiments, the correction of the registration mapping relationship by the processor 40 further includes: the processor 40 calculates the breathing information of the checked object according to the pose change information of the checked object caused by breathing. Correct the changes in the registration mapping relationship caused by the changes in the ultrasound image data;

The processor 40 registers the real-time ultrasound image data and the detected image data of the object under inspection according to the corrected registration mapping relationship.

Analyzing the above process, it can be seen that when any of the pose information of the ultrasonic probe 10 and the pose information of the object to be inspected changes, the registration mapping relationship will change, so it needs to be corrected and updated, and the registration mapping relationship The change of the detected object essentially includes several categories. The first type is caused by the change of the pose information of the detected object (for the magnetic navigation system, the change of the pose information of the detected object includes the change of the patient's body position, or the magnetic field The position of the generator has changed) the change of the registration mapping relationship; the second type is based on the first type, the detected object is caused by the interference caused by breathing on the pose information obtained by the body surface locator 13, This will also lead to changes in the registration mapping relationship; further, the third type is the change of the ultrasound image data due to the breathing of the detected object, which will also cause changes in the registration mapping relationship, so it can be targeted Correction is performed on one or more of the third categories, which will be described in detail below.

In some embodiments, the ultrasonic probe 10 transmits ultrasonic waves to the object to be inspected and receives ultrasonic echoes returned from the object to be inspected to obtain ultrasonic echo signals, and the processor 40 obtains real-time ultrasonic waves of the object to be inspected according to the ultrasonic echo signals. Image data; the processor 40 acquires the non-real-time detection image data of the object to be inspected, such as three-dimensional ultrasonic image data or image data of a different modality from the ultrasonic image data, and the image data of a different modality from the ultrasonic image data can be CT or MR image data, etc.; the processor 40 obtains the real-time pose information of the ultrasonic probe 10 through the probe locator 11, and obtains the real-time pose information of the inspected object through the body surface locator 13; the processor 40 obtains the real-time ultrasonic image of the inspected object The data and the detection image data are registered. The specific registration process in some embodiments can be as follows:

The processor 40 acquires the initial registration mapping relationship between the ultrasound image data and the detection image data according to the real-time pose information of the ultrasound probe. The acquisition of the initial registration mapping relationship can refer to the description of the initial registration above. For example, the operator selects an ultrasound image, then finds the corresponding slice in the detection image, and then superimposes the two, and makes the ultrasound The image is superimposed with the same anatomical structure in the detection image of the corresponding section to obtain the T matrix at this time, and then the P matrix can be calculated by the formula P=T×A ^-1 ×R ^-1 , and the P matrix is the ultrasound image The initial registration mapping relationship between the data and the detected image data. The processor 40 corrects the above-mentioned registration mapping relationship, and then performs registration on the real-time ultrasonic image data and detection image data of the above-mentioned inspected object according to the corrected registration mapping relationship.

In some embodiments, the processor 40 correcting the above-mentioned registration mapping relationship includes: the processor 40 performs a first-type correction on the above-mentioned registration mapping relationship according to the real-time pose information of the inspected object; and, the processor 40 through the above-mentioned The body surface locator 13 acquires the pose change information of the inspected object caused by respiration, and performs a second type of correction on the above-mentioned registration mapping relationship according to the pose change information caused by respiration. In some embodiments, the above-mentioned first type of correction is used to correct the change of the registration mapping relationship caused by the change of the body position of the inspected object, and the above-mentioned second type of correction is used to eliminate the position of the inspected object during the first type of correction. The pose information contains the pose change information caused by breathing. In some embodiments, the processor 40 correcting the above-mentioned registration mapping relationship further includes: the processor 40 performs a third type of correction on the above-mentioned registration mapping relationship through the above-mentioned posture change information caused by breathing, and the above-mentioned third type of correction is used for It is used to correct the change of the registration mapping relationship caused by the change of the ultrasonic image data caused by the respiration of the examined object.

A more detailed description of the second type of correction is given below.

In some embodiments, the processor 40 obtains information on the pose change of the object under inspection caused by respiration through the above-mentioned body surface locator 13, for example, it may be to obtain information on the pose change of the object under inspection caused by respiration within at least one breathing cycle . In some specific embodiments, the processor 40 determines the respiratory phase corresponding to the initial registration mapping relationship, and its corresponding pose information caused by breathing; the processor 40 determines each respiratory phase in the respiratory cycle, and its Corresponding pose information caused by breathing; processor 40, according to the breathing phase corresponding to the initial registration mapping relationship and the corresponding pose information caused by breathing, each breathing phase and The corresponding pose information caused by breathing is calculated to obtain the corresponding pose change matrix for each breath in the breathing cycle caused by breathing, which is used as the above pose change information caused by breathing; The pose change information of , an example is the pose change ΔR _t mentioned in formula (15) above.

In some embodiments, the processor 40 performs the second type of correction on the above-mentioned registration mapping relationship according to the above-mentioned pose change information caused by breathing, including: the processor 40 determines the real-time breathing phase of the detected object; the processor 40 according to the above-mentioned The real-time breathing phase of the detected object, the pose change information caused by breathing of the detected object within at least one breathing cycle, the breathing phase corresponding to the initial registration mapping relationship and the pose information caused by breathing, for The second type of correction is performed on the registration mapping relationship, so as to eliminate the pose change information caused by breathing contained in the pose information of the inspected object when the first type of correction is performed. In some specific embodiments, the processor 40 obtains the real-time breathing phase of the detected object caused by the breathing according to the real-time breathing phase of the detected object and the corresponding pose change matrix for each breathing time in the breathing cycle caused by the breathing. The corresponding pose variation matrix during breathing; the real-time pose information of the detected object includes the real-time pose matrix of the detected object, and the processor 40 multiplies the real-time pose matrix of the detected object by the detected The inverse matrix of the pose change matrix corresponding to the real-time breathing of the object, so as to eliminate the pose change information caused by the breathing contained in the pose information of the inspected object when the first type of correction is performed.

In some embodiments, the processor 40 also controls the display module 50 to perform fusion display or comparative display of the registered real-time ultrasound image data and detection image data.

The above are some descriptions of the ultrasound imaging system.

Some embodiments of the present invention also disclose an image registration method, which will be described in detail below.

Please refer to FIG. 4 or FIG. 5, the image registration method in some embodiments includes the following steps:

Step 100: Obtain real-time first image data of the inspected object through the ultrasonic probe. The first image data is, for example, ultrasound image data.

Step 110: Obtain the second image data of the above-mentioned inspected object. The second image data is, for example, three-dimensional ultrasound image data or image data of a different modality from the ultrasound image data, such as CT or MR image data.

Step 120: Obtain the real-time pose information of the ultrasonic probe.

For example, the real-time position and orientation information of the ultrasonic probe can be obtained by the probe positioner arranged on the ultrasonic probe. In some embodiments, the position and position information can include six degrees of freedom (that is, vertical, lateral, longitudinal, pitch, roll and swing) spatial orientation information.

Step 130: Obtain the real-time pose information of the object under inspection.

For example, the real-time pose information of the ultrasonic probe can be obtained by setting a body surface locator on the object to be inspected. In some embodiments, the pose information can include six degrees of freedom (that is, vertical, lateral, longitudinal, pitch, roll, and Swing) spatial orientation information.

Step 140: Acquiring information on pose changes of the above-mentioned inspected object caused by breathing.

In some embodiments, step 140 acquiring information on pose changes of the inspected object caused by breathing includes: acquiring information on pose changes of the inspected object caused by breathing within at least one breathing cycle.

A specific way can be as in step (3) above to collect body surface Sensor positioning information for a period of time to extract respiratory cycle information, for example, step 140 determines the respiratory phase corresponding to the initial registration mapping relationship, and its corresponding Pose information caused by breathing; step 140 determines each breathing phase in the breathing cycle, and its corresponding pose information caused by breathing; step 140, according to the breathing phase corresponding to the above-mentioned initial registration mapping relationship, and Its corresponding pose information caused by breathing, each breathing phase in the above breathing cycle and its corresponding pose information caused by breathing, calculate the corresponding pose change in each breathing cycle caused by breathing Quantity matrix, as the above-mentioned pose change information caused by breathing.

Step 150: According to the real-time pose information of the ultrasonic probe, the real-time pose information of the inspected object, and the pose change information of the inspected object caused by breathing, analyze the real-time first image data and the second image data of the inspected object. The two image data are registered.

In some embodiments, please refer to FIG. 6, step 150 includes the following steps:

Step 151: Obtain an initial registration mapping relationship between the first image data and the second image data according to the real-time pose information of the ultrasound probe.

The acquisition of the initial registration mapping relationship can refer to the description of the initial registration above. For example, the operator selects a first image, then finds the corresponding slice in the second image, and then superimposes the two, through translation or rotation Make the first image coincide with the same anatomical structure in the second modality image of the corresponding section to obtain the T matrix at this time, and then the P matrix can be calculated by the formula P=T×A ^-1 ×R ^-1 , the The P matrix is the initial registration mapping relationship between the first image data and the second image data.

Step 152: Correct the registration mapping relationship according to the pose change information of the inspected object caused by breathing and the real-time pose information of the inspected object. In some embodiments, step 152 corrects the real-time pose information of the inspected object according to the posture change information of the inspected object caused by breathing, for example, identifying and removing all the detected objects from the real-time pose information of the inspected object. Included pose change information caused by respiration; step 152 corrects the registration mapping relationship based on the corrected real-time pose information of the inspected object.

In some embodiments, please refer to FIG. 7, step 152 identifies and removes the pose change information caused by breathing from the above-mentioned real-time pose information of the detected object, including the following steps:

Step 152.1: Determine the real-time breathing phase of the detected object;

Step 152.3: According to the real-time breathing phase of the detected object and the pose change information caused by breathing of the detected object within at least one breathing cycle, the real-time pose information of the detected object contained in the breathing The resulting pose change information is removed from the real-time pose information of the detected object. For example, in step 152.3, according to the real-time breathing phase of the detected object and the corresponding pose change matrix for each breathing time in the breathing cycle caused by the breathing, obtain the corresponding real-time breathing time of the detected object caused by breathing Pose variation matrix; the real-time pose information of the detected object includes the real-time pose matrix of the detected object, and step 152.3 multiplies the real-time pose matrix of the detected object by the real-time breathing time of the detected object to correspond to The inverse matrix of the pose change matrix is used to remove the pose change information caused by breathing from the real-time pose information of the detected object.

Step 153: Register the real-time first image data and second image data of the object under inspection according to the corrected registration mapping relationship.

During real-time registration, especially for some abdominal breathing patients, when the ultrasound images of abdominal organs are registered with other modal images, the displacement, rotation and deformation of the organs caused by respiratory movement will affect the changes in the ultrasound image data, thus The registration result is unsatisfactory. In order to eliminate or reduce this effect, in some embodiments, step 150 further includes: according to the above-mentioned posture change information of the inspected object caused by breathing, the ultrasound image data of the inspected object caused by breathing Correction for changes in the registration mapping relationship caused by changes in .

Step 160: control to perform fusion display or comparative display of the registered real-time first image data and second image data.

Therefore, the image registration method in some embodiments may be as follows:

(1) Control the ultrasonic probe to transmit ultrasonic waves to the object under inspection and receive the ultrasonic echo returned from the object under inspection to obtain ultrasonic echo signals, and obtain real-time ultrasonic image data of the object under inspection according to the above-mentioned ultrasonic echo signals;

(2) Obtain the non-real-time detection image data of the above-mentioned inspected object;

(3) Obtain the real-time pose information of the above-mentioned ultrasonic probe through the probe locator, and obtain the real-time pose information of the above-mentioned inspected object and the pose change information of the above-mentioned inspected object caused by breathing through the body surface locator; The body surface locator acquires information on pose changes of the inspected object caused by breathing, including: acquiring information on pose changes of the inspected object caused by breathing within at least one breathing cycle. In some specific embodiments, the acquisition of the breathing-induced pose change information of the inspected object within at least one breathing cycle is further implemented as follows: determine the breathing phase corresponding to the initial registration mapping relationship, and the corresponding breathing phase caused by breathing pose information; determine each breathing phase in the breathing cycle, and its corresponding pose information caused by breathing; according to the above initial registration mapping relationship, the corresponding breathing phase and its corresponding breathing-induced Pose information, each breathing phase in the above breathing cycle and its corresponding pose information caused by breathing, calculate the corresponding pose change matrix for each breathing in the breathing cycle caused by breathing, as the above-mentioned breathing caused by The pose change information caused by this method; the pose change information caused by breathing calculated in this way, an example is the pose change ΔR _t mentioned in the above formula (15);

(4) According to the real-time pose information of the ultrasound probe, the initial registration mapping relationship between the real-time ultrasound image data of the inspected object and the detection image data is obtained. The acquisition of the initial registration mapping relationship can refer to the description of the initial registration above. For example, the operator selects an ultrasound image, then finds the corresponding slice in the detection image, and then superimposes the two, and makes the ultrasound The image is superimposed with the same anatomical structure in the detection image of the corresponding section to obtain the T matrix at this time, and then the P matrix can be calculated by the formula P=T×A ^-1 ×R ^-1 , and the P matrix is the ultrasound image The initial registration mapping relationship between the data and the detected image data;

(5) Remove the pose change information caused by breathing from the real-time pose information of the detected object; In some embodiments, the real-time pose information of the inspected object is corrected according to the pose change information of the inspected object caused by breathing, for example, the real-time pose information of the inspected object is identified and eliminated. Included pose change information caused by breathing; in some specific embodiments, identifying and removing the pose change information contained in the detected object from the real-time pose information may be carried out as follows: determine The real-time respiratory phase of the detected object; according to the real-time respiratory phase of the detected object and the posture change information caused by breathing of the detected object within at least one respiratory cycle, the real-time pose of the detected object The pose change information caused by breathing contained in the information is removed from the real-time pose information of the detected object; The pose change matrix corresponding to each breath is obtained to obtain the pose change matrix corresponding to the real-time breath of the detected object caused by breathing; the real-time pose information of the detected object includes the real-time position of the detected object Pose matrix, the real-time pose matrix of the detected object is multiplied by the inverse matrix of the pose change matrix corresponding to the real-time breathing of the detected object, so as to remove all its components from the real-time pose information of the detected object Included pose change information caused by breathing; during real-time registration, especially for some patients with abdominal breathing, ultrasonic images and other images of abdominal organs (such as three-dimensional ultrasound, CT, MR or PET and other three-dimensional data ) during registration, the organ displacement, rotation and deformation caused by respiratory movement will affect the change of ultrasound image data, thus making the registration result unsatisfactory. In order to eliminate or weaken this effect, in some embodiments, the registration mapping relationship The correction also includes: the processor 40 corrects the change of the registration mapping relationship caused by the change of the ultrasonic image data of the inspected object caused by the breathing according to the pose change information of the inspected object caused by the breathing;

(6) According to the corrected registration mapping relationship, register the real-time ultrasonic image data and detection image data of the object to be inspected.

Analyzing the above process, it can be seen that when any one of the pose information of the ultrasonic probe and the pose information of the inspected object changes, the registration mapping relationship will change, so it needs to be corrected and updated. The changes essentially include several categories. The first category is caused by changes in the pose information of the detected object (for a magnetic navigation system, changes in the pose information of the detected object include changes in the patient's position, or changes in the magnetic field. The position of the device has changed) the change of the registration mapping relationship; the second type is based on the first type, the detected object is due to the interference caused by the breath on the pose information obtained by the body surface locator 13, which It will also lead to changes in the registration mapping relationship; furthermore, the third type is the change of the ultrasound image data due to the breathing of the detected object, which will also cause changes in the registration mapping relationship, so it can be targeted. One or more of these third categories are corrected, as described below.

Please refer to FIG. 8 or FIG. 9, the image registration method in some embodiments includes the following steps:

Step 200: Obtain real-time ultrasound image data. For example, step 200 controls the ultrasonic probe to transmit ultrasonic waves to the inspected object and receive ultrasonic echoes returned from the inspected object to obtain ultrasonic echo signals, and obtain real-time ultrasonic image data of the inspected object according to the ultrasonic echo signals.

Step 210: Obtain non-real-time detection image data of the object to be inspected, such as three-dimensional ultrasound image data or image data of a different modality from the ultrasound image data, and the image data of a different modality from the ultrasound image data may be CT or MR image data, etc. .

Step 220: Obtain two types of pose information. Specifically, step 220 obtains the real-time pose information of the ultrasonic probe through the probe locator, and obtains the real-time pose information of the inspected object through the body surface locator; wherein the probe locator is set on the ultrasonic probe, and the body surface locator The locator is used for setting on the above-mentioned inspected object.

Step 230: Registering the real-time ultrasound image data and detection image data of the object under inspection.

Please refer to FIG. 10 , in some embodiments, step 230 registers the real-time ultrasound image data and detection image data of the object under inspection, including the following steps:

Step 231: According to the real-time pose information of the ultrasound probe, the initial registration mapping relationship between the ultrasound image data and the detection image data is acquired.

The acquisition of the initial registration mapping relationship can refer to the description of the initial registration above. For example, the operator selects an ultrasound image, then finds the corresponding slice in the detection image, and then superimposes the two, and makes the ultrasound The image is superimposed with the same anatomical structure in the detection image of the corresponding section to obtain the T matrix at this time, and then the P matrix can be calculated by the formula P=T×A ^-1 ×R ^-1 , and the P matrix is the ultrasound image The initial registration mapping relationship between the data and the detected image data.

Step 233: Correct the above registration mapping relationship. Step 233 correcting the registration mapping relationship includes performing one or more of the first type of correction, the second type of correction and the third type of correction.

In some embodiments, step 233 performs a first type of correction on the above-mentioned registration mapping relationship according to the real-time pose information of the inspected object. In some embodiments, the above-mentioned first type of correction is used to correct the change of the registration mapping relationship caused by the change of the body position of the inspected object.

In some embodiments, step 233 obtains information on pose changes of the object under inspection caused by respiration through a body surface locator, for example, obtains information on pose changes of the object under inspection caused by respiration within at least one breathing cycle; some specific embodiments In step 233, determine the respiratory phase corresponding to the initial registration mapping relationship, and its corresponding pose information caused by breathing; step 233 determines each respiratory phase in the respiratory cycle, and its corresponding breathing-induced Pose information; Step 233 According to the respiratory phase corresponding to the above-mentioned initial registration mapping relationship and the corresponding pose information caused by breathing, each breathing phase in the above-mentioned breathing cycle and its corresponding breathing phase Pose information, calculate the corresponding pose change matrix for each breath in the breathing cycle caused by breathing, as the above pose change information caused by breathing; the calculated pose change information caused by breathing, a An example is the pose variation ΔR _t mentioned in formula (15) above.

Step 233 performs a second type of correction on the above-mentioned registration mapping relationship according to the above-mentioned pose change information caused by breathing. In some embodiments, the above-mentioned second type of correction is used to eliminate the pose change information caused by breathing contained in the pose information of the inspected object when the first type of correction is performed.

A more detailed description of the second type of correction is given below.

Please refer to FIG. 11 , in some embodiments, step 233 performs the second type of correction on the above-mentioned registration mapping relationship according to the above-mentioned pose change information caused by breathing, including the following steps:

Step 233.1: Determine the real-time breathing phase of the detected object.

Step 233.3: According to the real-time respiratory phase of the detected object, the posture change information caused by breathing of the detected object within at least one breathing cycle, the respiratory phase corresponding to the initial registration mapping relationship and the breathing phase caused by breathing Pose information, the second type of correction is performed on the registration mapping relationship, so as to eliminate the pose change information caused by breathing contained in the pose information of the inspected object when the first type of correction is performed. In some specific embodiments, step 233.3 is based on the real-time breathing phase of the detected object and the pose change matrix corresponding to each breath in the breathing cycle caused by the breathing, to obtain the real-time breathing of the detected object caused by breathing. The corresponding pose change matrix during breathing; the real-time pose information of the detected object includes the real-time pose matrix of the detected object, and step 233.3 multiplies the real-time pose matrix of the detected object by the real-time The inverse matrix of the pose change matrix corresponding to the respiration, so as to eliminate the pose change information caused by respiration contained in the pose information of the inspected object when the first type of correction is performed.

In some embodiments, step 233 performs a third type of correction on the above registration mapping relationship through the above pose change information caused by respiration. The changes in the registration mapping relationship caused by it.

Step 235: According to the corrected registration mapping relationship, register the real-time ultrasound image data and the detected image data of the object under inspection.

Step 240: control to perform fusion display or comparative display of the registered real-time ultrasound image data and detection image data.

In some embodiments of the present invention, the body position change information contained in the signal of the body surface locator (the position and posture information of the inspected object) and the breathing cycle information are decoupled, and the body position change information is used to realize the accurate body position tracking function, ensuring During the image registration process, even if the patient's body position changes or the position of the magnetic field generator moves, it can still maintain a good registration effect; while the respiratory cycle information can be discarded or used for respiratory compensation, real-time correction of liver and other organs during fusion. Displacement within the body caused by breathing movement.

This document is described with reference to various exemplary embodiments. However, those skilled in the art will recognize that changes and modifications can be made to the exemplary embodiments without departing from the scope herein. For example, the various operational steps, as well as the components used to perform the operational steps, may be implemented in different ways depending on the particular application or considering any number of cost functions associated with the operation of the system (e.g., one or more steps may be deleted, modified or incorporated into other steps).

In the above embodiments, all or part of them may be implemented by software, hardware, firmware or any combination thereof. In addition, the principles herein may be embodied in a computer program product on a computer-readable storage medium having computer-readable program code preloaded thereon, as understood by those skilled in the art. Any tangible, non-transitory computer readable storage medium may be used, including magnetic storage devices (hard disk, floppy disk, etc.), optical storage devices (CD to ROM, DVD, Blu Ray disk, etc.), flash memory, and/or the like . These computer program instructions can be loaded into a general purpose computer, special purpose computer or other programmable data processing apparatus to form a machine, so that these instructions executed on the computer or other programmable data processing apparatus can generate an apparatus for realizing specified functions. These computer program instructions may also be stored in a computer-readable memory which can instruct a computer or other programmable data processing device to operate in a particular manner such that the instructions stored in the computer-readable memory form a Manufactures, including implementing devices for implementing specified functions. Computer program instructions can also be loaded on a computer or other programmable data processing device, thereby performing a series of operational steps on the computer or other programmable device to produce a computer-implemented process, so that the computer or other programmable device Instructions may provide steps for performing specified functions.

While the principles herein have been shown in various embodiments, many modifications in structure, arrangement, proportions, elements, materials and components, particularly suited to particular circumstances and operational requirements may be made without departing from the principles and scope of this disclosure use. The above modifications and other changes or amendments are intended to be included within the scope of this document.

The foregoing detailed description has been described with reference to various embodiments. However, those skilled in the art will recognize that various modifications and changes can be made without departing from the scope of the present disclosure. Accordingly, the disclosure is to be considered in an illustrative rather than a restrictive sense, and all such modifications are intended to be embraced within its scope. Also, advantages, other advantages and solutions to problems have been described above with respect to various embodiments. However, neither benefits, advantages, solutions to problems, nor any elements that lead to these, or make the solutions more definite, should be construed as critical, required, or necessary. As used herein, the term "comprises" and any other variants thereof are non-exclusive, such that a process, method, article, or apparatus that includes a list of elements includes not only those elements, but also elements not expressly listed or not part of the process. , method, system, article or other element of a device. Additionally, the term "coupled" and any other variations thereof, as used herein, refers to a physical connection, an electrical connection, a magnetic connection, an optical connection, a communicative connection, a functional connection, and/or any other connection.

Those skilled in the art will recognize that many changes may be made to the details of the above-described embodiments without departing from the underlying principles of the invention. Accordingly, the scope of the invention should be determined only by the claims.

Claims (20)

1. A method of image registration, comprising:

controlling an ultrasonic probe to emit ultrasonic waves to an inspected object, receiving ultrasonic echoes returned from the inspected object to acquire ultrasonic echo signals, and acquiring real-time ultrasonic image data of the inspected object according to the ultrasonic echo signals;

acquiring non-real-time detection image data of the checked object;

acquiring real-time pose information of the ultrasonic probe through a probe positioner, and acquiring real-time pose information of the inspected object and pose change information of the inspected object caused by breathing through a body surface positioner; the probe positioner is arranged on the ultrasonic probe, and the body surface positioner is arranged on the inspected object;

acquiring the real-time ultrasonic image data of the checked object and the initial registration mapping relation of the detection image data according to the real-time pose information of the ultrasonic probe;

Removing pose change information caused by respiration from the real-time pose information of the detected object; correcting the registration mapping relation according to pose information of the detected object after removing pose change information caused by respiration;

and registering the ultrasonic image data and the detection image data of the checked object in real time according to the corrected registration mapping relation.

2. The image registration method according to claim 1, wherein acquiring pose change information of the inspected object caused by respiration by the body surface locator includes:

and acquiring pose change information caused by breathing in at least one breathing cycle of the checked object.

3. The image registration method according to claim 2, wherein the acquiring of respiratory-induced pose change information of the subject over at least one respiratory cycle comprises:

determining a respiration phase corresponding to the initial registration mapping relation and corresponding pose information caused by respiration;

determining each breathing phase in the breathing cycle and corresponding pose information caused by breathing;

and calculating to obtain a pose change amount matrix corresponding to each breathing time phase in the breathing period caused by breathing according to the breathing time phase corresponding to the initial registration mapping relation and the pose information corresponding to the breathing time phase and the pose information corresponding to the breathing in the breathing period caused by breathing.

4. The image registration method according to claim 2 or 3, wherein the removing pose change information caused by breathing from the real-time pose information of the detected object includes:

determining a breathing phase of the detected subject in real time;

and removing pose change information caused by breathing contained in the real-time pose information of the detected object from the real-time pose information of the detected object according to the real-time breathing time phase of the detected object and the pose change information caused by breathing in at least one breathing period of the detected object.

5. The image registration method of claim 4, wherein,

the removing the pose change information caused by breathing included in the pose information of the detected object in real time from the pose information of the detected object in real time according to the respiratory phase of the detected object in real time and the pose change information caused by breathing in at least one respiratory period of the detected object, includes:

obtaining a pose change matrix corresponding to the real-time breathing time phase of the detected object caused by breathing according to the real-time breathing time phase of the detected object and the pose change matrix corresponding to each breathing time phase in the breathing period caused by breathing;

The real-time pose information of the detected object comprises a real-time pose matrix of the detected object, and the real-time pose matrix of the detected object is multiplied by an inverse matrix of a pose change amount matrix corresponding to the real-time breathing time phase of the detected object so as to remove the pose change information caused by breathing from the real-time pose information of the detected object.

6. The image registration method according to claim 1, wherein the correcting the registration mapping relationship further includes: and correcting the change of the registration mapping relation caused by the change of the ultrasonic image data of the checked object caused by the respiration according to the pose change information of the checked object caused by the respiration.

7. The image registration method according to any one of claims 1 to 6, characterized by further comprising: and controlling fusion display or contrast display of the registered real-time ultrasonic image data and the detected image data.

8. The image registration method according to any one of claims 1 to 6, wherein the detection image data is three-dimensional ultrasound image data or image data of a different modality from ultrasound image data.

9. A method of image registration, comprising:

acquiring real-time first image data of an inspected object through an ultrasonic probe;

acquiring second image data of the inspected object;

acquiring real-time pose information of the ultrasonic probe;

acquiring real-time pose information of the checked object;

acquiring pose change information of the checked object caused by respiration;

and registering the first image data and the second image data of the inspected object in real time according to the real-time pose information of the ultrasonic probe, the real-time pose information of the inspected object and the pose change information of the inspected object caused by breathing.

10. The image registration method according to claim 9, wherein the acquiring pose change information of the inspected object caused by respiration includes:

and acquiring pose change information caused by breathing in at least one breathing cycle of the checked object.

11. The image registration method according to claim 10, wherein the acquiring of respiratory-induced pose change information of the subject over at least one respiratory cycle comprises:

determining a respiration phase corresponding to the initial registration mapping relation and corresponding pose information caused by respiration;

Determining each breathing phase in the breathing cycle and corresponding pose information caused by breathing;

and calculating to obtain a pose change amount matrix corresponding to each breathing time phase in the breathing period caused by breathing according to the breathing time phase corresponding to the initial registration mapping relation and the pose information corresponding to the breathing time phase and the pose information corresponding to the breathing in the breathing period caused by breathing.

12. The image registration method according to any one of claims 9 to 11, wherein registering the first image data and the second image data of the inspected object in real time includes:

acquiring an initial registration mapping relation between the real-time first image data and the second image data of the detected object according to the real-time pose information of the ultrasonic probe;

correcting the registration mapping relation according to the pose change information of the checked object caused by breathing and the real-time pose information of the checked object;

and registering the first image data and the second image data of the checked object in real time according to the corrected registration mapping relation.

13. The image registration method according to claim 12, wherein the correcting the registration map according to pose change information of the inspected object caused by breathing and the real-time pose information of the inspected object includes:

correcting the real-time pose information of the checked object according to the pose change information of the checked object caused by breathing;

and correcting the registration mapping relation according to the real-time pose information of the checked object after correction.

14. The image registration method according to claim 13, wherein correcting the pose information of the inspected object in real time according to the pose change information of the inspected object caused by respiration includes:

and identifying and removing the pose change information caused by breathing from the real-time pose information of the detected object.

15. The image registration method according to claim 14, wherein the identifying and removing pose change information caused by breathing contained in the detected object from the real-time pose information of the detected object includes:

determining a breathing phase of the detected subject in real time;

And removing pose change information caused by breathing contained in the real-time pose information of the detected object from the real-time pose information of the detected object according to the real-time breathing time phase of the detected object and the pose change information caused by breathing in at least one breathing period of the detected object.

16. The image registration method according to claim 15, wherein the removing pose change information caused by breathing included in the real-time pose information of the detected object from the real-time pose information of the detected object based on the real-time breathing phase of the detected object and the pose change information caused by breathing in at least one breathing cycle of the detected object includes:

obtaining a pose change matrix corresponding to the real-time breathing time phase of the detected object caused by breathing according to the real-time breathing time phase of the detected object and the pose change matrix corresponding to each breathing time phase in the breathing period caused by breathing;

the real-time pose information of the detected object comprises a real-time pose matrix of the detected object, and the real-time pose matrix of the detected object is multiplied by an inverse matrix of a pose change amount matrix corresponding to the real-time breathing time phase of the detected object so as to remove the pose change information caused by breathing from the real-time pose information of the detected object.

17. The image registration method according to any one of claims 12 to 16, wherein the correcting the registration map according to respiratory-induced pose change information of the subject under examination and real-time pose information of the subject under examination further includes:

and correcting the change of the registration mapping relation caused by the change of the first image data of the checked object caused by the respiration according to the pose change information of the checked object caused by the respiration.

18. The image registration method according to any one of claims 9 to 11, characterized by further comprising: and controlling fusion display or contrast display of the registered real-time first image data and the registered real-time second image data, wherein the first image data comprises two-dimensional ultrasonic image data, and the second image data comprises three-dimensional ultrasonic image data or image data of a different mode from the ultrasonic image data.

19. An ultrasound imaging system, comprising:

the ultrasonic probe is used for transmitting ultrasonic waves to the detected object and receiving corresponding ultrasonic wave echoes so as to acquire ultrasonic wave echo signals;

a transmission and reception control circuit for controlling the ultrasonic probe to perform transmission of ultrasonic waves and reception of ultrasonic echo signals;

The probe locator is arranged on the ultrasonic probe and used for acquiring real-time pose information of the ultrasonic probe;

the body surface locator is used for being arranged on the checked object to acquire real-time pose information of the checked object;

a processor for performing the image registration method as claimed in any one of claims 1 to 18.

20. A computer readable storage medium having stored thereon a program executable by a processor to implement the method of any one of claims 1 to 18.

Images