CN116128936A - Registration method, registration device, registration equipment and storage medium - Google Patents

Abstract

The embodiment of the disclosure relates to a registration method, a device, equipment and a storage medium, wherein the registration method comprises the following steps: acquiring an initial three-dimensional image template and a reference image corresponding to a target user, wherein the reference image is a two-dimensional image shot for the target user in operation; and elastically registering the initial three-dimensional image template and the reference image to obtain a target three-dimensional deformation field corresponding to the initial three-dimensional image template, wherein the target three-dimensional deformation field is used for representing the displacement of points on the initial three-dimensional image template. According to the embodiment of the disclosure, the radiation to the target user can be reduced.

Description

Registration method, device, equipment and storage medium

technical field

Embodiments of the present disclosure relate to the field of computer technology, and in particular, to a registration method, device, equipment, and storage medium.

Background technique

With the development of computer technology and medical technology, image registration has become a key technology in image-guided surgery. Among them, a wide application of image registration in image-guided surgery is: to register the user's preoperative image and intraoperative image, so as to map the structure of interest in the preoperative image to the intraoperative image, so that the doctor Know the position of the structure of interest in the intraoperative image in the preoperative image. However, the preoperative images are usually three-dimensional computer tomography (Computed Tomography, CT) images and other images that will generate radiation to the user during the shooting process, which will affect the user's health. For example, surgery based on spinal surgery robots usually requires preoperative The nail placement plan is to make the nail placement plan on the user's preoperative 3D CT image first, and then map the nail placement plan to the user's intraoperative 2D X-ray image through image registration. However, the shooting of the preoperative 3D CT image It will bring radiation to users and affect their health.

At present, there is no effective solution to the above-mentioned problems.

Contents of the invention

In order to solve the above technical problem or at least partly solve the above technical problem, embodiments of the present disclosure provide a registration method, device, device and storage medium.

The first aspect of the embodiments of the present disclosure provides a registration method, the method includes:

Acquiring an initial three-dimensional image template and a reference image corresponding to the target user, wherein the reference image is a two-dimensional image taken for the target user during the operation;

performing elastic registration on the initial 3D image template and the reference image to obtain a target 3D deformation field corresponding to the initial 3D image template, wherein the target 3D deformation field is used to characterize the displacement of points on the initial 3D image template quantity.

A second aspect of an embodiment of the present disclosure provides a registration device, which includes:

An acquisition module, configured to acquire an initial three-dimensional image template and a reference image corresponding to the target user, wherein the reference image is a two-dimensional image taken for the target user during the operation;

A registration module, configured to register the initial 3D image template and the reference image to obtain a target 3D deformation field corresponding to the initial 3D image template, wherein the 3D deformation field is used to characterize points on the 3D image template the amount of displacement.

A third aspect of the embodiments of the present disclosure provides an electronic device, the server includes: a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the The processor executes the method of the first aspect above.

A fourth aspect of the embodiments of the present disclosure provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method of the above-mentioned first aspect can be implemented.

Compared with the prior art, the technical solutions provided by the embodiments of the present disclosure have the following advantages:

In the embodiment of the present disclosure, the initial 3D image template and the reference image corresponding to the target user can be acquired, wherein the reference image is a 2D image taken for the target user during the operation, so as to perform elastic registration on the initial 3D image template and the reference image , to obtain a target three-dimensional deformation field corresponding to the initial three-dimensional image template, wherein the target three-dimensional deformation field is used to represent the displacement of a point on the initial three-dimensional image template. It can be seen that with the above technical solution, the initial 3D image template can be used to replace the preoperative image that would cause radiation to the user during the shooting process, thus reducing the radiation to the target user, and by elastically adjusting the initial 3D image template and the reference image Registration, the target 3D deformation field can be obtained, and when the initial 3D image template is transformed according to the target 3D deformation field, the human body in the transformed initial 3D image template can be matched with the target user's human body reconstructed from the reference image , to realize the mapping from the initial 3D image template to the reference image.

Description of drawings

The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the disclosure and together with the description serve to explain the principles of the disclosure.

In order to more clearly illustrate the technical solutions in the embodiments of the present disclosure or the prior art, the following will briefly introduce the drawings that need to be used in the description of the embodiments or the prior art. Obviously, for those of ordinary skill in the art, In other words, other drawings can also be obtained from these drawings without paying creative labor.

FIG. 1 is a schematic flowchart of a registration method provided by an embodiment of the present disclosure;

FIG. 2 is a schematic flowchart of another registration method provided by an embodiment of the present disclosure;

FIG. 3 is a schematic flowchart of a rough registration process provided by an embodiment of the present disclosure;

FIG. 4 is a schematic flowchart of a fine registration process provided by an embodiment of the present disclosure;

Fig. 5 is a schematic structural diagram of a registration device provided by an embodiment of the present disclosure;

Fig. 6 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure.

Detailed ways

In order to more clearly understand the above objects, features and advantages of the present disclosure, the solutions of the present disclosure will be further described below. It should be noted that, in the case of no conflict, the embodiments of the present disclosure and the features in the embodiments can be combined with each other.

In the following description, many specific details are set forth in order to fully understand the present disclosure, but the present disclosure can also be implemented in other ways than described here; obviously, the embodiments in the description are only some of the embodiments of the present disclosure, and Not all examples.

Fig. 1 is a schematic flowchart of a registration method provided by an embodiment of the present disclosure, and the method may be executed by an electronic device. The electronic device can be understood as an exemplary device having a page display function, such as a mobile phone, a tablet computer, a notebook computer, a desktop computer, and a smart TV. As shown in Figure 1, the method provided in this embodiment includes the following steps:

S110. Acquire an initial three-dimensional image template and a reference image corresponding to the target user, wherein the reference image is a two-dimensional image taken for the target user during the operation.

Specifically, the initial three-dimensional image template is a three-dimensional image used as a template, therefore, the human body in the initial three-dimensional image template is not the human body of the target user. The initial three-dimensional image template may include three-dimensional medical images such as three-dimensional CT images, but is not limited thereto.

In some embodiments, optionally, acquiring the initial three-dimensional image template may include: acquiring user information of the target user; searching for an initial three-dimensional image template matching the user information from a preset template library.

Specifically, the user information may include height, weight, gender, age, race, and other information related to the physical characteristics of the user and available for authorization by the user, but is not limited thereto.

Specifically, a template library including a plurality of candidate three-dimensional image templates can be established in advance, wherein different candidate three-dimensional image templates correspond to different user information, so that a candidate that matches the user information of the target user can be searched from the preset template library. 3D image template and use it as the initial 3D image template. Wherein, the matching mentioned here includes that the user information corresponding to the candidate 3D image template is completely the same as the user information of the target user, or the difference between the user information of the target user and the user information corresponding to the candidate 3D image template is within a preset Within the difference range, the preset difference range can be set according to the actual situation.

It can be understood that, according to the user information of the target user, the initial three-dimensional image template matching the user information can be searched from the preset template library, so that the human body in the initial three-dimensional image template (that is, the human body before deformation) can be compared with the target user. The human body is more similar or closer, which is beneficial to improve the degree of coincidence between the human body (that is, the deformed human body) and the target user's human body after the initial 3D image template is transformed according to the target 3D deformation field, thereby improving the initial 3D image template to the reference The mapping accuracy of the image.

Of course, only one 3D image template can also be set in advance, which can be used as an initial 3D image template for any user.

Specifically, the reference image is a two-dimensional image used as a reference when determining the target three-dimensional deformation field corresponding to the initial three-dimensional image template. The reference image may include two-dimensional medical images such as X-rays, but is not limited thereto.

It should be noted that the number of reference images and the direction in which the reference images are taken of the target user can be set by those skilled in the art according to actual conditions, and are not limited here. For example, the reference image may include an anteroposterior image (such as an anteroposterior X-ray image) and a lateral image (such as a lateral X-ray image) of the target user, but is not limited thereto.

In some embodiments, acquiring the reference image corresponding to the target user may include: receiving the reference image corresponding to the target user sent by other electronic devices, or reading the reference image corresponding to the target user from a storage device (such as a U disk).

S120. Perform elastic registration on the initial 3D image template and the reference image to obtain a target 3D deformation field corresponding to the initial 3D image template, wherein the target 3D deformation field is used to represent the displacement of points on the initial 3D image template.

Specifically, the target 3D deformation field is used to characterize the displacement of points (ie image pixels) on the initial 3D image template. For the point on the initial three-dimensional image template, when it is displaced according to the corresponding displacement amount, the position error between it and the corresponding reconstruction point is within the preset error range, wherein the corresponding reconstruction point is the corresponding point on the reference image Backprojection reconstruction to points in the 3D coordinate system where the original 3D image template resides.

Specifically, the elastic registration mentioned here refers to using the reference image as a reference to obtain the target 3D deformation field corresponding to the initial 3D image template, so that the initial 3D image template undergoes spatial transformation according to the target 3D deformation field. The human body is deformed, and the degree of coincidence between the deformed human body and the human body reconstructed by the reference image is within the preset first error range (that is, the image pixels of the same anatomical structure are correspondingly matched in the same reference system), wherein, the predetermined The set first error range can be set according to actual conditions. It can be seen that the mapping from the initial 3D image template to the reference image can be realized according to the target 3D deformation field.

In some embodiments, optionally, S120 may include: performing elastic registration on the initial 3D image template and the reference image using the trained elastic registration model to obtain a target 3D deformation field corresponding to the initial 3D image template.

It can be understood that, using the initial three-dimensional image template to replace the preoperative image that will cause radiation to the target user can reduce the steps of taking images before the operation, and can reduce the radiation dose for the target user. Moreover, planning can be done on the initial 3D image template, and when the initial 3D image template is mapped to the reference image, the planning on the initial 3D image template can be mapped to the reference image, compared to the intraoperative 2D reference image Doing planning on the initial 3D image template can significantly improve the accuracy.

In the embodiment of the present disclosure, the initial 3D image template and the reference image corresponding to the target user can be acquired, wherein the reference image is a 2D image taken for the target user during the operation, so as to perform elastic registration on the initial 3D image template and the reference image , to obtain a target three-dimensional deformation field corresponding to the initial three-dimensional image template, where the three-dimensional deformation field is used to represent the displacement of a point on the three-dimensional image template. It can be seen that with the above technical solution, the initial 3D image template can be used to replace the preoperative image that would cause radiation to the user during the shooting process, thus reducing the radiation to the target user, and by elastically adjusting the initial 3D image template and the reference image Registration, the target 3D deformation field can be obtained, and when the initial 3D image template is transformed according to the target 3D deformation field, the human body in the transformed initial 3D image template can be matched with the target user's human body reconstructed from the reference image , to realize the mapping from the initial 3D image template to the reference image.

Fig. 2 is a schematic flowchart of another registration method provided by an embodiment of the present disclosure. The embodiments of the present disclosure are optimized on the basis of the foregoing embodiments, and the embodiments of the present disclosure may be combined with various optional solutions in the foregoing one or more embodiments.

As shown in Fig. 2, the registration method may include the following steps.

S210. Acquire an initial three-dimensional image template and a reference image corresponding to the target user, wherein the reference image is a two-dimensional image taken for the target user during the operation.

Specifically, S210 is similar to S110 and will not be repeated here.

S220. Perform rigid rough registration on the initial three-dimensional image template and the reference image to obtain a transformation matrix.

Specifically, the transformation matrix is used to characterize the displacement of the initial 3D image template. Rigid coarse registration refers to using the reference image as a reference to obtain the transformation matrix corresponding to the initial 3D image template, so that the human body in the initial 3D image template will not be deformed after being transformed according to the transformation matrix (that is, after displacement), but The positions of the transformed human body and the reconstructed human body from the reference image are approximately aligned (that is, the positions of the image pixels of the same anatomical structure are approximately aligned in the same reference frame).

In some embodiments, optionally, S220 may include:

S221. Extract matching feature point pairs from the initial 3D image template and the reference image, where the feature point pairs include 3D feature points on the initial 3D image template and 2D feature points on the reference image.

Specifically, the number of reference images is greater than or equal to 2, and the same feature point pair includes one 3D feature point and at least two 2D feature points, wherein the 3D feature point and the 2D feature point represent the same anatomical marker point, that is, the same A human anatomy marker point is a 3D feature point on the initial 3D image template, and a 2D feature point on the reference image. For example, the reference image usually includes two kinds of front view image and side view image, therefore, the same feature point pair includes one 3D feature point and two 2D feature points.

Specifically, in response to the feature point selection operation, the feature point pair selected by the feature point selection operation can be extracted from the initial 3D image template and the reference image; or, any possible feature point extraction algorithm (such as SIFT, Harris and based on The key point detection algorithm of deep learning technology, etc.) extracts matching feature point pairs from the initial 3D image template and the reference image.

S222. Back-project the two-dimensional feature points into the three-dimensional coordinate system where the original three-dimensional image template is located, and obtain corresponding three-dimensional reference points.

S223. Perform point set registration on the 3D feature points and the 3D reference points to obtain a transformation matrix.

Specifically, the SVD algorithm or other algorithms known to those skilled in the art may be used to perform point set registration, which is not limited here.

It can be understood that the rigid coarse registration of the initial 3D image template and the reference image based on the feature point pairs is simple and easy to implement, which is beneficial to reduce the difficulty of overall image registration.

Of course, rigid rough registration can also be performed on the initial 3D image template and the reference image based on the grayscale or by using the trained rough registration model.

S230. Transform the initial 3D image template according to the transformation matrix to obtain an intermediate 3D image template.

Specifically, the intermediate 3D image template is a 3D image transformed from the initial 3D image template according to the transformation matrix.

S240. Perform elastic fine registration on the intermediate 3D image template and the reference image to obtain an initial 3D deformation field, wherein the initial 3D deformation field is used to represent the displacement of points on the intermediate 3D image template.

Specifically, the intermediate three-dimensional deformation field is used to characterize the displacement of points (ie, image pixels) on the intermediate three-dimensional image template. For the point on the intermediate three-dimensional image template, when it is displaced according to the corresponding displacement amount, the position error between it and the corresponding reconstruction point is within the preset error range, wherein the corresponding reconstruction point is the corresponding point on the reference image Backprojection reconstruction to points in the 3D coordinate system where the intermediate 3D image template resides.

Specifically, the elastic fine registration mentioned here refers to using the reference image as a reference to obtain the initial 3D deformation field corresponding to the intermediate 3D image template, so that the intermediate 3D image template is transformed according to the target initial 3D deformation field. The human body is deformed, and the coincidence degree of the deformed human body and the reconstructed human body from the reference image is within the preset second error range (that is, the image pixels of the same anatomical structure match correspondingly when they are in the same reference system), wherein, The preset second error range can be set according to actual conditions. It can be seen that the mapping from the intermediate 3D image template to the reference image can be realized according to the initial 3D deformation field.

In some embodiments, optionally, S240 may include:

S241. Project the intermediate 3D image template to the 2D coordinate system where the reference image is located, to obtain a corresponding projected 2D image.

Specifically, if the number of reference images is greater than or equal to 2 (for example, N), then for each of the N reference images, the intermediate 3D image template is projected to the two-dimensional coordinate system where the reference image is located to obtain the corresponding projection 2D image.

For example, if the reference image includes the front view image and the side view image of the target user, the middle three-dimensional image template is projected to the two-dimensional coordinate system where the front view image is located (that is, the middle three-dimensional image template is orthographically projected), and the front view image is obtained. The projected two-dimensional image corresponding to the image (that is, the front projection image), and project the middle three-dimensional image template to the two-dimensional coordinate system where the side shot image is located (that is, the side projection of the middle three-dimensional image template), and obtain the side shot image corresponding to The projected two-dimensional image (ie lateral projection image).

S242. Select initial 3D control points on the intermediate 3D image template, and project the initial 3D control points to the 2D coordinate system where the reference image is located, to obtain corresponding initial 2D control points.

Specifically, the initial 3D control point is a point (ie, an image pixel) selected from the intermediate 3D image template.

Specifically, if the number of reference images is greater than or equal to 2 (for example, N), then for each of the N reference images, the initial three-dimensional control point is projected to the two-dimensional coordinate system where the reference image is located to obtain the corresponding initial 2D control points.

For example, if the reference image includes the target user's front view image and side view image, the initial three-dimensional control point is projected to the two-dimensional coordinate system where the front view image is located (that is, the initial three-dimensional control point is orthographically projected), and the orthographic projection is obtained The initial two-dimensional control points on the image, and project the initial three-dimensional control points to the two-dimensional coordinate system where the side shot image is located (that is, the initial three-dimensional control point template side projection), to obtain the initial two-dimensional control points on the lateral projection image point.

In some embodiments, optionally, selecting an initial three-dimensional control point on the intermediate three-dimensional image template includes: performing edge detection on the intermediate three-dimensional image template to obtain a plurality of edge points; selecting an initial three-dimensional control point from the plurality of edge points .

Specifically, edge detection is a basic problem in image processing and computer vision, and the purpose of edge detection is to identify points (ie, edge points) with obvious brightness changes in an image.

Edge detection algorithms known to those skilled in the art, such as differential method and differential edge detection method, can be used to perform edge detection on the intermediate three-dimensional image template, which is not limited here.

Specifically, the initial three-dimensional control point may be selected from multiple edge points based on random sampling, uniform sampling, etc., but is not limited thereto.

Of course, the initial 3D control points selected by the control point selection operation may also be selected on the intermediate 3D image template in response to the control point selection operation.

S243. Perform elastic registration on the projected two-dimensional image and the reference image to obtain a target two-dimensional deformation field corresponding to the projected two-dimensional image, wherein the two-dimensional deformation field is used to represent a displacement of a point on the two-dimensional image.

Specifically, the target two-dimensional deformation field is used to characterize the displacement of a point (ie image pixel) on the projected two-dimensional image. For the point on the projected two-dimensional image, when it is displaced according to the corresponding displacement amount, the position error between it and the corresponding point on the reference image is within the preset third error range.

Specifically, the elastic fine registration mentioned here refers to using the reference image as a reference to obtain the target two-dimensional deformation field corresponding to the projected two-dimensional image, so that the projected two-dimensional image undergoes spatial transformation according to the target two-dimensional deformation field Afterwards, the human body is deformed, and the coincidence degree of the deformed human body and the human body in the reference image is within the preset third error range (that is, the image pixels of the same anatomical structure are matched correspondingly when they are in the same reference system), wherein, The preset third error range can be set according to actual conditions. It can be seen that the mapping from the projected 2D image to the reference image can be realized according to the target 2D deformation field.

S244. Transform the initial two-dimensional control points according to the target two-dimensional deformation field to obtain corresponding intermediate two-dimensional control points.

Specifically, for each initial two-dimensional control point, space transformation is performed on it according to its corresponding displacement to obtain its corresponding intermediate two-dimensional control point.

S245. Reconstruct the intermediate two-dimensional control point into the three-dimensional coordinate system where the intermediate three-dimensional image template is located by back-projection, and obtain corresponding reference three-dimensional control points.

S246. Determine an initial three-dimensional deformation field according to the initial three-dimensional control point and the reference three-dimensional control point.

In some embodiments, optionally, S246 may include: making a difference between the coordinates of the initial 3D control point and the coordinates of the reference 3D control point to obtain the displacement corresponding to the initial 3D control point; The displacement corresponding to the points other than the initial three-dimensional control points on the three-dimensional image template, wherein the specific implementation method of interpolation may include: first, delauny generates a triangle set based on all initial three-dimensional control points, and the set of vertices of all triangles is all initial three-dimensional control points, Since the displacement of the three vertices of the triangle (that is, the initial three-dimensional control point) is known, the displacement of the point (that is, the pixel point) within the range of each triangle can be obtained by calculating the distance from the point to each vertex of the triangle and taking the sum of the weights .

It is understandable that, compared to treating each point on the intermediate 3D image template as an initial 3D control point, the displacement corresponding to a part of the points (that is, the initial 3D control point) is determined through elastic fine-fitting, and then determined by interpolation The displacements corresponding to points other than the initial three-dimensional control points can reduce the amount of calculation, thereby improving the overall efficiency of image registration.

It can also be understood that the flexible and precise registration of the intermediate 3D image template and the reference image in S241-S246 above has simple steps and is easy to implement, which is beneficial to reduce the difficulty of overall image registration.

Of course, the trained fine registration model can also be used to perform elastic fine registration on the intermediate 3D image template and the reference image to obtain the initial 3D deformation field.

S250. Determine a target three-dimensional deformation field according to the transformation matrix and the initial three-dimensional deformation field.

In the embodiment of the present disclosure, the method of combining rigid coarse registration and elastic fine registration is used to perform elastic registration on the initial 3D image template and the reference image to obtain the target 3D deformation field corresponding to the initial 3D image template, which can improve The accuracy of the registration algorithm improves the accuracy of subsequent planning and navigation.

Below, based on a specific example, the image registration process provided by the embodiment of the present disclosure will be described in detail.

Exemplarily, FIG. 3 is a schematic flowchart of a rough registration process provided by an embodiment of the present disclosure. Fig. 4 is a schematic flowchart of a fine registration process provided by an embodiment of the present disclosure.

See Figure 3. During rough registration, the 3D volume data template (i.e., the initial 3D image template, such as a 3D CT image) is used as a floating image, and the intraoperative 2D frontal view and intraoperative 2D lateral view are used (i.e., the reference image, such as the frontal Two-dimensional X-ray images and lateral two-dimensional X-ray images) are fixed images, and the purpose of rough registration is to obtain the initial transformation matrix from the template space of 3D volume data to the space of intraoperative 2D front view and intraoperative 2D lateral view. The specific steps include: 3.1) Extract the corresponding feature points on the intraoperative 2D front view film, the intraoperative 2D lateral view film and the 3D volume data template to obtain feature point pairs, wherein the feature point extraction is performed manually or automatically. Any method of identification is acceptable. 3.2) The 2D feature points corresponding to the intraoperative 2D frontal view and the intraoperative 2D lateral view are back-projected and reconstructed to obtain a series of 3D feature points (ie, three-dimensional reference points) in 3D space. 3.3) Perform point set registration between the reconstructed 3D feature points and the feature points (ie, three-dimensional feature points) extracted from the 3D volume data template to obtain an initial transformation matrix.

Referring to Figure 4, the 3D volume data template image is transformed using the initial transformation matrix obtained from the above-mentioned rough registration, and the transformed 3D volume data template image (ie, the intermediate 3D image template) is used as a finely registered floating image (ie, input) . Fine registration mainly includes the following steps: 4.1) Perform frontal and lateral front projection on the transformed 3D volume data template, respectively, to obtain frontal DRR and lateral DRR (that is, project two-dimensional images). 4.2) Generate a series of 3D control points (i.e. initial 3D control points) on the transformed 3D volume data template, and generate the corresponding 2D control points in the projection space (i.e. initial 2D control points) through the same front projection process as step 4.1). dimension control point). 4.3) The anterior DRR and lateral DRR obtained in 4.1) are elastically registered with the intraoperative 2D anteroposterior film and intraoperative 2D lateral film in 2D space to obtain a 2D deformation field (i.e., the target two-dimensional deformation field ), which is also the displacement transformation matrix of each pixel in 2D space. 4.4) Use the 2D deformation field in step 4.3) to transform the 2D control point in step 4.4), and obtain the updated coordinates of the 2D control point (ie, the intermediate two-dimensional control point). 4.5) Reconstruct the 2D control points updated in step 4.4) into the 3D space through back projection, and obtain the coordinates of the corresponding updated 3D control points (ie, refer to the 3D control points). 4.6) Subtract the coordinates of the 3D control points generated in step 4.2) from the coordinates of the 3D control points updated in step 4.5) to obtain the displacement of each 3D control point, and then obtain the 3D control points by interpolation The final 3D deformation field (that is, the initial 3D deformation field) can be obtained.

It is understandable that the use of 3D volume data templates instead of preoperative 3D CT images can reduce the steps of preoperative CT imaging and reduce the radiation dose for patients. Compared with planning on intraoperative 2D X-ray images, planning on 3D volume data templates can significantly improve accuracy.

Fig. 5 is a schematic structural diagram of a registration device provided by an embodiment of the present disclosure. The registration device may be understood as the above-mentioned electronic device or some functional modules in the above-mentioned electronic device. As shown in Figure 5, the registration device 500 includes:

An acquisition module 510, configured to acquire an initial three-dimensional image template and a reference image corresponding to the target user, wherein the reference image is a two-dimensional image taken for the target user during the operation;

The registration module 520 is configured to elastically register the initial 3D image template and the reference image to obtain a target 3D deformation field corresponding to the initial 3D image template, wherein the target 3D deformation field is used to represent the points on the initial 3D image template displacement.

In another implementation manner of the present disclosure, the acquisition module 510 may include a first acquisition submodule for acquiring an initial three-dimensional image template, wherein the first acquisition submodule may include:

an acquisition unit, configured to acquire user information of the target user;

The search unit is configured to search for an initial three-dimensional image template matching user information from a preset template library.

In yet another implementation manner of the present disclosure, the registration module 520 may include:

The coarse registration sub-module is used for performing rigid coarse registration on the initial 3D image template and the reference image to obtain a transformation matrix;

The transformation sub-module is used to transform the initial three-dimensional image template according to the transformation matrix to obtain the intermediate three-dimensional image template;

The fine registration sub-module is used for elastic fine registration of the intermediate 3D image template and the reference image to obtain an initial 3D deformation field, wherein the initial 3D deformation field is used to represent the displacement of points on the intermediate 3D image template;

The determining submodule is used to determine the target three-dimensional deformation field according to the transformation matrix and the initial three-dimensional deformation field.

In yet another embodiment of the present disclosure, the rough registration submodule may include:

An extraction unit, configured to extract matched feature point pairs from the initial 3D image template and the reference image, wherein the feature point pairs include 3D feature points located on the initial 3D image template and 2D feature points located on the reference image;

The first back-projection reconstruction unit is used for back-projection reconstruction of two-dimensional feature points to the three-dimensional coordinate system where the initial three-dimensional image template is located, to obtain corresponding three-dimensional reference points;

The first registration unit is configured to perform point set registration on the 3D feature points and the 3D reference points to obtain a transformation matrix.

In yet another embodiment of the present disclosure, the fine registration submodule may include:

A projection unit, configured to project the intermediate three-dimensional image template to the two-dimensional coordinate system where the reference image is located, to obtain a corresponding projected two-dimensional image;

A selection unit is used to select an initial three-dimensional control point on the intermediate three-dimensional image template, and project the initial three-dimensional control point to the two-dimensional coordinate system where the reference image is located to obtain a corresponding initial two-dimensional control point;

The second registration unit is configured to elastically register the projected two-dimensional image and the reference image to obtain a target two-dimensional deformation field corresponding to the projected two-dimensional image, wherein the two-dimensional deformation field is used to represent points on the two-dimensional image displacement;

A transformation unit, configured to transform the initial two-dimensional control points according to the target two-dimensional deformation field, to obtain corresponding intermediate two-dimensional control points;

The second back-projection reconstruction unit is used to reconstruct the back-projection of the middle two-dimensional control point to the three-dimensional coordinate system where the middle three-dimensional image template is located, and obtain the corresponding reference three-dimensional control point;

A determination unit is used to determine an initial three-dimensional deformation field according to the initial three-dimensional control point and the reference three-dimensional control point.

In yet another embodiment of the present disclosure, the determination unit is specifically configured to make a difference between the coordinates of the initial three-dimensional control point and the coordinates of the reference three-dimensional control point to obtain the displacement corresponding to the initial three-dimensional control point; according to a preset interpolation algorithm, Determine displacements corresponding to points other than the initial 3D control points on the intermediate 3D image template.

In yet another embodiment of the present disclosure, the device may also include a selection module for selecting initial 3D control points on the intermediate 3D image template, wherein the selection module may include:

The edge detection sub-module is used to perform edge detection on the intermediate three-dimensional image template to obtain a plurality of edge points;

The selection submodule is used to select initial 3D control points from multiple edge points.

The device provided in this embodiment can execute the method in any of the foregoing embodiments, and its execution mode and beneficial effect are similar, and details are not repeated here.

An embodiment of the present disclosure also provides an electronic device, which includes: a memory, a computer program is stored in the memory; a processor, configured to execute the computer program, and when the computer program is executed by the processor, the computer program can be Implement the method of any one of the above embodiments.

As an example, FIG. 6 is a schematic structural diagram of an electronic device in an embodiment of the present disclosure. Referring to FIG. 6 in detail below, it shows a schematic structural diagram of an electronic device 600 suitable for implementing an embodiment of the present disclosure. The electronic device 600 in the embodiment of the present disclosure may include, but is not limited to, mobile phones, notebook computers, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Tablet Computers), PMPs (Portable Multimedia Players), vehicle-mounted terminals ( Mobile terminals such as car navigation terminals) and stationary terminals such as digital TVs, desktop computers and the like. The electronic device shown in FIG. 6 is only an example, and should not limit the functions and application scope of the embodiments of the present disclosure.

As shown in FIG. 6, an electronic device 600 may include a processing device (such as a central processing unit, a graphics processing unit, etc.) 601, which may be randomly accessed according to a program stored in a read-only memory (ROM) 602 or loaded from a storage device 608. Various appropriate actions and processes are executed by programs in the memory (RAM) 603 . In the RAM 603, various programs and data necessary for the operation of the electronic device 600 are also stored. The processing device 601, ROM 602, and RAM 603 are connected to each other through a bus 604. An input/output (I/O) interface 605 is also connected to the bus 604 .

Typically, the following devices can be connected to the I/O interface 605: input devices 606 including, for example, a touch screen, touchpad, keyboard, mouse, camera, microphone, accelerometer, gyroscope, etc.; including, for example, a liquid crystal display (LCD), speaker, vibration an output device 607 such as a computer; a storage device 608 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 609. The communication means 609 may allow the electronic device 600 to communicate with other devices wirelessly or by wire to exchange data. While FIG. 6 shows electronic device 600 having various means, it should be understood that implementing or having all of the means shown is not a requirement. More or fewer means may alternatively be implemented or provided.

In particular, according to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present disclosure include a computer program product, which includes a computer program carried on a non-transitory computer readable medium, where the computer program includes program code for executing the method shown in the flowchart. In such an embodiment, the computer program may be downloaded and installed from a network via communication means 609, or from storage means 608, or from ROM 602. When the computer program is executed by the processing device 601, the above-mentioned functions defined in the methods of the embodiments of the present disclosure are performed.

It should be noted that the computer-readable medium mentioned above in the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the two. A computer readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer diskettes, hard disks, random access memory (RAM), read-only memory (ROM), erasable Programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination of the above. In the present disclosure, a computer-readable storage medium may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In the present disclosure, however, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave carrying computer-readable program code therein. Such propagated data signals may take many forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the foregoing. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can transmit, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device . Program code embodied on a computer readable medium may be transmitted by any appropriate medium, including but not limited to wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

In some embodiments, the client and the server can communicate using any currently known or future-developed network protocols such as HTTP (HyperText Transfer Protocol), and can communicate with digital data in any form or medium (eg, communication network) interconnections. Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), internetworks (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future developed network of.

The above-mentioned computer-readable medium may be included in the above-mentioned electronic device, or may exist independently without being incorporated into the electronic device.

The above-mentioned computer-readable medium carries one or more programs, and when the above-mentioned one or more programs are executed by the electronic device, the electronic device:

Obtaining an initial three-dimensional image template and a reference image corresponding to the target user, wherein the reference image is a two-dimensional image taken for the target user during the operation;

Elastic registration is performed on the initial 3D image template and the reference image to obtain a target 3D deformation field corresponding to the initial 3D image template, wherein the target 3D deformation field is used to represent the displacement of points on the initial 3D image template.

Computer program code for carrying out operations of the present disclosure may be written in one or more programming languages, or combinations thereof, including but not limited to object-oriented programming languages—such as Java, Smalltalk, C++, and Includes conventional procedural programming languages - such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer or entirely on the remote computer or server. In cases involving a remote computer, the remote computer can be connected to the user computer through any kind of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computer (such as through an Internet service provider). Internet connection).

The flowchart and block diagrams in the Figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present disclosure. In this regard, each block in a flowchart or block diagram may represent a module, program segment, or portion of code that contains one or more logical functions for implementing specified executable instructions. It should also be noted that, in some alternative implementations, the functions noted in the block may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or they may sometimes be executed in the reverse order, depending upon the functionality involved. It should also be noted that each block of the block diagrams and/or flowchart illustrations, and combinations of blocks in the block diagrams and/or flowchart illustrations, can be implemented by a dedicated hardware-based system that performs the specified functions or operations , or may be implemented by a combination of dedicated hardware and computer instructions.

The units involved in the embodiments described in the present disclosure may be implemented by software or by hardware. Wherein, the name of a unit does not constitute a limitation of the unit itself under certain circumstances.

The functions described herein above may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: Field Programmable Gate Arrays (FPGAs), Application Specific Integrated Circuits (ASICs), Application Specific Standard Products (ASSPs), System on Chips (SOCs), Complex Programmable Logical device (CPLD) and so on.

In the context of the present disclosure, a machine-readable medium may be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium may be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media would include one or more wire-based electrical connections, portable computer discs, hard drives, random access memory (RAM), read only memory (ROM), erasable programmable read only memory (EPROM or flash memory), optical fiber, compact disk read only memory (CD-ROM), optical storage, magnetic storage, or any suitable combination of the foregoing.

An embodiment of the present disclosure also provides a computer-readable storage medium, in which a computer program is stored, and when the computer program is executed by a processor, the method of any of the above-mentioned embodiments can be implemented, and its execution mode and beneficial effects Similar and will not be repeated here.

It should be noted that in this article, relative terms such as "first" and "second" are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply these No such actual relationship or order exists between entities or operations. Furthermore, the term "comprises", "comprises" or any other variation thereof is intended to cover a non-exclusive inclusion such that a process, method, article, or apparatus comprising a set of elements includes not only those elements, but also includes elements not expressly listed. other elements of or also include elements inherent in such a process, method, article, or device. Without further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of additional identical elements in the process, method, article or apparatus comprising said element.

The above descriptions are only specific implementation manners of the present disclosure, so that those skilled in the art can understand or implement the present disclosure. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present disclosure. Therefore, the present disclosure will not be limited to the embodiments described herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

Claims (10)

1. A registration method, characterized in that, comprising: Acquiring an initial three-dimensional image template and a reference image corresponding to the target user, wherein the reference image is a two-dimensional image taken for the target user during the operation; performing elastic registration on the initial three-dimensional image template and the reference image to obtain a target three-dimensional deformation field corresponding to the initial three-dimensional image template, wherein the target three-dimensional deformation field is used to characterize the initial three-dimensional image template The displacement of the point of . 2. The method according to claim 1, wherein said obtaining an initial three-dimensional image template comprises: Obtain user information of the target user; Searching for the initial three-dimensional image template matching the user information from a preset template library. 3. The method according to claim 1, wherein the elastic registration of the initial three-dimensional image template and the reference image is performed to obtain a target three-dimensional deformation field corresponding to the initial three-dimensional image template, comprising : performing rigid rough registration on the initial three-dimensional image template and the reference image to obtain a transformation matrix; transforming the initial three-dimensional image template according to the transformation matrix to obtain an intermediate three-dimensional image template; performing elastic fine registration on the intermediate three-dimensional image template and the reference image to obtain an initial three-dimensional deformation field, wherein the initial three-dimensional deformation field is used to characterize the displacement of points on the intermediate three-dimensional image template; The target three-dimensional deformation field is determined according to the transformation matrix and the initial three-dimensional deformation field. 4. The method according to claim 3, wherein said initial three-dimensional image template and said reference image are subjected to rigid rough registration to obtain a transformation matrix, comprising: Extract matched feature point pairs from the initial 3D image template and the reference image, wherein the feature point pairs include 3D feature points located on the initial 3D image template and 2D feature points located on the reference image Feature points; Back-projecting the two-dimensional feature points to the three-dimensional coordinate system where the initial three-dimensional image template is located, to obtain corresponding three-dimensional reference points; Perform point set registration on the 3D feature points and the 3D reference points to obtain the transformation matrix. 5. The method according to claim 3, wherein the elastic fine registration of the intermediate 3D image template and the reference image to obtain an initial 3D deformation field comprises: Projecting the intermediate three-dimensional image template to the two-dimensional coordinate system where the reference image is located to obtain a corresponding projected two-dimensional image; Selecting initial three-dimensional control points on the intermediate three-dimensional image template, and projecting the initial three-dimensional control points to the two-dimensional coordinate system where the reference image is located, to obtain corresponding initial two-dimensional control points; performing elastic registration on the projected two-dimensional image and the reference image to obtain a target two-dimensional deformation field corresponding to the projected two-dimensional image, wherein the two-dimensional deformation field is used to represent the displacement of a point on the two-dimensional image quantity; transforming the initial two-dimensional control points according to the target two-dimensional deformation field to obtain corresponding intermediate two-dimensional control points; Back-projecting the intermediate two-dimensional control points to the three-dimensional coordinate system where the intermediate three-dimensional image template is located, to obtain corresponding reference three-dimensional control points; The initial three-dimensional deformation field is determined according to the initial three-dimensional control point and the reference three-dimensional control point. 6. The method according to claim 5, wherein the determining the initial three-dimensional deformation field according to the initial three-dimensional control point and the reference three-dimensional control point comprises: Making a difference between the coordinates of the initial three-dimensional control point and the coordinates of the reference three-dimensional control point to obtain the displacement corresponding to the initial three-dimensional control point; According to a preset interpolation algorithm, determine displacements corresponding to points other than the initial 3D control points on the intermediate 3D image template. 7. The method according to claim 5, wherein the selecting an initial three-dimensional control point on the intermediate three-dimensional image template comprises: Perform edge detection on the intermediate three-dimensional image template to obtain a plurality of edge points; The initial three-dimensional control point is selected from the plurality of edge points. 8. A registration device, characterized in that it comprises: An acquisition module, configured to acquire an initial three-dimensional image template and a reference image corresponding to the target user, wherein the reference image is a two-dimensional image taken for the target user during the operation; A registration module, configured to register the initial 3D image template and the reference image to obtain a target 3D deformation field corresponding to the initial 3D image template, wherein the 3D deformation field is used to characterize the initial 3D The amount of displacement of a point on the image template. 9. An electronic device, characterized in that it comprises: A processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the processor executes the method according to any one of claims 1-7. 10. A computer-readable storage medium, wherein a computer program is stored in the storage medium, and when the computer program is executed by a processor, the method according to any one of claims 1-7 is implemented .

Images