CN118657817B - Image processing method, system, medium and product - Google Patents

Abstract

The invention discloses an image processing method, an image processing system, a medium and a product. The method comprises the following steps: acquiring mouth scan data and a navigation CT image, and determining a first registration result between the mouth scan data and the navigation CT image; if the current navigation scene image accords with the image non-shielding condition, taking the current navigation scene image as a current key frame image, and determining a navigation rough registration result between the mouth scan data and the current key frame image; determining projection data of the mouth scan data on a plane where the current key frame image is located according to the navigation rough registration result to obtain a point cloud projection image; and determining a first matching result between the point cloud projection image and the current key frame image, and converting the navigation CT image into a coordinate system where the navigation system is located according to the first registration result and the first matching result. The technical scheme provided by the embodiment of the invention can accurately map the navigation CT image to the coordinate system where the navigation system is located without the participation of the marker.

Description

Image processing method, system, medium and product

Technical Field

The present invention relates to the technical field of medical image processing, and in particular to an image processing method, system, medium and product.

Background Art

Optical navigation implantation methods are widely used due to their high intraoperative flexibility. However, existing optical navigation implantation methods usually require the bonding of markers (such as designated reference plates) on the patient's teeth to facilitate image registration and surgical navigation. If the marker does not shake due to surgical collision, it can ensure a high surgical navigation effect; if the marker shakes due to collision during surgery, it is usually necessary to re-register based on the new position of the marker. Therefore, the optical navigation implantation method requires the user to perfectly avoid the marker during surgery.

In summary, the navigation accuracy of existing optical navigation implantation methods depends on the stability of the markers, and this dependence reduces the flexibility of navigation.

Summary of the invention

The present invention provides an image processing method to solve the problem that the navigation accuracy of the existing optical navigation implantation method depends on the degree of fixation of the marker.

According to one aspect of the present invention, there is provided an image processing method, comprising:

Acquire oral scan data and a navigation CT (Computed Tomography) image, and determine a first registration result between the oral scan data and the navigation CT image, wherein both the oral scan data and the navigation CT image include teeth of a target object;

If the current navigation scene image meets the image unobstructed condition, the current navigation scene image is used as the current key frame image, and a navigation rough registration result between the oral scan data and the current key frame image is determined;

Determine the projection data of the oral scan data on the plane where the current key frame image is located according to the navigation rough registration result, and obtain a point cloud projection image;

A first matching result between the point cloud projection image and the current key frame image is determined, and according to the first registration result and the first matching result, the navigation CT image is converted to the coordinate system of the navigation system.

According to another aspect of the present invention, there is provided an image processing apparatus, comprising:

A first registration module, used for acquiring oral scan data and a navigation CT image, and determining a first registration result between the oral scan data and the navigation CT image, wherein both the oral scan data and the navigation CT image include teeth of a target object;

An unobstructed module, used for taking the current navigation scene image as the current key frame image if the current navigation scene image meets the image unobstructed condition, and determining a navigation rough registration result between the oral scan data and the current key frame image;

A projection module, used to determine the projection data of the oral scan data on the plane where the current key frame image is located according to the navigation coarse registration result, and obtain a point cloud projection image;

The second registration module is used to determine a first matching result between the point cloud projection image and the current key frame image, and determine the position and posture of the navigation CT image in the navigation system according to the first registration result and the first matching result.

According to another aspect of the present invention, an electronic device is provided, wherein the navigation system comprises:

Surgical instruments;

A camera device, used to obtain a navigation scene image during surgery on the target object, wherein the navigation scene image includes the surgical instrument;

A processor, configured to execute the image processing method described in any embodiment;

The display device displays the navigation CT image and the navigation scene image at the coordinates of the navigation system.

According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method described in any embodiment of the present invention when executed.

According to another aspect of the present invention, a computer program product is provided. The computer program product comprises a computer program. When the computer program is executed by a processor, the image processing method according to any embodiment is implemented.

In the technical solution of the embodiment of the present invention, the current key frame image is a two-dimensional image of the teeth of the target object, and the oral scan data is a three-dimensional image of the teeth of the target object; first, the navigation rough registration result between the oral scan data and the current key frame image is determined, and then the projection angle of view is determined according to the navigation rough registration result, and according to the projection angle of view, the oral scan data is projected to the plane where the current key frame image is located to obtain a point cloud projection image; since the point cloud projection image and the current key frame image correspond to the same angle of view and are both two-dimensional images of the same object, the first matching result between the two has high accuracy; since the first matching result is the precise registration result of the current key image and part of the data in the oral scan data, the precise registration result of the current key frame image and the entire oral scan data can be obtained based on the first matching result, and then the precise registration result between the current key frame image and the navigation CT image is obtained with the oral scan data as the medium, and then based on the precise registration result and the first registration result between the navigation CT image and the oral scan data, the navigation CT image is mapped to the coordinate system where the navigation system is located. Since the first registration result, the first matching result and the precise registration result between the current key frame image and the entire oral scan data all have high accuracy, the final precise registration result between the current key frame image and the navigation CT image has high accuracy. Therefore, the navigation CT image can be accurately mapped to the coordinate system of the navigation system, thereby achieving precise registration between the navigation CT image and the navigation image without reference markers.

It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present invention, nor are they intended to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description.

BRIEF DESCRIPTION OF THE DRAWINGS

In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.

FIG1 is a flow chart of an image processing method provided according to an embodiment of the present invention;

FIG2 is a flow chart of another image processing method provided according to an embodiment of the present invention;

FIG3A is a schematic diagram of the structure of an image processing device provided according to an embodiment of the present invention;

FIG3B is another schematic diagram of the structure of an image processing device provided according to an embodiment of the present invention;

FIG3C is another schematic diagram of the structure of an image processing device provided according to an embodiment of the present invention;

FIG4 is a schematic diagram of the structure of a surgical navigation system provided according to an embodiment of the present invention;

FIG. 5 is another schematic diagram of the structure of a surgical navigation system according to an embodiment of the present invention.

DETAILED DESCRIPTION

In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations of the two are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

FIG1 is a flow chart of an image processing method provided by an embodiment of the present invention. This embodiment is applicable to the case of performing surgical navigation without introducing a reference plate. The method can be executed by an image processing device, which can be implemented in the form of hardware and/or software. The image processing device can be configured in a processor of a navigation system. As shown in FIG1 , the method includes:

S110, acquiring oral scan data and a navigation CT image, and determining a first registration result between the oral scan data and the navigation CT image, wherein both the oral scan data and the navigation CT image include teeth of the target object.

Navigation CT images are three-dimensional images acquired using methods such as CBCT (cone beam CT).

After the navigation CT image is acquired, a target position is determined thereon manually or automatically, and the target position is a surgical position, such as a dental implant position.

The first registration result is a precise registration result. In one embodiment, the first registration result is determined by the following steps:

Step a1: performing tooth segmentation on the oral scan data to obtain a first tooth segmentation result, and performing tooth segmentation on the navigation CT image to obtain a second tooth segmentation result.

The first tooth segmentation result and the second tooth segmentation result both include the tooth position number and three-dimensional shape of each tooth. In one embodiment, artificial intelligence technology is used to perform tooth segmentation on the oral scan data and the navigation CT image to obtain the three-dimensional shape and tooth position number of each tooth. The tooth position number is the tooth number. This embodiment can quickly and accurately complete the tooth segmentation of the oral scan data and the navigation CT image.

Step a2: respectively determine the first tooth position feature of each tooth in the first tooth position segmentation result and the second tooth position feature of each tooth in the second tooth position segmentation result.

The first tooth position feature and the second tooth position feature include the same feature items, such as the centroid, the occlusal surface center, the gum line vertex, etc.

Step a3: Determine a first rough registration result between the oral scan data and the navigation CT image based on the first tooth position feature of each tooth in the first tooth position segmentation result and the second tooth position feature of each tooth in the second tooth position segmentation result.

Step a4: Based on the three-dimensional morphology of each tooth in the first tooth segmentation result and the three-dimensional morphology of each tooth in the second tooth segmentation result, the first coarse registration result is adjusted to obtain a first registration result between the oral scan data and the navigation CT image.

In one embodiment, the first rough registration result is adjusted by using an iterative closest point algorithm, the three-dimensional morphology of each tooth in the first tooth segmentation result, and the three-dimensional morphology of each tooth in the second tooth segmentation result to obtain a precise registration result between the oral scan data and the navigation CT image, and the precise registration result is used as the first registration result. The determination of the first registration result marks the end of the preoperative preparation process.

S120: If the current navigation scene image meets the image unobstructed condition, the current navigation scene image is used as the current key frame image, and a navigation coarse registration result between the oral scan data and the current key frame image is determined.

After the preoperative process is completed and the intraoperative navigation begins, the current navigation scene image is captured by the camera device. The current navigation scene image is a two-dimensional image, which includes the teeth of the target object.

In order to ensure the surgical navigation results, it is necessary to obtain key frame images at the beginning of the surgery. The so-called key frame image refers to a navigation scene image that meets the image unobstructed condition, specifically, a navigation scene image in which the ratio of the obstructed tooth area to the total tooth area is less than or equal to the set ratio threshold. Therefore, at the beginning of intraoperative navigation, the key frame image must be acquired first, and then the navigation rough registration result between the key frame image and the oral scan data must be determined.

In one embodiment, the navigation rough registration result between the oral scan data and the current key frame image is determined by the following steps:

Step b1, determining each tooth region in the current key frame image, and a third tooth position feature of each tooth region in the current key frame image.

Perform tooth segmentation on the current key frame image to obtain various tooth areas, and determine the third tooth position feature of each tooth area. The feature content included in the third tooth position feature is the same as the feature items included in the first tooth position feature, such as both including the center of mass, occlusal surface center, gum line vertex, etc.

Step b2: determining a navigation rough registration result between the oral scan data and the current key frame image according to the first tooth position feature and the third tooth position feature.

Since the oral scan data is three-dimensional data and the current key frame image is a two-dimensional image, there must be a difference between the first tooth position feature determined based on the oral scan data and the third tooth position feature determined based on the current key frame image. Therefore, based on the first tooth position feature and the third tooth position feature, only the oral scan data and the current key frame image can be roughly aligned to obtain the navigation rough alignment result.

In one embodiment, the PnP (Perspective-n-Point) algorithm is used to determine the navigation rough registration result between the current key frame image and the oral scan data. PnP is a method for solving the correspondence between 3D and 2D points. It describes an algorithm for estimating the camera's pose when n 3D space points and their positions are known. It can be solved by direct linear transformation (DLT), effective PnP (Efficient PnP), bundle adjustment (BA), etc.

S130. Determine the projection data of the oral scan data on the plane where the current key frame image is located according to the navigation coarse registration result, and obtain a point cloud projection image.

Although the accuracy of the coarse registration result cannot meet the pixel-level registration requirements, the target projection angle can be determined, and then the mouth scan data can be projected according to the target projection angle, so that the mouth scan data can be projected to the plane where the current key frame image is located to obtain a point cloud projection image. Therefore, the point cloud projection image in this embodiment corresponds to the image and the current key frame image is a two-dimensional image at the same viewing angle.

S140, determining a first matching result between the point cloud projection image and the current key frame image, and converting the navigation CT image to a coordinate system where the navigation system is located according to the first registration result and the first matching result.

In one embodiment, each tooth region in the point cloud projection image and the fourth tooth position feature of each tooth region in the point cloud projection image are determined; and a first matching result between the point cloud projection image and the current key frame image is determined based on the fourth tooth position feature and the third tooth position feature. Since the image corresponding to the point cloud projection image and the current key frame image are two-dimensional images under the same viewing angle, the point cloud projection image and the current key frame image can be sub-pixel mapped based on the fourth tooth position feature of each tooth region in the point cloud projection image and the third tooth position feature of the current key frame image to obtain a first matching result.

After the first matching result is determined, a second registration result between the oral scan data and the current key frame image is determined according to the first matching result; and a position of the navigation CT image in the navigation system is determined according to the first registration result and the second registration result.

Specifically, the mapping between the oral scan data and the current key frame image is updated according to the first matching result, and then the PnP algorithm is used to optimize the mapping relationship between the two to obtain the second registration result. Since the first registration result is the registration result between the oral scan data and the navigation CT image, the registration result between the current key frame image and the navigation CT image can be obtained using the oral scan data as a medium, and then the position and posture of the navigation CT image in the navigation system is determined according to the registration result, that is, the navigation CT image is mapped to the coordinate system where the navigation system is located.

In one embodiment, the surgical instrument and the surgical instrument posture in the current navigation scene image, as well as the target coordinates of the target position predetermined in the navigation CT image under the navigation system are determined; the motion path of the surgical instrument is determined based on the target coordinates and the surgical instrument posture; and guidance information for guiding the surgical instrument to move to the target coordinates is generated based on the motion path.

Specifically, since dental implants require surgical instruments, the current navigation scene image includes surgical instruments. The positioning result of the surgical instrument in the current navigation scene image is determined, and the surgical instrument posture in the current navigation scene image and the target coordinates of the target position predetermined in the navigation CT image under the navigation system are determined; since the target coordinates are the surgical area, the surgical instrument needs to be moved to the target coordinates. Therefore, after the target coordinates and the surgical instrument posture are determined, the motion path of the surgical instrument is determined based on the two, and then the user is guided to move the surgical instrument to the target coordinates based on the motion path.

When it is necessary to explain, the existing technology can be used to determine the positioning result of the surgical instrument in the current navigation scene image, and this embodiment will not be described in detail here.

In the technical solution provided by the embodiment of the present invention, the current key frame image is a two-dimensional image of the teeth of the target object, and the oral scan data is a three-dimensional image of the teeth of the target object; first, the navigation rough registration result between the oral scan data and the current key frame image is determined, and then the projection angle of view is determined according to the navigation rough registration result, and according to the projection angle of view, the oral scan data is projected to the plane where the current key frame image is located to obtain a point cloud projection image; since the point cloud projection image and the current key frame image correspond to the same angle of view and are both two-dimensional images of the same object, the first matching result between the two has high accuracy; since the first matching result is the precise registration result of the current key image and part of the data in the oral scan data, the precise registration result of the current key frame image and the entire oral scan data can be obtained based on the first matching result, and then the precise registration result between the current key frame image and the navigation CT image is obtained with the oral scan data as the medium, and then based on the precise registration result and the first registration result between the navigation CT image and the oral scan data, the navigation CT image is mapped to the coordinate system where the navigation system is located. Since the first registration result, the first matching result and the precise registration result between the current key frame image and the entire oral scan data all have high accuracy, the final precise registration result between the current key frame image and the navigation CT image has high accuracy. Therefore, the navigation CT image can be accurately mapped to the coordinate system of the navigation system, thereby achieving precise registration between the navigation CT image and the navigation image without reference markers.

FIG2 is a flow chart of an image processing method provided by an embodiment of the present invention. The embodiment of the present invention adds a processing step for the current navigation scene image that does not meet the image unobstructed condition on the basis of the above-mentioned embodiment. As shown in FIG2, the method includes:

S210, acquiring oral scan data and a navigation CT image, determining a first registration result between the oral scan data and the navigation CT image, and determining whether the current navigation scene image does not meet an image unobstructed condition.

S220: If the current navigation scene image meets the image unobstructed condition, the current navigation scene image is used as the current key frame image, and a navigation coarse registration result between the oral scan data and the current key frame image is determined.

S230. Determine the projection data of the oral scan data on the plane where the current key frame image is located according to the navigation coarse registration result, and obtain a point cloud projection image.

S240, determining a first matching result between the point cloud projection image and the current key frame image, and converting the navigation CT image to a coordinate system where the navigation system is located according to the first registration result and the first matching result.

S250: If the current navigation scene image does not meet the image unobstructed condition, determine a second matching result between the current navigation scene image and the current key frame image.

In one embodiment, if the ratio of the obscured tooth area to all tooth areas in the current navigation scene image during the navigation process is greater than a set ratio threshold, it is determined that the current navigation scene image does not meet the image unobstructed condition, and a sub-pixel image processing method is used to determine a second matching result between the current navigation scene image and the current key frame image.

The current key frame image is the latest key frame image, the current navigation scene image is the latest acquired navigation image, and the refresh frequency of the navigation image is high during the navigation process. Therefore, the unobstructed tooth area in the current navigation scene image is relatively similar to the part of the tooth area in the current key frame image. Therefore, a sub-pixel image registration algorithm is used to match the current navigation scene image with the current key frame image to obtain a second matching result.

S260: Determine a third registration result between the current navigation scene image and the oral scan data according to the second registration result and the second matching result.

The second registration result and the second matching result, that is, the registration result between the mouth scan data and the current key frame image, and the registration result between the current navigation scene image and the current key frame image. Therefore, after the second registration result and the second matching result are determined, the mouth scan data can be used as a bridge to determine the precise registration result between the mouth scan data and the current navigation data, and the precise registration result is used as the third registration result.

S270: Determine the position and posture of the navigation CT image in the navigation system according to the first registration result and the third registration result.

Since the first registration result is a precise registration result between the oral scan data and the navigation CT image, the oral scan data can be used as a bridge to establish a precise registration result between the current navigation scene image and the navigation CT image, that is, the third registration result, and then the position and posture of the navigation CT image in the navigation system is determined based on the third registration result.

The technical solution provided by the embodiment of the present invention performs sub-pixel-level precise registration of the unobstructed tooth portion in the current navigation scene image that meets the occlusion condition with the corresponding portion of the current key frame image, so as to obtain a second matching result between the two. Then, using the current key frame as a bridge, a precise registration result between the current navigation scene image and the oral scan data, i.e., the third registration result, is established. Then, using the oral scan data as a bridge, a precise registration result between the current navigation scene image and the navigation CT image is established, so that the position and posture of the navigation CT image in the navigation system can be accurately determined. This achieves accurate registration between the current navigation scene image that meets the occlusion condition and the navigation CT image without the aid of any markers, thereby allowing the user to have higher flexibility during the operation.

FIG3A is a schematic diagram of the structure of an image processing device provided by an embodiment of the present invention. As shown in FIG3A , the device includes:

A first registration module 31 is used to obtain oral scan data and a navigation CT image, and determine a first registration result between the oral scan data and the navigation CT image, wherein both the oral scan data and the navigation CT image include teeth of a target object;

The unobstructed module 32 is used for taking the current navigation scene image as the current key frame image if the current navigation scene image meets the image unobstructed condition, and determining the navigation rough registration result between the oral scan data and the current key frame image;

A projection module 33 is used to determine the projection data of the oral scan data on the plane where the current key frame image is located according to the navigation rough registration result, and obtain a point cloud projection image;

The second registration module 34 is used to determine a first matching result between the point cloud projection image and the current key frame image, and convert the navigation CT image to the coordinate system of the navigation system according to the first registration result and the first matching result.

In one embodiment, the first registration module 31 is specifically used for:

Performing tooth segmentation on the oral scan data to obtain a first tooth segmentation result, and performing tooth segmentation on the navigation CT image to obtain a second tooth segmentation result;

respectively determining a first tooth position feature of each tooth in the first tooth position segmentation result and a second tooth position feature of each tooth in the second tooth position segmentation result;

Determine a first rough registration result between the oral scan data and the navigation CT image based on a first tooth position feature of each tooth in the first tooth position segmentation result and a second tooth position feature of each tooth in the second tooth position segmentation result;

Based on the first coarse registration result, the three-dimensional morphology of each tooth in the first tooth position segmentation result, and the three-dimensional morphology of each tooth in the second tooth position segmentation result, a first registration result between the oral scan data and the navigation CT image is determined.

In one embodiment, the non-blocking module 32 is specifically used for:

Determine each tooth region of the current key frame image, and a third tooth position feature of each tooth region in the current key frame image;

Determining a navigation rough registration result between the oral scan data and the current key frame image according to the first tooth position feature and the third tooth position feature;

In one embodiment, the second registration module 33 is specifically used for:

Determine each tooth region in the point cloud projection image, and a fourth tooth position feature of each tooth region in the point cloud projection image;

A first matching result between the point cloud projection image and the current key frame image is determined according to the fourth tooth position feature and the third tooth position feature.

In one embodiment, the second registration module 33 is specifically used for:

Determine a second registration result between the oral scan data and the current key frame image according to the first matching result;

The position and posture of the navigation CT image in the navigation system is determined according to the first registration result and the second registration result.

In one embodiment, as shown in FIG3B , the device further includes a shielding module 35, which is used to:

If the current navigation scene image during the navigation process does not meet the image unobstructed condition, determining a second matching result between the current navigation scene image and the current key frame image;

Determining a third registration result between the current navigation scene image and the oral scan data according to the second registration result and the second matching result;

The position and posture of the navigation CT image in the navigation system is determined according to the first registration result and the third registration result.

In one embodiment, the shielding module 35 is specifically used for:

If the ratio of the obscured tooth area to all tooth areas in the current navigation scene image during the navigation process is greater than a set ratio threshold, it is determined that the current navigation scene image does not meet the image unobstructed condition, and a sub-pixel image processing method is used to determine a second matching result between the current navigation scene image and the current key frame image.

In one embodiment, as shown in FIG. 3C , the device further includes a guiding module 36, and the guiding module 36 is used to:

Determine the surgical instrument and the surgical instrument posture in the current navigation scene image, and the target coordinates of the target position predetermined in the navigation CT image under the navigation system;

Determine the motion path of the surgical instrument according to the target coordinates and the position and posture of the surgical instrument;

Guidance information for guiding the movement of the surgical instrument is generated according to the movement path.

In the technical solution provided by the embodiment of the present invention, the current key frame image is a two-dimensional image of the teeth of the target object, and the oral scan data is a three-dimensional image of the teeth of the target object; first, the navigation rough registration result between the oral scan data and the current key frame image is determined, and then the projection angle of view is determined according to the navigation rough registration result, and according to the projection angle of view, the oral scan data is projected to the plane where the current key frame image is located to obtain a point cloud projection image; since the point cloud projection image and the current key frame image correspond to the same angle of view and are both two-dimensional images of the same object, the first matching result between the two has high accuracy; since the first matching result is the precise registration result of the current key image and part of the data in the oral scan data, the precise registration result of the current key frame image and the entire oral scan data can be obtained based on the first matching result, and then the precise registration result between the current key frame image and the navigation CT image is obtained with the oral scan data as the medium, and then based on the precise registration result and the first registration result between the navigation CT image and the oral scan data, the navigation CT image is mapped to the coordinate system where the navigation system is located. Since the first registration result, the first matching result and the precise registration result between the current key frame image and the entire oral scan data all have high accuracy, the final precise registration result between the current key frame image and the navigation CT image has high accuracy. Therefore, the navigation CT image can be accurately mapped to the coordinate system of the navigation system, thereby achieving precise registration between the navigation CT image and the navigation image without reference markers.

The image processing device provided by the embodiment of the present invention can execute the image processing method provided by any embodiment of the present invention, and has the corresponding functional modules and beneficial effects of the execution method.

Fig. 4 shows a schematic diagram of the structure of a navigation system 10 that can be used to implement an embodiment of the present invention. As shown in Fig. 4, the navigation system 10 includes a surgical instrument 101; a camera 102, which is used to obtain a navigation scene image of a target object during surgery, and the navigation scene image includes the surgical instrument; a processor 11, which is used to execute the image processing method described in any embodiment; and a display device 12, which displays the navigation CT image and the navigation scene image at the coordinates of the navigation system.

In one embodiment, as shown in FIG5 , the navigation system includes at least one processor 11, and a memory connected to the at least one processor 11 in communication, such as a read-only memory (ROM) 13, a random access memory (RAM) 14, etc., wherein the memory stores a computer program that can be executed by at least one processor, and the processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (ROM) 13 or the computer program loaded from the storage unit 18 to the random access memory (RAM) 14. In the RAM 14, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 13, and the RAM 14 are connected to each other through a bus 15. An input/output (I/O) interface 16 is also connected to the bus 15.

A number of components in the navigation system 10 are connected to the I/O interface 16, including: an input device 17, such as a keyboard, a mouse, etc.; an output device 12, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disk, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information/data with other devices through a computer network such as the Internet and/or various telecommunication networks.

The processor 11 may be a variety of general and/or special processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 performs the various methods and processes described above, such as an image processing method.

In some embodiments, the image processing method may be implemented as a computer program, which is tangibly contained in a computer-readable storage medium, such as a storage unit 18. In some embodiments, part or all of the computer program may be loaded and/or installed on the electronic device 10 via the ROM 13 and/or the communication unit 19. When the computer program is loaded into the RAM 14 and executed by the processor 11, one or more steps of the image processing method described above may be performed. Alternatively, in other embodiments, the processor 11 may be configured to perform the image processing method in any other appropriate manner (e.g., by means of firmware).

Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), load programmable logic devices (CPLDs), computer hardware, firmware, software, and/or combinations thereof. These various implementations can include: being implemented in one or more computer programs that can be executed and/or interpreted on a programmable system including at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

Computer programs for implementing the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, so that when the computer program is executed by the processor, the functions/operations specified in the flow chart and/or block diagram are implemented. The computer program may be executed entirely on the machine, partially on the machine, partially on the machine and partially on a remote machine as a stand-alone software package, or entirely on a remote machine or server.

In the context of the present invention, a computer-readable storage medium may be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, device, or equipment. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or equipment, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. A more specific example of a machine-readable storage medium may include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

To provide interaction with a user, the systems and techniques described herein may be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or trackball) through which the user can provide input to the electronic device. Other types of devices may also be used to provide interaction with the user; for example, the feedback provided to the user may be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user may be received in any form (including acoustic input, voice input, or tactile input).

The systems and techniques described herein may be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system may be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), a blockchain network, and the Internet.

A computing system may include a client and a server. The client and the server are generally remote from each other and usually interact through a communication network. The client and server relationship is generated by computer programs running on the corresponding computers and having a client-server relationship with each other. The server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in the cloud computing service system to solve the defects of difficult management and weak business scalability in traditional physical hosts and VPS services.

An embodiment of the present invention further provides a computer program product, including a computer program, which, when executed by a processor, implements the image processing method provided in any embodiment of the present application.

In the process of implementation, the computer program product can be written in one or more programming languages or a combination thereof to perform the computer program code of the present invention, including object-oriented programming languages, such as Java, Smalltalk, C++, and conventional procedural programming languages, such as "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as an independent software package, partially on the user's computer and partially on a remote computer, or completely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or can be connected to an external computer (for example, using an Internet service provider to connect through the Internet).

It should be understood that the various forms of processes shown above can be used to reorder, add or delete steps. For example, the steps described in the present invention can be executed in parallel, sequentially or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and this document does not limit this.

The above specific implementations do not constitute a limitation on the protection scope of the present invention. It should be understood by those skilled in the art that various modifications, combinations, sub-combinations and substitutions can be made according to design requirements and other factors. Any modification, equivalent substitution and improvement made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims (8)

1. An image processing method, comprising: Acquire oral scan data and a navigation CT image, and determine a first registration result between the oral scan data and the navigation CT image, wherein both the oral scan data and the navigation CT image include teeth of a target object; If the current navigation scene image meets the image unobstructed condition, the current navigation scene image is used as the current key frame image, and a navigation rough registration result between the oral scan data and the current key frame image is determined; Determine the projection data of the oral scan data on the plane where the current key frame image is located according to the navigation rough registration result, and obtain a point cloud projection image; Determine a first matching result between the point cloud projection image and the current key frame image, and transform the navigation CT image to a coordinate system where a navigation system is located according to the first registration result and the first matching result; Wherein, determining a first registration result between the oral scan data and the navigation CT image comprises: Performing tooth segmentation on the oral scan data to obtain a first tooth segmentation result, and performing tooth segmentation on the navigation CT image to obtain a second tooth segmentation result; respectively determining a first tooth position feature of each tooth in the first tooth position segmentation result and a second tooth position feature of each tooth in the second tooth position segmentation result; Determine a first rough registration result between the oral scan data and the navigation CT image based on a first tooth position feature of each tooth in the first tooth position segmentation result and a second tooth position feature of each tooth in the second tooth position segmentation result; Determine a first registration result between the oral scan data and the navigation CT image based on the first rough registration result, the three-dimensional morphology of each tooth in the first tooth segmentation result, and the three-dimensional morphology of each tooth in the second tooth segmentation result; Wherein, determining the navigation rough registration result between the oral scan data and the current key frame image includes: Determine each tooth region of the current key frame image, and a third tooth position feature of each tooth region in the current key frame image; Determining a navigation rough registration result between the oral scan data and the current key frame image according to the first tooth position feature and the third tooth position feature; Wherein, determining a first matching result between the point cloud projection image and the current key frame image includes: Determine each tooth region in the point cloud projection image, and a fourth tooth position feature of each tooth region in the point cloud projection image; A first matching result between the point cloud projection image and the current key frame image is determined according to the fourth tooth position feature and the third tooth position feature. 2. The image processing method according to claim 1, characterized in that the step of converting the navigation CT image to a coordinate system of a navigation system according to the first registration result and the first matching result comprises: Determine a second registration result between the oral scan data and the current key frame image according to the first matching result; The position and posture of the navigation CT image in the navigation system is determined according to the first registration result and the second registration result. 3. The image processing method according to claim 2, characterized in that after determining the first registration result between the oral scan data and the navigation CT image, it also includes: If the current navigation scene image during the navigation process does not meet the image unobstructed condition, determining a second matching result between the current navigation scene image and the current key frame image; Determining a third registration result between the current navigation scene image and the oral scan data according to the second registration result and the second matching result; The position and posture of the navigation CT image in the navigation system is determined according to the first registration result and the third registration result. 4. The image processing method according to claim 3, characterized in that if the current navigation scene image in the navigation process does not meet the image unobstructed condition, determining a second matching result between the current navigation scene image and the current key frame image comprises: If the ratio of the obscured tooth area to all tooth areas in the current navigation scene image during the navigation process is greater than the set ratio threshold, it is determined that the current navigation scene image does not meet the image unobstructed condition, and the fourth matching result between the current navigation scene image and the current key frame image is determined using a sub-pixel image matching method. 5. The image processing method according to claim 1, characterized in that after converting the navigation CT image to the coordinate system of the navigation system, it also includes: Determine the surgical instrument and the posture of the surgical instrument in the current navigation scene image, and the target coordinates of the target position predetermined in the navigation CT image under the navigation system; Determine a motion path of the surgical instrument according to the target coordinates and the position and posture; Guidance information for guiding the movement of the surgical instrument is generated according to the movement path. 6. A navigation system, comprising: Surgical instruments; A camera device, used to obtain a navigation scene image during surgery on the target object, wherein the navigation scene image includes the surgical instrument; A processor, configured to execute the image processing method according to any one of claims 1 to 5; The display device displays the navigation CT image and the navigation scene image at the coordinates of the navigation system. 7. A computer-readable storage medium, characterized in that the computer-readable storage medium stores computer instructions, and the computer instructions are used to enable a processor to implement the image processing method according to any one of claims 1 to 5 when executed. 8. A computer program product, characterized in that the computer program product comprises a computer program, and when the computer program is executed by a processor, the image processing method according to any one of claims 1 to 5 is implemented.