CN111508057A

CN111508057A - Trachea model reconstruction method and system by using computer vision and deep learning technology

Info

Publication number: CN111508057A
Application number: CN201910334929.0A
Authority: CN
Inventors: 卢昭全; 王友光; 陈威廷; 许斐凯
Original assignee: Individual
Current assignee: Individual
Priority date: 2019-01-31
Filing date: 2019-04-24
Publication date: 2020-08-07
Also published as: TWM584008U

Abstract

The invention relates to a trachea model reconstruction method by utilizing computer vision and deep learning technology, which comprises a step of obtaining a trachea wall image, a step of loading image data, a step of processing images, a step of capturing image characteristics, a step of comparing the images, a step of estimating pose and converting space, and a step of reconstructing a three-dimensional trachea model; therefore, the trachea model reconstruction system capable of reconstructing and recording the three-dimensional trachea model accurately and quickly is provided.

Description

Trachea model reconstruction method and system by using computer vision and deep learning technology

Technical Field

The invention relates to a trachea model reconstruction method and a trachea model reconstruction system by utilizing computer vision and deep learning technology, in particular to a trachea model reconstruction method and a trachea model reconstruction system which can accurately and quickly reconstruct and record a three-dimensional trachea model.

Background

When patients are under general anesthesia, cardiopulmonary resuscitation or cannot breathe by themselves during operation, intubation treatment is required to be carried out on the patients so as to insert the artificial airway into the trachea and ensure that medical gas is smoothly sent into the trachea of the patients. When the intubation treatment is carried out, medical staff cannot directly visually observe and adjust the artificial airway, and only can operate by relying on the touch feeling and past experience of the medical staff, so that the trachea of a patient is prevented from being stabbed, the operation is required to be carried out for multiple times, and the time for establishing the unobstructed airway is delayed. Therefore, the rapid and accurate establishment of the three-dimensional tracheal model for medical staff to assist intubation is a problem to be solved at present.

Disclosure of Invention

The present invention is directed to overcome the above-mentioned drawbacks of the prior art, and provides a method and a system for reconstructing a trachea model using computer vision and deep learning techniques, which can accurately and rapidly reconstruct and record a three-dimensional trachea model.

The technical scheme adopted by the invention for solving the technical problems is as follows:

a trachea model reconstruction method by using computer vision and deep learning technology comprises the following steps:

obtaining an image of the tracheal wall: the endoscope lens 70 is used to capture continuous images from the mouth to the trachea.

Loading the graph resources: loading and storing the continuous images shot and captured by the lens of the endoscope for subsequent processing;

image processing: denoising and denoising the shot and captured continuous images, and enhancing image details through image enhancement processing to obtain clear images;

image feature acquisition: capturing and screening the image through the characteristic capturing mode of the extreme value of the area of the clear image after the image processing step, and then storing the captured and screened image characteristic points;

image comparison: comparing the image characteristic points of the two continuous images which are connected after the image characteristic acquisition processing, finding out the common image characteristic points contained in the images and recording and storing the common image characteristic points;

pose estimation and spatial conversion: the common image feature points are subjected to auxiliary identification by deep learning, so that the position and the posture of the trachea in a three-dimensional space when the endoscope lens shoots the common image feature points are estimated, and the spatial information of the depth and the angle when the endoscope lens extends into the trachea for shooting is converted;

reconstructing a three-dimensional trachea model: projecting the common image characteristic points processed in the image comparison step to a three-dimensional space, and reconstructing and recording the spatial information of the shooting depth and angle of the endoscope lens obtained in the pose estimation and space conversion step into an actual three-dimensional trachea model;

by means of the method, the three-dimensional trachea model can be quickly and correctly reconstructed, and therefore, the method can assist personnel in intubation treatment.

Therefore, the trachea model reconstruction method can be used for accurately and quickly reconstructing and recording the three-dimensional trachea model for subsequent medical research or use.

The invention has the beneficial effect that the three-dimensional trachea model can be correctly and quickly reconstructed and recorded.

Drawings

The invention is further illustrated with reference to the following figures and examples.

FIG. 1 is a flow chart of the steps of the present invention.

Fig. 2 is a system block diagram of the present invention.

FIG. 3 is a block diagram of a system incorporating an endoscope lens according to the present invention.

The reference numbers in the figures illustrate:

10 figure data loading module

20 image processing module

30 image feature capturing module

40 image comparison module

50 pose estimation algorithm module

60 three-dimensional model reconstruction module

70 endoscope lens

Detailed Description

The technical means and the structure thereof applied to achieve the object of the present invention will be described in detail with reference to the embodiments shown in fig. 1 to 3 as follows:

as shown in fig. 1, the trachea model reconstruction method using computer vision and deep learning technology in the embodiment includes the following steps:

Loading the graph resources: the continuous images captured by the endoscope lens 70 are loaded and stored for subsequent processing.

Image processing: the continuous images captured are denoised and denoised, and the image is enhanced to emphasize the image details to obtain clear images.

Image feature acquisition: the method comprises extracting and screening feature points of the images of the continuous images after image processing by feature extraction method (such as SIFT, SURF, ORB, etc.) of local extremum, and storing the extracted and screened image feature points.

Image comparison: and comparing the image characteristic points of the two continuous images which are connected after the image characteristic acquisition processing, finding out the common image characteristic points contained in the images, and recording and storing the common image characteristic points.

Pose estimation and spatial conversion: the common image feature points are recognized in an auxiliary manner by deep learning, so that the position and the posture of the trachea in the three-dimensional space reached when the endoscope lens 70 shoots the common image feature points are estimated, and the spatial information of the depth and the angle when the endoscope lens 70 extends into the trachea to shoot is converted.

Reconstructing a three-dimensional trachea model: and projecting the common image characteristic points processed in the image comparison step to a three-dimensional space, and reconstructing and recording the spatial information of the shooting depth and angle of the endoscope lens 70 obtained in the pose estimation and space conversion steps into an actual three-dimensional trachea model.

In order to achieve the above method, the model reconstruction system of the present invention, with reference to the embodiments shown in fig. 2 to 3, is described in detail as follows:

as shown in fig. 2, the trachea model reconstruction system using computer vision and deep learning technology of the present invention includes a map data loading module 10, an image processing module 20, an image feature capturing module 30, an image comparison module 40, a pose estimation algorithm module 50 and a three-dimensional model reconstruction module 60; wherein:

the image loading module 10 (please refer to fig. 3) is connected to the endoscope lens 70 for loading and storing the continuous images captured by the endoscope lens 70 from the oral cavity into the trachea for subsequent processing.

The image processing module 20 (please refer to fig. 3) is connected to the image loading module 10, and is configured to receive the continuous images loaded by the image loading module 10, perform denoising and denoising on the continuous images, and emphasize image details by using an image enhancement technique to obtain a clear image.

The image feature capturing module 30 (please refer to fig. 3) is connected to the image processing module 20, and is used for capturing and screening the feature points of the clear image processed by the image processing module 20 through the feature capturing manner of the local extremum, and storing the captured and screened image feature points.

In view of the above, the feature extraction method of the local extremum may be Scale-invariant feature transform (SIFT), Speeded Up Robust Features (SURF), fast feature point extraction and description (orinterfast and rotaed BRIEF, ORB), and the like.

The image comparison module 40 (please refer to fig. 3) is connected to the image feature capturing module 30 for receiving the image feature points captured and screened by the image feature capturing module 30, and comparing two consecutive images to find out the common image feature points and record and store the common image feature points.

The pose estimation algorithm module 50 (please refer to fig. 3) has a deep learning function, is connected to the image comparison module 40, and is configured to receive the common image feature points found by the image comparison module 40, and utilize the deep learning model to achieve auxiliary identification, so as to estimate the position and posture of the trachea reached by the endoscope lens 70 in the three-dimensional space when capturing the captured image according to the common feature points in the continuous images, thereby further converting the spatial information of the depth and angle of the endoscope lens 70 extending into the trachea to capture the image.

The three-dimensional model reconstructing module 60 (please refer to fig. 3) is connected to the image comparing module 40 and the pose estimation algorithm module 50, and is configured to receive the common image feature points found by the image comparing module 40 and the spatial information converted by the pose estimation algorithm module 50, so as to project all the image feature points to a three-dimensional space, and to reconstruct and record a complete three-dimensional trachea model by using the spatial information obtained by the pose estimation algorithm module 50.

In the pose estimation and spatial conversion step and the pose estimation and calculation module 50, the trachea image data of a plurality of patients are captured to obtain image feature points, and the image feature points and the captured images are input into a deep learning model, wherein the deep learning model can be selected from the types of supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning and the like (such as neural network, random forest, SVM (support vector machine), decision tree or cluster and the like), and the path trajectory of the depth, angle, path position and direction of the endoscope lens extending into the trachea is identified through the deep learning model, and the features and the shape of the trachea wall can be identified.

Therefore, after the continuous images are shot by the endoscope lens, the images are subjected to denoising, denoising and image detail strengthening treatment, the common characteristic points are captured and compared by the image characteristic points, the position and posture information of the continuous images is obtained by the position and posture estimation with the deep learning function, the depth and angle information of the endoscope lens extending into the trachea is further obtained, the moving track of the endoscope lens can be described, and the three-dimensional trachea model is correctly and quickly reconstructed by using a characteristic capture mode of computer vision and Visual ranging (Visual Odometry) to be formed for intubation assistance and subsequent medical research or use.

The above description is only a preferred embodiment of the present invention, and is not intended to limit the present invention in any way, and all simple modifications, equivalent variations and modifications made to the above embodiment according to the technical spirit of the present invention still fall within the scope of the technical solution of the present invention.

Claims

1. a tracheal model reconstruction method utilizing computer vision and deep learning technology, is characterized in that, comprises following steps:

Obtain images of the tracheal wall: use an endoscopic lens to capture continuous images from the mouth to the trachea;

Image loading: load and store the continuous images captured by the endoscope lens for subsequent processing;

Image processing: de-noise and de-noise the captured continuous images, and perform image enhancement processing to emphasize image details;

Image feature extraction: extract and filter the image feature points of the continuous images after the image processing step through the feature extraction method of the regional extreme value, and then store the captured and filtered image feature points;

Image comparison: compare the image feature points of consecutive images that are connected in pairs after image feature extraction processing, find out the common image feature points contained and record and store them;

Pose estimation and space conversion: The common image feature points are used for auxiliary identification by deep learning, so as to estimate the position and posture of the trachea in the three-dimensional space of the trachea when the common image feature points are captured by the endoscope lens, and convert the endoscope Spatial information of depth and angle when the lens is inserted into the trachea;

Reconstruct the 3D trachea model: Project the common image feature points processed by the image comparison step into the 3D space, and reconstruct and record the spatial information of the depth and angle of the endoscope lens obtained in the pose estimation and space conversion steps. into an actual three-dimensional trachea model.

2. a tracheal model reconstruction system utilizing computer vision and deep learning technology, is characterized in that, is applied to the above-mentioned tracheal model reconstruction method utilizing computer vision and deep learning technology, comprises a picture material loading module, image processing module, Image feature extraction module, image comparison module, pose estimation algorithm module and 3D model reconstruction module; among them:

The image loading module is connected with the endoscope lens to load the continuous images obtained by the endoscope lens entering the trachea;

The image loading module is connected with the endoscope lens, and is used to load and store the continuous images captured by the endoscope lens from the oral cavity into the trachea for subsequent processing;

The image processing module is connected with the image loading module, and is used for receiving the continuous images loaded by the image loading module, so as to perform denoising and noise reduction processing on these continuous images, and use the image enhancement technology to emphasize the image details;

The image feature extraction module is connected with the image processing module, and is used for capturing and filtering the image feature points through the feature extraction method of the regional extreme value of the continuous images processed by the image processing module, and storing the image feature points. some captured and filtered image feature points;

The image comparison module is connected with the image feature capture module, and is used for receiving the image feature points captured and screened by the image feature capture module, and compares the image feature points for the continuous images connected in pairs, and finds the image feature points. Record and store the common image feature points contained in it;

The pose estimation algorithm module, connected with the image comparison module, has the function of deep learning, and is used to receive the common image feature points found by the image comparison module, and at the same time use deep learning to achieve auxiliary identification, and can be based on continuous The common feature points in the image estimate the position and posture of the trachea in the three-dimensional space that the endoscopic lens reaches when the image is captured, and then the spatial information of the depth and angle of the image taken by the endoscopic lens extending into the trachea can be converted;

The three-dimensional model reconstruction module is connected with the image comparison module and the pose estimation algorithm module, and is used to receive the common image feature points found by the image comparison module and receive the conversion from the pose estimation algorithm module The obtained spatial information can be used to project all image feature points into the three-dimensional space, and supplemented by the spatial information obtained by the pose estimation algorithm module, and a complete three-dimensional three-dimensional trachea model can be reconstructed and recorded.