JP5993221B2 - Endoscope system and endoscopic image generation method - Google Patents

Description

  The present invention relates to an endoscope system and an endoscope image generation method.

2. Description of the Related Art Conventionally, there is known an apparatus that synthesizes and displays a 3D image acquired by CT, MRI, or the like on an endoscopic image (for example, see Patent Document 1).
This apparatus sequentially detects the position of the endoscope, projects a 3D image acquired by CT, MRI, etc. onto the observation surface of the endoscope determined based on the position, creates a 2D image, The 2D image and the endoscopic image are superimposed.

Japanese Patent No. 4152402

  However, in the apparatus of Patent Document 1, the 3D image is used to generate a 2D image that can be superimposed on an endoscopic image acquired by an endoscope arranged at an arbitrary position, and the stereoscopic information There is an inconvenience that cannot be used. For example, during ventricular tachycardia ablation by high-frequency energization from the epicardium side, if the fat layer on the myocardium is thick (5 mm or more), sufficient ablation cannot be performed via the epicardium. In such a case, the 2D superimposed image obtained by the apparatus of Patent Document 1 has a disadvantage that the thickness of the fat layer cannot be confirmed and the procedure becomes difficult.

  The present invention has been made in view of the above-described circumstances, and is an endoscope system that can facilitate a procedure by effectively using stereoscopic information included in a 3D image acquired by CT, MRI, or the like. An object of the present invention is to provide an endoscopic image generation method.

In order to achieve the above object, the present invention provides the following means.
One aspect of the present invention includes an endoscope that is inserted into a subject and acquires an image of an observation site, an endoscope detection unit that detects the position and observation direction of the endoscope, and 3 in the vicinity of the observation site. A storage unit that associates and stores a three-dimensional image and three-dimensional feature values at a plurality of representative points on the three-dimensional image, and a position and an observation direction of the endoscope detected by the endoscope detection unit , Projecting the representative point on the three-dimensional image stored in the storage unit onto a plane orthogonal to the observation direction of the endoscope, and the stereoscopic feature at the position of each representative point projected on the plane An endoscope system is provided that includes an image processing unit that generates a feature amount image displaying the amount.

  In another aspect of the present invention, a three-dimensional image in the vicinity of the observation site and three-dimensional feature values at a plurality of representative points on the three-dimensional image are stored in association with each other, and the position of the endoscope and the observation are observed. The direction is detected, the representative point on the stored three-dimensional image is projected onto a plane orthogonal to the observation direction of the endoscope, and the three-dimensional feature is located at the position of each representative point projected on the plane. An endoscopic image generation method for generating a feature amount image displaying an amount is provided.

  According to this aspect, the endoscope detection unit detects the position and the observation direction of the endoscope that is inserted into the subject and acquires the image of the observation site that is arranged inside the subject. The In the image processing unit, the position of the endoscope is detected using the detected position and observation direction of the endoscope and the three-dimensional image position information and the three-dimensional feature value in the vicinity of the observation site stored in the storage unit. The three-dimensional feature value of the representative point on the three-dimensional image viewed in the observation direction is generated as a feature value image.

  That is, according to this aspect, it is possible to observe the distribution of the three-dimensional feature amount of the observation site viewed from the same direction as the endoscope as a feature amount image. Examples of the three-dimensional feature amount include the thickness of the tissue when the observation site is an organ or the like. That is, when performing tissue observation of an organ using an endoscope, three-dimensional feature amounts such as the thickness of fat tissue and muscle tissue in the vicinity of the observation site can be confirmed, thereby facilitating treatment. it can.

In the above aspect, the image processing unit may generate a composite image obtained by combining the image acquired by the endoscope and the feature amount image.
By doing in this way, the observer can confirm by contrasting, without switching both the image acquired by the endoscope, and the feature-value image only by confirming a synthesized image.

In the above aspect, the image processing unit may display the image acquired by the endoscope and the feature amount image in a parallel state.
By doing in this way, the observer can confirm both the image acquired by the endoscope and the feature amount image only by confirming the images displayed in parallel.

  Further, in the above aspect, the apparatus includes a three-dimensional measurement device that acquires the three-dimensional image, and the image processing unit calculates the stereoscopic feature amount based on the three-dimensional image sent from the three-dimensional measurement device. It may be calculated.

  Further, in the above aspect, the observation site by the endoscope is a surface of an organ, and the three-dimensional measurement device displays a three-dimensional image of the organ in a wider range than the observation site acquired by the endoscope. The acquired image processing unit may generate an image in which the position and direction of the endoscope detected by the endoscope detection unit are superimposed on the three-dimensional image acquired by the three-dimensional measurement apparatus.

Moreover, in the said aspect, the site | part observed with the said endoscope may be the surface of the heart, and the said three-dimensional feature-value may be acquired in the ventricular systole of the said heart.
In this way, the endoscopic image and the distribution of the three-dimensional feature amount can be compared and observed during the ventricular systole when the visual field in the pericardium is the widest.

In the above aspect, the three-dimensional feature amount may be a tissue thickness dimension, a thickness dimension ratio, or a tissue damage area dimension.
By doing so, in the procedure using the endoscope, the distribution of the thickness of the fat tissue and the muscle tissue in the observation site can be confirmed, and the procedure can be easily performed.

Another aspect of the present invention, the storage unit may leave association with each sterically feature quantity at a plurality of representative points on the three-dimensional image and the three-dimensional image of the observed region near the endoscope A detection unit detects a position and an observation direction of the endoscope, and an image processing unit is a plane orthogonal to the observation direction of the endoscope with the representative point on the three-dimensional image stored in the storage unit And an endoscope image generation method for generating a feature amount image in which the stereoscopic feature amount is displayed at the position of each representative point projected onto a plane.

In the above aspect, the image processing unit may generate a composite image obtained by combining the image acquired by the endoscope and the feature amount image.
Moreover, in the said aspect, the said image process part may parallelize the image acquired with the said endoscope, and the said feature-value image.

Moreover, in the said aspect, the said image process part may calculate the said three-dimensional feature-value based on the three-dimensional image sent from the three-dimensional measuring apparatus which acquires the said three-dimensional image.
Moreover, in the said aspect, the observation site | part by the said endoscope is the surface of an organ, and the said image process part is wider than the said observation site | part acquired by the said endoscope acquired by the said three-dimensional measuring apparatus. You may produce | generate the image which superimposed the position and direction of the endoscope detected by the said endoscope detection part on the three-dimensional image of the said organ of the range.
Moreover, in the said aspect, the site | part observed with the said endoscope may be the surface of the heart, and the said three-dimensional feature-value may be acquired in the ventricular systole of the said heart.

  According to the present invention, there is an effect that the procedure can be facilitated by effectively using stereoscopic information included in a 3D image acquired by CT, MRI, or the like.

1 is an overall configuration diagram showing an endoscope system according to an embodiment of the present invention. It is a figure explaining the feature-value and feature-value image by the endoscope system of FIG. It is a figure which shows an example of the endoscopic image acquired by the endoscope system of FIG. It is a figure which shows an example of the feature-value image acquired by the endoscope system of FIG. It is a whole block diagram which shows the 1st modification of the endoscope system of FIG. It is a figure which shows the example of an image acquired by the 2nd modification of the endoscope system of FIG. It is a whole block diagram which shows the endoscope system for acquiring the image of FIG.

An endoscope system and an endoscope image generation method according to an embodiment of the present invention will be described below with reference to the drawings.
In this embodiment, the heart is assumed as an object to be observed.

  As shown in FIG. 1, an endoscope system 1 according to the present embodiment is an endoscope having an insertion portion 2a that is inserted into a pericardial membrane of a patient to observe the heart A from the outer surface B side. 2, an endoscope detection unit 3 that detects the position and direction of the tip of the endoscope 2, an MRI system 4, and the MRI system 4, the endoscope 2, and the endoscope detection unit 3 An image processing unit 5 and monitors 6 and 7 for displaying images processed by the image processing unit 5 are provided.

  The endoscope 2 includes an objective optical system and an illumination optical system (not shown) on the distal end surface of the insertion portion 2a, and has an observation direction forward along the optical axis along the longitudinal direction. For example, excitation light is emitted from the illumination optical system, and fluorescence generated in each tissue of the heart A is collected by the objective optical system and acquired as a fluorescence image.

  The endoscope detection unit 3 is based on the magnetic sensor 3a disposed at the distal end of the endoscope 2, the three magnetic field generating members 3b disposed outside the body, and the intensity of the magnetic field detected by the magnetic sensor 3a. And a calculator 3c that calculates the position and the observation direction of the distal end of the endoscope 2. The endoscope detection unit 3 has a unique coordinate system, and calculates the position and direction of the tip of the endoscope 2 in the coordinate system.

  The MRI system 4 acquires three-dimensional image data (three-dimensional volume rendering data) of the patient's heart A prior to endoscopic surgery performed by inserting the endoscope 2 into the body. The three-dimensional image data acquired by the MRI system 4 includes the three-dimensional image data of the epicardium B located on the outer surface of the heart A, the adipose tissue C existing on the surface of the heart A, and the myocardium D existing behind it. 3D image data of the boundary surface E.

  The acquisition of three-dimensional image data by the MRI system 4 is preferably performed in synchronization with the ventricular systole, for example. In the ventricular systole, since the field of view of the endoscope arranged in the pericardium is the widest, an easy-to-see endoscopic image is obtained. Therefore, by acquiring the three-dimensional image data at this time, it is possible to generate a feature amount image that is easily contrasted with an easily viewable endoscopic image.

The MRI system 4 also has a unique coordinate system, and acquires three-dimensional image data defined according to the coordinate system.
The 3D image data acquired in the MRI system 4 is transmitted to the image processing unit 5.

  The image processing unit 5 receives the three-dimensional image data from the MRI system 4 and receives the position and the observation direction of the endoscope 2 detected by the endoscope detection unit 3 and calculates a three-dimensional feature amount. An amount calculation unit 8; a memory 9 for storing the stereoscopic feature amount calculated by the feature amount calculation unit 8; and a feature amount for generating a feature amount image indicating a distribution of the stereoscopic feature amount stored in the memory 9. And an image generation unit 10. Further, the image processing unit 5 includes an endoscope image generation unit 11 that receives an image signal of the heart A taken by the endoscope 2 and generates an endoscope image G1 as shown in FIG. Yes.

  More specifically, the feature quantity calculation unit 8 of the image processing unit 5 is in a state where the correspondence between the coordinate system of the endoscope detection unit 3 and the coordinate system of the MRI system 4 is calibrated by a known method. Based on the position and observation direction of the endoscope 2 sent from the calculation unit 3c of the endoscope detection unit 3 and the angle-of-view information of the endoscope 2 stored in advance, it is sent from the MRI system 4. The range of the 3D image data existing in the imaging range acquired as the endoscopic image G1 is specified from the 3D image data thus obtained, and a plurality of representative points arranged in the range are determined from the 3D image data. The feature amount is calculated.

  For example, as shown in FIG. 2, the feature amount is calculated as 3D image data sent from the MRI system 4 as 3D image data, and 3D shape data of the epicardium B located on the outermost surface of the heart A. The three-dimensional shape data of the boundary surface E between the adipose tissue C of the heart A and the myocardium D is received. Then, a plurality of representative points p1, p2, p3... Are set on the epicardium B, and the epicardium B in the direction perpendicular to the epicardium B of the heart A at each representative point p1, p2, p3. The distance from the boundary surface E is calculated.

  Thereby, the thickness of the adipose tissue C as the feature amount is calculated. The thickness of the adipose tissue C at the positions of the representative points p1, p2, p3,... As the calculated feature values is stored in the memory 9 in association with the coordinates of the representative points p1, p2, p3,. It has become.

  The feature amount image generation unit 10 sets the coordinates of the representative points p1, p2, p3... On the epicardium B set in the feature amount calculation unit 8 to calculate the feature amount in the observation direction of the endoscope 2. By converting to the two-dimensional coordinates projected on the orthogonal plane F, and adding the feature quantities of the representative points p1, p2, p3... To the positions of the representative points p1a, p2a, p3a. As shown in FIG. 4, a feature amount image G2 showing a distribution of feature amounts is generated.

As the monitors 6 and 7, a first monitor 6 for displaying the endoscope image G1 and a second monitor 7 for displaying the feature amount image G2 are prepared separately.
The endoscope image G1 generated in the endoscope image generation unit 11 is sent to the first monitor 6 and displayed, and the feature amount image G2 generated in the feature amount image generation unit 10 is displayed on the second monitor 7. It is sent and displayed.

The operation of the endoscope system 1 according to the present embodiment configured as described above will be described below.
In order to perform endoscopic surgery using the endoscope system 1 according to the present embodiment, first, three-dimensional image data of the patient's heart A is acquired by the MRI system 4 prior to surgery. In the present embodiment, the three-dimensional image data acquired by the MRI system 4 includes not only the shape of the epicardium B on the outermost surface of the heart A but also the 3 of the boundary surface E between the adipose tissue C and the myocardium D. Dimensional shape data is included.

  When the insertion section 2a of the endoscope 2 is inserted into a patient's body and an endoscopic operation is started, an endoscopic image G1 of the heart A acquired by the endoscope 2 is an endoscopic image generation section. 11 and displayed on the first monitor 6. Further, the operation of the endoscope detection unit 3 detects the tip position and the observation direction of the insertion unit 2 a, and the detected information is sent to the image processing unit 5.

In the image processing unit 5, the feature amount is calculated by the feature amount calculation unit 8.
That is, when the distal end position and the observation direction of the insertion unit 2a are sent from the endoscope detection unit 3, the imaging range acquired as the endoscope image G1 is calculated from the angle-of-view information of the endoscope 2. Is possible. Therefore, it is possible to specify the range of 3D image data that belongs to the range of the endoscopic image G1 among the 3D image data sent from the MRI system 4.

The thickness of the adipose tissue C is calculated for a plurality of representative points p1, p2, p3,... Defined on the epicardium B from the three-dimensional shape data included in the range of the three-dimensional image data specified in this way. The calculated thickness is stored in the memory 9 in association with the coordinates of the representative points p1, p2, p3,.
Thereafter, the feature amount image generation unit 10 projects the representative points p1, p2, p3,... So as to project the representative points p1, p2, p3,... On the plane F orthogonal to the observation direction of the endoscope 2. A feature amount image G2 is generated by converting the coordinates and assigning feature amounts to the coordinate positions of the converted representative points p1a, p2a, p3a,. The generated feature amount image G2 is displayed on the second monitor 7.

  The feature quantity image G2 generated in this way is in the same visual field range as the endoscope image G1 displayed on the first monitor 6, and the heart is directed from the same position as the endoscope 2 toward the same observation direction. This is an image in which the distribution of the feature amount is displayed on the two-dimensional image in which A is observed, and can be observed in contrast to the endoscopic image G1 displayed on the first monitor 6.

  In particular, in this embodiment, since the distribution image of the thickness of the adipose tissue C is displayed as the feature amount image G2, when performing a procedure such as ablation on the affected area specified by the endoscopic image G1, the affected area is displayed. Therefore, there is an advantage that the thickness of the adipose tissue C existing at the position can be easily confirmed, and the procedure can be facilitated.

  In the present embodiment, the feature amount is calculated and stored for the three-dimensional image data existing in the imaging range calculated based on the position and the observation direction of the endoscope 2 detected by the endoscope detection unit 3. Therefore, there is an advantage that the calculation amount of the feature amount calculation unit 8 and the storage capacity of the memory 9 can be reduced.

  In the present embodiment, a patient is assumed as a subject, the heart A is exemplified as an observation site, and the thickness of the fat tissue C is exemplified as a feature quantity. Instead, the thickness dimension of the tissue is exemplified. The ratio, the tissue damage area size, any other subject, the observation site, and the feature amount may be adopted.

  In addition, the feature amount is calculated from the three-dimensional image data in the range specified by the position and the observation direction of the endoscope 2 detected by the endoscope detection unit 3, but instead, this is shown in FIG. As described above, the feature amount calculation unit 8 may calculate the feature amount offline for the representative points of the entire range from which the three-dimensional image data has been acquired by the MRI system 4, and store this in the memory 9. Thus, when the feature amount image is generated every time the position and the observation direction of the distal end portion of the endoscope 2 are detected by the endoscope detection unit 3, the feature amount image is already calculated and stored in the memory 9. Therefore, there is an advantage that the amount of calculation can be reduced and the real-time property can be improved.

  In FIG. 1, the endoscope image G1 and the feature amount image G2 are displayed on the separate monitors 6 and 7, but instead of this, an image composition unit 12 is provided in the image processing unit 5, A composite image G3 in which both images G1 and G2 are superimposed may be displayed on a single monitor 6, or both images G1 and G2 may be displayed in parallel on a single monitor 6. In this case, the images may be displayed at the same magnification, or one may be enlarged and the other may be reduced.

  Further, as the feature amount image G2, a contour line indicating a distribution of feature amounts or a color-coded image can be adopted. What is necessary is just to be able to adjust arbitrarily about the space | interval of a contour line and the width | variety of color classification. For example, the upper limit value and / or the lower limit value may be changed by a dial (not shown) provided on the monitor.

  In the present embodiment, the endoscope image G1 and the feature amount image G2 are displayed in parallel or superimposed, but in addition to this, as shown in FIG. Alternatively, the entire image G4 displaying the position of the endoscope 2 (for example, indicated by an arrow in FIG. 6) may be displayed in parallel. In the figure, reference numeral 13 denotes a gauge indicating a feature amount.

  In this case, as shown in FIG. 7, based on the three-dimensional image data stored in the memory 9 and the position and observation direction of the endoscope 2 detected by the endoscope detection unit 3, What is necessary is just to provide the whole image production | generation part 14 which produces | generates the whole image G4 which displays the position of the endoscope 2 in an image.

A heart (observation site)
F plane G1 Endoscopic image (image)
G2 feature image G3 composite image p1, p2, p3 representative point 1 endoscope system 2 endoscope 3 endoscope detection unit,
4 MRI system (3D measuring device)
5 Image processing section 9 Memory (storage section)

Claims (13)

An endoscope that is inserted into the subject and acquires an image of the observation site;
An endoscope detection unit for detecting the position and observation direction of the endoscope;
A storage unit that associates and stores a three-dimensional image in the vicinity of the observation site and stereoscopic features at a plurality of representative points on the three-dimensional image;
Based on the endoscope position and observation direction detected by the endoscope detection unit, the representative point on the three-dimensional image stored in the storage unit is orthogonal to the observation direction of the endoscope. An endoscope system comprising: an image processing unit that generates a feature amount image that projects onto a plane and displays the stereoscopic feature amount at the position of each representative point projected onto the plane.   The endoscope system according to claim 1, wherein the image processing unit generates a composite image obtained by combining the image acquired by the endoscope and the feature amount image.   The endoscope system according to claim 2, wherein the image processing unit synthesizes the image acquired by the endoscope and the feature amount image in a parallel state. A three-dimensional measuring device for acquiring the three-dimensional image;
The endoscope system according to any one of claims 1 to 3, wherein the image processing unit calculates the three-dimensional feature amount based on a three-dimensional image sent from the three-dimensional measurement apparatus. The observation site by the endoscope is the surface of the organ,
The three-dimensional measuring device acquires a three-dimensional image of the organ in a wider range than the observation site acquired by the endoscope;
The internal processing unit according to claim 4, wherein the image processing unit generates an image in which a position and a direction of an endoscope detected by the endoscope detection unit are superimposed on a three-dimensional image acquired by the three-dimensional measurement device. Endoscopic system. The observation site by the endoscope is the surface of the heart,
The endoscope system according to any one of claims 1 to 5, wherein the three-dimensional feature amount is acquired in a ventricular systole of the heart.   The endoscope system according to any one of claims 1 to 6, wherein the three-dimensional feature amount is a thickness dimension of a tissue. A storage unit stores a three-dimensional image in the vicinity of the observation site in association with three-dimensional feature values at a plurality of representative points on the three-dimensional image,
The endoscope detector detects the position and observation direction of the endoscope,
An image processing unit projects the representative point on the three-dimensional image stored in the storage unit onto a plane orthogonal to the observation direction of the endoscope, and at the position of each representative point projected on the plane. An endoscopic image generation method for generating a feature amount image displaying the three-dimensional feature amount. The endoscopic image generation method according to claim 8, wherein the image processing unit generates a composite image obtained by combining the image acquired by the endoscope and the feature amount image. The endoscopic image generation method according to claim 9, wherein the image processing unit parallelizes the image acquired by the endoscope and the feature amount image. The said image processing part calculates the said three-dimensional feature-value based on the three-dimensional image sent from the three-dimensional measuring apparatus which acquires the said three-dimensional image. Endoscopic image generation method. The observation site by the endoscope is the surface of the organ,
The image processing unit acquires a three-dimensional image of the organ that is acquired by the three-dimensional measuring device and is wider than the observation site acquired by the endoscope, and is detected by the endoscope detection unit . The endoscopic image generation method according to claim 11, wherein an image in which the position and direction of the endoscope are superimposed is generated. The observation site by the endoscope is the surface of the heart,
The endoscopic image generation method according to any one of claims 8 to 12, wherein the three-dimensional feature amount is acquired in a ventricular systole of the heart.

Images