WO2025234035A1 - Medical device, endoscope system, control method, and control program - Google Patents

Abstract

A medical device 4 comprises a processor 409 that processes a captured image obtained by capturing fluorescence generated from a biological tissue through irradiation of the biological tissue with excitation light. The processor 409: acquires structure information of a region of interest, which is a site to be subjected to a cauterization process in the biological tissue; calculates a target cauterization depth, which is a cauterization depth that is desired at the region of interest in the cauterization process, on the basis of the structure information; calculates an estimated cauterization depth, which is a cauterization depth that results from the cauterization process actually performed on the region of interest, on the basis of the captured image; and outputs assistance information including at least one of information representing the target cauterization depth and the estimated cauterization depth, and comparison information representing the result of comparison between the target cauterization depth and the estimated cauterization depth.

Description

Medical device, endoscope system, control method, and control program

The present invention relates to a medical device, an endoscope system, a control method, and a control program.

Microwave Endonetrial Ablation (MEA) is a treatment method known for improving the symptoms of menorrhagia (see, for example, Patent Document 1). With MEA, a microwave applicator is used to heat the uterine cavity with microwaves, cauterizing the endometrium. The cauterized endometrium then reduces the amount of menstrual blood. In other words, the amount of menstrual bleeding can be reduced or eliminated.

Special table 2019-511292 publication

However, in ablation treatment, depending on the thickness of the endometrium, the ablation depth may be excessive or insufficient. Therefore, if a user such as a surgeon can recognize whether the ablation depth is appropriate for the thickness of the endometrium, the ablation treatment can be performed appropriately, and convenience can be improved.
However, with the technology described in Patent Document 1, a user such as an operator cannot recognize whether the ablation depth is appropriate for the thickness of the endometrium.

The present invention has been made in consideration of the above, and aims to provide a medical device, an endoscope system, a control method, and a control program that can improve convenience.

In order to solve the above-mentioned problems and achieve the objectives, the medical device of the present invention includes a processor that processes captured images of fluorescence emitted from biological tissue when the biological tissue is irradiated with excitation light. The processor acquires structural information of a region of interest that is the target site for ablation treatment in the biological tissue, calculates a target ablation depth, which is the ablation depth to be targeted for the ablation treatment in the region of interest, based on the structural information, calculates an estimated ablation depth, which is the ablation depth actually achieved by the ablation treatment in the region of interest, based on the captured image, and outputs support information including at least one of the target ablation depth, the estimated ablation depth, and comparison information showing a comparison result between the target ablation depth and the estimated ablation depth.

The endoscopic system of the present invention comprises a light source device that irradiates excitation light, an endoscope that is insertable into a subject and outputs captured images of fluorescence generated from biological tissue within the subject when the excitation light is irradiated onto the biological tissue, and a medical device having a processor that processes the captured images. The processor acquires structural information of a region of interest that is a target site for ablation treatment in the biological tissue, calculates a target ablation depth that is the ablation depth to be targeted for the ablation treatment in the region of interest based on the structural information, calculates an estimated ablation depth that is the ablation depth actually achieved by the ablation treatment in the region of interest based on the captured images, and outputs support information including at least one of the target ablation depth, the estimated ablation depth, and comparison information that shows a comparison result between the target ablation depth and the estimated ablation depth.

The control method of the present invention is a control method executed by a medical device, which acquires structural information of a region of interest that is a target site for ablation treatment in biological tissue, calculates a target ablation depth, which is the ablation depth targeted for ablation treatment in the region of interest, based on the structural information, calculates an estimated ablation depth, which is the ablation depth achieved by the actual ablation treatment in the region of interest, based on an image capturing fluorescence emitted from the biological tissue when irradiated with excitation light, and outputs support information including at least one of the target ablation depth, the estimated ablation depth, and comparison information showing a comparison result between the target ablation depth and the estimated ablation depth.

The control program of the present invention is a control program executed by a medical device, and the control program instructs the medical device to perform the following: acquire structural information of a region of interest that is a target site for ablation treatment in biological tissue; calculate a target ablation depth, which is the ablation depth to be targeted for ablation treatment in the region of interest, based on the structural information; calculate an estimated ablation depth, which is the ablation depth to be achieved by the actual ablation treatment in the region of interest, based on an image capturing fluorescence emitted from the biological tissue when irradiated with excitation light; and output support information including at least one of the target ablation depth, the estimated ablation depth, and comparison information indicating a comparison between the target ablation depth and the estimated ablation depth.

The medical device, endoscope system, control method, and control program according to the present invention can improve convenience.

FIG. 1 is a block diagram showing the configuration of an endoscope system according to the first embodiment. FIG. 2 is a diagram showing the wavelength characteristics of the excitation light emitted by the second light source unit. FIG. 3 is a diagram showing the transmission characteristics of the cut filter. FIG. 4 is a diagram illustrating the observation principle in the fluorescence observation mode. FIG. 5 is a diagram illustrating the observation principle in the normal light observation mode. FIG. 6 is a flowchart showing a control method according to the first embodiment. FIG. 7 is a diagram illustrating a control method according to the first embodiment. FIG. 8 is a diagram illustrating a control method according to the first embodiment. FIG. 9 is a diagram illustrating a control method according to the first embodiment. FIG. 10 is a flowchart showing a control method according to the second embodiment. FIG. 11 is a diagram illustrating a control method according to the second embodiment. FIG. 12 is a diagram illustrating a control method according to the second embodiment. FIG. 13 is a flowchart showing a control method according to the third embodiment. FIG. 14 is a diagram illustrating a control method according to the third embodiment.

Below, a description will be given of a mode for carrying out the present invention (hereinafter referred to as an embodiment) with reference to the drawings. Note that the present invention is not limited to the embodiment described below. Furthermore, in the drawings, the same parts are designated by the same reference numerals.

(Embodiment 1)
[Overall configuration of endoscope system]
FIG. 1 is a block diagram showing the configuration of an endoscope system 1 according to the first embodiment.
An endoscope system 1 according to the first embodiment is an endoscope system used in microwave intrauterine ablation (MEA). As shown in FIG. 1 , the endoscope system 1 includes an endoscope 2, a display device 3, a control device 4, and an applicator 5.

The endoscope 2 has an insertion section (not shown) that is inserted into the subject (in the uterine cavity in this first embodiment), generates image data (RAW data) of the subject's interior, and outputs the image data to the control device 4. The detailed configuration of the endoscope 2 will be explained later in the section "Configuration of the Endoscope."

The display device 3 is composed of a display monitor such as a liquid crystal or organic EL (Electro Luminescence) display, and under the control of the control device 4, displays images based on image data that has been image processed by the control device 4, as well as various information related to the endoscope system 1.

The control device 4 corresponds to the medical device according to the present invention. This control device 4 is realized using a processor, which is a processing device having hardware such as a GPU (Graphics Processing Unit), FPGA (Field Programmable Gate Array), or CPU (Central Processing Unit), and memory, which is a temporary storage area used by the processor. The control device 4 then comprehensively controls the operation of each part of the endoscope system 1 in accordance with the program recorded in the memory. The detailed configuration of the control device 4 will be explained in the "Configuration of the Control Device" section below.

The applicator 5 is a microwave applicator that cauterizes biological tissue by supplying microwaves to the biological tissue. Specifically, the applicator 5 is inserted into the uterine cavity via a treatment tool channel in the insertion section of the endoscope 2. The applicator 5 then cauterizes the endometrium in response to user operation by a user such as a surgeon.

[Configuration of the endoscope]
Next, the configuration of the endoscope 2 will be described with reference to FIG.
As shown in FIG. 1, the endoscope 2 includes an illumination optical system 201, an imaging optical system 202, a cut filter 203, an imaging element 204, an A/D conversion unit 205, a P/S conversion unit 206, an imaging recording unit 207, an imaging control unit 208, and a sensor unit 209.
Here, each of the illumination optical system 201, the imaging optical system 202, the cut filter 203, the imaging element 204, the A/D conversion unit 205, the P/S conversion unit 206, the imaging recording unit 207, the imaging control unit 208, and the sensor unit 209 is arranged inside the tip portion of the insertion section of the endoscope 2.

The illumination optical system 201 is composed of one or more lenses and irradiates the subject with illumination light supplied from a light guide 231 .
The imaging optical system 202 is composed of one or more lenses, etc., and focuses returned light from the subject (reflected light from the subject, fluorescent light emitted by the subject, etc.) to form an image of the subject on the light receiving surface of the imaging element 204.
The cut filter 203 is disposed on the optical axis O1 of the imaging optical system 202, between the imaging optical system 202 and the imaging element 204. The cut filter 203 blocks light in a predetermined wavelength band and transmits other light.
The transmission characteristics of the cut filter 203 will be explained in the "Configuration of the Control Device" section below.

The imaging element 204 is configured using a CCD (Charge Coupled Device) or CMOS (Complementary Metal Oxide Semiconductor) image sensor, in which one of color filters forming a Bayer array (RGGB) is arranged on each of a plurality of pixels arranged in a two-dimensional matrix. The imaging element 204 receives the subject image formed by the imaging optical system 202 and transmitted through the cut filter 203 under the control of the imaging control unit 208, and performs photoelectric conversion to generate a captured image (analog signal). In the first embodiment, the imaging element 204 is configured to integrate the image sensor with a Time of Flight (TOF) sensor that acquires subject distance information (hereinafter referred to as depth map information) using a TOF method. The depth map information is information on the subject distance, detected for each pixel position, from the position of the imaging element 204 (the position of the tip of the insertion portion of the endoscope 2) to a corresponding position on the observation target corresponding to the pixel position in the captured image.
The configuration for generating depth map information is not limited to the TOF sensor described above, and a phase difference sensor, a stereo camera, or the like may also be used.
Hereinafter, the depth map information and the captured image will be collectively referred to as image data.
The image sensor 204 then outputs the image data to the A/D converter 205 .

The A/D conversion unit 205 is configured using an A/D conversion circuit, etc., and, under the control of the imaging control unit 208, performs A/D conversion processing on the analog image data input from the imaging element 204 and outputs it to the P/S conversion unit 206.

The P/S conversion unit 206 is configured using a P/S conversion circuit, etc., and under the control of the imaging control unit 208, performs parallel/serial conversion of the digital image data input from the A/D conversion unit 205 and outputs it to the control device 4 via the first signal line 232.
Note that instead of the P/S conversion unit 206, an E/O conversion unit that converts image data into an optical signal may be provided, and the image data may be output to the control device 4 by the optical signal. Alternatively, the image data may be transmitted to the control device 4 by wireless communication such as Wi-Fi (Wireless Fidelity) (registered trademark).

The imaging and recording unit 207 is composed of non-volatile memory or volatile memory, and records various information related to the endoscope 2 (for example, pixel information of the imaging element 204, characteristics of the cut filter 203). The imaging and recording unit 207 also records various setting data and control parameters transmitted from the control device 4 via the second signal line 233.

The imaging control unit 208 is implemented using a TG (Timing Generator), a processor which is a processing device having hardware such as a CPU, and memory which is a temporary storage area used by the processor. The imaging control unit 208 controls the operation of the imaging element 204, A/D conversion unit 205, and P/S conversion unit 206 based on setting data received from the control device 4 via the second signal line 233.

The sensor unit 209 is a sensor used to calculate the position of the tip of the insertion portion of the endoscope 2 and the direction in which the tip of the insertion portion is facing (the imaging field of view of the tip). In this embodiment 1, the sensor unit 209 is composed of multiple magnetic coils that generate magnetism.

[Configuration of the control device]
Next, the configuration of the control device 4 will be described with reference to FIG.
As shown in FIG. 1, the control device 4 includes a condenser lens 401, a first light source unit 402, a second light source unit 403, a light source control unit 404, an S/P conversion unit 405, an image processing unit 406, an input unit 407, a recording unit 408, a control unit 409, a communication unit 410, and a receiving unit 411.
The condenser lens 401 condenses the light emitted by the first and second light source units 402 and 403 , and emits the light to the light guide 231 .

The first light source unit 402, under the control of the light source control unit 404, emits white light (normal light), which is visible light, and supplies the white light as illumination light to the light guide 231. The first light source unit 402 is configured using a collimator lens, a white LED (Light Emitting Diode) lamp, a drive driver, and the like.
The first light source unit 402 may be configured to supply visible white light by simultaneously emitting light from a red LED lamp, a green LED lamp, and a blue LED lamp, or may be configured with a halogen lamp, a xenon lamp, or the like.

Under the control of the light source control unit 404, the second light source unit 403 emits excitation light having a predetermined wavelength band and supplies the excitation light to the light guide 231 as illumination light.

Fig. 2 is a diagram showing the wavelength characteristics of the excitation light emitted by the second light source unit 403. Specifically, in Fig. 2, the horizontal axis represents wavelength (nm) and the vertical axis represents wavelength characteristics. Also in Fig. 2, curves L- _V represent the wavelength characteristics of the excitation light emitted by the second light source unit 403. Furthermore, in Fig. 2, curves L- _B represent the blue wavelength band, curves L- _G represent the green wavelength band, and curves L- _R represent the red wavelength band.
In this embodiment, the second light source unit 403 emits excitation light having a center wavelength (peak wavelength) of 415 nm and a wavelength band of 400 nm to 430 nm, as shown in Fig. 2. This second light source unit 403 is configured using a collimator lens, a semiconductor laser such as a violet LD (laser diode), a driver, etc.

Here, the transmission characteristics of the cut filter 203 will be described.
Fig. 3 is a diagram showing the transmission characteristics of the cut filter 203. Specifically, in Fig. 3, the horizontal axis represents wavelength (nm) and the vertical axis represents wavelength characteristics. Also in Fig. 3, curves L- _F represent the transmission characteristics of the cut filter 203, and curves L- _V represent the wavelength characteristics of excitation light. Furthermore, in Fig. 3, curves L- _NG represent the wavelength characteristics of fluorescence generated by irradiating excitation light onto advanced glycation endproducts generated by cauterization of biological tissue.
In this embodiment, the cut filter 203 blocks a portion of the excitation light reflected from the biological tissue in the observation area and transmits light in other wavelength bands including fluorescent components, as shown in Fig. 3. More specifically, the cut filter 203 blocks a portion of light in the short-wavelength band of 400 nm to less than 430 nm, including the excitation light, and transmits light in the long-wavelength band of more than 430 nm, including fluorescence generated by irradiating excitation light on advanced glycation endproducts generated by ablation treatment.

The light source control unit 404 is implemented using a processor, which is a processing device having hardware such as an FPGA or CPU, and memory, which is a temporary storage area used by the processor. The light source control unit 404 controls the light emission timing and light emission duration of each of the first and second light source units 402, 403 based on control data input from the control unit 409.

Under the control of the control unit 409 , the S/P conversion unit 405 performs serial/parallel conversion on the image data received from the endoscope 2 via the first signal line 232 and outputs the converted data to the image processing unit 406 .
When the endoscope 2 outputs image data as an optical signal, an O/E converter that converts an optical signal into an electrical signal may be provided instead of the S/P converter 405. When the endoscope 2 transmits image data via wireless communication, a communication module that can receive wireless signals may be provided instead of the S/P converter 405.

The image processing unit 406 is implemented using a processor, which is a processing device having hardware such as a GPU or FPGA, and memory, which is a temporary storage area used by the processor. Under the control of the control unit 409, the image processing unit 406 performs predetermined image processing on the image data of parallel data input from the S/P conversion unit 405, and outputs the result to the display device 3. Examples of predetermined image processing include demosaic processing, white balance processing, gain adjustment processing, gamma correction processing, and format conversion processing.

The input unit 407 is composed of a mouse, foot switch, keyboard, buttons, switches, touch panel, etc., and accepts user operations by a user such as a surgeon, and outputs an operation signal corresponding to the user operation to the control unit 409.

The recording unit 408 corresponds to the first recording unit and second recording unit of the present invention. This recording unit 408 is configured using recording media such as volatile memory, non-volatile memory, SSD (Solid State Drive), HDD (Hard Disk Drive), etc., or memory cards. The recording unit 408 records data including various programs for operating the endoscope system 1 and various parameters necessary for the operation of the endoscope system 1. Examples of data including various parameters necessary for the operation of the endoscope system 1 include an endometrial map and first and second related information. The endometrial map corresponds to the tissue image information of the present invention. Details of the endometrial map and the first and second related information will be explained in the "Control Method" section below.

The control unit 409 corresponds to the processor according to the present invention. This control unit 409 is realized using a processor, which is a processing device having hardware such as an FPGA or CPU, and memory, which is a temporary storage area used by the processor. The control unit 409 then performs overall control of each component that makes up the endoscope system 1.

The communication unit 410 is an interface that communicates various data with external medical imaging equipment, such as ultrasound observation devices, CT (Computed Tomography), MRI (Magnetic Resonance Imaging), and PET (Position Emission Tomography) scanners, according to a specified protocol. Under the control of the control unit 409, the communication unit 410 acquires structural information about the target area to be ablated, acquired by the external medical imaging equipment. This structural information includes the three-dimensional coordinates of each position on the observation target (the target area to be ablated). These three-dimensional coordinates are based on a specific coordinate system and are calculated by the medical imaging equipment.

Note that communication between the communication unit 410 and the external medical imaging device may be wireless or wired. Furthermore, a configuration may be adopted in which structural information acquired by the external medical imaging device is stored on a server or the like, and the communication unit 410 acquires the structural information from the server.

The receiving unit 411 receives the magnetic field emitted from the sensor unit 209. The receiving unit 411 then outputs a signal corresponding to the received magnetic field to the control unit 409.

[Observation principle in the observation mode of the endoscope system]
Here, the endoscope system 1 allows the observation mode to be switched to a fluorescent observation mode, a normal light observation mode, or a specific observation mode by operating an operation unit (not shown) of the endoscope 2.
The observation principles of the endoscope system 1 in the fluorescence observation mode and normal light observation mode will be described below.

[Observation principle in fluorescence observation mode]
First, the observation principle in the fluorescence observation mode will be described.
FIG. 4 is a diagram illustrating the observation principle in the fluorescence observation mode.
As shown in graph G11 of Fig. 4 , first, the control device 4 irradiates the biological tissue O10 with excitation light (center wavelength 415 nm) by causing the second light source unit 403 to emit light. In this case, as shown in graph G12 of Fig. 4 , reflected light (hereinafter referred to as reflected light W10) including at least components of the excitation light reflected by the biological tissue O10 and return light is blocked by the cut filter 203 and reduced in intensity, while a portion of components on the longer wavelength side than the blocked wavelength band enters the image sensor 204 without reducing in intensity.

More specifically, as shown in graph G12 of Fig. 4 , the cut filter 203 blocks most of the reflected light W10 in a wavelength band on the short wavelength side, including the wavelength band of the excitation light, that is incident on the G pixel of the image sensor 204, and transmits wavelength bands on the long wavelength side relative to the wavelength band that is blocked. Furthermore, as shown in graph G12 of Fig. 4 , the cut filter 203 transmits the fluorescence WF10 that is auto-emitted by advanced glycation endproducts that are produced by the cauterization treatment of the biological tissue O10. Therefore, the reflected light W10 and the fluorescence WF10, each with reduced intensity, are incident on the R pixel, G pixel, and B pixel of the image sensor 204.
Here, the G pixel in the image sensor 204 has sensitivity to the fluorescence WF10. However, as shown by the curve _LNG of the fluorescence characteristics in the graph G12 in Fig. 4, the fluorescence is a very small reaction. Therefore, the output value corresponding to the fluorescence WF10 at the G pixel is a small value.

Then, the image processing unit 406 acquires image data (RAW data) from the image sensor 204 and performs image processing on the output values of each of the G and B pixels contained in the image data to generate an observation image (fluorescence image). In this case, the output values of the G pixels include fluorescence information corresponding to the fluorescence WF10 emitted from the ablation treatment area (advanced glycation end products) where the ablation treatment has been performed on the biological tissue O10. Furthermore, the output values of the B pixels include background information from the biological tissue O10 of the subject, including the ablation treatment area. Then, by displaying the fluorescence image on the display device 3, it becomes possible to observe the ablation treatment area where the ablation treatment has been performed on the biological tissue O10.

[Observation principle in normal light observation mode]
Next, the observation principle in the normal light observation mode will be described.
FIG. 5 is a diagram illustrating the observation principle in the normal light observation mode.
As shown in graph G21 of Fig. 5 , the control device 4 first emits light from the first light source unit 402, thereby irradiating the biological tissue O10 with white light. In this case, a portion of the reflected light and return light (hereinafter referred to as reflected light WR30, WG30, and WB30) reflected by the biological tissue O10 is blocked by the cut filter 203, and the remainder is incident on the image sensor 204. Specifically, as shown in graph G22 of Fig. 5 , the cut filter 203 blocks reflected light in a wavelength band on the short wavelength side, including the wavelength band of the excitation light. Therefore, the light component in the blue wavelength band incident on the B pixel of the image sensor 204 is reduced compared to when the cut filter 203 is not provided.

Then, the image processing unit 406 acquires image data (RAW data) from the image sensor 204 and performs image processing on the output values of each of the R, G, and B pixels contained in the image data to generate an observation image (white light image). In this case, because the blue component contained in the image data is smaller than when the cut filter 203 is not in place, the image processing unit 406 performs white balance adjustment processing to adjust the white balance so that the ratio of the red, green, and blue components is constant. Then, by displaying the observation image (white light image) on the display device 3, it becomes possible to observe a natural observation image (white light image) even when the cut filter 203 is in place.

[Control Method]
Next, a control method executed by the control device 4 when the applicator 5 performs cauterization on living tissue (endometrium) will be described.
Fig. 6 is a flowchart showing a control method according to embodiment 1. Figs. 7 to 9 are diagrams explaining the control method according to embodiment 1. Specifically, Fig. 7 is a diagram showing an endometrial map M1. Fig. 8 is a diagram showing the correlation (line L _Y ) between the fluorescence intensity of fluorescence auto-emitted by advanced glycation endproducts in biological tissue and the ablation depth due to the ablation treatment of the biological tissue. In Fig. 8, the vertical axis represents the fluorescence intensity, and the horizontal axis represents the ablation depth. Fig. 9 is a diagram explaining support information D0.

The endoscope system 1 is already set in the following state.
That is, the insertion portion of the endoscope 2 is inserted into the uterine cavity, and the observation region of the endoscopic system 1 is the region within the uterine cavity. The applicator 5 is inserted into the uterine cavity via a treatment tool channel in the insertion portion of the endoscope 2, and is ready to cauterize the endometrium. Furthermore, it is assumed that the observation mode has been switched to the specific observation mode in response to an operation by a user, such as an operator, on the operation unit of the endoscope 2 to "switch the observation mode of the endoscopic system 1 to the specific observation mode."

First, the control unit 409 acquires the endometrial map M1 recorded in the recording unit 408 (step S1).
As shown in Figure 7, the endometrial map M1 is a model image that schematically illustrates the overall structure of the uterus in a typical woman. For each pixel in this endometrial map M1, three-dimensional coordinates of each position in the uterus corresponding to the pixel position are assigned. These three-dimensional coordinates are in the same coordinate system as the three-dimensional coordinates of each position on the observation target (the target region (endometrium) to be cauterized) included in the structural information.

After step S1, the control unit 409 controls the operation of the communication unit 410, and acquires endometrial structural information from an external medical imaging device via the communication unit 410 (step S2).
Here, the structural information acquired in step S2 is structural information of the endometrium acquired by previously examining the same subject as the subject to be cauterized by the applicator 5 using external medical imaging equipment.

After step S2, the control unit 409 associates the three-dimensional coordinates of each position on the uterus included in the endometrial map M1 acquired in step S1 with the three-dimensional coordinates of each position on the observation target (the target area (endometrium) to be cauterized) included in the endometrial structural information acquired in step S2 (step S3).

Note that steps S1 to S3 may be performed before the cauterization process.

After step S3, the image processing unit 406, under the control of the control unit 409, generates a thermal denaturation image that enables identification of a thermally denatured region in the endometrium (step S4). The thermally denatured region corresponds to the region of interest according to the present invention.
Incidentally, there is a correlation between the fluorescence intensity of the fluorescence auto-emitted by advanced glycation end products in biological tissue and the ablation depth (depth of thermal denaturation) of the biological tissue due to the ablation treatment, as shown in Fig. 8. Specifically, as shown by the straight lines L- _Y in Fig. 8, the fluorescence intensity increases as the depth of thermal denaturation increases (the greater the invasiveness of the ablation treatment to the biological tissue). The thermally denatured region is a region in the fluorescence image that is composed of pixels where the fluorescence intensity of the fluorescence auto-emitted by advanced glycation end products in the biological tissue exceeds a specific fluorescence intensity Th1 (Fig. 8).

Specifically, in the specific observation mode, fluorescent images and white light images are generated in a time-division manner by alternately switching between fluorescent observation mode and normal light observation mode. Then, in step S3, the image processing unit 406 performs a superimposition process to generate a thermally denatured image by superimposing the fluorescent image and the white light image generated at approximately the same time.

The superimposition processing executed by the image processing unit 406 can be exemplified by the following first and second superimposition processing.
The first superimposition process is a process of replacing an area in the white light image that is at the same pixel position as a thermally denatured area in the fluorescent image with an image of the thermally denatured area in the fluorescent image.
The second superposition process is a process (so-called alpha blending process) that changes the brightness of the color indicating the fluorescence applied to each pixel in the area that is at the same pixel position as the thermally denatured area in the white light image, depending on the fluorescence intensity at each pixel position in the thermally denatured area in the fluorescent image.

After step S4, the control unit 409 calculates the position on the observation object corresponding to the pixel position of the thermally denatured region (step S5).
Specifically, in step S5, the control unit 409 estimates the shape of the insertion portion of the endoscope 2 based on the magnetism emitted from the sensor unit 209 and received by the receiving unit 411, and calculates position information indicating the three-dimensional coordinates of the position of the tip of the insertion portion and directional information indicating the field of view of the tip. The three-dimensional coordinates are coordinates in the same coordinate system as the three-dimensional coordinates of each position of the uterus included in the endometrial map M1 and the three-dimensional coordinates of each position on the observation object included in the structural information. Furthermore, the control unit 409 calculates the three-dimensional coordinates of a position on the observation object (the target site (endometrium) to be subjected to ablation treatment) corresponding to the pixel position of the thermal denaturation region in the thermal denaturation image, based on the calculated position information and directional information and the depth map information included in the image data used to generate the thermal denaturation image.

After step S5, the control unit 409 recognizes the position of the thermally denatured region on the endometrial map M1 (step S6).
Specifically, in step S6, the control unit 409 uses the three-dimensional coordinates of each position of the uterus included in the endometrial map M1 acquired in step S1 and the three-dimensional coordinates of the position on the observation object corresponding to the pixel position of the thermally denatured region calculated in step S5. Then, the control unit 409 recognizes the position on the endometrial map M1 whose three-dimensional coordinates are approximately the same as the three-dimensional coordinates of the position on the observation object corresponding to the pixel position of the thermally denatured region as the position of the thermally denatured region.

After step S6, the control unit 409 calculates the thickness of the endometrium at the position of the thermally denatured region (step S7).
Specifically, in step S3, the control unit 409 associates the three-dimensional coordinates of each position on the uterus included in the endometrial map M1 with the three-dimensional coordinates of each position on the observation target (the target region (endometrium) to be cauterized) included in the structural information of the endometrium. Therefore, in step S7, the control unit 409 identifies the structural information of the endometrium corresponding to the position of the thermally denatured region on the endometrial map M1 recognized in step S6, and calculates the thickness of the endometrium at the position of the thermally denatured region based on the structural information.

After step S7, the control unit 409 calculates a target ablation depth, which is a target ablation depth for the ablation process in the thermally denatured region (step S8).
Specifically, in step S8, the control unit 409 uses the endometrial thickness calculated in step S7 and the first relationship information recorded in the recording unit 408.
Here, the first relationship information indicates the relationship between the thickness of the endometrium and the target ablation depth required to cauterize the endometrium without excess or deficiency. That is, in step S8, the control unit 409 refers to the first relationship information and calculates the target ablation depth corresponding to the endometrial thickness calculated in step S7.

After step S8, the control unit 409 calculates an estimated ablation depth, which is the ablation depth in the thermally denatured region due to the actual ablation treatment (step S9).
Specifically, in step S9, the control unit 409 uses the fluorescence intensity of each pixel in the thermally denatured region in the fluorescence image and the second relationship information recorded in the recording unit 408.
Here, the second relationship information is information indicating the relationship between the fluorescence intensity of each pixel in the fluorescence image and the ablation depth resulting from the actual ablation process, such as information indicating the relationship between the lines L and _Y in Fig. 8. That is, in step S9, the control unit 409 refers to the second relationship information and calculates the estimated ablation depth corresponding to the fluorescence intensity of each pixel in the thermally denatured region in the fluorescence image.

After step S9, the control unit 409 outputs the support information D0 (step S10). Then, the display device 3 displays the support information D0. After that, the control unit 409 returns to step S4.
Specifically, in step S9, the control unit 409 outputs support information D0 that associates at least one of the target ablation depth, the estimated ablation depth, and comparison information showing a comparison result between the target ablation depth and the estimated ablation depth with respect to the position of the thermally denatured region on the endometrial map M1. In the example of Fig. 9 , the support information D0 is information in which the endometrial thickness calculated in step S7, the target ablation depth calculated in step S8, the estimated ablation depth calculated in step S9, and comparison information showing a comparison result between the target ablation depth and the estimated ablation depth are linked as tables T1 and T2, respectively, with respect to positions P1 and P2 of the thermally denatured region on the endometrial map M1. Here, the comparison information listed in tables T1 and T2 is information indicating "ablation incomplete" indicating that the estimated ablation depth is less than the target ablation depth, and information indicating "ablation completed" indicating that the estimated ablation depth has reached the target ablation depth. Also, in Figure 9, the elliptical images C1 and C2 surrounding positions P1 and P2 correspond to the comparison information according to the present invention, and their color and brightness differ depending on whether the estimated ablation depth is less than the target ablation depth.

Note that the support information D0 according to the present invention may be any information that associates at least one of the target ablation depth, estimated ablation depth, and comparison information showing the comparison results between the target ablation depth and the estimated ablation depth with respect to the position of the thermally denatured region on the endometrial map M1. That is, in the support information D0 shown in FIG. 9, one of the target ablation depth, estimated ablation depth, and comparison information may be omitted. Furthermore, the comparison information according to the present invention is not limited to the comparison information shown in FIG. 9, and may be information showing the insufficient depth of the estimated ablation depth relative to the target ablation depth.

According to the first embodiment described above, the following effects are achieved.
In the control device 4 according to the first embodiment, the control unit 409 acquires structural information of the region of interest (thermally denatured region). The control unit 409 then calculates a target ablation depth, which is the ablation depth targeted by the ablation treatment in the thermally denatured region, based on the structural information. The control unit 409 then calculates an estimated ablation depth, which is the ablation depth actually achieved by the ablation treatment in the thermally denatured region, based on the thermally denatured image. The control unit 409 then outputs support information D0 including at least one of the target ablation depth, the estimated ablation depth, and comparison information indicating a comparison result between the target ablation depth and the estimated ablation depth.
Therefore, the user such as the surgeon can recognize from the support information D0 whether the ablation depth is appropriate for the thickness of the endometrium, and can perform the ablation process appropriately. Therefore, the control device 4 according to the first embodiment can improve convenience.

In particular, the support information D0 is information that associates at least one of a target ablation depth and an estimated ablation depth with the position of the thermally denatured region in the endometrial map M1, and comparison information that shows the comparison result between the target ablation depth and the estimated ablation depth.
This allows the user, such as an operator, to visually recognize positions where the ablation depth is sufficient and positions where it is insufficient, further improving convenience.

(Embodiment 2)
Next, a second embodiment will be described.
In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted or simplified.
In the above-described first embodiment, the support information D0 was in a state in which at least one of the target ablation depth, the estimated ablation depth, and the comparison information showing the comparison result between the target ablation depth and the estimated ablation depth was associated with the position of the thermally denatured region in the endometrial map M1.
In contrast, the support information D0 according to the second embodiment does not include the endometrial map M1.

Fig. 10 is a flowchart showing a control method according to embodiment 2. Fig. 11 and Fig. 12 are diagrams explaining the control method according to embodiment 2. Specifically, Fig. 11 and Fig. 12 are diagrams showing support information D0 according to embodiment 2.
10, in the control method according to the second embodiment, steps S1, S3, and S6 are omitted, and steps S7A and S10A are adopted instead of steps S7 and S10, compared to the control method according to the second embodiment described above. The control method according to the second embodiment executes steps S2, S4, S5, S7A, S8, S9, and S10A in this order, and returns to step S4 after step S10A.
The following mainly describes steps S7A and S10A.

In step S7A, the control unit 409 uses the three-dimensional coordinates of each position on the observation object (the target area (endometrium) to be cauterized) included in the structural information of the endometrium acquired in step S2 and the three-dimensional coordinates of the position on the observation object corresponding to the pixel position of the thermally denatured region calculated in step S5. Then, the control unit 409 identifies the structural information of the endometrium corresponding to the position on the observation object that corresponds to the pixel position of the thermally denatured region, and calculates the thickness of the endometrium at the position of the thermally denatured region based on the structural information.
In step S8, the endometrial thickness calculated in step S7A is used.

In step S10A, the control unit 409 outputs support information D0 that associates at least one of the target ablation depth and estimated ablation depth with the position of the thermally denatured region in the thermal denatured image, and comparison information showing the comparison results between the target ablation depth and the estimated ablation depth. In the example of Figures 11 and 12, support information D0 is information in which the endometrial thickness calculated in step S7A, the target ablation depth calculated in step S8, the estimated ablation depth calculated in step S9, and comparison information showing the comparison results between the target ablation depth and the estimated ablation depth are linked as tables T1 and T2, respectively, with respect to the position of the thermally denatured region Ar (the area indicated by dots in Figure 12) in the thermal denatured image F1 generated in step S4. The contents of tables T1 and T2 are as described in embodiment 1 above. Additionally, in Figures 11 and 12, rectangular images C1 and C2 surrounding the thermally altered region Ar correspond to the comparison information according to the present invention, and differ in color and brightness depending on whether the estimated ablation depth is less than the target ablation depth.

Note that the support information D0 according to the present invention may be information that associates at least one of the target ablation depth, estimated ablation depth, and comparison information indicating the comparison result between the target ablation depth and the estimated ablation depth with the position of the thermally denatured region in the thermal denaturation image. That is, in the support information D0 shown in FIG. 12, one of the target ablation depth, estimated ablation depth, and comparison information may be omitted. Furthermore, the comparison information according to the present invention is not limited to the comparison information shown in FIG. 12, and may be information indicating the insufficient depth of the estimated ablation depth relative to the target ablation depth.

Even when the support information D0 according to the second embodiment described above is adopted, the same effects as those of the above-described embodiment can be achieved.

(Embodiment 3)
Next, a third embodiment will be described.
In the following description, the same components as those in the first embodiment are denoted by the same reference numerals, and detailed description thereof will be omitted or simplified.
In the above-described first and second embodiments, the endoscope system 1 is used in microwave intrauterine ablation (MEA).
In contrast, the endoscope system 1 according to the third embodiment is used in cardiac catheter ablation therapy (percutaneous myocardial ablation). In cardiac catheter ablation therapy, a treatment catheter is inserted into the heart from the groin through a blood vessel, and a high-frequency current is passed from the tip of the catheter to ablate the arrhythmia. The area to be ablated for treatment is the heart wall, and the heart wall must be ablated from the inside to the outside (transmurally). However, the thickness of the heart wall varies from location to location, ranging from approximately 1 to 15 mm. Therefore, in the third embodiment, a user such as a surgeon is prompted to recognize whether the ablation depth is appropriate for the thickness of the heart wall.

Fig. 13 is a flowchart showing a control method according to embodiment 3. Fig. 14 is a diagram explaining the control method according to embodiment 3. Specifically, Fig. 14 is a diagram showing a cardiac wall map M2 used in step S1.
The control method according to the third embodiment is identical to the operation of the control method described in the first embodiment, except that "endometrium" is replaced with "heart wall." Therefore, detailed description will be omitted. In FIG. 13, the third embodiment is applied to the control method described in the first embodiment, but this is not limiting, and the third embodiment may also be applied to the control method described in the second embodiment. Even in this case, the control method according to the third embodiment is identical to the operation of the control method described in the second embodiment, except that "endometrium" is replaced with "heart wall."

Furthermore, the heart wall map M2 according to this third embodiment is a model image that schematically shows the overall structure of the heart of a typical person, as shown in FIG. 14. This heart wall map M2 has three-dimensional coordinates assigned to each pixel of each position on the heart corresponding to the pixel position. These three-dimensional coordinates are in the same coordinate system as the three-dimensional coordinates of each position on the observation target (the target area (heart wall) to be cauterized) included in the structural information.

As explained above, even when the endoscope system 1 is used for cardiac catheter ablation treatment as in the third embodiment, the same effects as those of the first embodiment can be achieved.

(Other embodiments)
Although the embodiments for carrying out the present invention have been described above, the present invention should not be limited to only the above-described embodiments.
In the above-described embodiment, the medical device according to the present invention is mounted on an endoscope system used in microwave intrauterine ablation (MEA) or cardiac catheter ablation treatment (percutaneous myocardial ablation), but the present invention is not limited to this and may be mounted on an endoscope system used in other procedures.

In the above-described embodiment, the medical device according to the present invention is mounted in an endoscope system using an endoscope, but this is not limiting and the device may also be mounted in an endoscope system using a medical surgical robot.

In the above-described embodiment, the support information according to the present invention is displayed on the display device 3, but this is not limiting, and the support information may also be announced audibly via a speaker or the like.

In the above-described embodiment, when calculating the position on the object of observation corresponding to the pixel position of the thermally denatured region (step S5), the configuration of the TOF sensor, sensor unit 209, and receiving unit 411 included in the image sensor 204 was used, but this is not limited to this.
For example, in step S5, SLAM (Simultaneous Localization and Mapping) technology is used to generate an image (hereinafter referred to as a concatenated image) by concatenating endoscopic images (observation images (white light images)) captured by the endoscope 2. Then, using the concatenated image, positions on the observation target corresponding to the pixel positions of the thermally denatured region are calculated.

REFERENCE SIGNS LIST 1 Endoscope system 2 Endoscope 3 Display device 4 Control device 5 Applicator 201 Illumination optical system 202 Imaging optical system 203 Cut filter 204 Imaging element 205 A/D conversion unit 206 P/S conversion unit 207 Imaging and recording unit 208 Imaging control unit 209 Sensor unit 231 Light guide 232 First signal line 233 Second signal line 401 Condenser lens 402 First light source unit 403 Second light source unit 404 Light source control unit 405 S/P conversion unit 406 Image processing unit 407 Input unit 408 Recording unit 409 Control unit 410 Communication unit 411 Receiving unit Ar Thermally denatured region C1, C2 Image D0 Support information F1 Thermally denatured image G11 to G13, G21 to G23 Graph L _B , L _G , L _R , L _V curve L Transmission characteristics of _F cut filter L Wavelength characteristics of _NG fluorescence L _Y line M1 Endometrial map M2 Heart wall map O1 Optical axis O10 Living tissue P1, P2 Position T1, T2 Table Th1 Specific fluorescence intensity W10, WB30, WG30, WR30 Reflected light WF10 Fluorescence

Claims (15)

a processor for processing captured images of fluorescence generated from biological tissue by irradiating the biological tissue with excitation light;
The processor:
acquiring structural information of a region of interest that is a target site for ablation treatment in the biological tissue;
calculating a target ablation depth, which is a target ablation depth of the ablation process in the region of interest, based on the structural information;
Calculating an estimated ablation depth, which is an ablation depth due to an actual ablation process in the region of interest, based on the captured image;
A medical device that outputs support information including at least one of the target ablation depth, the estimated ablation depth, and comparison information indicating a comparison result between the target ablation depth and the estimated ablation depth. The processor:
acquiring tissue image information relating to a tissue image showing the biological tissue;
The medical device according to claim 1 , wherein the support information is output in which at least one of the target ablation depth and the estimated ablation depth is associated with the comparison information for the region of interest in the tissue image. The processor:
The medical device according to claim 1 , wherein the support information is output in which at least one of the target ablation depth and the estimated ablation depth is associated with the comparison information for the region of interest in the captured image. The region of interest is
The medical device of claim 1 which is an endometrium. The region of interest is
The medical device of claim 1 , which is a cardiac wall. The structural information is
The medical device of claim 1 , wherein the information is about the thickness of the region of interest. a first recording unit configured to store first relationship information indicating a relationship between a thickness of the region of interest and the target ablation depth;
The processor:
The medical device according to claim 6 , wherein the target ablation depth corresponding to a thickness of the region of interest is calculated based on the structural information and the first relationship information. The comparative information is
The medical device according to claim 1 , wherein the information indicates whether the estimated ablation depth is less than the target ablation depth. The comparison information is
The medical device according to claim 1 , wherein the information indicates an insufficient depth of the estimated ablation depth relative to the target ablation depth. The fluorescence is
The medical device according to claim 1 , wherein the biomolecules are generated from advanced glycation end products produced by subjecting the biological tissue to cauterization. The processor:
The medical device according to claim 1 , wherein the region of interest is identified and the estimated ablation depth of the region of interest is calculated based on the fluorescence intensity of each pixel in the captured image. a second recording unit configured to store second relationship information indicating a relationship between the fluorescence intensity and the estimated ablation depth;
The processor:
The medical device of claim 11 , wherein the estimated ablation depth of the region of interest is calculated based on the fluorescence intensity and the second relationship information. a light source device that irradiates excitation light;
an endoscope that can be inserted into a subject and outputs a captured image of fluorescence generated from living tissue in the subject by irradiating the living tissue with the excitation light;
a medical device having a processor for processing the captured image;
The processor:
acquiring structural information of a region of interest that is a target site for ablation treatment in the biological tissue;
calculating a target ablation depth, which is a target ablation depth of the ablation process in the region of interest, based on the structural information;
Calculating an estimated ablation depth, which is an ablation depth due to an actual ablation process in the region of interest, based on the captured image;
An endoscope system that outputs support information including at least one of the target ablation depth, the estimated ablation depth, and comparison information indicating a comparison result between the target ablation depth and the estimated ablation depth. A control method executed by a medical device, comprising:
acquiring structural information of a region of interest that is a target site for ablation treatment in biological tissue;
calculating a target ablation depth, which is a target ablation depth of the ablation process in the region of interest, based on the structural information;
calculating an estimated ablation depth, which is the ablation depth achieved by the actual ablation process in the region of interest, based on a captured image of fluorescence generated from the biological tissue by irradiating the biological tissue with excitation light;
A control method for outputting support information including at least one of the target ablation depth, the estimated ablation depth, and comparison information indicating a comparison result between the target ablation depth and the estimated ablation depth. A control program to be executed by a medical device,
The control program instructs the medical device to:
acquiring structural information of a region of interest that is a target site for ablation treatment in biological tissue;
calculating a target ablation depth, which is a target ablation depth of the ablation process in the region of interest, based on the structural information;
calculating an estimated ablation depth, which is the ablation depth achieved by the actual ablation process in the region of interest, based on a captured image of fluorescence generated from the biological tissue by irradiating the biological tissue with excitation light;
A control program that outputs support information including at least one of the target ablation depth, the estimated ablation depth, and comparison information indicating a comparison result between the target ablation depth and the estimated ablation depth.