WO2025070479A1 - Endoscope system, method for operating same, and program for operating endoscope system - Google Patents

Abstract

To provide an endoscope system, a method for operating the same, and a program for an endoscope system with which it is possible, when a plurality of parts exist in an image when correcting an oxygen saturation calculation table by a corrective operation performed by a user, to perform the correction in a highly accurate manner.　In this endoscope system in which oxygen saturation is calculated: an image obtained by imaging an observation target is acquired; a plurality of parts are recognized from the image; a region-of-interest is set in each part; a reliability is calculated for each region-of-interest; a comparison is made between the reliability and a preset threshold; a determination process, of determining whether or not the region-of-interest is a high-reliability region-of-interest in which data correction is possible, is performed; and correction of data is performed by using the high-reliability region-of-interest for which it has been determined that data correction is possible.

Description

Endoscope system, operation method thereof, and operation program for endoscope system

This disclosure relates to an endoscope system with an oxygen saturation imaging function, an operating method thereof, and an operating program for the endoscope system.

In recent years, oxygen saturation imaging has become known in the medical field, where endoscopes are used. Oxygen saturation imaging is performed by illuminating the object to be observed with illumination light including a wavelength band whose absorption coefficient changes with changes in the oxygen saturation of hemoglobin in the blood, and capturing an image. Then, based on the image obtained by capturing the image, an oxygen saturation image is displayed on a display, with the color tone changing according to the oxygen saturation level.

In order to calculate oxygen saturation, the object is photographed using multiple illumination lights with different wavelength bands. Then, a predetermined calculation value is calculated using pixel values of the obtained image, and oxygen saturation is calculated using an oxygen saturation calculation table that shows the correlation that associates the calculation value with oxygen saturation, and in some cases, a corrected oxygen saturation calculation table is used to calculate the oxygen saturation. It may be difficult to accurately correct the oxygen saturation calculation table due to disturbances such as halation in the image obtained by the endoscope (hereinafter referred to as the endoscopic image), specific dyes (yellow dyes, etc.), and bleeding, and an endoscopic system that issues a warning when correction of oxygen saturation, etc., fails is known (Patent Document 1). Also known is an endoscopic system that can perform appropriate correction operations even if disturbances such as bleeding are present in the object of observation (Patent Document 2), and an endoscopic system that can calculate specific dye concentrations from multiple image signals and perform appropriate correction operations (Patent Document 3).

JP 2017-80246 A International Publication No. 2023/119795 International Publication No. 2023/119856

If the observation subject has multiple regions with different amounts of pigment, the accuracy of the correction may decrease. To solve this, it was necessary to change the field of view of the observation subject or to correct for specific pigments (yellow pigment, etc.) before calculating the oxygen saturation.

The present disclosure aims to provide an endoscope system, an operating method and an operating program thereof that can perform highly accurate corrections when multiple areas are present in an image during data correction processing related to oxygen saturation calculations performed by a user.

The endoscope system disclosed herein is an endoscope system that calculates the oxygen saturation of an object to be observed using data used to calculate the oxygen saturation, and is equipped with a processor, which acquires an image of the object to be observed, recognizes multiple parts of the object to be observed in the acquired image, sets a region of interest in the image for each recognized part, calculates a region of interest reliability for each region of interest based on the pixels contained in the region of interest, performs a comparison process that compares the region of interest reliability with a preset region of interest reliability threshold, displays a high reliability region of interest on a display for each region of interest based on the comparison result obtained by the comparison process, and corrects the data based on the high reliability region of interest selected by the user.

The processor preferably calculates the region of interest reliability based on the pixel reliability calculated for each pixel contained in the region of interest and the number of pixels contained in the region of interest.

The processor preferably calculates the pixel reliability modified according to the distance from the image center in the image.

The processor preferably calculates the confidence level of the region of interest by weighting the region of interest according to its area or its distance from the image center in the image.

The processor preferably changes the display mode of the high confidence region of interest in response to a user instruction.

It is preferable that the processor be able to change the display format of the high confidence region of interest in response to a user instruction.

It is preferable that the processor displays on the display a name representing the part of the body that appears in the high-confidence region of interest, superimposed on the high-confidence region of interest.

The processor preferably accepts a user selection of multiple high confidence regions of interest and corrects the data based on the multiple high confidence regions of interest.

The processor preferably performs a determination process in which, in the comparison result, if the region of interest reliability is equal to or greater than a region of interest reliability threshold, the region of interest is determined to be a high reliability region of interest, and, if the region of interest reliability is lower than the region of interest reliability threshold, the region of interest is determined to be a low reliability region of interest in which data correction is not possible.

If the user selects a low-confidence region of interest, it is preferable for the processor to perform control to notify the user of operational guidance for changing the low-confidence region of interest to a high-confidence region of interest.

If the processor determines that the image does not contain a high-confidence region of interest, it is preferable to control the processor to notify the user.

The processor preferably controls the display to superimpose areas where the pixel reliability is equal to or greater than a pixel reliability threshold on the high reliability region of interest.

If data correction has been performed, the processor preferably calculates the oxygen saturation of the image based on the corrected data.

The operation method of the endoscope system disclosed herein is an operation method of an endoscope system that calculates the oxygen saturation of an observation target using data used for calculating the oxygen saturation, and includes a processor, which recognizes multiple parts shown in an acquired image, sets a region of interest in the image for each recognized part, calculates a region of interest reliability for each region of interest based on the pixels contained in the region of interest, performs a comparison process that compares the region of interest reliability with a preset region of interest reliability threshold, and performs a determination process for each region of interest based on the comparison result obtained by the comparison process to determine whether or not it is a high-reliability region of interest for which data can be corrected, displays the high-reliability region of interest on a display, and performs data correction based on the high-reliability region of interest selected by the user.

The operating program of the endoscope system disclosed herein is an operating program of an endoscope system that calculates the oxygen saturation of an observation target using data used for calculating the oxygen saturation, and provides a computer with the following functions: acquiring an image of the observation target; recognizing multiple parts from the acquired image; setting a region of interest for each recognized part in the image; calculating a region of interest reliability for each region of interest based on the pixels contained in the region of interest; performing a comparison process that compares the region of interest reliability with a preset region of interest reliability threshold; performing a determination process that determines, based on the comparison result obtained by the comparison process, whether each region of interest is a high-reliability region of interest for which data can be corrected; displaying the high-reliability region of interest on a display; and correcting data based on the high-reliability region of interest selected by the user.

According to the present disclosure, when a user corrects data related to oxygen saturation calculations, highly accurate corrections can be made when multiple areas are present in an image.

FIG. 1 is a schematic diagram of an endoscope system for laparoscopy. FIG. 4A is an explanatory diagram illustrating the display state of the display in normal mode, and FIG. 4B is an explanatory diagram illustrating the display state of the extended display in normal mode. 13A is an explanatory diagram illustrating the display mode of the display in the oxygen saturation mode, and FIG. 13B is an explanatory diagram illustrating the display mode of the extended display in the oxygen saturation mode. 13 is an explanatory diagram illustrating the display mode of the extended display when the mode is switched to the oxygen saturation mode. FIG. FIG. 13A is an image diagram of an extended display showing an oxygen saturation image on the serosal side, and FIG. 13B is an image diagram of an extended display showing an oxygen saturation image inside the digestive tract. 2 is a block diagram showing the functions of the endoscope system. FIG. 1 is a graph showing an emission spectrum of white light. 13A is a graph showing the emission spectrum of the first illumination light, FIG. 13B is a graph showing the emission spectrum of the second illumination light, and FIG. 13C is a graph showing the emission spectrum of the green light G. 4 is a graph showing the spectral sensitivity of an image sensor. 11 is a table showing illumination and acquired image signals in a normal mode. 11 is a table showing illumination and acquired image signals in an oxygen saturation mode or a correction mode. 11A and 11B are explanatory diagrams for explaining light emission control and display control in the oxygen saturation mode or the correction mode. 1 is a graph showing the reflectance spectrum of hemoglobin that varies depending on blood concentration. 1 is a graph showing the reflection spectrum of hemoglobin and the absorption spectrum of a yellow dye, which vary depending on the concentration of the yellow dye. 13 is a table showing the oxygen saturation dependency, blood concentration dependency, and brightness dependency of a B1 image signal, a G2 image signal, and an R2 image signal in a case where there is no influence of a yellow pigment. 1 is a graph showing contour lines representing oxygen saturation. 13 is a table showing the oxygen saturation dependency, blood concentration dependency, and brightness dependency of the X-axis value indicating the signal ratio ln(R2/G2) and the Y-axis value indicating the signal ratio ln(B1/G2). 13 is a table showing the oxygen saturation dependency, blood concentration dependency, yellow pigment dependency, and brightness dependency of a B1 image signal, a G2 image signal, and an R2 image signal when there is an influence of a yellow pigment. FIG. 13 is an explanatory diagram showing the oxygen saturation with and without a yellow dye when the observation subject has the same oxygen saturation. 13 is a table showing the oxygen saturation dependency, blood concentration dependency, yellow pigment dependency, and brightness dependency of a B1 image signal, a B3 image signal, a G2 image signal, a G3 image signal, an R2 image signal, and a B2 image signal when influenced by yellow pigment. 1 is a graph showing a surface representing oxygen saturation as a function of yellow dye. Graph (A) shows the oxygen saturation state expressed in three-dimensional coordinates of X, Y, and Z, and graph (B) shows the oxygen saturation state expressed in two-dimensional coordinates of X, Y. 13 is a table showing the oxygen saturation dependency, blood concentration dependency, yellow pigment dependency, and brightness dependency of the X-axis value indicating the signal ratio ln(R2/G2), the Y-axis value indicating the signal ratio ln(B1/G2), and the Z-axis value indicating the signal ratio ln(B3/G3). FIG. 2 is a block diagram showing the functions of the extended processor device; FIG. 2 is a block diagram showing functions of an image processing unit. FIG. 11 is an explanatory diagram for explaining a method of calculating oxygen saturation. FIG. 11 is an explanatory diagram showing a method for generating a contour line corresponding to a specific dye concentration. FIG. 4 is a block diagram showing the functions of a table correction unit. FIG. 1A is an explanatory diagram showing a plurality of parts in a correction image, and FIG. 1B is an explanatory diagram for explaining recognition of a plurality of parts in a correction image. FIG. 11 is an explanatory diagram for explaining setting of a region of interest in a correction image. 1 is a graph showing the relationship between pixel value and pixel reliability. 13 is a graph showing pixel reliability due to bleeding or the like. 13 is a graph showing pixel reliability due to fat or the like. 11 is an explanatory diagram for explaining a comparison between a region of interest reliability of a region of interest and a threshold for region of interest reliability; FIG. FIG. 13 is an explanatory diagram for explaining determination of a high-confidence region of interest and a low-confidence region of interest. FIG. 13 is a pictorial diagram of an augmented display showing a high confidence region of interest representation. 10 is a flowchart showing a flow of a series of processes in a correction mode. 11 is a graph showing correction data for correcting pixel reliability according to the distance from the image center. 11 is a graph showing data for weighting calculation of region reliability based on the area of a region of interest. 11 is a graph showing data for weighting calculation of region reliability according to the distance of a region of interest from the center of an image. FIG. 13 is an explanatory diagram illustrating an example of a display mode of a high-confidence region of interest. FIG. 13 is an explanatory diagram illustrating an example of a display mode of a high-confidence region of interest. FIG. 13 is an explanatory diagram illustrating display of multiple high confidence regions of interest. FIG. 13 is a block diagram showing the functions of a high confidence region of interest display unit. (A) is an explanatory diagram showing a high-reliability region of interest and a low-reliability region of interest in a correction image, and (B) is an explanatory diagram showing a display in which operation guidance is provided when a low-reliability region of interest is selected. 13A is an explanatory diagram showing a low-confidence region of interest in a correction image, and FIG. 13B is an explanatory diagram explaining a display in which operation guidance is notified when there is no high-confidence region of interest; 1A is an explanatory diagram showing a plurality of high-confidence regions of interest and low-confidence regions of interest in a correction image, and FIG. 1B is an explanatory diagram showing pixels above a region-of-interest threshold superimposed thereon;

As shown in FIG. 1, the endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, a processor-side interface 16, an extended processor device 17, and an extended display 18. The endoscope 12 is optically or electrically connected to the light source device 13, and is electrically connected to the processor device 14. The extended processor device 17 is electrically connected to the light source device 13, the processor device 14, and the extended display 18. Each of these connections is not limited to being wired, and may be wireless. Also, they may be connected via a network.

The endoscope system 10 is a rigid endoscope in which the endoscope 12 is inserted into a body cavity of a subject to perform surgical treatment, and images of the organs in the body cavity are captured from the serous membrane side. The endoscope system 10 is particularly suitable for use as a laparoscope. The endoscope 12 may also be a flexible endoscope that is inserted through the nose, mouth, or anus of the subject.

When the endoscope 12 is a laparoscope, as shown in FIG. 1, the endoscope 12 includes an insertion section 12a that is inserted into the abdominal cavity of the subject, and an operation section 12b that is provided at the base end of the insertion section 12a. An optical system and an image sensor are built into the tip section 12e, which is the tip part of the insertion section 12a. The optical system includes an illumination optical system described below for irradiating the subject with illumination light, and an image pickup optical system described below for capturing an image of the subject (see FIG. 6). The image sensor generates an image signal by forming an image on an image plane of the reflected light from the observation object that has passed through the image pickup optical system. The generated image signal is output to the processor device 14. The operation section 12b includes a mode switching switch 12c and a zoom operation switch 12d for zoom operation, etc. The mode switching switch 12c is used to switch between observation modes described below.

The light source device 13 generates illumination light. The processor device 14 performs system control of the endoscope system 10, and further performs control such as generating endoscopic images by performing image processing on image signals transmitted from the endoscope 12. The display 15 displays medical images transmitted from the processor device 14. The processor-side interface 16 has a keyboard, mouse, microphone, tablet, foot switch, touch pen, etc., and accepts input operations such as function settings.

The endoscope system 10 has three observation modes: normal mode, oxygen saturation mode, and tissue color correction mode (hereafter referred to as correction mode), and these three modes can be switched automatically or by the user operating the mode switching switch 12c. As shown in FIG. 2, in normal mode, a white light image NP1 with natural colors obtained by capturing an image of the observation target using white light as illumination light is displayed on the display 15, while nothing is displayed on the extended display 18.

As shown in FIG. 3, in the oxygen saturation mode, the oxygen saturation of the object of observation is calculated, and an oxygen saturation image OP that visualizes the calculated oxygen saturation is displayed on the extended display 18. Also, in the oxygen saturation mode, a white light equivalent image NP2 that has fewer short wavelength components than the white light image NP1 is displayed on the display 15. In the correction mode, corrections are made to the oxygen saturation calculation based on pixels within a high-confidence region of interest, which will be described later.

Correction of the oxygen saturation calculation is performed in a correction mode, which will be described later, and the oxygen saturation calculation table is corrected using an area determined to be capable of being corrected (details of the area will be described later). When switching from normal mode to oxygen saturation mode, the mode may automatically switch to the correction mode once, and then switch to the oxygen saturation mode after completing the correction process in the correction mode. Once the correction process is completed, the mode switches to the oxygen saturation mode, and an oxygen saturation image OP is displayed on the extended display 18. For example, when switching to the oxygen saturation mode, as shown in FIG. 4, a message MS stating "Please perform correction process" is displayed on the extended display 18, and it is preferable that the oxygen saturation calculation is performed after the correction process.

In the case of a rigid endoscope type for abdominal cavities such as the serosa, the endoscope system 10 displays a serosal side oxygen saturation image, which is an image of the oxygen saturation state on the serosal side, on the extended display 18 in the oxygen saturation mode, as shown in FIG. 5(A). In the case of a flexible endoscope type for the digestive tract such as the stomach or large intestine, in the oxygen saturation mode, an internal digestive tract oxygen saturation image, which is an image of the oxygen saturation state inside the digestive tract, is displayed on the extended display 18 in the oxygen saturation mode, as shown in FIG. 5(B). It is preferable to use an image in which the saturation is adjusted for the white light equivalent image NP2 as the serosal side oxygen saturation image. It is preferable to adjust the saturation in the correction mode, regardless of whether it is a mucosa, a serosal membrane, a flexible scope, or a rigid scope. In this embodiment, a rigid scope type is used as the endoscope 12.

In the oxygen saturation mode, it is possible to accurately calculate the oxygen saturation in the following cases.
- When observing a predetermined target area (e.g. esophagus, stomach, large intestine) - When not in an external environment with surrounding lighting - When there is no residue, residual fluid, mucus, blood, or fat remaining on the mucosa or serosa - When dye is not sprayed onto the mucosa - When the endoscope is more than 7 mm away from the observation area - When the endoscope is observed at an appropriate distance from the observation area without being too far away - Areas that are sufficiently illuminated by illumination light - When there is little specular reflection from the observation area - Areas within 2/3 of the oxygen saturation image - When the movement of the endoscope is small, or when there is little patient movement such as pulsation or breathing - When blood vessels deep in the gastrointestinal mucosa cannot be observed

The processor device 14 is electrically connected to the display 15 and the processor-side interface 16. The processor device 14 receives image signals from the endoscope 12 and performs various processes based on the image signals. The display 15 outputs and displays images or information of the observation target processed by the processor device 14. The processor-side interface 16 has a keyboard, mouse, touchpad, microphone, foot pedal, etc., and has the function of accepting input operations such as function settings.

As shown in FIG. 6, the light source device 13 includes a light source section 20 and a light source processor 21 that controls the light source section 20. The light source section 20 has, for example, a plurality of semiconductor light sources, which are turned on or off, and when turned on, the amount of light emitted by each semiconductor light source is controlled to emit illumination light that illuminates the object of observation. In this embodiment, the light source section 20 has five colored LEDs: a V-LED (Violet Light Emitting Diode) 20a, a BS-LED (Blue Short-wavelength Light Emitting Diode) 20b, a BL-LED (Blue Long-wavelength Light Emitting Diode) 20c, a G-LED (Green Light Emitting Diode) 20d, and an R-LED (Red Light Emitting Diode) 20e.

The V-LED 20a emits violet light V of 410 nm ± 10 nm. The BS-LED 20b emits second blue light BS of 450 nm ± 10 nm. The BL-LED 20c emits first blue light BL of 470 nm ± 10 nm. The G-LED 20d emits green light G in the green band. The central wavelength of the green light G is preferably 540 nm. The R-LED 20e emits red light R in the red band. The central wavelength of the red light R is preferably 620 nm. The central wavelength and peak wavelength of each of the LEDs 20a to 20e may be the same or different.

Both BL-LED 20c and BS-LED 20b are blue light sources that emit blue light. However, as described above, the central wavelength and wavelength band of the blue light emitted by BL-LED 20c (hereafter referred to as BL light) and the blue light emitted by BS-LED 20b (hereafter referred to as BS light) are different. The central wavelength and wavelength band of BL light are the central wavelength and wavelength band in the blue wavelength band where the difference in the absorption coefficient between oxygenated hemoglobin and reduced hemoglobin is approximately maximized.

G-LED 20d is a green light source that emits broadband green light (hereafter referred to as G light) with a central wavelength of 540 nm. R-LED 20e is a red light source that emits broadband red light (hereafter referred to as R light).

The light source processor 21 independently controls the on/off and light emission amount of each of the LEDs 20a-20e by inputting control signals to each of the LEDs 20a-20e. The control of the on/off and other aspects of the LEDs in the light source processor 21 differs depending on the mode, and will be described in detail later.

The light emitted by each of the LEDs 20a to 20e is incident on the light guide 24 via the optical path coupling section 23, which is composed of a mirror, lens, etc. The light guide 24 is built into the endoscope 12 and the universal cord (a cord that connects the endoscope 12 with the light source device 13 and the processor device 14). The light guide 24 propagates the light from the optical path coupling section 23 to the tip of the endoscope 12.

The endoscope 12 is provided with an illumination optical system 30 and an imaging optical system 31. The illumination optical system 30 has an illumination lens 32, and the illumination light propagated by the light guide 24 is irradiated onto the observation object via the illumination lens 32. The imaging optical system 31 has an objective lens 35 and an imaging sensor 36. Light from the observation object irradiated with the illumination light is incident on the imaging sensor 36 via the objective lens 35. As a result, an image of the observation object is formed on the imaging sensor 36.

The imaging sensor 36 is a color imaging sensor that captures an image of an object being observed while being illuminated with illumination light. Each pixel of the imaging sensor 36 is provided with either a B pixel (blue pixel) having a B (blue) color filter, a G pixel (green pixel) having a G (green) color filter, or an R pixel (red pixel) having an R (red) color filter. Therefore, when an object being observed is photographed with the imaging sensor 36, three types of images are obtained: a B image, a G image, and an R image. It is preferable that the imaging sensor 36 be a color imaging sensor with a Bayer array in which the ratio of the number of B pixels, G pixels, and R pixels is 1:2:1.

As the image sensor 36, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor can be used. Also, instead of the primary color image sensor 36, a complementary color image sensor equipped with complementary color filters of C (cyan), M (magenta), Y (yellow) and G (green) may be used. When a complementary color image sensor is used, image signals of four colors CMYG are output, and by converting the four color image signals of CMYG into three color image signals of RGB by complementary color-primary color conversion, image signals of each of the RGB colors similar to those of the image sensor 36 can be obtained.

The imaging sensor 36 is driven and controlled by an imaging processor 37. The control of each mode in the imaging processor 37 will be described later. A CDS/AGC circuit 40 (Correlated Double Sampling/Automatic Gain Control) performs correlated double sampling (CDS) and automatic gain control (AGC) on the analog image signal obtained from the imaging sensor 36. The image signal that passes through the CDS/AGC circuit 40 is converted into a digital image signal by an A/D converter 41 (Analog/Digital). The digital image signal after A/D conversion is input to the processor device 14.

The processor unit 14 comprises a DSP (Digital Signal Processor) 45, an image processing unit 50, an image communication unit 51, a display control unit 52, and a central control unit 53. The processor unit 14 has programs relating to various processes built into a program memory (not shown). The functions of the DSP 45, image processing unit 50, image communication unit 51, display control unit 52, and central control unit 53 are realized by the central control unit 53, which is made up of a processor, executing the programs in the program memory.

The DSP 45 performs various signal processing such as defect correction, offset processing, gain correction, demosaic processing, linear matrix processing, white balance processing, gamma conversion processing, YC conversion processing, and noise reduction processing on the image signal received from the endoscope 12. In defect correction processing, the signal of a defective pixel of the imaging sensor 36 is corrected. In offset processing, dark current components are removed from the image signal that has been subjected to defect correction processing, and an accurate zero level is set. In gain correction processing, the signal level of each image signal is adjusted by multiplying the image signal of each color after offset processing by a specific gain. The image signal of each color after gain correction processing is subjected to demosaic processing and linear matrix processing to improve color reproducibility.

After linear matrix processing, white balance processing is performed, and then gamma conversion processing is performed to adjust the brightness and saturation of each image signal. Then, YC conversion processing is performed, and the luminance signal Y and the color difference signals Cb and Cr are output to the DSP 45. The DSP 45 performs noise reduction processing using, for example, the moving average method or the median filter method.

The image processing unit 50 performs various image processing on the image signal from the DSP 45. Image processing includes color conversion processing such as 3x3 matrix processing, tone conversion processing, and table processing for three-dimensional oxygen saturation calculation, color enhancement processing, and structural enhancement processing such as spatial frequency enhancement. The image processing unit 50 performs image processing according to the mode. In the normal mode, the image processing unit 50 performs image processing for the normal mode to generate a white light image NP1. In the oxygen saturation mode, the image processing unit 50 generates a white light equivalent image NP2 and transmits the image signal from the DSP 45 to the extension processor unit 17 via the image communication unit 51. The extension processor unit 17 generates an oxygen saturation image OP based on the image signal of the transmitted endoscopic image.

The display control unit 52 performs display control to display the white light image NP1 or image information such as the oxygen saturation image OP from the image processing unit 50, and other information, on the display 15 or the extended display 18. In accordance with the display control, the white light image NP1 or the white light equivalent image NP2 is displayed on the display 15.

The extended processor device 17 receives image signals from the processor device 14 and performs various image processing. The extended processor device 17 functions as an image processing device. Image processing in the extended processor device 17 is performed in a correction mode and an oxygen saturation mode, and is processing for generating an oxygen saturation image. The generated oxygen saturation image OP is displayed on the extended display 18. In addition, in the correction mode, the extended processor device 17 calculates the region of interest reliability for each part in accordance with user operation, and performs correction processing related to the calculation of oxygen saturation for each part based on the calculated region of interest reliability.

In the oxygen saturation mode, the extended processor device 17 calculates the oxygen saturation and generates an oxygen saturation image OP that visualizes the calculated oxygen saturation. The generated oxygen saturation image OP is displayed on the extended display 18. In the correction mode, the extended processor device 17 also performs a correction process related to the calculation of the oxygen saturation. More specifically, in the correction process, a specific dye concentration is calculated in accordance with user operations, and the oxygen saturation calculation table is corrected based on the calculated specific dye concentration. An example of the specific dye is a yellow dye. Therefore, the specific dye concentration is the concentration of the yellow dye in the image information contained in the endoscopic image. Details of the oxygen saturation mode and the correction mode performed by the extended processor device 17 will be described later.

The on/off control in each mode will now be explained. In normal mode, the V-LED 20a, BS-LED 20b, G-LED 20d, and R-LED 20e are simultaneously turned on to emit white light 55, as shown in FIG. 7, that includes purple light V with a central wavelength of 410 nm, second blue light BS with a central wavelength of 450 nm, broadband green light G in the green band, and red light R with a central wavelength of 620 nm. The graph in the figure shows a schematic representation of the light intensity of each wavelength band.

In the oxygen saturation mode and the correction mode, three frames of light emission with different light emission patterns are repeated. In the first frame, as shown in FIG. 8(A), the BL-LED 20c, the G-LED 20d, and the R-LED 20e are simultaneously turned on to emit a first broadband illumination light 56 including a first blue light BL with a central wavelength of 470 nm, a broadband green light G in the green band, and a red light R with a central wavelength of 620 nm. In the second frame, as shown in FIG. 8(B), the BS-LED 20b, the G-LED 20d, and the R-LED 20e are simultaneously turned on to emit a second blue light BS with a central wavelength of 450 nm, a broadband green light G in the green band, and a red light R with a central wavelength of 620 nm. In the third frame, as shown in FIG. 8(C), the G-LED 20d is turned on to emit a broadband green light G in the green band as a third illumination light 58. In addition, in the oxygen saturation mode, the first and second frames are the frames required to obtain the image signal necessary to calculate the oxygen saturation, so light may be emitted only in the first and second frames.

As shown in FIG. 9, the B color filter BF provided in the B pixel of the image sensor 36 transmits mainly light in the blue band, specifically light with a wavelength band of 380 to 560 nm (blue transmission band). The peak wavelength at which the transmittance is maximum is around 460 to 470 nm. The G color filter GF provided in the G pixel of the image sensor 36 transmits mainly light in the green band, specifically light with a wavelength band of 450 to 630 nm (green transmission band). The R color filter RF provided in the R pixel of the image sensor 36 transmits mainly light in the red band, specifically light with a wavelength band of 580 to 760 nm (red transmission band).

As shown in FIG. 10, in normal mode, the imaging processor 37 controls the imaging sensor 36 to capture an image of an object being observed that is illuminated with white light 55 consisting of purple light V, second blue light BS, green light G, and red light R, for each frame. This causes the B pixels of the imaging sensor 36 to output Bc image signals, the G pixels to output Gc image signals, and the R pixels to output Rc image signals.

As shown in FIG. 11, in the oxygen saturation mode, when the first illumination light 56 including the first blue light BL, green light G, and red light R is illuminated on the observation object in the first frame, the imaging processor 37 outputs a B1 image signal from the B pixel of the imaging sensor 36, a G1 image signal from the G pixel, and an R1 image signal from the R pixel as the first illumination light image. When the second illumination light 57 including the second blue light BS, green light G, and red light R is illuminated on the observation object in the second frame, the imaging processor 37 outputs a B2 image signal from the B pixel of the imaging sensor 36, a G2 image signal from the G pixel, and an R2 image signal from the R pixel as the second illumination light image.

In the third frame, when the third illumination light 58, which is green light G, is illuminated on the object to be observed, the imaging processor 37 outputs a B3 image signal from the B pixel of the imaging sensor 36, a G3 image signal from the G pixel, and an R3 image signal from the R pixel as the third illumination light image.

In the oxygen saturation mode, as shown in FIG. 12, the first illumination light 56 is emitted in the first frame (1stF), the second illumination light 57 is emitted in the second frame (2ndF), and the third illumination light 58 is emitted in the third frame (3rdF), after which the second illumination light 57 is emitted in the second frame and the first illumination light 56 is emitted in the first frame. A white light equivalent image NP2 obtained based on the emission of the second illumination light 57 in the second frame is displayed on the display 15. In addition, an oxygen saturation image OP obtained based on the emission of the first to third illumination lights in the first to third frames is displayed on the extended display 18.

In the oxygen saturation mode, of the image signals for the above three frames, the B1 image signal included in the first illumination light image, and the G2 and R2 image signals included in the second illumination light image are used. In the correction mode, in addition to the B1, G2, and R2 image signals, the B3 and G3 image signals included in the third illumination light image are used to measure the concentration of a specific pigment (such as a yellow pigment) that affects the accuracy of oxygen saturation calculation.

The B1 image signal includes image information on at least the first blue light BL from the light that has passed through the B color filter BF in the first illumination light 56. The B1 image signal (image signal for oxygen saturation) includes image information on the wavelength band B1, whose reflection spectrum changes with changes in the oxygen saturation of blood hemoglobin, as image information on the first blue light BL. As the wavelength band B1, for example, as shown in FIG. 13, it is preferable to set the wavelength band B1 to a wavelength band of 460 nm to 480 nm, including 470 nm, at which the difference between the reflection spectrum of oxygenated hemoglobin shown by curves 55b and 56b and the reflection spectrum of reduced hemoglobin shown by curves 55a and 56a is maximized.

In FIG. 13, curve 55a represents the reflectance spectrum of reduced hemoglobin when blood concentration is high, and curve 55b represents the reflectance spectrum of oxygenated hemoglobin when blood concentration is high. On the other hand, curve 56a represents the reflectance spectrum of reduced hemoglobin when blood concentration is low, and curve 56b represents the reflectance spectrum of oxygenated hemoglobin when blood concentration is low.

The G2 image signal contains image information of at least the wavelength band G2 relating to green light G from the light in the first illumination light 56 that has passed through the G color filter GF. The wavelength band G2 is preferably a wavelength band of 500 nm to 580 nm, for example, as shown in FIG. 13. The R2 image signal contains image information of at least the wavelength band R2 relating to red light R from the light in the first illumination light 56 that has passed through the R color filter RF. The wavelength band R2 is preferably a wavelength band of 610 nm to 630 nm, for example, as shown in FIG. 13.

Also, as shown in FIG. 14, the image information of wavelength band B1 contains image information on the first blue light BL, and the image information of wavelength band B3 contains image information on the green light G. The image information on the first blue light BL and the green light G is image information in which the absorption spectrum of a specific pigment changes due to a change in the concentration of the specific pigment such as a yellow pigment. The change in the absorption spectrum of the specific pigment also causes a change in the reflection spectrum of hemoglobin. Curve 55a represents the reflection spectrum of reduced hemoglobin when there is no influence of the yellow pigment, and curve 55c represents the reflection spectrum of reduced hemoglobin when there is an influence of the yellow pigment. As shown by these curves 55a and 55c, the reflection spectrum of reduced hemoglobin changes depending on the presence or absence of the yellow pigment (the same applies to the reflection spectrum of oxygenated hemoglobin). Therefore, the reflection spectrum of wavelength band B1 and wavelength band B3 changes due to a change in the oxygen saturation of blood hemoglobin under the influence of a specific pigment such as a yellow pigment.

In an ideal case where the object of observation using the endoscope 12 is not affected by specific pigments such as yellow pigments, as shown in FIG. 15, the B1 image signal (denoted as "B1"), G2 image signal (denoted as "G2"), and R2 image signal (denoted as "R2") are each affected by oxygen saturation dependency, blood concentration dependency, or brightness dependency. As described above, the B1 image signal includes the wavelength band B1 in which the difference between the reflectance spectrum of oxygenated hemoglobin and the reflectance spectrum of reduced hemoglobin is maximized, and therefore has a "large" degree of oxygen saturation dependency that changes with oxygen saturation. Furthermore, as shown by curves 55a, 55b and curves 56a, 56b, the B1 image signal has a "medium" degree of blood concentration dependency that changes with blood concentration. Furthermore, the B1 image signal has a "brightness dependency" that changes with the brightness of the object of observation. The degrees of dependency are indicated as "large," "medium," and "small," with "large" indicating a high degree of dependency compared to other image signals, "medium" indicating a medium degree of dependency compared to other image signals, and "small" indicating a low degree of dependency compared to other image signals.

The G2 image signal has a low dependency on oxygen saturation because the magnitude relationship between the reflectance spectrum of oxygenated hemoglobin and the reflectance spectrum of reduced hemoglobin is reversed over a wide wavelength band. Also, the G2 image signal has a high degree of dependency on blood concentration, as shown by curves 55a, 55b and curves 56a, 56b. Also, the G2 image signal has a dependency on brightness, similar to the B1 image signal.

The R2 image signal does not change as much with oxygen saturation as the B1 image signal, but it is moderately dependent on oxygen saturation. Also, the R2 image signal is slightly dependent on blood concentration, as shown by curves 55a, 55b and curves 56a, 56b. Also, the G2 image signal, like the B1 image signal, is dependent on brightness.

As described above, since the B1, G2, and R2 image signals all have brightness dependency, the G2 image signal is used as the normalized signal to create an oxygen saturation calculation table 73a (see FIG. 16) that is an oxygen saturation calculation table of data for calculating oxygen saturation using a signal ratio ln(B1/G2) obtained by normalizing the B1 image signal with the G2 image signal and a signal ratio ln(R2/G2) obtained by normalizing the R2 image signal with the G2 image signal. Note that the "ln" in the signal ratio ln(B1/G2) is the natural logarithm (the same applies to the signal ratio ln(R2/G2)).

If the relationship between the signal ratio ln(B1/G2) and the signal ratio ln(R2/G2) and the oxygen saturation is expressed in two-dimensional coordinates with the signal ratio ln(R2/G2) on the X-axis and the signal ratio ln(B1/G2) on the Y-axis, as shown in FIG. 16, the oxygen saturation is represented by the contour line EL along the Y-axis direction. The contour line ELH represents an oxygen saturation level of "100%", and the contour line ELL represents an oxygen saturation level of "0%". The contour lines are distributed so that the oxygen saturation level gradually decreases from the contour line ELH to the contour line ELL (in FIG. 16, the contour lines of "80%, " "60%, " "40%, " and "20%" are distributed).

The X-axis value (signal ratio ln(R2/G2)) and Y-axis value (signal ratio ln(B1/G2)) are affected by oxygen saturation dependency and blood concentration dependency, respectively. However, as for brightness dependency, as shown in FIG. 17, the X-axis value and the Y-axis value are normalized by the G2 image signal, and therefore are considered to be "none," meaning they are not affected. For the X-axis value (denoted as "X"), the oxygen saturation dependency is considered to be about "medium," and the blood concentration dependency is considered to be about "large." On the other hand, for the Y-axis value (denoted as "Y"), the oxygen saturation dependency is considered to be about "large," and the blood concentration dependency is considered to be about "medium."

On the other hand, in a realistic case where an object observed using the endoscope 12 is affected by a specific dye such as a yellow dye, as shown in FIG. 18, the B1 image signal (denoted as "B1"), G2 image signal (denoted as "G2"), and R2 image signal (denoted as "R2") are each affected by oxygen saturation dependency, blood concentration dependency, yellow dye dependency, or brightness dependency. The B1 image signal contains image information in which the absorption spectrum of a specific dye such as a yellow dye changes depending on the concentration of the specific dye, and therefore has a "large" degree of yellow dye dependency, which changes depending on the yellow dye. In contrast, the G2 image signal is less affected by the yellow dye compared to the B1 image signal, and therefore has a "small to medium" degree of yellow dye dependency. The R1 image signal is less affected by the yellow dye, and therefore has a "small" degree of yellow dye dependency.

In addition, when the signal ratio ln(R2/G2) is represented on the X-axis and the signal ratio ln(B1/G2) is represented on the Y-axis in two-dimensional coordinates, even if the observation subject has the same oxygen saturation, the oxygen saturation StO2A when there is no yellow dye and the oxygen saturation StO2B when there is yellow dye are represented differently, as shown in Figure 19. The oxygen saturation StO2B appears to be shifted higher than the oxygen saturation StO2A due to the presence of the yellow dye in the image signal.

In order to accurately calculate oxygen saturation even in the case of yellow pigment dependency as described above, the B3 image signal and G3 image signal included in the third illumination light image are used when correcting the oxygen saturation calculation table. The B3 image signal includes image information related to the light transmitted through the B color filter BF in the third illumination light 58. The B3 image signal (specific pigment image signal) includes image information of the wavelength band B3 that has sensitivity to specific pigments other than hemoglobin, such as yellow pigment (see FIG. 14). Although the B3 image signal is not as sensitive to the specific pigment as the B1 image signal, it has a certain degree of sensitivity to the specific pigment. Therefore, as shown in FIG. 20, the B1 image signal has a "large" dependency on yellow pigment, while the B3 image signal has a "medium" dependency on yellow pigment. The B3 image signal has a "small" dependency on oxygen saturation, a "large" dependency on blood concentration, and a "present" dependency on brightness.

The G3 image signal also does not have as much sensitivity to the specific dye as the B3 image signal, but does include image signals in the wavelength band G3 that have some degree of sensitivity to the specific dye (see Figure 14). Therefore, the yellow dye dependency of the G3 image signal is "small to medium". The G3 image signal has "small" oxygen saturation dependency, "large" blood concentration dependency, and "some" brightness dependency. The B2 image signal also has "large" yellow dye dependency, so the B2 image signal may be used instead of the B3 image signal when correcting the oxygen saturation calculation table. The B2 image signal has "small" oxygen saturation dependency, "large" blood concentration dependency, and "some" brightness dependency.

If the relationship between the signal ratio ln(B1/G2) and signal ratio ln(R2/G2), the yellow pigment, and oxygen saturation is expressed in three-dimensional coordinates with the signal ratio ln(R2/G2) on the X-axis, the signal ratio ln(B1/G2) on the Y-axis, and the signal ratio ln(B3/G3) on the Z-axis, as shown in Figure 21, the curved surfaces CV0 to CV4 representing oxygen saturation are distributed in the Z-axis direction according to the pigment concentration of the yellow pigment. The curved surface CV0 represents the oxygen saturation when the yellow pigment has a concentration of "0" (no effect of the yellow pigment). The curved surfaces CV1 to CV4 represent the oxygen saturation when the yellow pigment has concentrations of "1" to "4", respectively. The higher the concentration number, the higher the concentration of the yellow pigment. Note that, as shown in the curved surfaces CV0 to CV4, the higher the concentration of the yellow pigment, the lower the Z-axis value changes.

When the oxygen saturation state expressed in three-dimensional coordinates X, Y, and Z as shown in Figure 22 (A) is expressed in two-dimensional coordinates X and Y, as shown in Figure 22 (B), the areas AR0 to AR4 representing the oxygen saturation state are distributed at different positions according to the concentration of the yellow dye, respectively. Areas AR0 to AR4 represent the distribution of oxygen saturation when the concentration of the yellow dye is "0" to "4", respectively. By defining a contour line EL representing oxygen saturation for each of these areas AR0 to AR4, it is possible to obtain the oxygen saturation corresponding to the concentration of the yellow dye (see Figure 16). Note that, as shown in areas AR0 to AR4, the higher the concentration of the yellow dye, the higher the value on the X axis and the lower the value on the Y axis.

As shown in FIG. 23, the X-axis value (signal ratio ln(R2/G2)), Y-axis value (signal ratio ln(B1/G2)), and Z-axis value (signal ratio ln(B3/G3)) are subject to yellow pigment dependency. The X-axis value (denoted as "X") has a yellow pigment dependency of "small to medium", the Y-axis value (denoted as "Y") has a yellow pigment dependency of "large", and the Z-axis value (denoted as "Z") has a yellow pigment dependency of "medium". The Z-axis value has an oxygen saturation dependency of "small to medium" and a blood concentration dependency of "small to medium". The Z-axis value has no brightness dependency because it is normalized by the G3 image signal.

In this embodiment, in a realistic case where the object of observation is affected by a specific pigment such as a yellow pigment, a selection of multiple data representing oxygen saturation according to the concentration of the yellow pigment is adopted to correct the data used to calculate oxygen saturation. Therefore, in this case, correcting the data used to calculate oxygen saturation in the correction mode specifically means selecting one of the curved surfaces CV0 to CV4 (see FIG. 21) that represent oxygen saturation.

As shown in FIG. 24, the extended processor device 17 includes an image acquisition unit 60a and an image processing unit 60b. The image acquisition unit 60a receives an image signal transmitted from the processor device 14 via the image communication unit 51. The image processing unit 60b performs various image processing operations to generate an oxygen saturation image, etc.

As shown in FIG. 25, the image processing unit 60b includes an oxygen saturation image generating unit 61, a specific dye concentration calculating unit 62, a table correction unit 63, and a display control unit 64. In the extended processor device 17, programs related to various processes are incorporated in a program memory (not shown). The functions of the oxygen saturation image generating unit 61, the specific dye concentration calculating unit 62, the table correction unit 63, and the display control unit 64 are realized by a central control unit (not shown) formed by a processor executing the programs in the program memory.

The oxygen saturation image generating unit 61 includes a base image generating unit 70, an operation value calculating unit 71, an oxygen saturation calculating unit 72, an oxygen saturation calculation table 73, and a color adjusting unit 74. The base image generating unit 70 generates a base image based on an image signal from the processor device 14. The base image is used as the base of the oxygen saturation image OP. It is preferable that the base image is an image that can grasp morphological information such as the shape of the observed object. The base image is composed of a B2 image signal, a G2 image signal, and an R2 image signal. The base image may be a narrowband light image in which blood vessels or structures (ductal structures) are highlighted by narrowband light or the like.

The calculation value calculation unit 71 calculates a calculation value by calculation processing based on the B1 image signal, G2 image signal, and R2 image signal included in the oxygen saturation image signal. Specifically, the calculation value calculation unit 71 calculates the signal ratio B1/G2 between the B1 image signal and the G2 image signal, and the signal ratio R2/G2 between the R2 image signal and the G2 image signal as calculation values used to calculate the oxygen saturation. Note that it is preferable to logarithmize (ln) the signal ratios B1/G2 and R2/G2, respectively. In addition, the color difference signals Cr and Cb calculated from the B1 image signal, G2 image signal, and R2 image signal, or saturation S, hue H, etc. may be used as the calculation value.

The oxygen saturation calculation unit 72 refers to the oxygen saturation calculation table 73 and calculates the oxygen saturation based on the calculated value. The oxygen saturation calculation table 73 stores the correlation between the signal ratios B1/G2 and R2/G2, which are one of the calculated values, and the oxygen saturation. When the correlation is expressed in two-dimensional coordinates with the signal ratio ln(B1/G2) on the vertical axis and the signal ratio ln(R2/G2) on the horizontal axis, the state of oxygen saturation is expressed by the contour line EL extending in the horizontal axis direction, and when the oxygen saturation differs, the contour line EL is distributed at different positions in the vertical axis direction (oxygen saturation calculation table 73a (see FIG. 16)). The oxygen saturation calculation table 73 includes the oxygen saturation calculation table 73a expressed in two-dimensional coordinates.

The oxygen saturation calculation unit 72 refers to the oxygen saturation calculation table 73 and calculates the oxygen saturation corresponding to the signal ratios B1/G2 and R2/G2 for each pixel. For example, as shown in Fig. 26, the oxygen saturation calculation table 73a refers to the oxygen saturation calculation table 73a, and when the signal ratios of a specific pixel are ln(B1 ^* /G2 ^* ) and ln(R2 ^* /G2 ^* ), the oxygen saturation corresponding to the signal ratios ln(B1 ^* /G2 ^* ) and ln(R2 ^* /G2 ^* ) is "40%". Therefore, the oxygen saturation calculation unit 72 calculates the oxygen saturation of the specific pixel to be "40%".

The color tone adjustment unit 74 generates an oxygen saturation image by performing composite color processing that changes the color tone of the base image using the oxygen saturation calculated by the oxygen saturation calculation unit 72. The color tone adjustment unit 74 maintains the color tone of areas in the base image where the oxygen saturation exceeds a threshold, and changes the color tone of areas where the oxygen saturation is below the threshold to a color tone that changes depending on the oxygen saturation. This maintains the color tone of normal areas where the oxygen saturation exceeds the threshold, while only changing the color tone of abnormal areas where the oxygen saturation is below the threshold and becomes low, making it possible to grasp the oxygen status of abnormal areas under conditions where morphological information of normal areas can be observed.

In addition, the color tone adjustment unit 74 may generate an oxygen saturation image by pseudo-color processing in which a color is assigned according to the oxygen saturation level, regardless of the level of oxygen saturation. When performing pseudo-color processing, the base image is not necessary.

The specific dye concentration calculation unit 62 includes a specific dye concentration calculation table 75. In the correction mode, the specific dye concentration calculation unit 62 calculates the specific dye concentration based on a specific dye image signal including image information of a wavelength band that has sensitivity to a specific dye other than hemoglobin in blood among the dyes included in the observation subject. Examples of specific dyes include yellow dyes such as bilirubin. It is preferable that the specific dye image signal includes at least a B3 image signal. Specifically, the specific dye concentration calculation unit 62 calculates the signal ratios ln(B1/G2), ln(G2/R2), and ln(B3/G3). The specific dye concentration calculation unit 62 then refers to the specific dye concentration calculation table 75 to calculate the specific dye concentrations corresponding to the signal ratios ln(B1/G2), ln(G2/R2), and ln(B3/G3).

The specific dye concentration calculation table 75 stores the correlation between the signal ratios ln(B1/G2), ln(G2/R2), and ln(B3/G3) and the specific dye concentrations. For example, if the ranges of the signal ratios ln(B1/G2), ln(G2/R2), and ln(B3/G3) are divided into five stages, the specific dye concentrations "0" to "4" are associated with the signal ratios ln(B1/G2), ln(G2/R2), and ln(B3/G3) in the five stages, and are stored in the specific dye concentration calculation table 75. Note that it is preferable to logarithmize (ln) the signal ratio B3/G3.

The table correction unit 63 performs table correction processing to correct the oxygen saturation calculation table 73 based on the specific dye concentration as a correction processing performed in the correction mode. In the table correction processing, the correlation between the signal ratios B1/G2 and R2/G2 stored in the oxygen saturation calculation table 73 and the oxygen saturation is corrected. Specifically, when the specific dye concentration is "2", the table correction unit 63 generates a contour line EL representing the state of oxygen saturation in the area AR2 corresponding to the specific dye concentration of "2" among the areas AR0 to AR4 determined according to the specific dye concentration, as shown in FIG. 27. The table correction unit 63 generates a corrected oxygen saturation calculation table 73b by correcting the oxygen saturation calculation table 73 so that it becomes the generated contour line EL.

As shown in FIG. 28, the table correction unit 63 includes a recognition unit 81, a region setting unit 82, a reliability calculation unit 83, a comparison processing unit 84, a correction feasibility determination unit 85, a high reliability region of interest display unit 86, a region selection unit 87, and a correction unit 88.

The recognition unit 81 recognizes multiple parts that appear in the acquired first illumination light image or second illumination light image. Note that multiple parts refer to multiple types of parts, and multiple types of parts may be areas that appear to be different parts of the endoscopic image due to different colors, textures, etc., and do not necessarily have to be anatomically different parts. The recognition unit 81 recognizes multiple types of parts as different parts.

As shown in FIG. 29(A), a correction image 90, which is either a first illumination light image or a second illumination light image acquired by the image acquisition unit 60a, includes a transverse colon 91 and a peritoneum 92. The correction image 90 is an example of an endoscopic image acquired by the image acquisition unit 60a, and is an image that is subject to correction processing. As shown in FIG. 29(B), the recognition unit 81 recognizes the transverse colon 91 and the peritoneum 92 as different parts in the correction image 90. In FIG. 29(B), the recognized parts are shown with different shading.

The recognition method may use known technology, and for example, the method described in WO 2021/149552 may be adopted. That is, it is preferable to recognize the part or tissue from the endoscopic image using a trained model such as a neural network (a model trained using an image set consisting of images of a living body). For example, it is preferable to use a CNN (Convolutional Neural Network) as the neural network to perform multi-class classification (each class corresponds to a different part or tissue) and recognize the part or tissue. Note that the CNN may include a pooling layer in the intermediate layer.

As another recognition method, for example, the method described in JP 2011-218090 A can be adopted. That is, in order to more accurately recognize multiple parts of the object of observation that appear in the first illumination light image obtained by the image acquisition unit 60a, it is preferable to use a second image in which the illumination light is different from that of the first image, and to use high-confidence areas and low-confidence areas in the first image and second image that are set for determining the type of subject.

The region setting unit 82 sets a region of interest for each recognized part. As shown in FIG. 30, a region of interest is set for the transverse colon 91 and peritoneum 92 recognized by the recognition unit 81, a first region of interest 91a is set for the recognized transverse colon 91, and a second region of interest 92a is set for the recognized peritoneum 92. Each region of interest is set so that the recognized transverse colon 91 and peritoneum 92 do not overlap. Therefore, the region of interest is set so that one transverse colon 91 or peritoneum 92 is not included in multiple regions of interest. Note that the first region of interest 91a and the second region of interest 92a are actually set in the same manner as the regions shown by different shading in FIG. 30, but in order to avoid cluttering the figure, the figure may not show all of the first region of interest 91a or the second region of interest 92a. Examples of parts for which a region of interest is set include the large intestine, small intestine, liver, stomach, etc. The large intestine may also be further divided into the rectum, sigmoid colon, descending colon, transverse colon, ascending colon, cecum, etc. The set region of interest is not displayed on the extended display 18 at this stage.

The reliability calculation unit 83 calculates a region of interest reliability Rc for each of the first region of interest 91a and the second region of interest 92a based on the pixels contained in the first region of interest 91a or the second region of interest 92a. The region of interest reliability Rc is the reliability in the region of interest. Note that reliability is an index related to the ability to correct the oxygen saturation calculation table with higher accuracy. The higher the reliability, the more accurately the oxygen saturation calculation table can be corrected, while the lower the reliability, the more likely it is that a problem will occur in the accuracy of the correction of the oxygen saturation calculation table. The reliability calculation unit 83 calculates a region of interest reliability Rc for the region of interest and a pixel reliability Pc for the pixels, as described below.

The reliability calculation unit 83 first calculates pixel reliability Pc in order to calculate the region of interest reliability Rc. The pixel reliability Pc is a reliability calculated based on the pixels included in the region of interest. The pixel reliability Pc is calculated for each of the first region of interest 91a and the second region of interest 92a using the pixels included in each region of interest and factors that affect the correction of the oxygen saturation calculation table. The factors that affect the correction of the oxygen saturation calculation table will be described later. The region of interest reliability Rc is calculated based on the pixel reliability Pc and the number of pixels included in each of the first region of interest 91a and the second region of interest 92a. In addition to being used to calculate the region of interest reliability Rc, the pixel reliability Pc may also be used for the image to be displayed on the extended display 18 based on conditions set by the user.

Specifically, the reliability calculation unit 83 calculates at least one pixel reliability Pc that affects the correction of the oxygen saturation calculation table based on the B1 image signal, G1 image signal, and R1 image signal included in the first illumination light image, or the B2 image signal, G2 image signal, and R2 image signal included in the second illumination light image. Factors that affect the correction of the oxygen saturation calculation table include, for example, disturbances and pixel values including bleeding, fat, residue, mucus, or residual liquid. Therefore, it is preferable to calculate the pixel reliability Pc using these factors. The pixel reliability Pc is expressed, for example, as a decimal between 0 and 1. When the reliability calculation unit 83 calculates multiple types of pixel reliability Pc, it is preferable to adopt the minimum pixel reliability Pc among the multiple types of pixel reliability Pc as the pixel reliability Pc of each pixel.

For example, the pixel value of the G2 image signal can be used for the pixel value that affects the correction accuracy of the oxygen saturation calculation table. As shown in FIG. 31, the pixel reliability Pc of the pixel value of the G2 image signal outside the certain range Rx on the definition line 93 is lower than the pixel reliability Pc of the pixel value of the G2 image signal within the certain range Rx. The definition line 93 shows the relationship between the pixel value of the G2 image signal and the pixel reliability Pc related to the correction accuracy of the oxygen saturation calculation table, and is preset from past data. Outside the certain range Rx includes high pixel values such as halation, as well as extremely small pixel values such as dark areas. Outside the certain range Rx, the correction accuracy of the oxygen saturation calculation table is low, and the pixel reliability Pc is accordingly low. The G1 image signal may be used instead of the G2 image signal to calculate the pixel reliability Pc.

Also, disturbances that affect the accuracy of correction of the oxygen saturation calculation table include at least bleeding, fat, residue, mucus, or residual fluid, and these disturbances also cause pixel reliability Pc to fluctuate. For bleeding, which is one of the disturbances, as shown in FIG. 32, pixel reliability Pc is determined according to the distance from the defined line DFX on a two-dimensional plane consisting of the vertical axis ln (B2/G2) and the horizontal axis ln (R2/G2). Here, the pixel reliability Pc decreases as the coordinate plotted on the two-dimensional plane based on the B2 image signal, G2 image signal, and R2 image signal is located to the lower right. Also, when the coordinate is located in the upper left area of the defined line DFX, the reliability according to the degree of bleeding is set to a fixed value that is a high reliability. In the figure, ln represents the natural logarithm. B2/G2 represents the signal ratio between the B2 image signal and the G2 image signal, and R2/G2 represents the signal ratio between the R2 image signal and the G2 image signal.

Furthermore, for fat, residue, residual liquid, and mucus contained in the disturbance, as shown in FIG. 33, pixel reliability Pc is determined according to the distance from the defined line DFY on a two-dimensional plane consisting of the vertical axis ln (B1/G1) and the horizontal axis ln (R1/G1). Here, the pixel reliability Pc decreases as the coordinate plotted on the two-dimensional plane based on the B1, G1, and R1 image signals is located further to the lower left. Furthermore, when the coordinate is located in the area to the upper right of the defined line DFY, the reliability based on the degree of fat is set to a fixed value that is high reliability. B1/G1 represents the signal ratio between the B1 and G1 image signals, and R1/G1 represents the signal ratio between the R1 and G1 image signals.

Next, the reliability calculation unit 83 calculates the region of interest reliability Rc. The region of interest reliability Rc may be calculated based on the pixel reliability Pc calculated for each pixel included in each of the first region of interest 91a and the second region of interest 92a set by the region setting unit 82, and the number of pixels N included in the region of interest. That is, in each region of interest, as shown in the following formula (1), the pixel reliability Pc calculated for each of all pixels included in the region of interest is summed up, and the average pixel reliability Pc is calculated using the number of pixels N included in the region of interest, and this is set as the region of interest reliability Rc in this region of interest. Note that the region of interest reliability Rc may be calculated by a calculation method other than a simple average using the pixel reliability Pc of each pixel included in the region of interest. For example, it may be calculated by an average such as a weighted average or a trimmed average, or by other statistics such as a median or a mode.

　Region of interest reliability Rc = Σ (pixel reliability Pc of pixels in region of interest) / number of pixels in region of interest N (1)

The comparison processing unit 84 compares the region of interest reliability Rc calculated by the reliability calculation unit 83 with a preset region of interest threshold Rth for each of the first region of interest 91a and the second region of interest 92a set by the region setting unit 82. The region of interest threshold Rth is a value for evaluating the region of interest reliability Rc, and can be set to a specific value in advance. The region of interest threshold Rth is set in advance depending on the observation target, the type of reliability, etc.

As shown in FIG. 34, when the region of interest reliability Rc of the first region of interest 91a is calculated to be 0.8, the region of interest reliability Rc of the second region of interest 92a is calculated to be 0.2, and the region of interest threshold Rth is set to be 0.5, a comparison result 93a is obtained in which the region of interest reliability Rc is equal to or greater than the region of interest threshold Rth, and a comparison result 94b is obtained in which the region of interest reliability Rc is smaller than the region of interest threshold Rth.

The correction feasibility determination unit 85 determines whether each of the first region of interest 91a and the second region of interest 92a is a high reliability region of interest HRc for which correction of the oxygen saturation calculation table 73 used to calculate oxygen saturation is possible, based on the comparison result obtained by the comparison processing unit 84. As shown in FIG. 35, the first region of interest 91a whose region of interest reliability Rc is equal to or greater than the region of interest threshold value Rth is set as a high reliability region of interest HRc. It is also preferable to set the second region of interest 92a whose region of interest reliability Rc is smaller than the region of interest threshold value Rth as a low reliability region of interest LRc.

The high-reliability region of interest display unit 86 displays the high-reliability region of interest HRc on the extended display 18 so that the user can visually recognize the high-reliability region of interest HRc. When displaying the high-reliability region of interest HRc, it is sufficient to display it on the extended display 18 in a manner that allows the user to recognize the high-reliability region of interest HRc in the correction image 90. For example, the high-reliability region of interest HRc may be displayed in a color that allows it to be distinguished from other regions, or displayed with a boundary line that shows the outline of the high-reliability region of interest HRc.

As shown in FIG. 36, a boundary line 95 is displayed on the extended display 18 so that the user can visually recognize the high-reliability region of interest HRc set by the correction feasibility determination unit 85. As described above, it is not possible or not desirable to use the second region of interest 92a set in the low-reliability region of interest LRc to perform correction processing. Therefore, the boundary line indicating the second region of interest 92a is not displayed on the extended display 18.

The region selection unit 87 accepts the user's selection of a high-reliability region of interest HRc. After checking the high-reliability region of interest HRc displayed by the boundary line 95a on the extended display 18, the user can select a high-reliability region of interest HRc for performing correction processing. This allows the user to select a high-reliability region of interest HRc to be used for correction processing according to the site or tissue for which the user wishes to know the oxygen saturation with greater accuracy. Note that, if there are multiple high-reliability regions of interest HRc, the region selection unit 87 may accept the user's selection of multiple high-reliability regions of interest HRc.

The user can select the high reliability region of interest HRc, for example, by using the processor-side interface 16, the zoom switch 12d, etc. When the user selects the high reliability region of interest HRc displayed on the extended display 18 using the processor-side interface 16 or the zoom switch 12d, the correction unit 88 corrects the oxygen saturation calculation table 73 used to calculate oxygen saturation based on the high reliability region of interest HRc. When the correction process is completed, the mode is automatically switched to the oxygen saturation mode.

The correction unit 88 performs a correction process, which is a correction of the oxygen saturation calculation table 73, in response to a user instruction. The correction of the oxygen saturation calculation table 73 is performed using a high reliability region of interest HRc selected by the user in the correction image 90. When correcting the oxygen saturation calculation table 73 using the high reliability region of interest HRc, image information of the high reliability region of interest HRc can be used, and image information for each pixel included in the high reliability region of interest HRc can be used.

In this embodiment, the correction of the oxygen saturation calculation table 73 is performed by selecting one of the curved surfaces CV0 to CV4 (see FIG. 21) that represent the oxygen saturation. Therefore, the correction unit 88 uses the concentration of the yellow pigment in the high-reliability region of interest HRc selected by the user as the image information of this high-reliability region of interest HRc, and performs correction of the oxygen saturation calculation table 73 by selecting one of the curved surfaces CV0 to CV4 (see FIG. 21) that represent the oxygen saturation, as described above.

Furthermore, if there is no high-reliability region of interest HRc selected by the user, it is preferable that the oxygen saturation calculation unit 72 calculates the oxygen saturation of the observation subject by using the oxygen saturation calculation table 73 that is not corrected. This makes it possible to calculate a more accurate oxygen saturation without undesirable corrections being made to the oxygen saturation calculation table 73.

A series of processing steps in the correction mode by the endoscope system 10 will be described with reference to the flowchart of FIG. 37. The user operates the mode switching switch 12c to switch to the correction mode (step ST100). In the correction mode, the correction image 90 is displayed on the extended display 18 (step ST110). Based on the correction image 90, multiple parts are recognized (step ST120), and a region of interest 82a is set based on the multiple recognized parts (step ST130). A region of interest reliability Rc is calculated for each set region of interest (step ST140), and is compared with a region of interest threshold Rth for each region of interest (step ST150). If the region of interest reliability Rc is equal to or greater than the region of interest threshold Rth, the region of interest is set as a high reliability region of interest HRc (step ST160). When the user selects the high reliability region of interest HRc (step ST170), the high reliability region of interest HRc is used to perform a correction process for the oxygen saturation calculation table 73 related to the oxygen saturation calculation (step ST180). After the correction process is performed, the mode is switched to oxygen saturation mode (step ST190). In the oxygen saturation mode, the oxygen saturation for the endoscopic image is calculated using the corrected oxygen saturation calculation table 73.

The endoscope system 10, with the above configuration, can use the high reliability region of interest HRc in the correction process related to oxygen saturation calculation, so that highly accurate correction can be performed when multiple parts are present in the endoscope image. Furthermore, the endoscope system 10 can perform highly accurate correction related to oxygen saturation calculation through the correction process, so that more accurate oxygen saturation can be obtained regardless of the endoscope image.

When calculating pixel reliability Pc in the region of interest, the reliability calculation unit 83 may modify the pixel reliability Pc according to the distance from the center of the correction image 90 of the pixel to be calculated. The modified pixel reliability Pc is set as the modified pixel reliability. When the reliability calculation unit 83 modifies the pixel reliability Pc, it calculates the region of interest reliability Pc using the modified pixel reliability. If the horizontal axis of the correction image 90 is x and the vertical axis is y, the distance can be calculated as the square root of the sum of the squares of the differences between the x-axis value and the y-axis value between the coordinates of the center of the correction image 90 and the coordinates of each pixel to be calculated, where x is the horizontal axis and y is the vertical axis of the correction image 90.

The pixels for which pixel reliability Pc is calculated can be divided into two categories, for example "near distance" and "far distance", depending on the distance from the center of the correction image 90, and the pixel reliability Pc can be corrected by using different correction data for each category. The correction data is set in advance by examining the relationship between the pixel reliability Pc of multiple pixels at different positions in the correction image 90 and the oxygen saturation. At this time, the classification criteria for "near distance" pixels and "far distance" pixels are also set at the same time as creating the correction data.

As shown in FIG. 38, the correction data indicates the relationship between the original pixel reliability, which is the pixel reliability Pc before correction, and the corrected pixel reliability. For example, definition line 96a, which is close-distance correction data, is data in which the original pixel reliability and the corrected pixel reliability are in a proportional relationship regardless of the magnitude of the original pixel reliability. On the other hand, definition line 96b, which is long-distance correction data, is correction data that suppresses the increase in the corrected pixel reliability when the original pixel reliability is large compared to when the original pixel reliability is small.

By using the modified pixel reliability, it is possible to more appropriately calculate the pixel reliability Pc and the region reliability Rc calculated based on the pixel reliability Pc, thereby improving the accuracy of the final oxygen saturation.

In addition, when calculating the region of interest reliability Rc in the region of interest, the reliability calculation unit 83 may perform weighting according to the area of the region of interest and/or the distance of the region of interest from the center of the correction image 90. Such weighting can be performed using data that associates the area of the region of interest or the distance of the region of interest from the center of the correction image 90 with a weight.

As shown in FIG. 39, the area weight Aw for the area of the region of interest may be calculated according to a definition line 97, which is data that associates the area of the region of interest with the area weight Aw. The definition line 97 is set in advance by examining the relationship between the area of the region of interest and the oxygen saturation level.

As shown in FIG. 40, the distance weight Dw for the distance of the region of interest from the center of the correction image 90 (hereafter referred to as the region of interest distance) may be calculated according to a definition line 98. The definition line 98 is set in advance by examining the relationship between the region of interest distance and oxygen saturation. Note that, when the horizontal axis of the correction image 90 is x and the vertical axis is y, the region of interest distance can be calculated as the square root of the sum of the squares of the differences between the x-axis value and the y-axis value between the coordinates of the center of the correction image 90 and the coordinates of the center or center of gravity of the region of interest.

The weighting according to the area of the region of interest and/or the distance of the region of interest from the center of the correction image 90 may be done only according to the area of the region of interest, or only according to the distance of the region of interest from the center of the correction image 90, or both. When both are done, the weighted region of interest reliability Rc is calculated according to the following formula (2).

　Weighted region of interest reliability Rc = region of interest reliability Rc × area weight Aw × distance weight Dw　(2)

By weighting the region of interest reliability Rc, the area of the region of interest or the distance of the region of interest from the center of the correction image 90, which are factors that affect the calculation of the region of interest reliability Rc, are reflected. This makes it possible to calculate the region of interest reliability Rc more preferably, thereby improving the accuracy of the ultimately obtained oxygen saturation.

The high-reliability region of interest display unit 86 may change the display mode of the high-reliability region of interest HRc displayed on the extended display 18 in response to a user instruction. As shown in FIG. 41, the boundary line 95 that is displayed on the extended display 18 and indicates the high-reliability region of interest HRc may be erased, and instead, a region of interest display color 99 representing the high-reliability region of interest HRc may be displayed on the extended display 18, thereby allowing the user to recognize the high-reliability region of interest HRc.

By making it possible to change the display mode of the high-reliability region of interest HRc displayed on the extended display 18 at the user's command, it becomes possible to change the display mode to match the state of the image shown in the correction image 90, for example by displaying the region of interest display color 99 when the area of the high-reliability region of interest HRc shown in the correction image 90 is small, and displaying the boundary line 95 when the area is large, thereby improving user visibility.

Furthermore, the high reliability region of interest display unit 86 may display a name representing a region shown in the high reliability region of interest HRc on the extended display 18, superimposed on the high reliability region of interest HRc, in response to a user instruction. As shown in FIG. 42, a region name 100 may be displayed on the extended display 18, superimposed on the high reliability region of interest HRc. In the case of FIG. 42, "transverse colon" is displayed as the region name 100. As described above, the region name may be set to, for example, large intestine, small intestine, liver, stomach, etc. Furthermore, the large intestine may be further divided into the rectum, sigmoid colon, descending colon, transverse colon, ascending colon, cecum, etc.

By superimposing a name representing a part that appears in the high-reliability region of interest HRc on the extended display 18 at the user's instruction, for example, when multiple parts of similar shapes appear on the extended display 18 as high-reliability regions of interest HRc, the effect of further improving the visibility of each of the multiple parts is achieved by superimposing multiple names representing the multiple parts on the high-reliability regions of interest HRc. Furthermore, when the target part is known in advance, for example, by displaying only the high-reliability region of interest HRc of the previously set target part on the extended display 18, it becomes easy to determine whether the target part is set as a high-reliability region of interest HRc.

The region selection unit 87 may accept a selection of multiple high-reliability regions of interest HRc by the user and correct data based on the selected multiple high-reliability regions of interest HRc. As an example shown in FIG. 43, when the first high-reliability region of interest HRc1 and the second high-reliability region of interest HRc2 are displayed on the extended display 18 by the boundary lines 95a and 95b, respectively, when the user selects the first high-reliability region of interest HRc1 and the second high-reliability region of interest HRc2 using the processor-side interface 16 or the zoom operation switch 12d, the correction unit 88 performs correction based on the selected first high-reliability region of interest HRc1 and the second high-reliability region of interest HRc2. In this case, the correction unit 88 can perform correction processing on the region of interest that includes the selected first high-reliability region of interest HRc1 and the second high-reliability region of interest HRc2.

By making it possible to correct the data used to calculate oxygen saturation based on multiple high-reliability regions of interest HRc, for example, when there are multiple types of regions to be corrected, correction processing can be performed selectively using only the multiple desired regions without changing the field of view.

The high-reliability region of interest display unit 86 may include a determination notification unit 101, as shown in FIG. 44. When the target region of interest is determined to be a low-reliability region of interest LRc by the correction feasibility determination unit 85, the determination notification unit 101 performs control to display operation guidance or the like on the display 18 to notify the user so that the region of interest is determined to be a high-reliability region of interest HRc. The notification may be by voice or the like.

As shown in FIG. 45(A), when the user selects the low-reliability region of interest LRc determined by the correction feasibility determination unit 85 using the processor-side interface 16 or the zoom operation switch 12d, the user may be notified by displaying operation guidance GD1 on the extended display 18 to change the low-reliability region of interest HRc selected by the user to a high-reliability region of interest HRc, as shown in FIG. 45(B). When the operation guidance GD1 of "Avoid dark areas" is displayed on the extended display 18, the user is notified of the operation guidance GD1 and operates the endoscope 12 to make the observation target appear brighter. This changes the low-reliability region of interest LRc to a high-reliability region of interest HRc, and correction processing, etc. can be carried out.

In addition, when the correction feasibility determination unit 85 determines that the correction image 90 does not include the high-reliability region of interest HRc, the determination notification unit 101 may perform control to notify the user of this fact. The notification may be operation guidance to reacquire the correction image 90 so that the target area becomes the high-reliability region of interest HRc.

As shown in FIG. 46(A), if the correction possibility determination unit 85 determines that the correction image 90 includes low-reliability regions of interest LRc such as the first low-reliability region of interest LRc1 and the second low-reliability region of interest LRc2 in the correction process but does not include the high-reliability region of interest HRc, the user may be notified by displaying operation guidance GD2 on the extended display 18 to allow the user to set the transverse colon 91 and/or peritoneum 92, which are the target sites, as the high-reliability region of interest HRc, as shown in FIG. 46(B). When the operation guidance GD2 of "Avoid bleeding, residue, fat, etc." is displayed on the extended display 18, the user can understand the content of the operation guidance GD2 by being notified of this operation guidance GD2 and operate the endoscope 12 to re-acquire the correction image 90 with fewer factors that affect the correction.

If the correction image 90 does not include the high-reliability region of interest HRc, the user is notified of this fact, and when operating the endoscope 12 to reacquire the correction image 90, the user can reduce the burden of reacquiring the correction image 90 by following the instructions in the operation guidance without trial and error.

The high-reliability region of interest display unit 86 may superimpose and display on the extended display 18 a region including pixels in the high-reliability region of interest HRc whose pixel reliability Pc is equal to or greater than a preset pixel reliability threshold Pth.

As shown in FIG. 47(A), when the boundary lines 95a and 95b, the high-reliability region of interest HRc1 and HRc2, and the low-reliability region of interest LRc in the correction image 90 are displayed on the extended display 18, it is preferable that, as shown in FIG. 47(B), in the high-reliability region of interest HRc1, pixel overlapping regions 102a and 102b including pixels whose pixel reliability Pc is equal to or greater than a preset pixel reliability threshold Pth are superimposed on the extended display 18, and in the high-reliability region of interest HRc2, pixel overlapping region 102c including pixels whose pixel reliability Pc is equal to or greater than a preset pixel reliability threshold Pth is superimposed on the extended display 18.

By superimposing and displaying on the extended display 18 an area including pixels in the high-reliability region of interest HRc, where the pixel reliability Pc is equal to or greater than a preset pixel reliability threshold Pth, the user can easily and at a glance see at which position on the endoscopic image a more accurate oxygen saturation level can be calculated. Note that, even in the low-reliability region of interest LRc, an area including pixels that are equal to or greater than the pixel reliability threshold Pth may be superimposed and displayed on the extended display 18.

In the above embodiment, the hardware structure of the processing units that perform various processes, such as the DSP 45, image processing unit 50, image communication unit 51, display control unit 52, and central control unit 53, as well as the image acquisition unit 60a and image processing unit 60b, are various processors as shown below. The various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a GPU (Graphical Processing Unit), a Programmable Logic Device (PLD), which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), and a dedicated electrical circuit, which is a processor with a circuit configuration designed specifically to perform various processes.

A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU, etc.). Multiple processing units may also be configured with one processor. As an example of configuring multiple processing units with one processor, first, there is a form in which one processor is configured with a combination of one or more CPUs and software, as represented by computers such as clients and servers, and this processor functions as multiple processing units. Second, there is a form in which a processor is used that realizes the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip, as represented by System On Chip (SoC). In this way, the various processing units are configured using one or more of the above various processors as a hardware structure.

More specifically, the hardware structure of these various processors is an electric circuit (circuitry) that combines circuit elements such as semiconductor elements. The hardware structure of the memory unit is a storage device such as a hard disc drive (HDD) or solid state drive (SSD).

From the above description, the following additional points can be understood:
[Additional Note 1]
An endoscope system that calculates an oxygen saturation level of an observation object using data used for calculating an oxygen saturation level,
A processor is provided.
The processor,
Acquire an image of the object to be observed,
Recognizing a plurality of parts appearing in the acquired image,
A region of interest is set for each of the recognized parts in the image;
calculating a region of interest confidence for each of the regions of interest based on pixels contained in the region of interest;
performing a comparison process of comparing the region of interest reliability with a preset region of interest reliability threshold;
performing a determination process for determining whether each of the regions of interest is a high-reliability region of interest for which the data can be corrected based on a comparison result obtained by the comparison process;
displaying the high confidence region of interest on a display;
An endoscope system that corrects the data based on the high confidence region of interest selected by a user.
[Additional Note 2]
The endoscope system according to claim 1, wherein the processor calculates the region of interest reliability based on a pixel reliability calculated for each pixel included in the region of interest and the number of pixels included in the region of interest.
[Additional Note 3]
The endoscope system according to claim 2, wherein the processor calculates the pixel reliability modified according to a distance from a center of an image in the image.
[Additional Note 4]
An endoscopic system according to any one of claims 1 to 3, wherein the processor calculates the region of interest reliability by weighting the region of interest according to its area or its distance from the image center in the image.
[Additional Note 5]
The endoscope system according to any one of claims 1 to 4, wherein the processor changes a display mode of the high-reliability region of interest in response to an instruction from the user.
[Additional Note 6]
The endoscope system according to any one of claims 1 to 5, wherein the processor displays on the display a name indicating a region depicted in the high reliability region of interest superimposed on the high reliability region of interest.
[Additional Note 7]
The endoscopic system of any one of claims 1 to 6, wherein the processor accepts a selection of a plurality of the high reliability regions of interest by the user, and corrects the data based on the plurality of the high reliability regions of interest.
[Additional Note 8]
An endoscopic system as described in any one of appendix 1 to 7, wherein the processor performs a determination process in which, in the comparison result, if the region of interest reliability is equal to or greater than a threshold for the region of interest reliability, the region of interest is determined to be a high reliability region of interest, and if the region of interest reliability is lower than the threshold for the region of interest reliability, the region of interest is determined to be a low reliability region of interest in which correction of the data is not possible.
[Additional Note 9]
The endoscopic system described in Appendix 8, wherein the processor performs control to notify the user of operational guidance for changing the low-reliability region of interest to the high-reliability region of interest when the user selects the low-reliability region of interest.
[Additional Item 10]
The endoscope system according to any one of claims 1 to 9, wherein the processor performs control to notify the user when it is determined that the image does not include the high-reliability region of interest.
[Additional Note 11]
The endoscope system according to any one of claims 2 to 10, wherein the processor controls the display to superimpose an area in which the pixel reliability is equal to or greater than a pixel reliability threshold on the high reliability area of interest.
[Additional Item 12]
An endoscopic system according to any one of claims 1 to 11, wherein the processor, when correction of the data is performed, calculates the oxygen saturation of the image based on the corrected data.

10 Endoscope system 12 Endoscope 12a Insertion section 12b Operation section 12c Mode changeover switch 12d Zoom operation switch 13 Light source device 14 Processor device 15 Display 16 Processor side interface 17 Extended processor device 18 Extended display 20 Light source section 20a V-LED
20b BS-LED
20c BL-LED
20d G-LED
20e R-LED
21 Light source processor 23 Optical path coupling unit 24 Light guide 30 Illumination optical system 31 Imaging optical system 32 Illumination lens 35 Objective lens 36 Imaging sensor 37 Imaging processor 40 CDS/AGC circuit 41 A/D converter 45 DSP
50 Image processing unit 51 Image communication unit 52 Display control unit 53 Central control unit 55 White light 55a, 55b, 55c, 56a, 56b Curve 56 First illumination light 57 Second illumination light 58 Third illumination light 60a Image acquisition unit 60b Image processing unit 61 Oxygen saturation image generation unit 62 Specific dye concentration calculation unit 63 Table correction unit 64 Display control unit 70 Base image generation unit 71 Calculation value calculation unit 72 Oxygen saturation calculation unit 73, 73a, 73b Oxygen saturation calculation table 74 Color tone adjustment unit 75 Specific dye concentration calculation table 81 Recognition unit 82 Region setting unit 83 Reliability calculation unit 84 Comparison processing unit 85 Correction possibility determination unit 86 High reliability region of interest display unit 87 Region selection unit 88 Correction unit 90 Correction image 91 Transverse colon 92 Peritoneum 91a First region of interest 92a Second region of interest 93 Definition lines 93a, 94b Comparison results 95, 95a, 95b Boundaries 96a, 96b, 97, 98 Definition line 99 Region of interest display color 100 Part name 101 Determination notification units 102a, 102b Pixel overlap region NP1 White light image NP2 White light equivalent image OP Oxygen saturation image MS Message BF B color filter GF G color filter RF R color filter AR0 to AR4 Regions EL, ELL, ELH Contour lines CV0 to CV4 Curved surfaces DFX, DFY Definition line HRc High-reliability region of interest HRc1 First high-reliability region of interest HRc2 Second high-reliability region of interest LRc Low-reliability region of interest LRc1 First low-reliability region of interest LRc2 Second low-reliability region of interest Aw Area weight Dw Distance weights GD1, GD2 Operation Guidance ST100 to ST190 Steps

Claims (14)

An endoscope system that calculates an oxygen saturation level of an observation object using data used for calculating an oxygen saturation level,
A processor is provided.
The processor,
Acquire an image of the object to be observed,
Recognizing a plurality of parts appearing in the acquired image,
A region of interest is set for each of the recognized parts in the image;
calculating a region of interest confidence for each of the regions of interest based on pixels contained in the region of interest;
performing a comparison process of comparing the region of interest reliability with a preset region of interest reliability threshold;
performing a determination process for determining whether each of the regions of interest is a high-reliability region of interest for which the data can be corrected based on a comparison result obtained by the comparison process;
displaying the high confidence region of interest on a display;
An endoscope system that corrects the data based on the high confidence region of interest selected by a user. The endoscope system of claim 1, wherein the processor calculates the region of interest reliability based on the pixel reliability calculated for each pixel contained in the region of interest and the number of pixels contained in the region of interest. The endoscope system of claim 2, wherein the processor calculates the pixel reliability modified according to the distance from the image center in the image. The endoscope system of claim 1, wherein the processor calculates the reliability of the region of interest by weighting the region of interest according to its area or its distance from the center of the image in the image. The endoscope system of claim 1, wherein the processor changes the display mode of the high-confidence region of interest in response to an instruction from the user. The endoscope system of claim 1, wherein the processor displays on the display a name representing the part of the image captured in the high-reliability region of interest superimposed on the high-reliability region of interest. The endoscope system of claim 1, wherein the processor accepts a selection of multiple high-confidence regions of interest by the user and corrects the data based on the multiple high-confidence regions of interest. The endoscope system according to claim 1, wherein the processor performs a determination process in which, in the comparison result, if the region of interest reliability is equal to or greater than the region of interest reliability threshold, the region of interest is determined to be a high reliability region of interest, and if the region of interest reliability is lower than the region of interest reliability threshold, the region of interest is determined to be a low reliability region of interest in which the data cannot be corrected. The endoscope system according to claim 8, wherein the processor performs control to notify the user of operation guidance for changing the low-confidence region of interest to the high-confidence region of interest when the user selects the low-confidence region of interest. The endoscope system of claim 1, wherein the processor performs control to notify the user when it is determined that the image does not include the high-confidence region of interest. The endoscope system according to claim 2, wherein the processor controls to superimpose the region of interest having the pixel reliability equal to or greater than a pixel reliability threshold on the display. The endoscope system of claim 1, wherein the processor calculates the oxygen saturation of the image based on the corrected data when the data correction is performed. 1. A method for operating an endoscope system that calculates an oxygen saturation level of an observation object using data used for calculating an oxygen saturation level, comprising:
A processor is provided.
The processor,
Acquire an image of the object to be observed,
Recognizing a plurality of parts appearing in the acquired image,
A region of interest is set for each of the recognized parts in the image;
calculating a region of interest confidence for each of the regions of interest based on pixels contained in the region of interest;
performing a comparison process of comparing the region of interest reliability with a preset region of interest reliability threshold;
performing a determination process for determining whether each of the regions of interest is a high-reliability region of interest for which the data can be corrected based on a comparison result obtained by the comparison process;
displaying the high confidence region of interest on a display;
A method of operating an endoscopy system, comprising: correcting the data based on the high confidence region of interest selected by a user. An operation program for an endoscope system for calculating an oxygen saturation level of an observation target using data used for calculating an oxygen saturation level, the operation program comprising:
A function for acquiring images of the object to be observed;
A function of recognizing a plurality of parts appearing in the acquired image;
A function of setting a region of interest for each of the recognized parts in the image;
A function of calculating a region of interest reliability for each of the regions of interest based on pixels contained in the region of interest;
A function of performing a comparison process to compare the region of interest reliability with a preset region of interest reliability threshold;
a function of performing a determination process for determining whether each of the regions of interest is a high-reliability region of interest for which the data can be corrected based on a comparison result obtained by the comparison process;
displaying said high confidence region of interest on a display;
and a function of correcting the data based on the high-reliability region of interest selected by a user.

Images