[go: up one dir, main page]

WO2025070479A1 - Système d'endoscope, son procédé de fonctionnement et programme pour faire fonctionner un système d'endoscope - Google Patents

Système d'endoscope, son procédé de fonctionnement et programme pour faire fonctionner un système d'endoscope Download PDF

Info

Publication number
WO2025070479A1
WO2025070479A1 PCT/JP2024/034153 JP2024034153W WO2025070479A1 WO 2025070479 A1 WO2025070479 A1 WO 2025070479A1 JP 2024034153 W JP2024034153 W JP 2024034153W WO 2025070479 A1 WO2025070479 A1 WO 2025070479A1
Authority
WO
WIPO (PCT)
Prior art keywords
interest
region
reliability
image
oxygen saturation
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/JP2024/034153
Other languages
English (en)
Japanese (ja)
Inventor
高宏 岡本
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Fujifilm Corp
Original Assignee
Fujifilm Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Fujifilm Corp filed Critical Fujifilm Corp
Publication of WO2025070479A1 publication Critical patent/WO2025070479A1/fr
Pending legal-status Critical Current
Anticipated expiration legal-status Critical

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof

Definitions

  • This disclosure relates to an endoscope system with an oxygen saturation imaging function, an operating method thereof, and an operating program for the endoscope system.
  • oxygen saturation imaging has become known in the medical field, where endoscopes are used. Oxygen saturation imaging is performed by illuminating the object to be observed with illumination light including a wavelength band whose absorption coefficient changes with changes in the oxygen saturation of hemoglobin in the blood, and capturing an image. Then, based on the image obtained by capturing the image, an oxygen saturation image is displayed on a display, with the color tone changing according to the oxygen saturation level.
  • the object is photographed using multiple illumination lights with different wavelength bands. Then, a predetermined calculation value is calculated using pixel values of the obtained image, and oxygen saturation is calculated using an oxygen saturation calculation table that shows the correlation that associates the calculation value with oxygen saturation, and in some cases, a corrected oxygen saturation calculation table is used to calculate the oxygen saturation. It may be difficult to accurately correct the oxygen saturation calculation table due to disturbances such as halation in the image obtained by the endoscope (hereinafter referred to as the endoscopic image), specific dyes (yellow dyes, etc.), and bleeding, and an endoscopic system that issues a warning when correction of oxygen saturation, etc., fails is known (Patent Document 1). Also known is an endoscopic system that can perform appropriate correction operations even if disturbances such as bleeding are present in the object of observation (Patent Document 2), and an endoscopic system that can calculate specific dye concentrations from multiple image signals and perform appropriate correction operations (Patent Document 3).
  • the accuracy of the correction may decrease. To solve this, it was necessary to change the field of view of the observation subject or to correct for specific pigments (yellow pigment, etc.) before calculating the oxygen saturation.
  • the present disclosure aims to provide an endoscope system, an operating method and an operating program thereof that can perform highly accurate corrections when multiple areas are present in an image during data correction processing related to oxygen saturation calculations performed by a user.
  • the endoscope system disclosed herein is an endoscope system that calculates the oxygen saturation of an object to be observed using data used to calculate the oxygen saturation, and is equipped with a processor, which acquires an image of the object to be observed, recognizes multiple parts of the object to be observed in the acquired image, sets a region of interest in the image for each recognized part, calculates a region of interest reliability for each region of interest based on the pixels contained in the region of interest, performs a comparison process that compares the region of interest reliability with a preset region of interest reliability threshold, displays a high reliability region of interest on a display for each region of interest based on the comparison result obtained by the comparison process, and corrects the data based on the high reliability region of interest selected by the user.
  • the processor preferably calculates the region of interest reliability based on the pixel reliability calculated for each pixel contained in the region of interest and the number of pixels contained in the region of interest.
  • the processor preferably calculates the pixel reliability modified according to the distance from the image center in the image.
  • the processor preferably calculates the confidence level of the region of interest by weighting the region of interest according to its area or its distance from the image center in the image.
  • the processor preferably changes the display mode of the high confidence region of interest in response to a user instruction.
  • the processor be able to change the display format of the high confidence region of interest in response to a user instruction.
  • the processor displays on the display a name representing the part of the body that appears in the high-confidence region of interest, superimposed on the high-confidence region of interest.
  • the processor preferably accepts a user selection of multiple high confidence regions of interest and corrects the data based on the multiple high confidence regions of interest.
  • the processor preferably performs a determination process in which, in the comparison result, if the region of interest reliability is equal to or greater than a region of interest reliability threshold, the region of interest is determined to be a high reliability region of interest, and, if the region of interest reliability is lower than the region of interest reliability threshold, the region of interest is determined to be a low reliability region of interest in which data correction is not possible.
  • the processor If the user selects a low-confidence region of interest, it is preferable for the processor to perform control to notify the user of operational guidance for changing the low-confidence region of interest to a high-confidence region of interest.
  • the processor preferably controls the display to superimpose areas where the pixel reliability is equal to or greater than a pixel reliability threshold on the high reliability region of interest.
  • the processor preferably calculates the oxygen saturation of the image based on the corrected data.
  • the operation method of the endoscope system disclosed herein is an operation method of an endoscope system that calculates the oxygen saturation of an observation target using data used for calculating the oxygen saturation, and includes a processor, which recognizes multiple parts shown in an acquired image, sets a region of interest in the image for each recognized part, calculates a region of interest reliability for each region of interest based on the pixels contained in the region of interest, performs a comparison process that compares the region of interest reliability with a preset region of interest reliability threshold, and performs a determination process for each region of interest based on the comparison result obtained by the comparison process to determine whether or not it is a high-reliability region of interest for which data can be corrected, displays the high-reliability region of interest on a display, and performs data correction based on the high-reliability region of interest selected by the user.
  • the operating program of the endoscope system disclosed herein is an operating program of an endoscope system that calculates the oxygen saturation of an observation target using data used for calculating the oxygen saturation, and provides a computer with the following functions: acquiring an image of the observation target; recognizing multiple parts from the acquired image; setting a region of interest for each recognized part in the image; calculating a region of interest reliability for each region of interest based on the pixels contained in the region of interest; performing a comparison process that compares the region of interest reliability with a preset region of interest reliability threshold; performing a determination process that determines, based on the comparison result obtained by the comparison process, whether each region of interest is a high-reliability region of interest for which data can be corrected; displaying the high-reliability region of interest on a display; and correcting data based on the high-reliability region of interest selected by the user.
  • FIG. 1 is a schematic diagram of an endoscope system for laparoscopy.
  • FIG. 4A is an explanatory diagram illustrating the display state of the display in normal mode
  • FIG. 4B is an explanatory diagram illustrating the display state of the extended display in normal mode.
  • 13A is an explanatory diagram illustrating the display mode of the display in the oxygen saturation mode
  • FIG. 13B is an explanatory diagram illustrating the display mode of the extended display in the oxygen saturation mode
  • 13 is an explanatory diagram illustrating the display mode of the extended display when the mode is switched to the oxygen saturation mode.
  • FIG. FIG. 13A is an image diagram of an extended display showing an oxygen saturation image on the serosal side
  • FIG. 13B is an image diagram of an extended display showing an oxygen saturation image inside the digestive tract.
  • FIG. 2 is a block diagram showing the functions of the endoscope system.
  • FIG. 1 is a graph showing an emission spectrum of white light.
  • 13A is a graph showing the emission spectrum of the first illumination light
  • FIG. 13B is a graph showing the emission spectrum of the second illumination light
  • FIG. 13C is a graph showing the emission spectrum of the green light G.
  • 4 is a graph showing the spectral sensitivity of an image sensor.
  • 11 is a table showing illumination and acquired image signals in a normal mode.
  • 11 is a table showing illumination and acquired image signals in an oxygen saturation mode or a correction mode.
  • 11A and 11B are explanatory diagrams for explaining light emission control and display control in the oxygen saturation mode or the correction mode.
  • 1 is a graph showing the reflectance spectrum of hemoglobin that varies depending on blood concentration.
  • 1 is a graph showing the reflection spectrum of hemoglobin and the absorption spectrum of a yellow dye, which vary depending on the concentration of the yellow dye.
  • 13 is a table showing the oxygen saturation dependency, blood concentration dependency, and brightness dependency of a B1 image signal, a G2 image signal, and an R2 image signal in a case where there is no influence of a yellow pigment.
  • 1 is a graph showing contour lines representing oxygen saturation.
  • 13 is a table showing the oxygen saturation dependency, blood concentration dependency, and brightness dependency of the X-axis value indicating the signal ratio ln(R2/G2) and the Y-axis value indicating the signal ratio ln(B1/G2).
  • FIG. 13 is a table showing the oxygen saturation dependency, blood concentration dependency, yellow pigment dependency, and brightness dependency of a B1 image signal, a G2 image signal, and an R2 image signal when there is an influence of a yellow pigment.
  • FIG. 13 is an explanatory diagram showing the oxygen saturation with and without a yellow dye when the observation subject has the same oxygen saturation.
  • 13 is a table showing the oxygen saturation dependency, blood concentration dependency, yellow pigment dependency, and brightness dependency of a B1 image signal, a B3 image signal, a G2 image signal, a G3 image signal, an R2 image signal, and a B2 image signal when influenced by yellow pigment.
  • 1 is a graph showing a surface representing oxygen saturation as a function of yellow dye.
  • FIG. 2 is a block diagram showing the functions of the extended processor device;
  • FIG. 2 is a block diagram showing functions of an image processing unit.
  • FIG. 11 is an explanatory diagram for explaining a method of calculating oxygen saturation.
  • FIG. 11 is an explanatory diagram showing a method for generating a contour line corresponding to a specific dye concentration.
  • FIG. 4 is a block diagram showing the functions of a table correction unit.
  • FIG. 1A is an explanatory diagram showing a plurality of parts in a correction image
  • FIG. 1B is an explanatory diagram for explaining recognition of a plurality of parts in a correction image.
  • FIG. 11 is an explanatory diagram for explaining setting of a region of interest in a correction image.
  • 1 is a graph showing the relationship between pixel value and pixel reliability.
  • 13 is a graph showing pixel reliability due to bleeding or the like.
  • 13 is a graph showing pixel reliability due to fat or the like.
  • FIG. 11 is an explanatory diagram for explaining a comparison between a region of interest reliability of a region of interest and a threshold for region of interest reliability;
  • FIG. 13 is an explanatory diagram for explaining determination of a high-confidence region of interest and a low-confidence region of interest.
  • FIG. 13 is a pictorial diagram of an augmented display showing a high confidence region of interest representation.
  • 10 is a flowchart showing a flow of a series of processes in a correction mode.
  • 11 is a graph showing correction data for correcting pixel reliability according to the distance from the image center.
  • 11 is a graph showing data for weighting calculation of region reliability based on the area of a region of interest.
  • FIG. 11 is a graph showing data for weighting calculation of region reliability according to the distance of a region of interest from the center of an image.
  • FIG. 13 is an explanatory diagram illustrating an example of a display mode of a high-confidence region of interest.
  • FIG. 13 is an explanatory diagram illustrating an example of a display mode of a high-confidence region of interest.
  • FIG. 13 is an explanatory diagram illustrating display of multiple high confidence regions of interest.
  • FIG. 13 is a block diagram showing the functions of a high confidence region of interest display unit.
  • (A) is an explanatory diagram showing a high-reliability region of interest and a low-reliability region of interest in a correction image
  • (B) is an explanatory diagram showing a display in which operation guidance is provided when a low-reliability region of interest is selected.
  • 13A is an explanatory diagram showing a low-confidence region of interest in a correction image
  • FIG. 13B is an explanatory diagram explaining a display in which operation guidance is notified when there is no high-confidence region of interest
  • 1A is an explanatory diagram showing a plurality of high-confidence regions of interest and low-confidence regions of interest in a correction image
  • FIG. 1B is an explanatory diagram showing pixels above a region-of-interest threshold superimposed thereon;
  • the endoscope system 10 includes an endoscope 12, a light source device 13, a processor device 14, a display 15, a processor-side interface 16, an extended processor device 17, and an extended display 18.
  • the endoscope 12 is optically or electrically connected to the light source device 13, and is electrically connected to the processor device 14.
  • the extended processor device 17 is electrically connected to the light source device 13, the processor device 14, and the extended display 18.
  • Each of these connections is not limited to being wired, and may be wireless. Also, they may be connected via a network.
  • the endoscope system 10 is a rigid endoscope in which the endoscope 12 is inserted into a body cavity of a subject to perform surgical treatment, and images of the organs in the body cavity are captured from the serous membrane side.
  • the endoscope system 10 is particularly suitable for use as a laparoscope.
  • the endoscope 12 may also be a flexible endoscope that is inserted through the nose, mouth, or anus of the subject.
  • the endoscope 12 When the endoscope 12 is a laparoscope, as shown in FIG. 1, the endoscope 12 includes an insertion section 12a that is inserted into the abdominal cavity of the subject, and an operation section 12b that is provided at the base end of the insertion section 12a.
  • An optical system and an image sensor are built into the tip section 12e, which is the tip part of the insertion section 12a.
  • the optical system includes an illumination optical system described below for irradiating the subject with illumination light, and an image pickup optical system described below for capturing an image of the subject (see FIG. 6).
  • the image sensor generates an image signal by forming an image on an image plane of the reflected light from the observation object that has passed through the image pickup optical system.
  • the generated image signal is output to the processor device 14.
  • the operation section 12b includes a mode switching switch 12c and a zoom operation switch 12d for zoom operation, etc.
  • the mode switching switch 12c is used to switch between observation modes described below.
  • the light source device 13 generates illumination light.
  • the processor device 14 performs system control of the endoscope system 10, and further performs control such as generating endoscopic images by performing image processing on image signals transmitted from the endoscope 12.
  • the display 15 displays medical images transmitted from the processor device 14.
  • the processor-side interface 16 has a keyboard, mouse, microphone, tablet, foot switch, touch pen, etc., and accepts input operations such as function settings.
  • the endoscope system 10 has three observation modes: normal mode, oxygen saturation mode, and tissue color correction mode (hereafter referred to as correction mode), and these three modes can be switched automatically or by the user operating the mode switching switch 12c.
  • normal mode a white light image NP1 with natural colors obtained by capturing an image of the observation target using white light as illumination light is displayed on the display 15, while nothing is displayed on the extended display 18.
  • the oxygen saturation of the object of observation is calculated, and an oxygen saturation image OP that visualizes the calculated oxygen saturation is displayed on the extended display 18.
  • an oxygen saturation image OP that visualizes the calculated oxygen saturation is displayed on the extended display 18.
  • a white light equivalent image NP2 that has fewer short wavelength components than the white light image NP1 is displayed on the display 15.
  • corrections are made to the oxygen saturation calculation based on pixels within a high-confidence region of interest, which will be described later.
  • Correction of the oxygen saturation calculation is performed in a correction mode, which will be described later, and the oxygen saturation calculation table is corrected using an area determined to be capable of being corrected (details of the area will be described later).
  • the mode may automatically switch to the correction mode once, and then switch to the oxygen saturation mode after completing the correction process in the correction mode.
  • the mode switches to the oxygen saturation mode, and an oxygen saturation image OP is displayed on the extended display 18. For example, when switching to the oxygen saturation mode, as shown in FIG. 4, a message MS stating "Please perform correction process" is displayed on the extended display 18, and it is preferable that the oxygen saturation calculation is performed after the correction process.
  • the endoscope system 10 displays a serosal side oxygen saturation image, which is an image of the oxygen saturation state on the serosal side, on the extended display 18 in the oxygen saturation mode, as shown in FIG. 5(A).
  • a serosal side oxygen saturation image which is an image of the oxygen saturation state on the serosal side
  • an internal digestive tract oxygen saturation image which is an image of the oxygen saturation state inside the digestive tract
  • FIG. 5(B) It is preferable to use an image in which the saturation is adjusted for the white light equivalent image NP2 as the serosal side oxygen saturation image.
  • It is preferable to adjust the saturation in the correction mode regardless of whether it is a mucosa, a serosal membrane, a flexible scope, or a rigid scope.
  • a rigid scope type is used as the endoscope 12.
  • the processor device 14 is electrically connected to the display 15 and the processor-side interface 16.
  • the processor device 14 receives image signals from the endoscope 12 and performs various processes based on the image signals.
  • the display 15 outputs and displays images or information of the observation target processed by the processor device 14.
  • the processor-side interface 16 has a keyboard, mouse, touchpad, microphone, foot pedal, etc., and has the function of accepting input operations such as function settings.
  • the light source device 13 includes a light source section 20 and a light source processor 21 that controls the light source section 20.
  • the light source section 20 has, for example, a plurality of semiconductor light sources, which are turned on or off, and when turned on, the amount of light emitted by each semiconductor light source is controlled to emit illumination light that illuminates the object of observation.
  • the light source section 20 has five colored LEDs: a V-LED (Violet Light Emitting Diode) 20a, a BS-LED (Blue Short-wavelength Light Emitting Diode) 20b, a BL-LED (Blue Long-wavelength Light Emitting Diode) 20c, a G-LED (Green Light Emitting Diode) 20d, and an R-LED (Red Light Emitting Diode) 20e.
  • V-LED Volt Light Emitting Diode
  • BS-LED Blue Short-wavelength Light Emitting Diode
  • BL-LED Blue Long-wavelength Light Emitting Diode
  • G-LED Green Light Emitting Diode
  • R-LED Red Light Emitting Diode
  • the V-LED 20a emits violet light V of 410 nm ⁇ 10 nm.
  • the BS-LED 20b emits second blue light BS of 450 nm ⁇ 10 nm.
  • the BL-LED 20c emits first blue light BL of 470 nm ⁇ 10 nm.
  • the G-LED 20d emits green light G in the green band.
  • the central wavelength of the green light G is preferably 540 nm.
  • the R-LED 20e emits red light R in the red band.
  • the central wavelength of the red light R is preferably 620 nm.
  • the central wavelength and peak wavelength of each of the LEDs 20a to 20e may be the same or different.
  • Both BL-LED 20c and BS-LED 20b are blue light sources that emit blue light.
  • the central wavelength and wavelength band of the blue light emitted by BL-LED 20c (hereafter referred to as BL light) and the blue light emitted by BS-LED 20b (hereafter referred to as BS light) are different.
  • the central wavelength and wavelength band of BL light are the central wavelength and wavelength band in the blue wavelength band where the difference in the absorption coefficient between oxygenated hemoglobin and reduced hemoglobin is approximately maximized.
  • G-LED 20d is a green light source that emits broadband green light (hereafter referred to as G light) with a central wavelength of 540 nm.
  • R-LED 20e is a red light source that emits broadband red light (hereafter referred to as R light).
  • the light source processor 21 independently controls the on/off and light emission amount of each of the LEDs 20a-20e by inputting control signals to each of the LEDs 20a-20e.
  • the control of the on/off and other aspects of the LEDs in the light source processor 21 differs depending on the mode, and will be described in detail later.
  • the light emitted by each of the LEDs 20a to 20e is incident on the light guide 24 via the optical path coupling section 23, which is composed of a mirror, lens, etc.
  • the light guide 24 is built into the endoscope 12 and the universal cord (a cord that connects the endoscope 12 with the light source device 13 and the processor device 14).
  • the light guide 24 propagates the light from the optical path coupling section 23 to the tip of the endoscope 12.
  • the endoscope 12 is provided with an illumination optical system 30 and an imaging optical system 31.
  • the illumination optical system 30 has an illumination lens 32, and the illumination light propagated by the light guide 24 is irradiated onto the observation object via the illumination lens 32.
  • the imaging optical system 31 has an objective lens 35 and an imaging sensor 36. Light from the observation object irradiated with the illumination light is incident on the imaging sensor 36 via the objective lens 35. As a result, an image of the observation object is formed on the imaging sensor 36.
  • the imaging sensor 36 is a color imaging sensor that captures an image of an object being observed while being illuminated with illumination light.
  • Each pixel of the imaging sensor 36 is provided with either a B pixel (blue pixel) having a B (blue) color filter, a G pixel (green pixel) having a G (green) color filter, or an R pixel (red pixel) having an R (red) color filter. Therefore, when an object being observed is photographed with the imaging sensor 36, three types of images are obtained: a B image, a G image, and an R image.
  • the imaging sensor 36 be a color imaging sensor with a Bayer array in which the ratio of the number of B pixels, G pixels, and R pixels is 1:2:1.
  • a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal-Oxide Semiconductor) image sensor can be used.
  • a complementary color image sensor equipped with complementary color filters of C (cyan), M (magenta), Y (yellow) and G (green) may be used.
  • image signals of four colors CMYG are output, and by converting the four color image signals of CMYG into three color image signals of RGB by complementary color-primary color conversion, image signals of each of the RGB colors similar to those of the image sensor 36 can be obtained.
  • the imaging sensor 36 is driven and controlled by an imaging processor 37.
  • the control of each mode in the imaging processor 37 will be described later.
  • a CDS/AGC circuit 40 Correlated Double Sampling/Automatic Gain Control
  • CDS correlated double sampling
  • AGC automatic gain control
  • the image signal that passes through the CDS/AGC circuit 40 is converted into a digital image signal by an A/D converter 41 (Analog/Digital).
  • the digital image signal after A/D conversion is input to the processor device 14.
  • the processor unit 14 comprises a DSP (Digital Signal Processor) 45, an image processing unit 50, an image communication unit 51, a display control unit 52, and a central control unit 53.
  • the processor unit 14 has programs relating to various processes built into a program memory (not shown).
  • the functions of the DSP 45, image processing unit 50, image communication unit 51, display control unit 52, and central control unit 53 are realized by the central control unit 53, which is made up of a processor, executing the programs in the program memory.
  • the DSP 45 performs various signal processing such as defect correction, offset processing, gain correction, demosaic processing, linear matrix processing, white balance processing, gamma conversion processing, YC conversion processing, and noise reduction processing on the image signal received from the endoscope 12.
  • defect correction processing the signal of a defective pixel of the imaging sensor 36 is corrected.
  • offset processing dark current components are removed from the image signal that has been subjected to defect correction processing, and an accurate zero level is set.
  • gain correction processing the signal level of each image signal is adjusted by multiplying the image signal of each color after offset processing by a specific gain. The image signal of each color after gain correction processing is subjected to demosaic processing and linear matrix processing to improve color reproducibility.
  • the DSP 45 performs noise reduction processing using, for example, the moving average method or the median filter method.
  • the image processing unit 50 performs various image processing on the image signal from the DSP 45.
  • Image processing includes color conversion processing such as 3x3 matrix processing, tone conversion processing, and table processing for three-dimensional oxygen saturation calculation, color enhancement processing, and structural enhancement processing such as spatial frequency enhancement.
  • the image processing unit 50 performs image processing according to the mode. In the normal mode, the image processing unit 50 performs image processing for the normal mode to generate a white light image NP1. In the oxygen saturation mode, the image processing unit 50 generates a white light equivalent image NP2 and transmits the image signal from the DSP 45 to the extension processor unit 17 via the image communication unit 51.
  • the extension processor unit 17 generates an oxygen saturation image OP based on the image signal of the transmitted endoscopic image.
  • the display control unit 52 performs display control to display the white light image NP1 or image information such as the oxygen saturation image OP from the image processing unit 50, and other information, on the display 15 or the extended display 18. In accordance with the display control, the white light image NP1 or the white light equivalent image NP2 is displayed on the display 15.
  • the extended processor device 17 receives image signals from the processor device 14 and performs various image processing.
  • the extended processor device 17 functions as an image processing device. Image processing in the extended processor device 17 is performed in a correction mode and an oxygen saturation mode, and is processing for generating an oxygen saturation image.
  • the generated oxygen saturation image OP is displayed on the extended display 18.
  • the extended processor device 17 calculates the region of interest reliability for each part in accordance with user operation, and performs correction processing related to the calculation of oxygen saturation for each part based on the calculated region of interest reliability.
  • the extended processor device 17 calculates the oxygen saturation and generates an oxygen saturation image OP that visualizes the calculated oxygen saturation.
  • the generated oxygen saturation image OP is displayed on the extended display 18.
  • the extended processor device 17 also performs a correction process related to the calculation of the oxygen saturation. More specifically, in the correction process, a specific dye concentration is calculated in accordance with user operations, and the oxygen saturation calculation table is corrected based on the calculated specific dye concentration.
  • a specific dye concentration is calculated in accordance with user operations, and the oxygen saturation calculation table is corrected based on the calculated specific dye concentration.
  • An example of the specific dye is a yellow dye. Therefore, the specific dye concentration is the concentration of the yellow dye in the image information contained in the endoscopic image. Details of the oxygen saturation mode and the correction mode performed by the extended processor device 17 will be described later.
  • the V-LED 20a, BS-LED 20b, G-LED 20d, and R-LED 20e are simultaneously turned on to emit white light 55, as shown in FIG. 7, that includes purple light V with a central wavelength of 410 nm, second blue light BS with a central wavelength of 450 nm, broadband green light G in the green band, and red light R with a central wavelength of 620 nm.
  • the graph in the figure shows a schematic representation of the light intensity of each wavelength band.
  • the oxygen saturation mode and the correction mode three frames of light emission with different light emission patterns are repeated.
  • the BL-LED 20c, the G-LED 20d, and the R-LED 20e are simultaneously turned on to emit a first broadband illumination light 56 including a first blue light BL with a central wavelength of 470 nm, a broadband green light G in the green band, and a red light R with a central wavelength of 620 nm.
  • the second frame as shown in FIG.
  • the BS-LED 20b, the G-LED 20d, and the R-LED 20e are simultaneously turned on to emit a second blue light BS with a central wavelength of 450 nm, a broadband green light G in the green band, and a red light R with a central wavelength of 620 nm.
  • the G-LED 20d is turned on to emit a broadband green light G in the green band as a third illumination light 58.
  • the first and second frames are the frames required to obtain the image signal necessary to calculate the oxygen saturation, so light may be emitted only in the first and second frames.
  • the imaging processor 37 controls the imaging sensor 36 to capture an image of an object being observed that is illuminated with white light 55 consisting of purple light V, second blue light BS, green light G, and red light R, for each frame. This causes the B pixels of the imaging sensor 36 to output Bc image signals, the G pixels to output Gc image signals, and the R pixels to output Rc image signals.
  • the imaging processor 37 In the oxygen saturation mode, when the first illumination light 56 including the first blue light BL, green light G, and red light R is illuminated on the observation object in the first frame, the imaging processor 37 outputs a B1 image signal from the B pixel of the imaging sensor 36, a G1 image signal from the G pixel, and an R1 image signal from the R pixel as the first illumination light image.
  • the imaging processor 37 When the second illumination light 57 including the second blue light BS, green light G, and red light R is illuminated on the observation object in the second frame, the imaging processor 37 outputs a B2 image signal from the B pixel of the imaging sensor 36, a G2 image signal from the G pixel, and an R2 image signal from the R pixel as the second illumination light image.
  • the imaging processor 37 In the third frame, when the third illumination light 58, which is green light G, is illuminated on the object to be observed, the imaging processor 37 outputs a B3 image signal from the B pixel of the imaging sensor 36, a G3 image signal from the G pixel, and an R3 image signal from the R pixel as the third illumination light image.
  • the third illumination light 58 which is green light G
  • the imaging processor 37 outputs a B3 image signal from the B pixel of the imaging sensor 36, a G3 image signal from the G pixel, and an R3 image signal from the R pixel as the third illumination light image.
  • the first illumination light 56 is emitted in the first frame (1stF)
  • the second illumination light 57 is emitted in the second frame (2ndF)
  • the third illumination light 58 is emitted in the third frame (3rdF)
  • a white light equivalent image NP2 obtained based on the emission of the second illumination light 57 in the second frame is displayed on the display 15.
  • an oxygen saturation image OP obtained based on the emission of the first to third illumination lights in the first to third frames is displayed on the extended display 18.
  • the B1 image signal included in the first illumination light image, and the G2 and R2 image signals included in the second illumination light image are used.
  • the B3 and G3 image signals included in the third illumination light image are used to measure the concentration of a specific pigment (such as a yellow pigment) that affects the accuracy of oxygen saturation calculation.
  • the B1 image signal includes image information on at least the first blue light BL from the light that has passed through the B color filter BF in the first illumination light 56.
  • the B1 image signal (image signal for oxygen saturation) includes image information on the wavelength band B1, whose reflection spectrum changes with changes in the oxygen saturation of blood hemoglobin, as image information on the first blue light BL.
  • the wavelength band B1 for example, as shown in FIG. 13, it is preferable to set the wavelength band B1 to a wavelength band of 460 nm to 480 nm, including 470 nm, at which the difference between the reflection spectrum of oxygenated hemoglobin shown by curves 55b and 56b and the reflection spectrum of reduced hemoglobin shown by curves 55a and 56a is maximized.
  • curve 55a represents the reflectance spectrum of reduced hemoglobin when blood concentration is high
  • curve 55b represents the reflectance spectrum of oxygenated hemoglobin when blood concentration is high
  • curve 56a represents the reflectance spectrum of reduced hemoglobin when blood concentration is low
  • curve 56b represents the reflectance spectrum of oxygenated hemoglobin when blood concentration is low.
  • the G2 image signal contains image information of at least the wavelength band G2 relating to green light G from the light in the first illumination light 56 that has passed through the G color filter GF.
  • the wavelength band G2 is preferably a wavelength band of 500 nm to 580 nm, for example, as shown in FIG. 13.
  • the R2 image signal contains image information of at least the wavelength band R2 relating to red light R from the light in the first illumination light 56 that has passed through the R color filter RF.
  • the wavelength band R2 is preferably a wavelength band of 610 nm to 630 nm, for example, as shown in FIG. 13.
  • the image information of wavelength band B1 contains image information on the first blue light BL
  • the image information of wavelength band B3 contains image information on the green light G.
  • the image information on the first blue light BL and the green light G is image information in which the absorption spectrum of a specific pigment changes due to a change in the concentration of the specific pigment such as a yellow pigment.
  • the change in the absorption spectrum of the specific pigment also causes a change in the reflection spectrum of hemoglobin.
  • Curve 55a represents the reflection spectrum of reduced hemoglobin when there is no influence of the yellow pigment
  • curve 55c represents the reflection spectrum of reduced hemoglobin when there is an influence of the yellow pigment.
  • the reflection spectrum of reduced hemoglobin changes depending on the presence or absence of the yellow pigment (the same applies to the reflection spectrum of oxygenated hemoglobin). Therefore, the reflection spectrum of wavelength band B1 and wavelength band B3 changes due to a change in the oxygen saturation of blood hemoglobin under the influence of a specific pigment such as a yellow pigment.
  • the B1 image signal (denoted as “B1"), G2 image signal (denoted as “G2”), and R2 image signal (denoted as “R2”) are each affected by oxygen saturation dependency, blood concentration dependency, or brightness dependency.
  • the B1 image signal includes the wavelength band B1 in which the difference between the reflectance spectrum of oxygenated hemoglobin and the reflectance spectrum of reduced hemoglobin is maximized, and therefore has a "large" degree of oxygen saturation dependency that changes with oxygen saturation.
  • the B1 image signal has a "medium” degree of blood concentration dependency that changes with blood concentration. Furthermore, the B1 image signal has a "brightness dependency” that changes with the brightness of the object of observation.
  • the degrees of dependency are indicated as “large,” “medium,” and “small,” with “large” indicating a high degree of dependency compared to other image signals, “medium” indicating a medium degree of dependency compared to other image signals, and “small” indicating a low degree of dependency compared to other image signals.
  • the G2 image signal has a low dependency on oxygen saturation because the magnitude relationship between the reflectance spectrum of oxygenated hemoglobin and the reflectance spectrum of reduced hemoglobin is reversed over a wide wavelength band. Also, the G2 image signal has a high degree of dependency on blood concentration, as shown by curves 55a, 55b and curves 56a, 56b. Also, the G2 image signal has a dependency on brightness, similar to the B1 image signal.
  • the R2 image signal does not change as much with oxygen saturation as the B1 image signal, but it is moderately dependent on oxygen saturation. Also, the R2 image signal is slightly dependent on blood concentration, as shown by curves 55a, 55b and curves 56a, 56b. Also, the G2 image signal, like the B1 image signal, is dependent on brightness.
  • the G2 image signal is used as the normalized signal to create an oxygen saturation calculation table 73a (see FIG. 16) that is an oxygen saturation calculation table of data for calculating oxygen saturation using a signal ratio ln(B1/G2) obtained by normalizing the B1 image signal with the G2 image signal and a signal ratio ln(R2/G2) obtained by normalizing the R2 image signal with the G2 image signal.
  • ln in the signal ratio ln(B1/G2) is the natural logarithm (the same applies to the signal ratio ln(R2/G2)).
  • the oxygen saturation is represented by the contour line EL along the Y-axis direction.
  • the contour line ELH represents an oxygen saturation level of "100%”
  • the contour line ELL represents an oxygen saturation level of "0%”.
  • the contour lines are distributed so that the oxygen saturation level gradually decreases from the contour line ELH to the contour line ELL (in FIG. 16, the contour lines of "80%, " “60%, “ “40%, “ and “20%” are distributed).
  • the X-axis value (signal ratio ln(R2/G2)) and Y-axis value (signal ratio ln(B1/G2)) are affected by oxygen saturation dependency and blood concentration dependency, respectively.
  • the X-axis value and the Y-axis value are normalized by the G2 image signal, and therefore are considered to be "none,” meaning they are not affected.
  • the oxygen saturation dependency is considered to be about “medium,” and the blood concentration dependency is considered to be about “large.”
  • the oxygen saturation dependency is considered to be about "large,” and the blood concentration dependency is considered to be about "medium.”
  • the B1 image signal (denoted as "B1"), G2 image signal (denoted as “G2”), and R2 image signal (denoted as “R2”) are each affected by oxygen saturation dependency, blood concentration dependency, yellow dye dependency, or brightness dependency.
  • the B1 image signal contains image information in which the absorption spectrum of a specific dye such as a yellow dye changes depending on the concentration of the specific dye, and therefore has a "large" degree of yellow dye dependency, which changes depending on the yellow dye.
  • the G2 image signal is less affected by the yellow dye compared to the B1 image signal, and therefore has a "small to medium” degree of yellow dye dependency.
  • the R1 image signal is less affected by the yellow dye, and therefore has a "small” degree of yellow dye dependency.
  • the signal ratio ln(R2/G2) is represented on the X-axis and the signal ratio ln(B1/G2) is represented on the Y-axis in two-dimensional coordinates
  • the oxygen saturation StO2A when there is no yellow dye and the oxygen saturation StO2B when there is yellow dye are represented differently, as shown in Figure 19.
  • the oxygen saturation StO2B appears to be shifted higher than the oxygen saturation StO2A due to the presence of the yellow dye in the image signal.
  • the B3 image signal and G3 image signal included in the third illumination light image are used when correcting the oxygen saturation calculation table.
  • the B3 image signal includes image information related to the light transmitted through the B color filter BF in the third illumination light 58.
  • the B3 image signal (specific pigment image signal) includes image information of the wavelength band B3 that has sensitivity to specific pigments other than hemoglobin, such as yellow pigment (see FIG. 14). Although the B3 image signal is not as sensitive to the specific pigment as the B1 image signal, it has a certain degree of sensitivity to the specific pigment. Therefore, as shown in FIG.
  • the B1 image signal has a "large” dependency on yellow pigment
  • the B3 image signal has a “medium” dependency on yellow pigment.
  • the B3 image signal has a "small” dependency on oxygen saturation, a “large” dependency on blood concentration, and a “present” dependency on brightness.
  • the G3 image signal also does not have as much sensitivity to the specific dye as the B3 image signal, but does include image signals in the wavelength band G3 that have some degree of sensitivity to the specific dye (see Figure 14). Therefore, the yellow dye dependency of the G3 image signal is "small to medium”.
  • the G3 image signal has "small” oxygen saturation dependency, "large” blood concentration dependency, and "some” brightness dependency.
  • the B2 image signal also has "large” yellow dye dependency, so the B2 image signal may be used instead of the B3 image signal when correcting the oxygen saturation calculation table.
  • the B2 image signal has "small” oxygen saturation dependency, "large” blood concentration dependency, and "some” brightness dependency.
  • the curved surfaces CV0 to CV4 representing oxygen saturation are distributed in the Z-axis direction according to the pigment concentration of the yellow pigment.
  • the curved surface CV0 represents the oxygen saturation when the yellow pigment has a concentration of "0" (no effect of the yellow pigment).
  • the curved surfaces CV1 to CV4 represent the oxygen saturation when the yellow pigment has concentrations of "1" to "4", respectively.
  • the higher the concentration number the higher the concentration of the yellow pigment. Note that, as shown in the curved surfaces CV0 to CV4, the higher the concentration of the yellow pigment, the lower the Z-axis value changes.
  • the areas AR0 to AR4 representing the oxygen saturation state are distributed at different positions according to the concentration of the yellow dye, respectively.
  • Areas AR0 to AR4 represent the distribution of oxygen saturation when the concentration of the yellow dye is "0" to "4", respectively.
  • a contour line EL representing oxygen saturation for each of these areas AR0 to AR4 it is possible to obtain the oxygen saturation corresponding to the concentration of the yellow dye (see Figure 16). Note that, as shown in areas AR0 to AR4, the higher the concentration of the yellow dye, the higher the value on the X axis and the lower the value on the Y axis.
  • the X-axis value (signal ratio ln(R2/G2)), Y-axis value (signal ratio ln(B1/G2)), and Z-axis value (signal ratio ln(B3/G3)) are subject to yellow pigment dependency.
  • the X-axis value (denoted as "X") has a yellow pigment dependency of "small to medium”
  • the Y-axis value (denoted as "Y") has a yellow pigment dependency of "large”
  • the Z-axis value (denoted as "Z”) has a yellow pigment dependency of "medium”.
  • the Z-axis value has an oxygen saturation dependency of "small to medium” and a blood concentration dependency of "small to medium”.
  • the Z-axis value has no brightness dependency because it is normalized by the G3 image signal.
  • correcting the data used to calculate oxygen saturation in the correction mode specifically means selecting one of the curved surfaces CV0 to CV4 (see FIG. 21) that represent oxygen saturation.
  • the extended processor device 17 includes an image acquisition unit 60a and an image processing unit 60b.
  • the image acquisition unit 60a receives an image signal transmitted from the processor device 14 via the image communication unit 51.
  • the image processing unit 60b performs various image processing operations to generate an oxygen saturation image, etc.
  • the image processing unit 60b includes an oxygen saturation image generating unit 61, a specific dye concentration calculating unit 62, a table correction unit 63, and a display control unit 64.
  • programs related to various processes are incorporated in a program memory (not shown).
  • the functions of the oxygen saturation image generating unit 61, the specific dye concentration calculating unit 62, the table correction unit 63, and the display control unit 64 are realized by a central control unit (not shown) formed by a processor executing the programs in the program memory.
  • the oxygen saturation image generating unit 61 includes a base image generating unit 70, an operation value calculating unit 71, an oxygen saturation calculating unit 72, an oxygen saturation calculation table 73, and a color adjusting unit 74.
  • the base image generating unit 70 generates a base image based on an image signal from the processor device 14.
  • the base image is used as the base of the oxygen saturation image OP. It is preferable that the base image is an image that can grasp morphological information such as the shape of the observed object.
  • the base image is composed of a B2 image signal, a G2 image signal, and an R2 image signal.
  • the base image may be a narrowband light image in which blood vessels or structures (ductal structures) are highlighted by narrowband light or the like.
  • the oxygen saturation calculation unit 72 refers to the oxygen saturation calculation table 73 and calculates the oxygen saturation based on the calculated value.
  • the oxygen saturation calculation table 73 stores the correlation between the signal ratios B1/G2 and R2/G2, which are one of the calculated values, and the oxygen saturation.
  • the correlation is expressed in two-dimensional coordinates with the signal ratio ln(B1/G2) on the vertical axis and the signal ratio ln(R2/G2) on the horizontal axis
  • the state of oxygen saturation is expressed by the contour line EL extending in the horizontal axis direction, and when the oxygen saturation differs, the contour line EL is distributed at different positions in the vertical axis direction (oxygen saturation calculation table 73a (see FIG. 16)).
  • the oxygen saturation calculation table 73 includes the oxygen saturation calculation table 73a expressed in two-dimensional coordinates.
  • the oxygen saturation calculation unit 72 refers to the oxygen saturation calculation table 73 and calculates the oxygen saturation corresponding to the signal ratios B1/G2 and R2/G2 for each pixel.
  • the oxygen saturation calculation table 73a refers to the oxygen saturation calculation table 73a, and when the signal ratios of a specific pixel are ln(B1 * /G2 * ) and ln(R2 * /G2 * ), the oxygen saturation corresponding to the signal ratios ln(B1 * /G2 * ) and ln(R2 * /G2 * ) is "40%". Therefore, the oxygen saturation calculation unit 72 calculates the oxygen saturation of the specific pixel to be "40%".
  • the color tone adjustment unit 74 generates an oxygen saturation image by performing composite color processing that changes the color tone of the base image using the oxygen saturation calculated by the oxygen saturation calculation unit 72.
  • the color tone adjustment unit 74 maintains the color tone of areas in the base image where the oxygen saturation exceeds a threshold, and changes the color tone of areas where the oxygen saturation is below the threshold to a color tone that changes depending on the oxygen saturation. This maintains the color tone of normal areas where the oxygen saturation exceeds the threshold, while only changing the color tone of abnormal areas where the oxygen saturation is below the threshold and becomes low, making it possible to grasp the oxygen status of abnormal areas under conditions where morphological information of normal areas can be observed.
  • the color tone adjustment unit 74 may generate an oxygen saturation image by pseudo-color processing in which a color is assigned according to the oxygen saturation level, regardless of the level of oxygen saturation.
  • pseudo-color processing the base image is not necessary.
  • the specific dye concentration calculation unit 62 includes a specific dye concentration calculation table 75.
  • the specific dye concentration calculation unit 62 calculates the specific dye concentration based on a specific dye image signal including image information of a wavelength band that has sensitivity to a specific dye other than hemoglobin in blood among the dyes included in the observation subject. Examples of specific dyes include yellow dyes such as bilirubin. It is preferable that the specific dye image signal includes at least a B3 image signal.
  • the specific dye concentration calculation unit 62 calculates the signal ratios ln(B1/G2), ln(G2/R2), and ln(B3/G3).
  • the specific dye concentration calculation unit 62 then refers to the specific dye concentration calculation table 75 to calculate the specific dye concentrations corresponding to the signal ratios ln(B1/G2), ln(G2/R2), and ln(B3/G3).
  • the specific dye concentration calculation table 75 stores the correlation between the signal ratios ln(B1/G2), ln(G2/R2), and ln(B3/G3) and the specific dye concentrations. For example, if the ranges of the signal ratios ln(B1/G2), ln(G2/R2), and ln(B3/G3) are divided into five stages, the specific dye concentrations "0" to "4" are associated with the signal ratios ln(B1/G2), ln(G2/R2), and ln(B3/G3) in the five stages, and are stored in the specific dye concentration calculation table 75. Note that it is preferable to logarithmize (ln) the signal ratio B3/G3.
  • the table correction unit 63 performs table correction processing to correct the oxygen saturation calculation table 73 based on the specific dye concentration as a correction processing performed in the correction mode.
  • the table correction processing the correlation between the signal ratios B1/G2 and R2/G2 stored in the oxygen saturation calculation table 73 and the oxygen saturation is corrected.
  • the table correction unit 63 when the specific dye concentration is "2", the table correction unit 63 generates a contour line EL representing the state of oxygen saturation in the area AR2 corresponding to the specific dye concentration of "2" among the areas AR0 to AR4 determined according to the specific dye concentration, as shown in FIG. 27.
  • the table correction unit 63 generates a corrected oxygen saturation calculation table 73b by correcting the oxygen saturation calculation table 73 so that it becomes the generated contour line EL.
  • the table correction unit 63 includes a recognition unit 81, a region setting unit 82, a reliability calculation unit 83, a comparison processing unit 84, a correction feasibility determination unit 85, a high reliability region of interest display unit 86, a region selection unit 87, and a correction unit 88.
  • the recognition unit 81 recognizes multiple parts that appear in the acquired first illumination light image or second illumination light image. Note that multiple parts refer to multiple types of parts, and multiple types of parts may be areas that appear to be different parts of the endoscopic image due to different colors, textures, etc., and do not necessarily have to be anatomically different parts. The recognition unit 81 recognizes multiple types of parts as different parts.
  • the recognition method may use known technology, and for example, the method described in WO 2021/149552 may be adopted. That is, it is preferable to recognize the part or tissue from the endoscopic image using a trained model such as a neural network (a model trained using an image set consisting of images of a living body). For example, it is preferable to use a CNN (Convolutional Neural Network) as the neural network to perform multi-class classification (each class corresponds to a different part or tissue) and recognize the part or tissue. Note that the CNN may include a pooling layer in the intermediate layer.
  • a trained model such as a neural network (a model trained using an image set consisting of images of a living body).
  • a CNN Convolutional Neural Network
  • the CNN may include a pooling layer in the intermediate layer.
  • JP 2011-218090 A As another recognition method, for example, the method described in JP 2011-218090 A can be adopted. That is, in order to more accurately recognize multiple parts of the object of observation that appear in the first illumination light image obtained by the image acquisition unit 60a, it is preferable to use a second image in which the illumination light is different from that of the first image, and to use high-confidence areas and low-confidence areas in the first image and second image that are set for determining the type of subject.
  • the region setting unit 82 sets a region of interest for each recognized part. As shown in FIG. 30, a region of interest is set for the transverse colon 91 and peritoneum 92 recognized by the recognition unit 81, a first region of interest 91a is set for the recognized transverse colon 91, and a second region of interest 92a is set for the recognized peritoneum 92. Each region of interest is set so that the recognized transverse colon 91 and peritoneum 92 do not overlap. Therefore, the region of interest is set so that one transverse colon 91 or peritoneum 92 is not included in multiple regions of interest. Note that the first region of interest 91a and the second region of interest 92a are actually set in the same manner as the regions shown by different shading in FIG.
  • the figure may not show all of the first region of interest 91a or the second region of interest 92a.
  • parts for which a region of interest is set include the large intestine, small intestine, liver, stomach, etc.
  • the large intestine may also be further divided into the rectum, sigmoid colon, descending colon, transverse colon, ascending colon, cecum, etc.
  • the set region of interest is not displayed on the extended display 18 at this stage.
  • the reliability calculation unit 83 calculates a region of interest reliability Rc for each of the first region of interest 91a and the second region of interest 92a based on the pixels contained in the first region of interest 91a or the second region of interest 92a.
  • the region of interest reliability Rc is the reliability in the region of interest. Note that reliability is an index related to the ability to correct the oxygen saturation calculation table with higher accuracy. The higher the reliability, the more accurately the oxygen saturation calculation table can be corrected, while the lower the reliability, the more likely it is that a problem will occur in the accuracy of the correction of the oxygen saturation calculation table.
  • the reliability calculation unit 83 calculates a region of interest reliability Rc for the region of interest and a pixel reliability Pc for the pixels, as described below.
  • the reliability calculation unit 83 first calculates pixel reliability Pc in order to calculate the region of interest reliability Rc.
  • the pixel reliability Pc is a reliability calculated based on the pixels included in the region of interest.
  • the pixel reliability Pc is calculated for each of the first region of interest 91a and the second region of interest 92a using the pixels included in each region of interest and factors that affect the correction of the oxygen saturation calculation table. The factors that affect the correction of the oxygen saturation calculation table will be described later.
  • the region of interest reliability Rc is calculated based on the pixel reliability Pc and the number of pixels included in each of the first region of interest 91a and the second region of interest 92a.
  • the pixel reliability Pc may also be used for the image to be displayed on the extended display 18 based on conditions set by the user.
  • the reliability calculation unit 83 calculates at least one pixel reliability Pc that affects the correction of the oxygen saturation calculation table based on the B1 image signal, G1 image signal, and R1 image signal included in the first illumination light image, or the B2 image signal, G2 image signal, and R2 image signal included in the second illumination light image.
  • Factors that affect the correction of the oxygen saturation calculation table include, for example, disturbances and pixel values including bleeding, fat, residue, mucus, or residual liquid. Therefore, it is preferable to calculate the pixel reliability Pc using these factors.
  • the pixel reliability Pc is expressed, for example, as a decimal between 0 and 1.
  • the reliability calculation unit 83 calculates multiple types of pixel reliability Pc, it is preferable to adopt the minimum pixel reliability Pc among the multiple types of pixel reliability Pc as the pixel reliability Pc of each pixel.
  • the pixel value of the G2 image signal can be used for the pixel value that affects the correction accuracy of the oxygen saturation calculation table.
  • the pixel reliability Pc of the pixel value of the G2 image signal outside the certain range Rx on the definition line 93 is lower than the pixel reliability Pc of the pixel value of the G2 image signal within the certain range Rx.
  • the definition line 93 shows the relationship between the pixel value of the G2 image signal and the pixel reliability Pc related to the correction accuracy of the oxygen saturation calculation table, and is preset from past data. Outside the certain range Rx includes high pixel values such as halation, as well as extremely small pixel values such as dark areas. Outside the certain range Rx, the correction accuracy of the oxygen saturation calculation table is low, and the pixel reliability Pc is accordingly low.
  • the G1 image signal may be used instead of the G2 image signal to calculate the pixel reliability Pc.
  • the reliability according to the degree of bleeding is set to a fixed value that is a high reliability.
  • ln represents the natural logarithm.
  • B2/G2 represents the signal ratio between the B2 image signal and the G2 image signal
  • R2/G2 represents the signal ratio between the R2 image signal and the G2 image signal.
  • pixel reliability Pc is determined according to the distance from the defined line DFY on a two-dimensional plane consisting of the vertical axis ln (B1/G1) and the horizontal axis ln (R1/G1).
  • the pixel reliability Pc decreases as the coordinate plotted on the two-dimensional plane based on the B1, G1, and R1 image signals is located further to the lower left.
  • the reliability based on the degree of fat is set to a fixed value that is high reliability.
  • B1/G1 represents the signal ratio between the B1 and G1 image signals
  • R1/G1 represents the signal ratio between the R1 and G1 image signals.
  • the reliability calculation unit 83 calculates the region of interest reliability Rc.
  • the region of interest reliability Rc may be calculated based on the pixel reliability Pc calculated for each pixel included in each of the first region of interest 91a and the second region of interest 92a set by the region setting unit 82, and the number of pixels N included in the region of interest. That is, in each region of interest, as shown in the following formula (1), the pixel reliability Pc calculated for each of all pixels included in the region of interest is summed up, and the average pixel reliability Pc is calculated using the number of pixels N included in the region of interest, and this is set as the region of interest reliability Rc in this region of interest.
  • the region of interest reliability Rc may be calculated by a calculation method other than a simple average using the pixel reliability Pc of each pixel included in the region of interest. For example, it may be calculated by an average such as a weighted average or a trimmed average, or by other statistics such as a median or a mode.
  • Region of interest reliability Rc ⁇ (pixel reliability Pc of pixels in region of interest) / number of pixels in region of interest N (1)
  • the comparison processing unit 84 compares the region of interest reliability Rc calculated by the reliability calculation unit 83 with a preset region of interest threshold Rth for each of the first region of interest 91a and the second region of interest 92a set by the region setting unit 82.
  • the region of interest threshold Rth is a value for evaluating the region of interest reliability Rc, and can be set to a specific value in advance.
  • the region of interest threshold Rth is set in advance depending on the observation target, the type of reliability, etc.
  • the region of interest reliability Rc of the first region of interest 91a is calculated to be 0.8
  • the region of interest reliability Rc of the second region of interest 92a is calculated to be 0.2
  • the region of interest threshold Rth is set to be 0.5
  • a comparison result 93a is obtained in which the region of interest reliability Rc is equal to or greater than the region of interest threshold Rth
  • a comparison result 94b is obtained in which the region of interest reliability Rc is smaller than the region of interest threshold Rth.
  • the correction feasibility determination unit 85 determines whether each of the first region of interest 91a and the second region of interest 92a is a high reliability region of interest HRc for which correction of the oxygen saturation calculation table 73 used to calculate oxygen saturation is possible, based on the comparison result obtained by the comparison processing unit 84.
  • the first region of interest 91a whose region of interest reliability Rc is equal to or greater than the region of interest threshold value Rth is set as a high reliability region of interest HRc. It is also preferable to set the second region of interest 92a whose region of interest reliability Rc is smaller than the region of interest threshold value Rth as a low reliability region of interest LRc.
  • the high-reliability region of interest display unit 86 displays the high-reliability region of interest HRc on the extended display 18 so that the user can visually recognize the high-reliability region of interest HRc.
  • it is sufficient to display it on the extended display 18 in a manner that allows the user to recognize the high-reliability region of interest HRc in the correction image 90.
  • the high-reliability region of interest HRc may be displayed in a color that allows it to be distinguished from other regions, or displayed with a boundary line that shows the outline of the high-reliability region of interest HRc.
  • a boundary line 95 is displayed on the extended display 18 so that the user can visually recognize the high-reliability region of interest HRc set by the correction feasibility determination unit 85.
  • the boundary line indicating the second region of interest 92a is not displayed on the extended display 18.
  • the region selection unit 87 accepts the user's selection of a high-reliability region of interest HRc. After checking the high-reliability region of interest HRc displayed by the boundary line 95a on the extended display 18, the user can select a high-reliability region of interest HRc for performing correction processing. This allows the user to select a high-reliability region of interest HRc to be used for correction processing according to the site or tissue for which the user wishes to know the oxygen saturation with greater accuracy. Note that, if there are multiple high-reliability regions of interest HRc, the region selection unit 87 may accept the user's selection of multiple high-reliability regions of interest HRc.
  • the user can select the high reliability region of interest HRc, for example, by using the processor-side interface 16, the zoom switch 12d, etc.
  • the correction unit 88 corrects the oxygen saturation calculation table 73 used to calculate oxygen saturation based on the high reliability region of interest HRc.
  • the mode is automatically switched to the oxygen saturation mode.
  • the correction unit 88 performs a correction process, which is a correction of the oxygen saturation calculation table 73, in response to a user instruction.
  • the correction of the oxygen saturation calculation table 73 is performed using a high reliability region of interest HRc selected by the user in the correction image 90.
  • image information of the high reliability region of interest HRc can be used, and image information for each pixel included in the high reliability region of interest HRc can be used.
  • the correction of the oxygen saturation calculation table 73 is performed by selecting one of the curved surfaces CV0 to CV4 (see FIG. 21) that represent the oxygen saturation. Therefore, the correction unit 88 uses the concentration of the yellow pigment in the high-reliability region of interest HRc selected by the user as the image information of this high-reliability region of interest HRc, and performs correction of the oxygen saturation calculation table 73 by selecting one of the curved surfaces CV0 to CV4 (see FIG. 21) that represent the oxygen saturation, as described above.
  • the oxygen saturation calculation unit 72 calculates the oxygen saturation of the observation subject by using the oxygen saturation calculation table 73 that is not corrected. This makes it possible to calculate a more accurate oxygen saturation without undesirable corrections being made to the oxygen saturation calculation table 73.
  • a series of processing steps in the correction mode by the endoscope system 10 will be described with reference to the flowchart of FIG. 37.
  • the user operates the mode switching switch 12c to switch to the correction mode (step ST100).
  • the correction image 90 is displayed on the extended display 18 (step ST110).
  • multiple parts are recognized (step ST120), and a region of interest 82a is set based on the multiple recognized parts (step ST130).
  • a region of interest reliability Rc is calculated for each set region of interest (step ST140), and is compared with a region of interest threshold Rth for each region of interest (step ST150).
  • the region of interest reliability Rc is equal to or greater than the region of interest threshold Rth.
  • the high reliability region of interest HRc is used to perform a correction process for the oxygen saturation calculation table 73 related to the oxygen saturation calculation (step ST180).
  • the mode is switched to oxygen saturation mode (step ST190). In the oxygen saturation mode, the oxygen saturation for the endoscopic image is calculated using the corrected oxygen saturation calculation table 73.
  • the endoscope system 10 can use the high reliability region of interest HRc in the correction process related to oxygen saturation calculation, so that highly accurate correction can be performed when multiple parts are present in the endoscope image. Furthermore, the endoscope system 10 can perform highly accurate correction related to oxygen saturation calculation through the correction process, so that more accurate oxygen saturation can be obtained regardless of the endoscope image.
  • the reliability calculation unit 83 may modify the pixel reliability Pc according to the distance from the center of the correction image 90 of the pixel to be calculated.
  • the modified pixel reliability Pc is set as the modified pixel reliability.
  • the reliability calculation unit 83 modifies the pixel reliability Pc, it calculates the region of interest reliability Pc using the modified pixel reliability.
  • the distance can be calculated as the square root of the sum of the squares of the differences between the x-axis value and the y-axis value between the coordinates of the center of the correction image 90 and the coordinates of each pixel to be calculated, where x is the horizontal axis and y is the vertical axis of the correction image 90.
  • the pixels for which pixel reliability Pc is calculated can be divided into two categories, for example "near distance” and "far distance”, depending on the distance from the center of the correction image 90, and the pixel reliability Pc can be corrected by using different correction data for each category.
  • the correction data is set in advance by examining the relationship between the pixel reliability Pc of multiple pixels at different positions in the correction image 90 and the oxygen saturation. At this time, the classification criteria for "near distance" pixels and "far distance” pixels are also set at the same time as creating the correction data.
  • the correction data indicates the relationship between the original pixel reliability, which is the pixel reliability Pc before correction, and the corrected pixel reliability.
  • definition line 96a which is close-distance correction data
  • definition line 96b which is long-distance correction data
  • the reliability calculation unit 83 may perform weighting according to the area of the region of interest and/or the distance of the region of interest from the center of the correction image 90. Such weighting can be performed using data that associates the area of the region of interest or the distance of the region of interest from the center of the correction image 90 with a weight.
  • the area weight Aw for the area of the region of interest may be calculated according to a definition line 97, which is data that associates the area of the region of interest with the area weight Aw.
  • the definition line 97 is set in advance by examining the relationship between the area of the region of interest and the oxygen saturation level.
  • the distance weight Dw for the distance of the region of interest from the center of the correction image 90 may be calculated according to a definition line 98.
  • the definition line 98 is set in advance by examining the relationship between the region of interest distance and oxygen saturation. Note that, when the horizontal axis of the correction image 90 is x and the vertical axis is y, the region of interest distance can be calculated as the square root of the sum of the squares of the differences between the x-axis value and the y-axis value between the coordinates of the center of the correction image 90 and the coordinates of the center or center of gravity of the region of interest.
  • the weighting according to the area of the region of interest and/or the distance of the region of interest from the center of the correction image 90 may be done only according to the area of the region of interest, or only according to the distance of the region of interest from the center of the correction image 90, or both.
  • the weighted region of interest reliability Rc is calculated according to the following formula (2).
  • the region of interest reliability Rc By weighting the region of interest reliability Rc, the area of the region of interest or the distance of the region of interest from the center of the correction image 90, which are factors that affect the calculation of the region of interest reliability Rc, are reflected. This makes it possible to calculate the region of interest reliability Rc more preferably, thereby improving the accuracy of the ultimately obtained oxygen saturation.
  • the high-reliability region of interest display unit 86 may change the display mode of the high-reliability region of interest HRc displayed on the extended display 18 in response to a user instruction. As shown in FIG. 41, the boundary line 95 that is displayed on the extended display 18 and indicates the high-reliability region of interest HRc may be erased, and instead, a region of interest display color 99 representing the high-reliability region of interest HRc may be displayed on the extended display 18, thereby allowing the user to recognize the high-reliability region of interest HRc.
  • the high reliability region of interest display unit 86 may display a name representing a region shown in the high reliability region of interest HRc on the extended display 18, superimposed on the high reliability region of interest HRc, in response to a user instruction.
  • a region name 100 may be displayed on the extended display 18, superimposed on the high reliability region of interest HRc.
  • "transverse colon” is displayed as the region name 100.
  • the region name may be set to, for example, large intestine, small intestine, liver, stomach, etc.
  • the large intestine may be further divided into the rectum, sigmoid colon, descending colon, transverse colon, ascending colon, cecum, etc.
  • the effect of further improving the visibility of each of the multiple parts is achieved by superimposing multiple names representing the multiple parts on the high-reliability regions of interest HRc.
  • the target part is known in advance, for example, by displaying only the high-reliability region of interest HRc of the previously set target part on the extended display 18, it becomes easy to determine whether the target part is set as a high-reliability region of interest HRc.
  • the region selection unit 87 may accept a selection of multiple high-reliability regions of interest HRc by the user and correct data based on the selected multiple high-reliability regions of interest HRc.
  • the correction unit 88 performs correction based on the selected first high-reliability region of interest HRc1 and the second high-reliability region of interest HRc2.
  • the correction unit 88 can perform correction processing on the region of interest that includes the selected first high-reliability region of interest HRc1 and the second high-reliability region of interest HRc2.
  • correction processing can be performed selectively using only the multiple desired regions without changing the field of view.
  • the high-reliability region of interest display unit 86 may include a determination notification unit 101, as shown in FIG. 44.
  • the determination notification unit 101 performs control to display operation guidance or the like on the display 18 to notify the user so that the region of interest is determined to be a high-reliability region of interest HRc.
  • the notification may be by voice or the like.
  • the user when the user selects the low-reliability region of interest LRc determined by the correction feasibility determination unit 85 using the processor-side interface 16 or the zoom operation switch 12d, the user may be notified by displaying operation guidance GD1 on the extended display 18 to change the low-reliability region of interest HRc selected by the user to a high-reliability region of interest HRc, as shown in FIG. 45(B).
  • the operation guidance GD1 of "Avoid dark areas" is displayed on the extended display 18, the user is notified of the operation guidance GD1 and operates the endoscope 12 to make the observation target appear brighter. This changes the low-reliability region of interest LRc to a high-reliability region of interest HRc, and correction processing, etc. can be carried out.
  • the determination notification unit 101 may perform control to notify the user of this fact.
  • the notification may be operation guidance to reacquire the correction image 90 so that the target area becomes the high-reliability region of interest HRc.
  • the operation guidance GD2 of "Avoid bleeding, residue, fat, etc.” is displayed on the extended display 18, the user can understand the content of the operation guidance GD2 by being notified of this operation guidance GD2 and operate the endoscope 12 to re-acquire the correction image 90 with fewer factors that affect the correction.
  • the user is notified of this fact, and when operating the endoscope 12 to reacquire the correction image 90, the user can reduce the burden of reacquiring the correction image 90 by following the instructions in the operation guidance without trial and error.
  • the high-reliability region of interest display unit 86 may superimpose and display on the extended display 18 a region including pixels in the high-reliability region of interest HRc whose pixel reliability Pc is equal to or greater than a preset pixel reliability threshold Pth.
  • the hardware structure of the processing units that perform various processes are various processors as shown below.
  • the various processors include a CPU (Central Processing Unit), which is a general-purpose processor that executes software (programs) and functions as various processing units, a GPU (Graphical Processing Unit), a Programmable Logic Device (PLD), which is a processor whose circuit configuration can be changed after manufacture such as an FPGA (Field Programmable Gate Array), and a dedicated electrical circuit, which is a processor with a circuit configuration designed specifically to perform various processes.
  • a CPU Central Processing Unit
  • GPU Graphic Processing Unit
  • PLD Programmable Logic Device
  • FPGA Field Programmable Gate Array
  • dedicated electrical circuit which is a processor with a circuit configuration designed specifically to perform various processes.
  • a single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, multiple FPGAs, a combination of a CPU and an FPGA, or a combination of a CPU and a GPU, etc.). Multiple processing units may also be configured with one processor.
  • multiple processing units may also be configured with one processor.
  • first there is a form in which one processor is configured with a combination of one or more CPUs and software, as represented by computers such as clients and servers, and this processor functions as multiple processing units.
  • a processor is used that realizes the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip, as represented by System On Chip (SoC).
  • SoC System On Chip
  • the hardware structure of these various processors is an electric circuit (circuitry) that combines circuit elements such as semiconductor elements.
  • the hardware structure of the memory unit is a storage device such as a hard disc drive (HDD) or solid state drive (SSD).
  • An endoscope system that calculates an oxygen saturation level of an observation object using data used for calculating an oxygen saturation level, A processor is provided.
  • the processor Acquire an image of the object to be observed, Recognizing a plurality of parts appearing in the acquired image, A region of interest is set for each of the recognized parts in the image; calculating a region of interest confidence for each of the regions of interest based on pixels contained in the region of interest; performing a comparison process of comparing the region of interest reliability with a preset region of interest reliability threshold; performing a determination process for determining whether each of the regions of interest is a high-reliability region of interest for which the data can be corrected based on a comparison result obtained by the comparison process; displaying the high confidence region of interest on a display; An endoscope system that corrects the data based on the high confidence region of interest selected by a user.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biomedical Technology (AREA)
  • Optics & Photonics (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biophysics (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Endoscopes (AREA)

Abstract

L'invention concerne un système d'endoscope, son procédé de fonctionnement, et un programme pour un système d'endoscope avec lesquels il est possible, lorsqu'une pluralité de parties existent dans une image lors de la correction d'une table de calcul de saturation en oxygène par une opération corrective effectuée par un utilisateur, d'effectuer la correction d'une manière très précise.　Dans ce système d'endoscope dans lequel la saturation en oxygène est calculée : une image obtenue par imagerie d'une cible d'observation est acquise ; une pluralité de parties sont reconnues à partir de l'image ; une région d'intérêt est définie dans chaque partie ; une fiabilité est calculée pour chaque région d'intérêt ; une comparaison est effectuée entre la fiabilité et un seuil prédéfini ; un processus de détermination, consistant à déterminer si la région d'intérêt est ou non une région d'intérêt à haute fiabilité dans laquelle une correction de données est possible, est effectué ; et la correction de données est effectuée à l'aide de la région d'intérêt à haute fiabilité pour laquelle il a été déterminé que la correction de données est possible.
PCT/JP2024/034153 2023-09-28 2024-09-25 Système d'endoscope, son procédé de fonctionnement et programme pour faire fonctionner un système d'endoscope Pending WO2025070479A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
JP2023168895 2023-09-28
JP2023-168895 2023-09-28

Publications (1)

Publication Number Publication Date
WO2025070479A1 true WO2025070479A1 (fr) 2025-04-03

Family

ID=95203727

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2024/034153 Pending WO2025070479A1 (fr) 2023-09-28 2024-09-25 Système d'endoscope, son procédé de fonctionnement et programme pour faire fonctionner un système d'endoscope

Country Status (1)

Country Link
WO (1) WO2025070479A1 (fr)

Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
WO2021210331A1 (fr) * 2020-04-17 2021-10-21 富士フイルム株式会社 Dispositif de traitement d'image et son procédé de fonctionnement
JP2023026480A (ja) * 2019-02-19 2023-02-24 富士フイルム株式会社 医療画像処理装置、内視鏡システム、及び医療画像処理装置の作動方法
WO2023132138A1 (fr) * 2022-01-07 2023-07-13 富士フイルム株式会社 Dispositif de processeur, son procédé de fonctionnement et système d'endoscope
WO2023132188A1 (fr) * 2022-01-05 2023-07-13 富士フイルム株式会社 Système d'endoscopie et son procédé de fonctionnement

Patent Citations (4)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2023026480A (ja) * 2019-02-19 2023-02-24 富士フイルム株式会社 医療画像処理装置、内視鏡システム、及び医療画像処理装置の作動方法
WO2021210331A1 (fr) * 2020-04-17 2021-10-21 富士フイルム株式会社 Dispositif de traitement d'image et son procédé de fonctionnement
WO2023132188A1 (fr) * 2022-01-05 2023-07-13 富士フイルム株式会社 Système d'endoscopie et son procédé de fonctionnement
WO2023132138A1 (fr) * 2022-01-07 2023-07-13 富士フイルム株式会社 Dispositif de processeur, son procédé de fonctionnement et système d'endoscope

Similar Documents

Publication Publication Date Title
US20240389829A1 (en) Endoscope system
US11179024B2 (en) Endoscope system capable of correcting image, processor device, and method for operating endoscope system
US12329351B2 (en) Endoscope system and method of operating endoscope system
JP7021183B2 (ja) 内視鏡システム、プロセッサ装置、及び、内視鏡システムの作動方法
US11185214B2 (en) Endoscope system with image data correction, processor device, and method for operating endoscope system
US10264955B2 (en) Processor device and method for operating same, and endoscopic system and method for operating same
US20180317754A1 (en) Endoscopic system and endoscopic system operating method
JP7692961B2 (ja) 内視鏡システム及びその作動方法
JP2017086303A (ja) プロセッサ装置、内視鏡システム、及び画像処理方法
US20240358245A1 (en) Processor device, method for operating the same, and endoscope system
US20240358244A1 (en) Endoscope system and method of operating the same
US11363943B2 (en) Endoscope system and operating method thereof
WO2025070479A1 (fr) Système d'endoscope, son procédé de fonctionnement et programme pour faire fonctionner un système d'endoscope
US20240335092A1 (en) Endoscope system and method for operating the same
JP2025058760A (ja) 内視鏡システム、画像処理装置、内視鏡システムの作動方法、及び内視鏡システム用プログラム
US20240341641A1 (en) Endoscope system and method for operating the same
US20250169706A1 (en) Endoscope system, operation method for endoscope system, and non-transitory computer readable medium
US20250176876A1 (en) Endoscope system and method for operating the same
WO2020121868A1 (fr) Système d'endoscope
JP6999597B2 (ja) プロセッサ装置及びその作動方法並びに内視鏡システム
JP2025135460A (ja) 内視鏡システム、生体パラメータ画像の生成方法及びプログラム
JP2024052487A (ja) プロセッサ装置及び内視鏡システム並びにその作動方法
WO2025187496A1 (fr) Système d'endoscope et son procédé de fonctionnement
JP2020178738A (ja) プロセッサ装置及び内視鏡システム並びにキャリブレーション方法

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24872265

Country of ref document: EP

Kind code of ref document: A1