WO2017057680A1 - 内視鏡システム及び評価値計算装置 - Google Patents
内視鏡システム及び評価値計算装置 Download PDFInfo
- Publication number
- WO2017057680A1 WO2017057680A1 PCT/JP2016/079024 JP2016079024W WO2017057680A1 WO 2017057680 A1 WO2017057680 A1 WO 2017057680A1 JP 2016079024 W JP2016079024 W JP 2016079024W WO 2017057680 A1 WO2017057680 A1 WO 2017057680A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- axis
- color
- pixel
- image
- severity
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
- A61B1/0661—Endoscope light sources
- A61B1/0684—Endoscope light sources using light emitting diodes [LED]
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00043—Operational features of endoscopes provided with output arrangements
- A61B1/00045—Display arrangement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
- A61B1/045—Control thereof
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
- A61B1/05—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances characterised by the image sensor, e.g. camera, being in the distal end portion
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
- A61B1/0646—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements with illumination filters
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
- A61B1/0655—Control therefor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
- A61B1/0661—Endoscope light sources
- A61B1/0669—Endoscope light sources at proximal end of an endoscope
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
- A61B1/07—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements using light-conductive means, e.g. optical fibres
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B23/00—Telescopes, e.g. binoculars; Periscopes; Instruments for viewing the inside of hollow bodies; Viewfinders; Optical aiming or sighting devices
- G02B23/24—Instruments or systems for viewing the inside of hollow bodies, e.g. fibrescopes
- G02B23/2407—Optical details
- G02B23/2461—Illumination
- G02B23/2469—Illumination using optical fibres
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/0002—Inspection of images, e.g. flaw detection
- G06T7/0012—Biomedical image inspection
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T7/00—Image analysis
- G06T7/80—Analysis of captured images to determine intrinsic or extrinsic camera parameters, i.e. camera calibration
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00163—Optical arrangements
- A61B1/00186—Optical arrangements with imaging filters
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10024—Color image
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/10—Image acquisition modality
- G06T2207/10068—Endoscopic image
-
- G—PHYSICS
- G06—COMPUTING OR CALCULATING; COUNTING
- G06T—IMAGE DATA PROCESSING OR GENERATION, IN GENERAL
- G06T2207/00—Indexing scheme for image analysis or image enhancement
- G06T2207/30—Subject of image; Context of image processing
- G06T2207/30004—Biomedical image processing
- G06T2207/30096—Tumor; Lesion
Definitions
- the present invention relates to an endoscope system and an evaluation value calculation device for calculating a predetermined evaluation value.
- the lesion is generally a color different from that of normal mucosal tissue.
- color endoscope apparatuses it is possible for an operator to grasp and diagnose a lesion part slightly different in color from a normal tissue.
- long-term training is performed under the guidance of an expert. It is necessary to receive.
- even a skilled operator cannot easily diagnose and diagnose a lesion from a slight color difference, and requires careful work.
- Patent Document 1 Japanese Patent Application Laid-Open No. 2014-18332
- Patent Document 1 describes an apparatus for scoring a lesion part appearing in a photographed image in order to assist an operator in diagnosing the lesion part. ing.
- the device described in Patent Document 1 performs a tone enhancement process that gives a non-linear gain to a pixel value for each pixel constituting a captured image by an endoscope, and a pixel value that is determined as a lesioned part.
- the tone-enhanced pixel data in the RGB space defined by the three primary colors of RGB is converted into a predetermined color space such as the HSI color space or HSV color space, and the hue and saturation are converted.
- Information is acquired, and it is determined whether or not the pixel is a lesioned part based on the acquired hue and saturation information, and an evaluation value (lesion index) is calculated based on the determined number of pixels.
- tone emphasis processing and color space conversion processing are heavy, and a large amount of hardware resources are required to execute the processing.
- tone emphasis processing a problem is pointed out that the evaluation value of the photographed image varies depending on the photographing condition that affects the brightness of the image (for example, how the irradiated light hits).
- the present invention has been made in view of the above circumstances, and an object of the present invention is to provide an endoscope system capable of suppressing the fluctuation of the evaluation value due to the brightness of the image and suppressing the processing load of the evaluation value calculation. And an evaluation value calculation device.
- An endoscope system provided by an embodiment of the present invention includes a light source device that irradiates illumination light toward a subject, an image of reflected light from the subject using an imaging device, and includes at least three or more color components.
- An image acquisition unit that acquires a color image and a predetermined reference point set in the color plane and an image acquisition unit within a color plane defined by at least two color components of at least three or more color components Obtain an evaluation result related to the target disease of each pixel based on an angle formed by a line segment connecting the pixel corresponding points in the color plane of each pixel constituting the acquired color image and a reference axis correlated with the target disease.
- the reference axis is set so as to pass through a predetermined reference point.
- the reference axis may be at least one of an axis that correlates with a target disease whose severity is a predetermined value or less and an axis that correlates with a target disease whose severity is a predetermined value or more in the color plane.
- the reference axis is a first axis that correlates with a high-severity disease whose severity is a predetermined level or higher among the target diseases in the color plane, and a lower severity and severity than the high-severity disease in the color plane May include at least a second axis having a correlation with a low-severity disease in which is less than or equal to a predetermined value.
- the evaluation unit obtains an evaluation result based on an angle formed by at least one of the first axis and the second axis and the above-described line segment.
- the reference point may be an intersection of the first axis and the second axis.
- the evaluation unit takes a maximum value when the angle between the line segment and the first axis is zero, and the angle between the line segment and the first axis is the first axis and the second axis.
- the angle between the above-mentioned line segment and the first axis is normalized so that a minimum value is obtained when the angle is equal to the angle between and the evaluation regarding the target disease based on the angle corresponding to each normalized pixel You may ask for the result.
- the endoscope system may further include a display unit that displays the evaluation result superimposed on the normal image acquired by the image acquisition unit.
- the evaluation result may be an evaluation result on the severity of inflammation as an example.
- the first axis may be an axis having a high correlation with the hemoglobin pigment as compared with other constituent substances in the body cavity.
- the second axis may be an axis having a high correlation with the color of the mucous membrane in the body cavity as compared with other constituent substances in the body cavity.
- the endoscope system may further include a display unit that displays the evaluation result superimposed on the normal image acquired by the image acquisition unit.
- the evaluation unit obtains evaluation results regarding a plurality of target diseases, and the color plane, the reference point, and the reference axis may be set for each of the plurality of target diseases.
- the display unit may display a plurality of evaluation results so as to be identifiable on the normal image.
- the color component may exclude hue and saturation. In other words, the color component may not include hue and saturation.
- An evaluation value calculation apparatus uses a pixel corresponding point corresponding to each pixel constituting a color image having color components of R (Red), G (Green), and B (Blue) as its color.
- an arrangement means for arranging in a plane including a first axis that is an axis of the R component and a second axis that is an axis of the G component or the B component orthogonal to the first axis;
- a third axis having a high correlation with the lesion having the highest severity in the plane, and for each pixel, a line segment connecting a predetermined reference point located on the third axis and the pixel corresponding point;
- Angle calculating means for calculating an angle formed by the third axis, and evaluation value calculating means for calculating a predetermined evaluation value for the color image based on the calculated angle corresponding to each pixel.
- the angle calculation means sets a fourth axis having a high correlation with a lesion having the lowest severity in a plane, and an intersection of the fourth axis and the third axis.
- the angle corresponding to each pixel may be calculated using the reference point.
- the evaluation value calculation means takes a maximum value when the angle calculated by the angle calculation means is zero, and the angle is an angle between the third axis and the fourth axis.
- the angle calculated by the angle calculation unit may be normalized so that a minimum value is obtained when they are equal, and a predetermined evaluation value for the color image may be calculated based on the angle corresponding to each normalized pixel. .
- the third axis is an axis having a high correlation with a hemoglobin pigment, for example, as compared with other constituent substances in the body cavity.
- the fourth axis is an axis having a high correlation with the color of the mucous membrane, for example, as compared with other constituent substances in the body cavity.
- the predetermined evaluation value is, for example, a numerical value indicating an abnormal part in a body cavity.
- evaluation value calculation apparatus may be configured to be incorporated in an electronic endoscope system.
- the evaluation value calculation apparatus determines a pixel corresponding point corresponding to each pixel constituting a color image having each color component of R, G, and B according to the color component.
- An arrangement means arranged in a plane including a first axis that is an axis of the second axis and a second axis that is an axis of the G component or the B component orthogonal to the first axis, and the most severe in the plane
- a third axis having a high correlation with a low lesion is set, and for each pixel, a line segment connecting a predetermined reference point located on the third axis and the pixel corresponding point, and the third axis are Angle calculation means for calculating an angle formed, and evaluation value calculation means for calculating a predetermined evaluation value for the color image based on the calculated angle corresponding to each pixel.
- an endoscope system and an evaluation value calculation device capable of suppressing evaluation value variation due to image brightness and suppressing the processing load of evaluation value calculation.
- FIG. 1 It is a block diagram which shows the structure of the electronic endoscope system which concerns on one Embodiment of this invention. It is a figure which shows the flowchart of the special image generation process by the special image processing circuit with which the processor which concerns on one Embodiment of this invention is equipped. It is a figure which shows the RG plane where a pixel corresponding point is plotted in one Embodiment of invention. It is a figure explaining the reference axis set in an RG plane. It is a figure which shows the example of a display screen displayed on the display screen of a monitor at the time of special mode in one Embodiment of this invention.
- FIG. 1 is a block diagram showing a configuration of an electronic endoscope system 1 according to an embodiment of the present invention.
- the electronic endoscope system 1 is a system specialized for medical use, and includes an electronic scope 100, a processor 200, and a monitor 300.
- the electronic scope 100 has an insertion portion, and the insertion portion has a distal end portion and a bending portion.
- An LCB (Light Carrying Bundle) 102 is extended in the insertion portion, and a light distribution lens 104, an objective lens 106, a solid-state image sensor 108, and the like are disposed at the distal end portion.
- LCB Light Carrying Bundle
- the monitor 300 is an example of a display unit provided in the electronic endoscope system of the present embodiment.
- the electronic endoscope system shown in FIG. 1 will be described as an example, but the electronic endoscope system of the present invention is not limited to the electronic endoscope system shown in FIG.
- inflammation will be described as an example of a target disease for which an evaluation value is obtained.
- the target disease is not limited to inflammation, and the present invention can be applied to any disease that can be evaluated by color components. .
- the processor 200 includes a system controller 202 and a timing controller 204.
- the system controller 202 executes various programs stored in the memory 222 and controls the entire electronic endoscope system 1 in an integrated manner.
- the system controller 202 is connected to the operation panel 218.
- the system controller 202 changes each operation of the electronic endoscope system 1 and parameters for each operation in accordance with an instruction from the operator input from the operation panel 218.
- the input instruction by the operator includes, for example, an instruction to switch the operation mode of the electronic endoscope system 1. In the present embodiment, there are a normal mode and a special mode as operation modes.
- the timing controller 204 outputs a clock pulse for adjusting the operation timing of each unit to each circuit in the electronic endoscope system 1.
- the electronic endoscope system 1 includes a light source device.
- a lamp 208 for example, a lamp 208, a semiconductor light emitting element, or the like can be applied, but the type thereof is not particularly limited.
- the lamp 208 shown in FIG. 1 emits white light L after being started by the lamp power igniter 206.
- the lamp 208 is a high-intensity lamp such as a xenon lamp, a halogen lamp, a mercury lamp, or a metal halide lamp.
- the white light L emitted from the lamp 208 is limited to an appropriate amount of light through the diaphragm 212 while being collected by the condenser lens 210.
- the lamp 208 may be replaced with a semiconductor light emitting element such as an LD (Laser Diode) or an LED (Light Emitting Diode).
- a semiconductor light emitting element such as an LD (Laser Diode) or an LED (Light Emitting Diode).
- the semiconductor light emitting device has advantages such as low power consumption and small amount of heat generation compared to other light sources, and thus has a merit that a bright image can be acquired while suppressing power consumption and heat generation amount. The ability to obtain a bright image leads to an improvement in the accuracy of the inflammation evaluation value described later.
- the semiconductor light emitting element is not limited to the processor 200 and may be incorporated in the electronic scope 100. As an example, the semiconductor light emitting element may be provided in the distal end portion of the electronic scope 100.
- the motor 214 is mechanically connected to the diaphragm 212 via a transmission mechanism such as an arm or gear not shown.
- the motor 214 is a DC motor, for example, and is driven under the drive control of the driver 216.
- the aperture 212 is operated by the motor 214 to change the opening degree so that the image displayed on the display screen of the monitor 300 has an appropriate brightness.
- the amount of white light L emitted from the lamp 208 is limited according to the opening degree of the diaphragm 212.
- the appropriate reference for the brightness of the image is changed according to the brightness adjustment operation of the operation panel 218 by the operator.
- the dimming circuit that controls the brightness by controlling the driver 216 is a well-known circuit and is omitted in this specification.
- the white light L that has passed through the diaphragm 212 is condensed on the incident end face of the LCB 102 and is incident on the LCB 102.
- White light L incident on the LCB 102 from the incident end face propagates in the LCB 102.
- the white light L that has propagated through the LCB 102 is emitted from the emission end face of the LCB 102 disposed at the tip of the electronic scope 100 and irradiates the living tissue via the light distribution lens 104.
- the return light from the living tissue irradiated with the white light L forms an optical image on the light receiving surface of the solid-state image sensor 108 via the objective lens 106.
- the solid-state imaging device 108 is, for example, a single-plate color CCD (Charge Coupled Device) image sensor having a Bayer pixel arrangement.
- the solid-state image sensor 108 accumulates an optical image formed by each pixel on the light receiving surface as a charge corresponding to the amount of light, and generates R (Red), G (Green), and B (Blue) image signals. Output.
- R Red
- G Green
- B Blue
- an image signal of each pixel (each pixel address) sequentially output from the solid-state image sensor 108 is referred to as a “pixel signal”.
- the solid-state imaging element 108 is not limited to a CCD image sensor, and may be replaced with a CMOS (Complementary Metal Oxide Semiconductor) image sensor or other types of imaging devices.
- CMOS Complementary Metal Oxide Semiconductor
- the solid-state image sensor 108 may be a device equipped with a primary color (RGB) filter, or a device equipped with a complementary color filter.
- RGB primary color
- a complementary color filter is a CMYG (cyan, magenta, yellow, green) filter.
- RGB Primary color
- complementary color filters have better color development than complementary color filters. Therefore, the evaluation accuracy can be improved by using the RGB image signal from the image sensor equipped with the primary color system filter for calculation of the evaluation value. Further, by using the primary color filter, it is not necessary to convert the signal in the evaluation value calculation process described later. Therefore, it is possible to suppress the processing load of evaluation value calculation.
- a driver signal processing circuit 112 is provided in the connection part of the electronic scope 100.
- the pixel signal of the living tissue irradiated with the white light L is input to the driver signal processing circuit 112 from the solid-state imaging device 108 at a frame period.
- the driver signal processing circuit 112 outputs the pixel signal input from the solid-state image sensor 108 to the previous signal processing circuit 220 of the processor 200.
- “frame” may be replaced with “field”.
- the frame period and the field period are 1/30 seconds and 1/60 seconds, respectively.
- the driver signal processing circuit 112 also accesses the memory 114 and reads the unique information of the electronic scope 100.
- the unique information of the electronic scope 100 recorded in the memory 114 includes, for example, the number and sensitivity of the solid-state image sensor 108, the operable frame rate, the model number, and the like.
- the driver signal processing circuit 112 outputs the unique information read from the memory 114 to the system controller 202.
- the system controller 202 performs various calculations based on the unique information of the electronic scope 100 and generates a control signal.
- the system controller 202 controls the operation and timing of various circuits in the processor 200 using the generated control signal so that processing suitable for the electronic scope connected to the processor 200 is performed.
- the timing controller 204 supplies clock pulses to the driver signal processing circuit 112 according to the timing control by the system controller 202.
- the driver signal processing circuit 112 drives and controls the solid-state imaging device 108 at a timing synchronized with the frame rate of the video processed on the processor 200 side, according to the clock pulse supplied from the timing controller 204.
- the pre-stage signal processing circuit 220 performs demosaic processing on the R, G, and B pixel signals input from the driver signal processing circuit 112 at a frame period. Specifically, interpolation processing by G and B peripheral pixels is performed on each R pixel signal, interpolation processing by R and B peripheral pixels is performed on each G pixel signal, and R processing is performed on each B pixel signal. , G interpolation processing is performed on the peripheral pixels. As a result, all pixel signals having only information of one color component are converted into pixel data having information of three color components of R, G, and B. In the present embodiment, the pixel data after demosaicing has 8 bits (0 to 255) of information for each of the R, G, and B color components.
- the pre-stage signal processing circuit 220 performs predetermined signal processing such as matrix operation, white balance adjustment processing, and gamma correction processing on the pixel data after demosaic processing, and outputs the result to the special image processing circuit 230.
- predetermined signal processing such as matrix operation, white balance adjustment processing, and gamma correction processing
- the special image processing circuit 230 outputs the pixel data input from the upstream signal processing circuit 220 to the downstream signal processing circuit 240 through.
- the post-stage signal processing circuit 240 performs predetermined signal processing on the pixel data input from the special image processing circuit 230 to generate screen data for monitor display, and the generated screen data for monitor display is converted to a predetermined video format. Convert to signal.
- the converted video format signal is output to the monitor 300. Thereby, a color image of the living tissue is displayed on the display screen of the monitor 300.
- the color image of the living tissue is a normal image.
- the color image of the living tissue includes at least three color components.
- the above-described operation of capturing an image of a subject by the solid-state image sensor 108 and acquiring a color image of the subject is performed by an image acquisition unit.
- the image in a color plane defined by at least two color components of color components included in a color image of a living tissue, the image is acquired by a predetermined reference point set in the color plane and an image acquisition unit.
- An evaluation result related to the target disease of each pixel is obtained based on an angle formed by a line segment connecting the pixel corresponding points in the color plane of each pixel constituting the color image and a reference axis having a correlation with the target disease.
- the R component and the G component are exemplified as color components that define the color plane, and an axis having a correlation with an inflammatory disease is described as an example of the reference axis, but the present invention is not limited to this embodiment. .
- the pre-stage signal processing circuit 220 performs predetermined image processing such as demosaic processing, matrix calculation, white balance adjustment processing, and gamma correction processing on the pixel signal input from the driver signal processing circuit 112 at a frame period to perform special image processing. Output to the circuit 230.
- FIG. 2 shows an example of a flowchart of special image generation processing by the special image processing circuit 230.
- the special image generation process of FIG. 2 is started when the operation mode of the electronic endoscope system 1 is switched to the special mode.
- FIG. 3 shows an RG plane (more specifically, a section in the RG plane defined by two axes of the R axis and the G axis) defined by the R axis and the G axis orthogonal to each other.
- the R axis is the axis of the R component (R pixel value)
- the G axis is the axis of the G component (G pixel value).
- pixel data three-dimensional data of each pixel in the RGB space defined by the RGB three primary colors is converted into two-dimensional data of RG, and as shown in FIG. Is plotted (orthographically projected) in the RG plane.
- pixel corresponding points plotted in the RG plane are referred to as “pixel corresponding points”.
- FIG. 3 for the sake of clarity, only the pixel corresponding points of some pixels are shown instead of the pixel corresponding points of all the pixels.
- pixel data (three-dimensional data) in the color space is orthogonally projected onto the color plane, and the perpendicular foot drawn from the point in the color space corresponding to the pixel data to the color plane is added. It becomes a pixel corresponding point (two-dimensional data).
- FIG. 4 is a diagram for assisting the explanation of the reference axis.
- the R component In the body cavity of a patient to be imaged, the R component is dominant over other components (G component and B component) due to the influence of hemoglobin pigment or the like. Component) becomes strong against other colors (G component and B component).
- the color of the captured image in the body cavity changes according to the imaging condition that affects the brightness (for example, the degree of hitting of the white light L).
- the shaded portion where the white light L does not reach is black (achromatic color, for example, R, G, B is zero or a value close to zero), and the portion where the white light L strikes strongly and is specularly reflected is White (achromatic color, for example, R, G, B is a value close to 255 or 255).
- the pixel value of the abnormal part image increases as the white light L strikes stronger. Therefore, depending on how the white light L hits, the pixel value may take a value that has no correlation with the intensity of inflammation.
- a normal site in a body cavity where inflammation has not occurred is covered with sufficient mucosa.
- an abnormal site in a body cavity where inflammation occurs is not covered with sufficient mucosa.
- the mucous membrane becomes thinner as the inflammation becomes stronger.
- the mucous membrane is basically a white tone, but has a slightly yellowish color, and the color (yellow color) reflected on the image changes depending on the shade (thickness of the mucous membrane). Therefore, the density of the mucous membrane is considered to be one index for evaluating the intensity of inflammation.
- a straight line passing through (50, 0) and (255, 76) is set as one of the reference axes, and (0 , 0) and (255, 192) are set as one of the reference axes.
- the former reference axis is referred to as “hemoglobin change axis AX1”, and the latter reference axis is referred to as “mucosal change axis AX2”.
- the setting data of the hemoglobin change axis AX1 and the mucosa change axis AX2 may be stored in advance in a storage medium such as the memory 222, for example.
- (50, 0) and (255, 76), (0, 0) and (255, 192) are given as examples of points through which the reference axis passes, it is not limited to this. The above points can be changed as appropriate.
- the plot shown in FIG. 4 is obtained as a result of analysis of a large number of sample images in the body cavity by the inventor.
- Sample images used for analysis include inflammatory image examples at each stage, such as inflammatory image examples with the highest severity and inflammatory image examples with the lowest severity (image examples that are considered to be substantially normal sites). included.
- inflammatory image examples at each stage such as inflammatory image examples with the highest severity and inflammatory image examples with the lowest severity (image examples that are considered to be substantially normal sites). included.
- FIG. 4 only a part of the plot obtained as a result of the analysis is shown for the sake of clarity.
- the number of plots actually obtained as a result of the analysis is much larger than the number of plots shown in FIG.
- the R component becomes stronger with respect to other components (G component and B component) as the abnormal site is more intensely inflamed. Therefore, it is a boundary line between a region where the plot is distributed and a region where the plot is not distributed, and is an axis on the boundary line closer to the R axis than the G axis, in the example of FIG. 4, (50, 0) and (255, 76).
- This axis is the hemoglobin change axis AX1.
- On the boundary line passing through) is set as an axis having a high correlation with the least severe lesion (the least severely affected inflammation (abnormal) site that is considered to be a substantially normal site) Is done.
- This axis is the mucosa changing axis AX2.
- the mucosal change axis AX2 is overlaid with plots corresponding to the least severely inflamed sites (substantially regarded as normal sites) imaged under various imaging conditions (for example, the degree of white light L). .
- the most severe inflammatory site may be accompanied by bleeding.
- the inflammatory site with the lowest severity is a substantially normal site and is therefore covered with sufficient mucosa. Therefore, the plot in the RG plane shown in FIG. 4 can be understood as being distributed in a region sandwiched between an axis having the highest correlation with blood (hemoglobin pigment) and an axis having the highest correlation with mucous color. . Therefore, of the boundary lines between the region where the plot is distributed and the region where the plot is not distributed, the boundary line closer to the R axis (the R component is strong) indicates the inflammatory site with the highest severity (hemoglobin change axis AX1). The boundary line closer to the G axis (the G component is stronger) corresponds to the axis (mucosal change axis AX2) indicating the inflammation site with the lowest severity.
- the reference axis may be an axis having a correlation with at least the inflammation to be evaluated.
- at least an axis having a correlation with a pixel indicating a site where the severity of inflammation is not more than a predetermined value in a color plane, and an axis having a correlation with a pixel indicating a site having the severity of inflammation not less than a predetermined value may be set as a reference axis.
- the reference axis may be set as an axis indicating a site of high severity inflammation and an axis indicating a site of low severity inflammation.
- the angle formed by the plurality of reference axes is smaller than the angle formed by the axes indicated by the color components defining the color plane. Since the resolution is improved as the angle formed by the reference axes is smaller, a highly accurate evaluation value can be obtained.
- the angle formed by the reference axes is preferably less than 90 degrees, and more preferably 60 degrees or less. *
- an angle for calculating the inflammation intensity is calculated for the target pixel selected in process step S14 (selection of target pixel). Specifically, in this processing step S15, the line segment L connecting the intersection (reference point) O ′ between the hemoglobin change axis AX1 and the mucosa change axis AX2 and the pixel corresponding point of the target pixel, and the hemoglobin change axis AX1 are obtained. An angle ⁇ formed is calculated (see FIG. 3). The reference point O ′ is located at the coordinates ( ⁇ 150, ⁇ 75).
- the reference point O ′ is set to the coordinates ( ⁇ 150, ⁇ 75) is given, it is not limited to this.
- the reference point O ′ can be changed as appropriate.
- a coordinate position suitable as a reference point is, for example, a position that can reduce errors in evaluation results due to variations in brightness.
- the reference point is set by obtaining in advance a point that minimizes an error between an evaluation result in a dark part (luminance is less than a predetermined value) and an evaluation result in a non-dark part (luminance is a predetermined value or more). It is preferable.
- the reference point O ′ is set between the coordinates ( ⁇ 10, ⁇ 10) to (10, 10), it is compared with the case where the coordinates ( ⁇ 150, ⁇ 75) etc. are set as the reference point O ′.
- the resolution is improved, so that a highly accurate evaluation result can be obtained.
- the angle ⁇ is a parameter that is substantially unaffected by changes in the brightness of the captured image.
- the angle ⁇ is normalized so that the value 255 is obtained when the angle ⁇ is zero and the value is zero when the angle ⁇ is ⁇ MAX .
- ⁇ MAX is equal to the angle formed by the hemoglobin change axis AX1 and the mucosa change axis AX2.
- this processing step S17 the display color on the color map image of the target pixel selected in the processing step S14 (selection of target pixel) is obtained in the processing step S16 (normalization processing) based on the above table.
- the color is determined according to the value of the inflammation intensity of the obtained target pixel.
- the special image generation processing in FIG. 2 executes the processing steps S14 to S17 for the next pixel of interest. Return to (Selecting the pixel of interest).
- This processing step S19 is executed when it is determined that the processing steps S14 to S17 have been executed for all the pixels of the current frame in the processing step S18 (determination of processing completion for all pixels) (S18: YES). Is done.
- an average value obtained by averaging the inflammation intensities of all pixels in the current frame is calculated as an inflammation evaluation value of the entire photographed image, and display data of the calculated inflammation evaluation value (display data example: Score: ⁇ ⁇ ) is generated.
- display data example: Score: ⁇ ⁇ display data example: Score: ⁇ ⁇
- the operation for obtaining the above-described inflammation evaluation value as the evaluation result of the target disease is performed by the evaluation unit.
- a normal image based on the pixel data that is, pixel data having three color components of RGB
- processing step S17 display color on the color map image
- the coefficient setting can be appropriately changed by a user operation. If the normal image is to be displayed darker, the coefficient of the normal pixel data is set higher. If the color map image is to be displayed darker, the coefficient of the color map pixel data is set higher.
- step S21 it is determined whether or not the operation mode of the electronic endoscope system 1 has been switched to a mode different from the special mode. If it is determined that the mode has not been switched to another mode (S21: NO), the special image generation processing in FIG. 2 returns to processing step S11 (input of pixel data of the current frame). On the other hand, when it is determined that the mode has been switched to another mode (S21: YES), the special image generation process in FIG. 2 ends.
- the post-stage signal processing circuit 240 generates display data of the overlay image of the normal image and the color map image based on the pixel data added in the processing step S20 (overlay processing) of FIG. A masking process for masking the peripheral area (periphery of the image display area) is performed, and screen data for monitor display is generated by superimposing an inflammation evaluation value on the mask area generated by the masking process.
- the post-stage signal processing circuit 240 converts the generated monitor display screen data into a predetermined video format signal and outputs it to the monitor 300.
- Fig. 5 shows a screen display example in the special mode.
- a captured image in the body cavity an overlay image in which a normal image and a color map image are displayed in an overlay manner
- an image display area A screen with a mask around is displayed.
- an inflammation evaluation value (score) is displayed in the mask area.
- the inflammation evaluation value (here, the region to be imaged) can be obtained only by performing simple calculation processing without performing nonlinear calculation processing such as tone enhancement processing or complicated color space conversion processing. A value correlated with the increase or decrease of the hemoglobin pigment). That is, the hardware resources necessary for calculating the inflammation evaluation value can be greatly reduced.
- the inflammation evaluation value does not substantially vary depending on the imaging conditions that affect the brightness of the captured image in the body cavity (for example, how the irradiated light hits), the surgeon can make a more objective and accurate determination of inflammation. Can be reduced.
- the electronic endoscope system according to the present embodiment brings about the following effects and solutions in the technical field.
- the electronic endoscope system is a diagnostic aid for early detection of inflammatory diseases.
- the degree of inflammation can be displayed on the screen, or the image of the region where the inflammation has occurred can be emphasized so that the surgeon can find mild inflammation that is difficult to visually recognize. .
- mild inflammation is difficult to distinguish from a normal part, the effects brought about by the configuration of the present embodiment regarding the evaluation of mild inflammation become significant.
- an objective evaluation value can be provided to the surgeon as an evaluation of the degree of inflammation, so that the diagnostic difference between the surgeons can be reduced.
- the merit of providing an objective evaluation value according to the configuration of the present embodiment to an inexperienced surgeon is great.
- the configuration of the present embodiment it is possible to display the inflamed part as an image in real time by reducing the load of image processing. Therefore, diagnostic accuracy can be improved.
- a color map image an image showing the degree of inflammation
- a normal image can be obtained without delay. Can be displayed side by side or synthesized. Therefore, it is possible to display a color map image without extending the examination time, and as a result, it is possible to avoid an increase in patient burden.
- the observation target part in the present embodiment is, for example, a respiratory organ, a digestive organ, or the like.
- the respiratory organ or the like is, for example, the lung or the ENT.
- Examples of digestive organs include the large intestine, the small intestine, the stomach, the duodenum, and the uterus.
- the electronic endoscope system according to the present embodiment is considered to be more effective when the observation target is the large intestine. Specifically, this is due to the following reasons.
- the inflammation evaluation value according to the present embodiment is effective as an index of inflammatory bowel disease (IBD) represented by ulcerative colitis. Since a treatment method for ulcerative colitis has not been established, the use of the electronic endoscope system having the configuration of the present embodiment has an extremely great effect of early detection and suppression of progression.
- IBD inflammatory bowel disease
- the large intestine is a long and slender organ compared to the stomach and the like, and the obtained image has a depth and becomes darker at the back. According to the configuration of the present embodiment, it is possible to suppress the fluctuation of the evaluation value due to the change in brightness in the image. Therefore, when the electronic endoscope system according to the present embodiment is applied to the observation of the large intestine, the effect of the present embodiment becomes remarkable. That is, the electronic endoscope system according to the present embodiment is preferably an electronic endoscope system for respiratory organs or an electronic endoscope system for digestive organs, and more preferably an electronic endoscope system for large intestine.
- mild inflammation is generally difficult to diagnose
- the configuration of the present embodiment for example, by displaying the result of evaluating the degree of inflammation on the screen, it is possible to prevent the operator from overlooking the mild inflammation. it can.
- the judgment criteria are not clear, and this is a factor that increases individual differences among surgeons.
- an objective evaluation value can be provided to the surgeon, so that variation in diagnosis due to individual differences can be reduced.
- the above-described configuration of the present embodiment can be applied not only to the degree of inflammation but also to the evaluation value of various lesions accompanied by cancer, polyps, and other color changes. In these cases, the same advantages as described above can be applied. Effects can be achieved. That is, the evaluation value of the present embodiment is preferably an evaluation value of a lesion accompanied by a color change, and preferably includes at least one of an inflammation level, a cancer, and a polyp.
- Embodiments of the present invention are not limited to those described above, and various modifications are possible within the scope of the technical idea of the present invention.
- the embodiment of the present application also includes an embodiment that is exemplarily specified in the specification or a combination of obvious embodiments and the like as appropriate.
- the inflammation evaluation value is calculated using the R component and the G component included in each pixel.
- the R component and the B component are used instead of the R component and the G component.
- evaluation values such as inflammation are calculated using R, G, and B primary color components.
- the configuration for calculating the evaluation value according to the present invention includes the R, G, and B primary color components. It is not limited to use. Instead of using the R, G, B primary color components, for example, complementary color components of C, M, Y, G (cyan, magenta, yellow, green) are used, and inflammation or the like is performed by the same method as in the above embodiment.
- the evaluation value may be calculated.
- pixel data in the RGB color space is converted to RG plane pixel data, and the inflammation evaluation value is calculated using the R component and the G component included in the converted pixel data.
- CIEC1976 L * a * b * color space CIE LCh color space, CIE 1976 L * u * v * color space, HSB color space, sRGB color space, CMK color space, CMYK Pixel data of another color space (a color space defined by n (n ⁇ 3) types of color components) such as a color space, a CMYG color space, or the like is converted into a lower-order color space (m (n> m ⁇ 2). )
- the target disease stomach atrophy, colon tumor, etc.
- the reference axis may be set as an axis passing through (0, 0) of the color plane. This has the effect of simplifying the implementation.
- the light source unit including the lamp power igniter 206, the lamp 208, the condenser lens 210, the diaphragm 212, the motor 214, and the like is provided integrally with the processor, but the light source unit is separate from the processor. It may be provided as a device.
- a CMOS image sensor may be used as the solid-state imaging device 108 instead of the CCD image sensor.
- a CMOS image sensor tends to darken an image as a whole as compared with a CCD image sensor. Therefore, the advantageous effect that the fluctuation of the evaluation value due to the brightness of the image by the configuration of the above-described embodiment can be suppressed becomes more prominent in a situation where a CMOS image sensor is used as the solid-state imaging device.
- the resolution of the image is preferably 1 million pixels or more, more preferably 2 million pixels, and even more preferably 8 million pixels or more.
- the processing load for performing the above-described evaluation value calculation for all pixels increases.
- the processing load of evaluation value calculation can be suppressed. Therefore, the advantageous effect of the configuration of the present embodiment is remarkably exhibited in a situation where a high-definition image is processed.
- all the pixels in the image are processed, but for example, extremely high luminance pixels and extremely low luminance pixels may be excluded from the processing targets. .
- the accuracy of the evaluation value can be improved by setting only the pixels determined to have the luminance within a predetermined reference luminance range as the evaluation value calculation target.
- various types of light sources can be used as the light source used in the electronic endoscope system 1.
- the type of the light source is limited (for example, a laser is excluded as the light source type).
- the color component used for calculating the evaluation value it may be excluded to calculate the evaluation value using the hue and the saturation.
- the angle ⁇ formed by the line segment L connecting the reference point O ′ and the pixel corresponding point of the target pixel and the hemoglobin change axis AX1 is calculated, and inflammation evaluation is performed based on the calculated angle ⁇ .
- the present invention is not limited to this.
- an angle formed by the line segment L and the mucous membrane change axis AX2 may be calculated, and the inflammation evaluation value may be calculated based on the calculated angle.
- the smaller the calculated angle the stronger the G component with respect to the R component, indicating that the severity of the inflammatory site is lower, and the larger the calculated angle, the stronger the R component with respect to the G component. Indicates a high degree. Therefore, in processing step S16 FIG. 2 (normalization) becomes a value zero when calculating the angle is zero, calculating the angle to a value 255 when a theta MAX, to normalize the computed angle.
- the reference axis may be at least one of an axis that correlates with a target disease whose severity is a predetermined value or less and an axis that correlates with a target disease whose severity is a predetermined value or more in the color plane. More specifically, the reference axis is a first axis correlated with a high-severity disease whose severity is a predetermined level or higher among target diseases in the color plane, and more severe than a high-severity disease in the color plane. It may include at least a second axis that is correlated with a low-severity disease having a low degree of severity and a severity of not more than a predetermined level.
- the evaluation unit for obtaining the evaluation result is defined by at least one of the first axis and the second axis and the above-described line segment (at least two color components among the color components included in the color image of the living tissue).
- the angle formed by a predetermined reference point set in the color plane and a line segment connecting pixel corresponding points in the color plane of each pixel constituting the color image acquired by the image acquisition unit in the color plane The evaluation result is obtained based on the above.
- the intersection of the hemoglobin change axis AX1 and the mucosa change axis AX2 is set as the reference point O ′ in order to suppress the influence of the brightness of the captured image on the inflammation evaluation value as much as possible.
- the invention is not limited to this.
- the origin (0, 0) of the RG plane located on the mucous membrane change axis AX2 may be set as the reference point O ′.
- the minimum required reference axis is sufficient (mucosal change axis AX2), the load of the special image generation process in FIG. 2 is reduced, and the processing speed is improved.
- the evaluation unit may be configured to obtain evaluation results for a plurality of target diseases, respectively.
- the color plane, the reference point, and the reference axis are set for each of a plurality of target diseases.
- a plurality of evaluation results may be displayed on the normal image so as to be identifiable on the monitor constituting the display unit. According to such a configuration, it is possible for the surgeon to recognize a portion having a possibility of a plurality of diseases on one screen without switching settings and the like. As a result, it is possible to assist the operator in diagnosis without increasing the burden on the patient.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Surgery (AREA)
- Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- General Health & Medical Sciences (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Medical Informatics (AREA)
- Radiology & Medical Imaging (AREA)
- Molecular Biology (AREA)
- Heart & Thoracic Surgery (AREA)
- Pathology (AREA)
- Biophysics (AREA)
- Animal Behavior & Ethology (AREA)
- Public Health (AREA)
- Veterinary Medicine (AREA)
- Biomedical Technology (AREA)
- General Physics & Mathematics (AREA)
- Signal Processing (AREA)
- Theoretical Computer Science (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Microelectronics & Electronic Packaging (AREA)
- Quality & Reliability (AREA)
- Astronomy & Astrophysics (AREA)
- Multimedia (AREA)
- Endoscopes (AREA)
- Image Processing (AREA)
Abstract
Description
図1は、本発明の一実施形態に係る電子内視鏡システム1の構成を示すブロック図である。図1に示されるように、電子内視鏡システム1は、医療用に特化されたシステムであり、電子スコープ100、プロセッサ200及びモニタ300を備えている。電子スコープ100は、挿入部を有し、挿入部は、先端部及び湾曲部を有する。挿入部内には、LCB(Light Carrying Bundle)102が延設されており、また、先端部には、配光レンズ104、対物レンズ106、固体撮像素子108等が配置される。モニタ300は、本実施形態の電子内視鏡システムが備える表示部の一例である。なお、以下では図1に示す電子内視鏡システムを例に挙げて説明するが、本発明の電子内視鏡システムが図1に示す電子内視鏡システムに限定されるものではない。また、以下では評価値を求める対象疾患として炎症を例に挙げて説明するが、対象疾患は炎症に限定されるものではなく、色成分によって評価可能な疾患であれば本発明を適用可能である。
通常モード時のプロセッサ200での信号処理動作を説明する。
次に、特殊モード時のプロセッサ200での信号処理動作を説明する。
図2は、特殊画像処理回路230による特殊画像生成処理のフローチャートの例を示す。図2の特殊画像生成処理は、電子内視鏡システム1の動作モードが特殊モードに切り替えられた時点で開始される。
本処理ステップS11では、前段信号処理回路220より現フレームの各画素の画素データが入力される。
図3に、互いに直交するR軸とG軸とによって定義されるRG平面(より詳細には、R軸、G軸の二軸によって規定されるRG平面内の区画)を示す。なお、R軸は、R成分(Rの画素値)の軸であり、G軸は、G成分(Gの画素値)の軸である。本処理ステップS12では、RGB3原色で定義されるRGB空間の各画素の画素データ(三次元データ)がRGの二次元データに変換されて、図3に示されるように、R、Gの画素値に応じてRG平面内にプロット(正射影)される。以下、説明の便宜上、RG平面内にプロットされた画素データの点を「画素対応点」と記す。なお、図3においては、図面を明瞭化する便宜上、全ての画素の画素対応点を示すのではなく一部の画素の画素対応点のみ示している。
本処理ステップS13では、炎症強度を計算するために必要なRG平面内の基準軸が設定される。図4に、基準軸の説明を補助する図を示す。
本処理ステップS14では、全ての画素の中から所定の順序に従い一つの注目画素が選択される。
本処理ステップS15では、処理ステップS14(注目画素の選択)にて選択された注目画素について炎症強度を計算するための角度が算出される。具体的には、本処理ステップS15では、ヘモグロビン変化軸AX1と粘膜変化軸AX2との交点(基準点)O’と注目画素の画素対応点とを結ぶ線分Lと、ヘモグロビン変化軸AX1とがなす角度θが算出される(図3参照)。なお、基準点O’は座標(-150,-75)に位置する。
体腔内の撮影画像の明るさが白色光Lの当たり具合によって変化すると、撮影画像の色味は、個人差、撮影箇所、炎症の状態等の影響があるものの、RG平面内において、概ね、重症度の最も高い炎症部位ではヘモグロビン変化軸AX1上に沿って変化し、重症度の最も低い炎症部位では粘膜変化軸AX2上に沿って変化する。また、中間の重症度の炎症部位の撮影画像の色味も同じ傾向で変化するものと推定される。すなわち、炎症部位に対応する画素対応点は、白色光Lの当たり具合によって変化すると、基準点O’を起点とした方位角方向にシフトする。言い換えると、炎症部位に対応する画素対応点は、白色光Lの当たり具合によって変化すると、角度θが一定のまま移動して基準点O’との距離が変わる。これは、角度θが撮影画像の明るさの変化に実質的に影響を受けないパラメータであることを意味する。
本実施形態では、炎症強度に応じた表示色で撮影画像をモザイク化したカラーマップ画像を表示することができる。カラーマップ画像を表示可能とするため、炎症強度の値と所定の表示色とを対応付けたテーブルがメモリ222等の記憶領域に記憶されている。本テーブルでは、例えば、値5刻みで異なる表示色が対応付けられている。例示的には、炎症強度の値が0~5の範囲では黄色が対応付けられており、該値が5増える毎に色相環での色の並び順に従って異なる表示色が対応付けられており、該値が250~255の範囲では赤色が対応付けられている。
本処理ステップS18では、現フレームの全ての画素に対して処理ステップS14~S17が実行されたか否かが判定される。
本処理ステップS19は、処理ステップS18(全画素に対する処理の実行完了判定)において、現フレームの全ての画素に対して処理ステップS14~S17が実行されたと判定された場合(S18:YES)に実行される。本処理ステップS19では、現フレームの全ての画素の炎症強度を平均化した平均値が撮影画像全体の炎症評価値として計算され、計算された炎症評価値の表示データ(表示データ例:Score:○○)が生成される。対象疾患の評価結果としての上述の炎症評価値を求める動作は、評価部により行われる。
本処理ステップS20では、前段信号処理回路220より入力される画素データ(すなわち、RGBの3つの色成分を持つ画素データ)に基づく通常画像と、処理ステップS17(カラーマップ画像上での表示色の決定)にて所定の表示色に決定された画素データに基づくカラーマップ画像とをオーバレイさせる割合を係数として、前者の画素データ(通常の画素データ)と後者の画素データ(カラーマップ用の画素データ)とが加算される。係数の設定は、ユーザ操作により適宜設定変更することが可能である。通常画像の方を濃く表示したい場合は、通常の画素データの係数が高く設定され、カラーマップ画像の方を濃く表示したい場合は、カラーマップ用の画素データの係数が高く設定される。
本処理ステップS21では、電子内視鏡システム1の動作モードが特殊モードとは別のモードに切り替えられたか否かが判定される。別のモードに切り替えられていないと判定される場合(S21:NO)、図2の特殊画像生成処理は、処理ステップS11(現フレームの画素データの入力)に戻る。一方、別のモードに切り替えられたと判定される場合(S21:YES)、図2の特殊画像生成処理は終了する。
後段信号処理回路240は、図2の処理ステップS20(オーバレイ処理)にて加算処理された画素データに基づいて通常画像とカラーマップ画像とのオーバレイ画像の表示データを生成すると共にモニタ300の表示画面の周辺領域(画像表示領域の周囲)をマスクするマスキング処理を行い、更に、マスキング処理により生成されるマスク領域に炎症評価値を重畳した、モニタ表示用の画面データを生成する。後段信号処理回路240は、生成されたモニタ表示用の画面データを所定のビデオフォーマット信号に変換して、モニタ300に出力する。
100 電子スコープ
102 LCB
104 配光レンズ
106 対物レンズ
108 固体撮像素子
112 ドライバ信号処理回路
114 メモリ
200 プロセッサ
202 システムコントローラ
204 タイミングコントローラ
206 ランプ電源イグナイタ
208 ランプ
210 集光レンズ
212 絞り
214 モータ
216 ドライバ
218 操作パネル
220 前段信号処理回路
222 メモリ
230 特殊画像処理回路
240 後段信号処理回路
Claims (13)
- 被写体に向けて照明光を照射する光源装置と、
前記被写体からの反射光を撮像素子により撮像し、少なくとも3つ以上の色成分を含むカラー画像を取得する画像取得部と、
前記少なくとも3つ以上の色成分のうちの少なくとも2つの色成分によって定義される色平面内において、前記色平面内に設定された所定の基準点及び前記画像取得部で取得されるカラー画像を構成する各画素の前記色平面内における画素対応点を結ぶ線分と、前記対象疾患に相関を有する基準軸と、がなす角度に基づいて前記各画素の前記対象疾患に関する評価結果を求める評価部と、を備え、
前記基準軸は、前記所定の基準点を通るように設定される、内視鏡システム。 - 前記基準軸は、前記色平面内において重症度が所定値以下の前記対象疾患と相関を有する軸及び重症度が所定値以上である前記対象疾患と相関を有する軸の少なくとも一方である、請求項1に記載の内視鏡システム。
- 前記基準軸は、前記色平面内において前記対象疾患のうち重症度が所定以上である高重症度疾患と相関を有する第1の軸と、前記色平面内において前記高重症度疾患よりも重症度が低くかつ重症度が所定以下である低重症度疾患と相関を有する第2軸と、を少なくとも含み、
前記評価部は、前記第1の軸及び第2の軸の少なくとも一方と、前記線分と、がなす角度に基づいて前記評価結果を求める、
請求項1又は請求項2に記載の内視鏡システム。 - 前記基準点は、前記第1の軸と前記第2の軸の交点である、
請求項3に記載の内視鏡システム。 - 前記評価部は、
前記線分と前記第1の軸とがなす角度がゼロのときに最大値を取り、前記線分と前記第1の軸とがなす角度が前記第1の軸と前記第2の軸との間の角度と等しいときに最小値を取るように、前記線分と前記第1の軸とがなす角度を正規化し、正規化された各画素に対応する角度に基づいて前記対象疾患に関する評価結果を求める、
請求項3又は請求項4に記載の内視鏡システム。 - 前記評価結果を、前記画像取得部により取得される通常画像に重ねて表示する表示部を更に備える、
請求項1から請求項5のいずれか一項に記載の内視鏡システム。 - 前記評価結果は、炎症の重症度についての評価結果である、
請求項1から請求項6のいずれか一項に記載の内視鏡システム。 - 前記第1の軸は、ヘモグロビン色素と相関の高い軸である、
請求項3から請求項5、請求項3を引用する請求項6、請求項3を引用する請求項7、のいずれか一項に記載の内視鏡システム。 - 前記第2の軸は、体腔内の粘膜の色味と相関の高い軸である、
請求項3から請求項5、請求項3を引用する請求項6、請求項3を引用する請求項7、請求項8、のいずれか一項に記載の内視鏡システム。 - 前記評価結果を、前記画像取得部により取得される通常画像に重ねて表示する表示部を更に備え、
前記評価部は、複数の対象疾患に関する評価結果を夫々求め、
前記色平面と、前記基準点と、前記基準軸とは、前記複数の対象疾患毎に設定され、
前記表示部は、前記複数の評価結果を識別可能に前記通常画像に重ねて表示する、
請求項1から請求項6のいずれか一項に記載の内視鏡システム。 - R(Red)、G(Green)、B(Blue)の各色成分を持つカラー画像を構成する各画素に対応する画素対応点を、その色成分に応じて、R成分の軸である第一の軸と、該第一の軸と直交するG成分又はB成分の軸である第二の軸とを含む平面内に配置する配置手段と、
前記平面内において重症度の最も高い病変部と相関の高い第三の軸を設定し、各画素について、該第三の軸上に位置する所定の基準点と前記画素対応点とを結ぶ線分と、該第三の軸とがなす角度を算出する角度算出手段と、
算出された各画素に対応する角度に基づいて前記カラー画像に対する所定の評価値を計算する評価値計算手段と、
を備える、
評価値計算装置。 - 前記角度算出手段は、
前記平面内におい重症度の最も低い病変部と相関の高い第四の軸を設定し、該第四の軸と前記第三の軸との交点を前記基準点として、前記各画素に対応する角度を算出する、
請求項11に記載の評価値計算装置。 - 前記第三の軸は、ヘモグロビン色素と相関の高い軸であり、
前記第四の軸は、体腔内の粘膜の色味と相関の高い軸である、
請求項12に記載の評価値計算装置。
Priority Applications (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US15/764,821 US10918270B2 (en) | 2015-09-30 | 2016-09-30 | Endoscope system and evaluation value calculation device |
| CN201680002634.8A CN107072509B (zh) | 2015-09-30 | 2016-09-30 | 内窥镜系统以及评价值计算装置 |
| JP2017503972A JP6125740B1 (ja) | 2015-09-30 | 2016-09-30 | 内視鏡システム及び評価値計算装置 |
| DE112016004454.2T DE112016004454B4 (de) | 2015-09-30 | 2016-09-30 | Endoskopsystem und Bewertungswertberechnungseinrichtung |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2015192431 | 2015-09-30 | ||
| JP2015-192431 | 2015-09-30 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2017057680A1 true WO2017057680A1 (ja) | 2017-04-06 |
Family
ID=58423695
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2016/079024 Ceased WO2017057680A1 (ja) | 2015-09-30 | 2016-09-30 | 内視鏡システム及び評価値計算装置 |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US10918270B2 (ja) |
| JP (1) | JP6125740B1 (ja) |
| CN (1) | CN107072509B (ja) |
| DE (1) | DE112016004454B4 (ja) |
| WO (1) | WO2017057680A1 (ja) |
Cited By (11)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018181156A1 (ja) * | 2017-03-31 | 2018-10-04 | Hoya株式会社 | 電子内視鏡システム |
| WO2019156022A1 (ja) * | 2018-02-07 | 2019-08-15 | Hoya株式会社 | 内視鏡システム |
| WO2019159435A1 (ja) * | 2018-02-13 | 2019-08-22 | Hoya株式会社 | 内視鏡システム |
| WO2020188825A1 (ja) * | 2019-03-20 | 2020-09-24 | Hoya株式会社 | 内視鏡システム |
| JP2020156903A (ja) * | 2019-03-27 | 2020-10-01 | Hoya株式会社 | 内視鏡用プロセッサ、情報処理装置、プログラム、情報処理方法および学習モデルの生成方法 |
| DE112020001788T5 (de) | 2019-04-02 | 2022-01-27 | Hoya Corporation | Elektronisches endoskopsystem und datenverarbeitungsvorrichtung |
| DE112020002107T5 (de) | 2019-04-26 | 2022-03-03 | Hoya Corporation | Elektronisches endoskopsystem und datenverarbeitungsvorrichtung |
| DE112020004617T5 (de) | 2019-09-27 | 2022-06-09 | Hoya Corporation | Endoskopsystem |
| DE112020004678T5 (de) | 2019-09-30 | 2022-06-15 | Hoya Corporation | Elektronisches endoskopsystem |
| US11944261B2 (en) | 2018-09-27 | 2024-04-02 | Hoya Corporation | Electronic endoscope system and data processing device |
| US12029385B2 (en) | 2018-09-27 | 2024-07-09 | Hoya Corporation | Electronic endoscope system |
Families Citing this family (5)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| DE112017000025B4 (de) | 2016-03-03 | 2025-02-20 | Hoya Corporation | Verfahren zur Erzeugung von Korrekturdaten und Vorrichtung zur Erzeugung von Korrekturdaten |
| US11961224B2 (en) * | 2019-01-04 | 2024-04-16 | Stella Surgical | Device for the qualitative evaluation of human organs |
| CN111489325B (zh) * | 2019-12-16 | 2023-07-28 | 杭州电子科技大学 | 一种甲亢疾病即时鉴定装置及其控制方法 |
| WO2021141048A1 (ja) | 2020-01-07 | 2021-07-15 | Hoya株式会社 | 内視鏡システム、プロセッサ装置、診断支援方法、及びコンピュータプログラム |
| EP4218535A4 (en) * | 2020-09-25 | 2025-03-12 | Hoya Corporation | ENDOSCOPE PROCESSOR AND ENDOSCOPE SYSTEM |
Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010187756A (ja) * | 2009-02-16 | 2010-09-02 | Olympus Corp | 画像処理装置、画像処理方法および画像処理プログラム |
| JP2014018332A (ja) * | 2012-07-17 | 2014-02-03 | Hoya Corp | 画像処理装置及び内視鏡装置 |
| JP2014213094A (ja) * | 2013-04-26 | 2014-11-17 | Hoya株式会社 | 病変評価情報生成装置 |
Family Cites Families (22)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP3716466B2 (ja) * | 1995-10-20 | 2005-11-16 | ソニー株式会社 | 画像処理装置 |
| JP2000330858A (ja) * | 1999-05-25 | 2000-11-30 | Fujitsu Ltd | 画像処理装置およびプログラム記憶媒体 |
| JP4534340B2 (ja) * | 2000-10-31 | 2010-09-01 | ソニー株式会社 | 色再現補正装置 |
| US7042488B2 (en) * | 2001-09-27 | 2006-05-09 | Fujinon Corporation | Electronic endoscope for highlighting blood vessel |
| JP4375781B2 (ja) * | 2002-11-29 | 2009-12-02 | 株式会社リコー | 画像処理装置および画像処理方法並びにプログラムおよび記録媒体 |
| JP4615963B2 (ja) * | 2004-10-29 | 2011-01-19 | オリンパス株式会社 | カプセル型内視鏡装置 |
| JP4241643B2 (ja) * | 2005-02-28 | 2009-03-18 | 村田機械株式会社 | 色判定装置及び色判定方法 |
| EP1938274A2 (en) * | 2005-09-12 | 2008-07-02 | D.V.P. Technologies Ltd. | Medical image processing |
| JP5121204B2 (ja) * | 2006-10-11 | 2013-01-16 | オリンパス株式会社 | 画像処理装置、画像処理方法、および画像処理プログラム |
| JP5006759B2 (ja) | 2007-10-29 | 2012-08-22 | Hoya株式会社 | 電子内視鏡用信号処理装置および電子内視鏡装置 |
| WO2010019515A2 (en) * | 2008-08-10 | 2010-02-18 | Board Of Regents, The University Of Texas System | Digital light processing hyperspectral imaging apparatus |
| JP5380973B2 (ja) * | 2008-09-25 | 2014-01-08 | 株式会社ニコン | 画像処理装置及び画像処理プログラム |
| WO2010122884A1 (ja) * | 2009-04-21 | 2010-10-28 | オリンパスメディカルシステムズ株式会社 | 蛍光画像装置及び蛍光画像取得方法 |
| EP2287807A1 (en) * | 2009-07-21 | 2011-02-23 | Nikon Corporation | Image processing device, image processing program, and imaging device |
| JP5721940B2 (ja) * | 2009-11-04 | 2015-05-20 | オリンパス株式会社 | 光スペクトル検出方法 |
| JP5802364B2 (ja) * | 2009-11-13 | 2015-10-28 | オリンパス株式会社 | 画像処理装置、電子機器、内視鏡システム及びプログラム |
| JP5303012B2 (ja) * | 2010-12-15 | 2013-10-02 | 富士フイルム株式会社 | 内視鏡システム、内視鏡システムのプロセッサ装置及び内視鏡システムの作動方法 |
| JP5851160B2 (ja) * | 2011-08-31 | 2016-02-03 | オリンパス株式会社 | 画像処理装置、画像処理装置の作動方法、及び画像処理プログラム |
| JP6027803B2 (ja) * | 2012-07-17 | 2016-11-16 | Hoya株式会社 | 画像処理装置及び内視鏡装置 |
| EP2979618B1 (en) * | 2013-03-27 | 2018-11-28 | FUJIFILM Corporation | Image processing device for operating endoscope system |
| JP6140056B2 (ja) * | 2013-09-26 | 2017-05-31 | 富士フイルム株式会社 | 内視鏡システム、内視鏡システムのプロセッサ装置、内視鏡システムの作動方法、プロセッサ装置の作動方法 |
| CN108109134B (zh) * | 2013-10-28 | 2021-12-03 | 富士胶片株式会社 | 图像处理装置及其工作方法 |
-
2016
- 2016-09-30 CN CN201680002634.8A patent/CN107072509B/zh active Active
- 2016-09-30 WO PCT/JP2016/079024 patent/WO2017057680A1/ja not_active Ceased
- 2016-09-30 DE DE112016004454.2T patent/DE112016004454B4/de active Active
- 2016-09-30 US US15/764,821 patent/US10918270B2/en active Active
- 2016-09-30 JP JP2017503972A patent/JP6125740B1/ja active Active
Patent Citations (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2010187756A (ja) * | 2009-02-16 | 2010-09-02 | Olympus Corp | 画像処理装置、画像処理方法および画像処理プログラム |
| JP2014018332A (ja) * | 2012-07-17 | 2014-02-03 | Hoya Corp | 画像処理装置及び内視鏡装置 |
| JP2014213094A (ja) * | 2013-04-26 | 2014-11-17 | Hoya株式会社 | 病変評価情報生成装置 |
Cited By (26)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| WO2018181156A1 (ja) * | 2017-03-31 | 2018-10-04 | Hoya株式会社 | 電子内視鏡システム |
| US10939801B2 (en) | 2017-03-31 | 2021-03-09 | Hoya Corporation | Electronic endoscope system |
| WO2019156022A1 (ja) * | 2018-02-07 | 2019-08-15 | Hoya株式会社 | 内視鏡システム |
| US11521319B2 (en) | 2018-02-07 | 2022-12-06 | Hoya Corporation | Endoscope system |
| JPWO2019156022A1 (ja) * | 2018-02-07 | 2020-12-17 | Hoya株式会社 | 内視鏡システム |
| US11436728B2 (en) | 2018-02-13 | 2022-09-06 | Hoya Corporation | Endoscope system |
| WO2019159435A1 (ja) * | 2018-02-13 | 2019-08-22 | Hoya株式会社 | 内視鏡システム |
| WO2019159770A1 (ja) * | 2018-02-13 | 2019-08-22 | Hoya株式会社 | 内視鏡システム |
| JPWO2019159770A1 (ja) * | 2018-02-13 | 2020-12-17 | Hoya株式会社 | 内視鏡システム |
| US11944261B2 (en) | 2018-09-27 | 2024-04-02 | Hoya Corporation | Electronic endoscope system and data processing device |
| US12029385B2 (en) | 2018-09-27 | 2024-07-09 | Hoya Corporation | Electronic endoscope system |
| WO2020188825A1 (ja) * | 2019-03-20 | 2020-09-24 | Hoya株式会社 | 内視鏡システム |
| JPWO2020188825A1 (ja) * | 2019-03-20 | 2021-09-13 | Hoya株式会社 | 内視鏡システム |
| US11612310B2 (en) | 2019-03-20 | 2023-03-28 | Hoya Corporation | Endoscope system |
| JP7041318B2 (ja) | 2019-03-20 | 2022-03-23 | Hoya株式会社 | 内視鏡システム |
| US12022991B2 (en) | 2019-03-27 | 2024-07-02 | Hoya Corporation | Endoscope processor, information processing device, and program |
| JP2020156903A (ja) * | 2019-03-27 | 2020-10-01 | Hoya株式会社 | 内視鏡用プロセッサ、情報処理装置、プログラム、情報処理方法および学習モデルの生成方法 |
| JP7555181B2 (ja) | 2019-03-27 | 2024-09-24 | Hoya株式会社 | 内視鏡用プロセッサ、情報処理装置、プログラム、情報処理方法および学習モデルの生成方法 |
| US12011142B2 (en) | 2019-04-02 | 2024-06-18 | Hoya Corporation | Electronic endoscope system |
| DE112020001788T5 (de) | 2019-04-02 | 2022-01-27 | Hoya Corporation | Elektronisches endoskopsystem und datenverarbeitungsvorrichtung |
| US11950762B2 (en) | 2019-04-26 | 2024-04-09 | Hoya Corporation | Electronic endoscope system and data processing device |
| DE112020002107T5 (de) | 2019-04-26 | 2022-03-03 | Hoya Corporation | Elektronisches endoskopsystem und datenverarbeitungsvorrichtung |
| DE112020004617T5 (de) | 2019-09-27 | 2022-06-09 | Hoya Corporation | Endoskopsystem |
| US12053146B2 (en) | 2019-09-27 | 2024-08-06 | Hoya Corporation | Endoscope system |
| DE112020004678T5 (de) | 2019-09-30 | 2022-06-15 | Hoya Corporation | Elektronisches endoskopsystem |
| US12108929B2 (en) | 2019-09-30 | 2024-10-08 | Hoya Corporation | Electronic endoscope system |
Also Published As
| Publication number | Publication date |
|---|---|
| CN107072509A (zh) | 2017-08-18 |
| DE112016004454T5 (de) | 2018-06-14 |
| DE112016004454B4 (de) | 2019-04-25 |
| JP6125740B1 (ja) | 2017-05-10 |
| JPWO2017057680A1 (ja) | 2017-10-05 |
| US20180279866A1 (en) | 2018-10-04 |
| US10918270B2 (en) | 2021-02-16 |
| CN107072509B (zh) | 2019-03-19 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| JP6125740B1 (ja) | 内視鏡システム及び評価値計算装置 | |
| US11701032B2 (en) | Electronic endoscope processor and electronic endoscopic system | |
| US20230112696A1 (en) | Evaluation value calculation device and electronic endoscope system | |
| US20230113382A1 (en) | Evaluation value calculation device and electronic endoscope system | |
| JP6427280B2 (ja) | 補正データ生成方法及び補正データ生成装置 | |
| JP6591688B2 (ja) | 電子内視鏡用プロセッサ及び電子内視鏡システム | |
| JP6433634B2 (ja) | 電子内視鏡用プロセッサ及び電子内視鏡システム | |
| JP6420358B2 (ja) | 内視鏡システム及び評価値計算装置 | |
| JP6926242B2 (ja) | 電子内視鏡用プロセッサ及び電子内視鏡システム | |
| JP2018023497A (ja) | 色補正用治具及び電子内視鏡システム |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| ENP | Entry into the national phase |
Ref document number: 2017503972 Country of ref document: JP Kind code of ref document: A |
|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 16851849 Country of ref document: EP Kind code of ref document: A1 |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 15764821 Country of ref document: US |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 112016004454 Country of ref document: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 16851849 Country of ref document: EP Kind code of ref document: A1 |