[go: up one dir, main page]

WO2024188159A1 - Procédé d'imagerie et d'affichage pour système d'endoscope, et système d'endoscope - Google Patents

Procédé d'imagerie et d'affichage pour système d'endoscope, et système d'endoscope Download PDF

Info

Publication number
WO2024188159A1
WO2024188159A1 PCT/CN2024/080633 CN2024080633W WO2024188159A1 WO 2024188159 A1 WO2024188159 A1 WO 2024188159A1 CN 2024080633 W CN2024080633 W CN 2024080633W WO 2024188159 A1 WO2024188159 A1 WO 2024188159A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
endoscopic
display area
display
endoscopic image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/080633
Other languages
English (en)
Chinese (zh)
Inventor
李洋
潘维枫
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Mindray Bio Medical Scientific Co Ltd
Shenzhen Mindray Bio Medical Electronics Co Ltd
Original Assignee
Wuhan Mindray Bio Medical Scientific Co Ltd
Shenzhen Mindray Bio Medical Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Mindray Bio Medical Scientific Co Ltd, Shenzhen Mindray Bio Medical Electronics Co Ltd filed Critical Wuhan Mindray Bio Medical Scientific Co Ltd
Publication of WO2024188159A1 publication Critical patent/WO2024188159A1/fr
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • A61B1/0005—Display arrangement combining images e.g. side-by-side, superimposed or tiled
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00163—Optical arrangements
    • A61B1/00193—Optical arrangements adapted for stereoscopic vision
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/043—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances for fluorescence imaging
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0661—Endoscope light sources

Definitions

  • the present invention relates to the field of medical equipment, and more particularly to an imaging and display method for an endoscope system and an endoscope system.
  • Endoscopes can present the tissue morphology of a patient's internal organs and the pathological conditions in the body during minimally invasive surgery, facilitating diagnosis and implementation of surgery. They are one of the important tools for diagnosis and treatment in modern medicine.
  • Traditional endoscopes only provide two-dimensional images, and doctors can only rely on experience to judge the depth information of various organs in the human body cavity during surgery, so they are only suitable for simple operations.
  • stereo endoscopes that can observe areas in three dimensions have been developed. Compared with traditional endoscopes that can provide three-dimensional information, they can better reflect the real situation of the scene, allowing doctors to feel the visual effects of three-dimensional images during surgery, providing realistic visual experience, and helping doctors to operate endoscopes more accurately.
  • Endoscopic imaging systems are usually capable of both visible light imaging and fluorescence imaging.
  • Visible light imaging is imaging the tissue of the part to be observed under visible light irradiation
  • fluorescence imaging is imaging the fluorescence generated by the excited fluorescent reagent injected into the tissue.
  • Stereoscopic endoscopes can only display stereoscopic visible light images or stereoscopic fluorescence images. In stereoscopic display mode, users cannot view endoscopic images of different image types at the same time.
  • the present application provides an imaging and display method for an endoscope system, including:
  • the endoscopic image at least includes a first endoscopic image of a first image type, the first endoscopic image is a stereoscopic endoscopic image, and the first endoscopic image is generated based on the left-path image data and the right-path image data;
  • the at least two endoscopic images of different image types are respectively displayed simultaneously in at least two display areas of the display interface, wherein the display areas at least include a first display area, and the first endoscopic image is displayed in the first display area.
  • the endoscopic image further includes a second endoscopic image of a second image type, the second endoscopic image is a stereoscopic endoscopic image, and the second endoscopic image is generated based on the left image data and the right image data;
  • the display area also includes a second display area, and the second endoscopic image is displayed in the second display area.
  • an area of the first display region is greater than an area of the second display region.
  • the method further includes: performing parallax adjustment on the first endoscopic image to reduce the parallax of the first endoscopic image; or performing parallax adjustment on the second endoscopic image to increase the parallax of the second endoscopic image.
  • the endoscopic image also includes a third endoscopic image of a third image type and a fourth endoscopic image of a fourth image type, the third endoscopic image and the fourth endoscopic image are both stereoscopic images, the third endoscopic image is generated based on the left image data and the right image data, and the fourth endoscopic image is generated based on the left image data and the right image data.
  • the display area also includes a third display area and a fourth display area, the third endoscopic image is displayed in the third display area, the fourth endoscopic image is displayed in the fourth display area, and the area of the first display area is larger than the areas of the second display area, the third display area, and the fourth display area.
  • the first display area has a first area
  • the second display area The third display area and the fourth display area both have a second area, and the first area is larger than the second area.
  • the method further includes: performing parallax adjustment on the first endoscopic image to reduce the parallax of the first endoscopic image; or, performing parallax adjustment on the second endoscopic image, the third endoscopic image and the fourth endoscopic image respectively to increase the parallax of the second endoscopic image, the third endoscopic image and the fourth endoscopic image.
  • the image type of at least one of the first endoscopic image, the second endoscopic image, the third endoscopic image and the fourth endoscopic image is a monochrome fluorescence image or a color fluorescence image generated based on the fluorescence image signal, or is a monochrome fusion image or a color fusion image generated based on the visible light image signal and the fluorescence image signal.
  • the endoscopic image further comprises a third endoscopic image of a third image type, the third endoscopic image being a two-dimensional endoscopic image;
  • the display area further includes a third display area in which the third endoscopic image is displayed.
  • one or more of the endoscopic images of other image types are two-dimensional endoscopic images.
  • the area of the first display region is larger than that of other display regions.
  • the at least two display regions have the same area.
  • the method further comprises:
  • endoscopic images of different image types displayed in the different display areas are switched for display.
  • the method further comprises:
  • An external image from an external image source is acquired, and the external image is displayed in a display area of the display interface.
  • the method further comprises:
  • the other medical equipment includes at least one of an ultrasound device, a monitoring device, and a ventilation device.
  • the medical image includes at least one of an ultrasound examination image of an ultrasound device, a patient physiological parameter waveform of a monitoring device, and a breathing control parameter waveform of a ventilation device.
  • the present application also provides an endoscope system, including an endoscope, a light source, and a connection
  • the endoscope comprises an insertion portion and an operation portion, wherein the insertion portion is used to be inserted into a patient's part to be observed;
  • the light source is used to provide illumination to the part to be observed
  • the endoscope is provided with at least one image sensor, and the at least one image sensor is used to collect visible light image signals and fluorescent image signals of the part to be observed;
  • the camera host is used to obtain the visible light image signal and the fluorescent image signal from the at least one image sensor to perform the imaging and display method for an endoscope system in any of the above embodiments, and obtain at least two endoscopic images of different image types;
  • a display is used to display the at least two endoscopic images of different image types.
  • the present application also provides another imaging and display method for an endoscope system, including:
  • the endoscopic images at least include a first endoscopic image of a first image type, the first endoscopic image is a stereoscopic endoscopic image, the first endoscopic image is generated based on left-path image data and right-path image data, and there is parallax between the left-path image data and the right-path image data;
  • the at least two endoscopic images of different image types are respectively displayed simultaneously in at least two display areas of the display interface, wherein the display areas at least include a first display area, and the first endoscopic image is displayed in the first display area.
  • the imaging and display method for an endoscope system and the endoscope system according to the embodiments of the present invention can simultaneously display at least two endoscopic images of different image types, thereby facilitating doctors to observe between different types of endoscopic images and providing a stereoscopic visual effect.
  • FIG1 is a schematic block diagram of an endoscope system according to an embodiment of the present invention.
  • FIG. 2 is a schematic diagram showing an imaging and display method for an endoscope system according to an embodiment of the present invention.
  • Sexual flow chart
  • FIG3 is a schematic diagram showing a display interface according to an embodiment of the present invention.
  • 4A-4C are schematic diagrams showing a display interface layout according to an embodiment of the present invention.
  • 5A and 5B are schematic diagrams showing a display interface layout according to another embodiment of the present invention.
  • FIG6 is a schematic diagram showing the parallax sizes of different display areas according to an embodiment of the present invention.
  • FIG. 7 is a schematic diagram showing a principle of parallax adjustment according to an embodiment of the present invention.
  • FIG. 1 shows a schematic structural block diagram of an endoscope system 100 according to an embodiment of the present invention.
  • the endoscope system 100 includes at least an endoscope and a camera host 150 connected to the endoscope.
  • the endoscope includes an insertion portion 130 and an operating portion 160.
  • the insertion portion 130 is used to be inserted into the patient's part to be observed.
  • the insertion portion 130 and the operating portion 160 can be an integral structure or a detachable structure.
  • the endoscope also includes at least one image sensor (not shown). Exemplarily, the image sensor can be arranged at the front end of the insertion portion 130 of the endoscope.
  • the camera host 150 is used to obtain an image signal from the endoscope to perform signal processing on it.
  • the endoscope system 100 also includes a light source 110, a light guide 120, a cable 140, and the light source 110 is connected to the endoscope through the light guide 120.
  • the operating portion 160 is connected to the camera host 150 through the cable 140, and is connected to the light source 110 through the light guide 120.
  • the light source 110 is used to provide an illumination light source to the part to be observed.
  • the light source 110 may include a visible light source and a special light source.
  • the visible light source is an LED light source, which can provide multiple monochromatic lights of different wavelength ranges, a combination of multiple monochromatic lights, or a wide-spectrum white light source.
  • the special light source can be a laser light source corresponding to a fluorescent agent, such as near-infrared light.
  • a fluorescent agent is injected into the part to be observed, and the fluorescent agent can generate fluorescence after absorbing the laser generated by the laser light source.
  • the insertion part 130 of the endoscope includes a mirror tube, an image sensor and an illumination light path.
  • the front end of the mirror tube is used to be inserted into the human body and penetrate into the part to be inspected.
  • the illumination light path is connected to the light guide 120, and is used to irradiate the light generated by the light source 110 to the part to be inspected of the target object.
  • the image sensor can specifically include a first image sensor and a second image sensor, which are used to convert optical signals into electrical signals, including but not limited to CCD sensors, CMOS sensors, etc.
  • the image signal collected by the image sensor is sent to the camera host 150 for subsequent image processing after preliminary signal processing by the operating part 160, and the preliminary signal processing includes amplification, filtering and other processing.
  • the optical axes of the first image sensor and the second image sensor can be set in parallel or at a certain angle.
  • the left image signal and the right image signal collected by the first image sensor and the second image sensor can correspond to the stereo pair image observed by the left and right eyes of the human body, thereby simulating the binocular stereoscopic vision of the human eye.
  • the other end of the operation unit 160 is connected to the camera host 150 through the cable 140, and transmits the image signal to the camera host 150 for processing through the cable 140.
  • the operation unit 160 can also send the image data to the camera host 150 by wireless transmission.
  • the camera host 150 is provided with a processor, which obtains the image signal output by the operation unit 160, processes the image signal, and outputs the processed image data.
  • the endoscope system 100 further includes a display 170 , and the camera host 150 is connected to the display 170 via a video connection line, so as to send the endoscopic image to the display 170 for display.
  • FIG1 is merely an example of the endoscope system 100 and does not constitute a limitation on the endoscope system 100.
  • the endoscope system 100 may include more or fewer components than those shown in FIG1, or a combination of certain components, or different components.
  • the endoscope system 100 may also include a dilator, a smoke control device, input and output devices, a network access device, etc.
  • FIG2 is a schematic flow chart of an imaging and display method 200 for an endoscope system according to an embodiment of the present invention, which specifically includes the following steps:
  • step S210 a visible light image signal and a fluorescent image signal of a part to be observed are acquired.
  • step S220 at least two endoscopic images of different image types are generated based on the visible light image signal and the fluorescent image signal, wherein the endoscopic image at least includes a first endoscopic image of a first image type, the first endoscopic image is a stereoscopic endoscopic image, and the first endoscopic image is generated based on left-path image data and right-path image data, and there is parallax between the left-path image data and the right-path image data.
  • the "image type" described in the embodiments of the present application refers to a visible light image, a fluorescent image, or a type of image obtained by fusing the two, and these image types can be stereoscopic images or two-dimensional images.
  • step S230 the at least two endoscopic images of different image types are respectively displayed simultaneously in at least two display areas of the display interface, wherein the display areas include at least a first display area, and the first endoscopic image is displayed in the first display area.
  • the imaging and display method 200 of the endoscope system of the embodiment of the present invention can simultaneously display endoscopic images of different image types, and display endoscopic images of at least one image type as a stereoscopic endoscopic image, so that the user can obtain a stereoscopic visual effect while viewing multiple types of endoscopic images.
  • the endoscopic system achieves a stereoscopic imaging effect based on the binocular parallax principle.
  • the binocular parallax principle means that when the human eye observes an object, the object is projected onto the retina of the left and right eyes respectively, forming two images with parallax.
  • the human brain can automatically calculate the depth information of the object based on the parallax, thereby forming a stereoscopic visual effect.
  • the display of the endoscope system alternately or simultaneously displays the left image and the right image with horizontal parallax, and uses the corresponding image separation technology to make the left eye image and the right eye image with horizontal parallax enter the human eye respectively.
  • the stereoscopic endoscope image in the embodiment of the present invention refers to an image that can make the observer produce a stereoscopic visual effect, which essentially includes a left-path image output to the left eye and a right-path image output to the right eye.
  • Dual monitor output includes helmet-mounted monitors, dual-screen monitors, etc., so that the observer's left and right eyes can view the left and right images displayed on different monitors respectively.
  • Single monitor output mainly includes active shutter stereo display mode, passive polarization stereo display mode, etc. Active shutter stereo display mode needs to be used in conjunction with liquid crystal light valve glasses, and passive polarization stereo display mode needs to be used in conjunction with polarization glasses. Both of them realize left and right eye image separation through the cooperation of monitors and glasses.
  • a visible light image signal and a fluorescent image signal of the part to be observed are acquired.
  • the visible light image signal includes a left visible light image signal and a right visible light image signal
  • the fluorescent image signal includes a left fluorescent image signal and a right fluorescent image signal.
  • the left visible light image signal and the left fluorescent image signal may be acquired by the same image sensor or by different image sensors; similarly, the right visible light image signal and the right fluorescent image signal may be acquired by the same image sensor or by different image sensors.
  • the image sensor for acquiring visible light image signals and fluorescent image signals may be disposed at the front end of the endoscope insertion portion. When the endoscope insertion portion is inserted into the patient's part to be observed, the image sensor approaches the part to be observed and acquires the visible light image signal and the fluorescent image signal of the part to be observed.
  • the light source of the endoscopic system can provide multiple monochromatic lights of different wavelength ranges in time-sharing, and the provided monochromatic light can be blue light, green light, red light, etc.; or the light source can also provide a combination of multiple monochromatic lights, or a wide-spectrum white light source.
  • the light source provides a narrow-band laser to excite the fluorescent agent to produce fluorescence.
  • a fluorescent agent is injected into the part to be observed.
  • special light illumination for exciting the fluorescent agent can be provided to the part to be observed, so that the fluorescent agent is excited to produce fluorescence; the part to be observed includes but is not limited to the blood circulation system, lymphatic system and tumor tissue, etc.
  • step S220 at least two endoscopic images of different image types are generated based on the visible light image signal and the fluorescent image signal, that is, in the at least two endoscopic images of different image types, the endoscopic image of each image type can be generated based on the visible light image signal alone, can be generated based on the fluorescent image signal alone, or can be generated based on both the visible light image signal and the fluorescent image signal.
  • Endoscopic images of different image types can provide different tissue-related information.
  • fluorescent images can show high contrast between diseased tissues such as tumor cells and normal tissue images, so that doctors can observe information such as tumor size and contour to achieve the purpose of precise treatment.
  • the at least two endoscopic images of different image types include at least two of the following: a visible light image generated based on a visible light image signal, a fluorescent image generated based on a fluorescent image signal, and a fused image generated based on a visible light image signal and a fluorescent image signal.
  • the visible light image may be a color RGB image or a grayscale image.
  • the fluorescence image may be a grayscale image or a monochrome image, or a color image obtained according to the mapping relationship between signal intensity and color value.
  • a fused image may be generated based on the visible light image and the fluorescence image. Specifically, the visible light image and the fluorescence image may be fused to obtain a fused image, which contains both the information of the visible light image and the information of the fluorescence image.
  • the fused image may be generated by superimposing the layers of the visible light image and the fluorescence image.
  • the fluorescence image may be processed to have a certain degree of transparency, and then the fluorescence image may be superimposed on the visible light image to form a fused image.
  • each pixel point on each fused image may be synthesized according to the pixel value of each pixel point on the visible light image and the pixel value of the corresponding pixel point in the fluorescence image.
  • certain weights may be assigned to the visible light image and the fluorescence image respectively, and the sum of the weights is 1.
  • the pixel values of each pixel point of the two are weighted summed to obtain the pixel value of each pixel point of the fused image.
  • the proportion of visible light image information and fluorescent image information in the information reflected by the fused image can be adjusted.
  • At least two endoscopic images of different image types may include different types of fused images.
  • a monochrome fused image may be obtained by fusing a monochrome fluorescent image (including but not limited to a green fluorescent image) with a visible light image;
  • a color fused image may be obtained by fusing a color fluorescent image with a visible light image.
  • at least two endoscopic images of different image types include a monochrome fused image displayed in a first display area 310, a visible light image 320 displayed in a second display area 320, a color fused image displayed in a third display area 330, and a fluorescent image displayed in a fourth display area 340.
  • the endoscopic image of the embodiment of the present invention includes at least one type of stereoscopic endoscopic image. That is, the at least two types of endoscopic images include at least a first endoscopic image of a first type, the first endoscopic image is a stereoscopic endoscopic image, the first endoscopic image is generated based on left-path image data and right-path image data, and there is parallax between the left-path image data and the right-path image data.
  • the first endoscopic image may refer to any one of a visible light image, a fluorescence image, or a fused image. indivual.
  • the left image data can be generated based on the first image signal collected by the first image sensor, and the right image data can be generated based on the second image signal collected by the second image sensor. Since the first image sensor and the second image sensor correspond to different viewing angles, there is parallax between the left image data and the right image data.
  • a three-dimensional simulation can be performed based on at least one of the first image signal and the second image signal to obtain the left image data and the right image data with parallax.
  • the endoscopic image may be scaled, cropped, etc. to make the endoscopic image fit the size of the display area.
  • step S230 the at least two endoscopic images of different image types are displayed simultaneously in at least two display areas of the display interface.
  • the display area includes at least a first display area, and the first endoscopic image is displayed in the first display area, that is, at least the first display area in the display interface is used to display a stereoscopic endoscopic image.
  • different display areas can be seamlessly spliced to make the layout of the display interface more compact; or, different display areas can be set with a certain interval.
  • the position and size of each display area can be fixed, or the user can be allowed to adjust it to a certain extent.
  • At least one of the at least two display areas is a main display area with a larger area
  • the other display areas are secondary display areas, whose areas are smaller than those of the main display area, and the areas of the various secondary display areas may be equal or unequal.
  • the first display area 310 is the main display area, which is dominant in the display interface and can provide more details for the user
  • the second display area 320 , the third display area 330 , and the fourth display area 340 are secondary display areas, which are mainly for users to refer to.
  • the second display area 320 , the third display area 330 , and the fourth display area 340 may be displayed on the same side of the first display area 310 , making it easier for users to view without frequently switching their sights.
  • the main display area of the embodiment of the present invention is not limited to the first display area, that is, the endoscopic image displayed in the main display area may be a stereoscopic endoscopic image or a two-dimensional endoscopic image.
  • At least two display areas have the same area, for example, referring to FIG5A , the first display area 510, the second display area 520, the third display area 530, and the fourth display area 540 may divide the display interface into four equal parts.
  • the areas of the display areas are the same, it is possible to avoid different parallaxes of the stereoscopic endoscopic images displayed in different display areas.
  • the number of display areas in the display interface may not be limited to four, but may have more or fewer display areas.
  • the display interface includes three display areas, namely, a first display area 510, a second display area 520, and a third display area 530.
  • the number of display areas may be fixed or may be set by the user as required.
  • the display interface of the embodiment of the present invention displays at least two endoscopic images of different image types, that is, in addition to the first endoscopic image, the endoscopic image displayed in the display interface also includes at least a second endoscopic image of a second image type.
  • the second endoscopic image is a stereoscopic endoscopic image, that is, in this embodiment, the display interface displays at least two stereoscopic endoscopic images of different image types. Similar to the first endoscopic image, the second endoscopic image is generated based on left-path image data and right-path image data, and there is parallax between the left-path image data and the right-path image data.
  • the display area also includes a second display area, and the second endoscopic image is displayed in the second display area.
  • the area of the first display area is larger than the area of the second display area, that is, stereoscopic endoscopic images of different image types are displayed in display areas of different areas, wherein the first display area may be a primary display area and the second display area may be a secondary display area.
  • the first endoscopic image displayed in the first display area may be 2880*2160 pixels
  • the second endoscopic image displayed in the second display area may be 960*720 pixels.
  • the stereoscopic visual effect is generated by outputting the left image and the right image with parallax to the left and right eyes of the user respectively, so that the human brain automatically fuses the stereoscopic effect.
  • the first endoscopic image e.g., a stereoscopic monochrome fusion image
  • the second endoscopic image e.g., a stereoscopic visible light image
  • the convergence and adjustment states of the eyes are different.
  • the line of sight frequently rotates between observing the first display area and the second display area, the eyes will easily become tired and
  • the embodiment of the present invention performs parallax adjustment on the first endoscopic image to The parallax of the first endoscopic image is reduced so that the parallax of the first endoscopic image is the same as or similar to the parallax of the second endoscopic image; or the parallax of the second endoscopic image is adjusted to increase the parallax of the second endoscopic image so that the parallax of the second endoscopic image is the same as or similar to the parallax of the first endoscopic image.
  • the parallax ⁇ 1 of the first endoscopic image can be reduced to ⁇ 2, or close to ⁇ 2, by translating the right image data to the left and the left image data to the right.
  • the right image data can be translated to the right and the left image data can be translated to the left, thereby increasing the parallax ⁇ 2 of the second endoscopic image to ⁇ 1, or close to ⁇ 1.
  • the parallax of the first endoscopic image can be reduced and the parallax of the second endoscopic image can be increased at the same time, so that after the parallax adjustment, the parallax of the two is close to the average value of ⁇ 1 and ⁇ 2.
  • the parallax adjustment is explained by taking the first endoscopic image and the second endoscopic image as examples. It can be understood that when more stereoscopic endoscopic images are displayed in the display interface, the parallax adjustment can also be performed based on a similar principle.
  • the display interface also includes a third display area and a fourth display area, the third display area is used to display a third endoscopic image of a third image type (e.g., a stereoscopic color fusion image), and the fourth display area is used to display a fourth endoscopic image of a fourth image type (e.g., a stereoscopic fluorescence image), and the third endoscopic image and the fourth endoscopic image are both stereoscopic images.
  • a third image type e.g., a stereoscopic color fusion image
  • the fourth display area is used to display a fourth endoscopic image of a fourth image type (e.g., a stereoscopic fluorescence image)
  • the third endoscopic image and the fourth endoscopic image are both
  • the third endoscopic image is generated based on the left image data and the right image data, and there is a parallax between the left image data and the right image data
  • the fourth endoscopic image is generated based on the left image data and the right image data, and there is a parallax between the left image data and the right image data.
  • the first display area has a first area
  • the second display area, the third display area, and the fourth display area all have a second area
  • the first area is larger than the second area, that is, the first display area is the main display area
  • the second display area, the third display area, and the fourth display area are secondary display areas.
  • the first endoscopic image displayed in the first display area has a larger parallax
  • the second endoscopic image, the third endoscopic image, and the fourth endoscopic image displayed in the second display area, the third display area, and the fourth display area have a smaller parallax
  • the first endoscopic image may be parallax-adjusted to reduce the parallax of the first endoscopic image; or, the second endoscopic image, the third endoscopic image, and the fourth endoscopic image may be parallax-adjusted to increase the parallax of the second endoscopic image, the third endoscopic image, and the fourth endoscopic image, respectively, so that the first endoscopic image,
  • the second endoscopic image, the third endoscopic image and the fourth endoscopic image are the same or close to each other. Since the second display area, the third display area and the fourth display area have the same area, when the second endoscopic image, the third endoscopic image and the fourth endoscopic image are selected for parallax adjustment, the same parallax adjustment parameter can be used.
  • At least one other display area can be used to display a two-dimensional endoscopic image.
  • at least the third endoscopic image displayed in the third display area can be a two-dimensional endoscopic image. Since a two-dimensional endoscopic image does not have parallax, even if the area of the third display area is not equal to that of the first display area or the second display area, there is no need to adjust the parallax of the third endoscopic image.
  • only the first endoscopic image of the first image type may be a stereoscopic endoscopic image, and one or more of the endoscopic images of other image types may be two-dimensional endoscopic images. In this case, no parallax adjustment is required, and eye discomfort of the user due to different parallax sizes can be avoided.
  • the visible light image, the fluorescence image and the color fused image can all be stereoscopic endoscopic images, that is, all endoscopic images displayed in the display interface are stereoscopic endoscopic images; or, as shown in Figure 4B, the visible light image, the fluorescence image and the color fused image can all be two-dimensional endoscopic images, that is, only the first endoscopic image in the display interface is a stereoscopic endoscopic image, and endoscopic images of other image types are all two-dimensional endoscopic images; or, as shown in Figure 4C, the visible light image, the fluorescence image and the color fused image can include both stereoscopic endoscopic images and two-dimensional endoscopic endoscopic images
  • the image types of endoscopic images displayed in different display areas can also be switched in response to the user's operation instructions, that is, endoscopic images of different image types displayed in different display areas are switched and displayed.
  • the user can switch the type of endoscopic images displayed in the main display area to meet the user's actual clinical needs.
  • the switching method can be to rotate the endoscopic images displayed in each display area in a preset order, or the user can directly specify the endoscopic image displayed in the current display area.
  • the endoscope system can also receive medical images sent by other medical devices through the communication connection between the endoscope system and other medical devices, and display the medical images from other medical devices in the display area of the display interface.
  • Other medical devices include ultrasound equipment, monitoring equipment, ventilation equipment, etc.
  • at least one display area can be allocated to the other.
  • the display area is used to display medical images of other medical devices.
  • the display area can be used to display endoscope images, or no images may be displayed.
  • the medical image includes at least one of an ultrasound examination image of an ultrasound device, a patient physiological parameter waveform of a monitoring device (such as an electrocardiogram signal waveform, a blood pressure waveform, a blood oxygen waveform, etc.), and a respiratory control parameter waveform of a ventilation device (such as a ventilation pressure waveform).
  • a monitoring device such as an electrocardiogram signal waveform, a blood pressure waveform, a blood oxygen waveform, etc.
  • a respiratory control parameter waveform of a ventilation device such as a ventilation pressure waveform
  • the imaging and display method 200 for an endoscope system can simultaneously display at least two endoscopic images of different image types, providing the user with more comprehensive information; at the same time, it can also produce a stereoscopic visual effect, provide a more realistic visual experience, and help the user perform surgery more accurately.
  • an embodiment of the present invention further provides an endoscope system 100, comprising an endoscope, a light source 110 and a camera host 150 connected to the endoscope, wherein the endoscope comprises an insertion portion 130 and an operating portion 160, wherein the insertion portion 130 is used to be inserted into a patient's part to be observed; the light source 110 is used to provide illumination to the part to be observed; the endoscope is provided with at least one image sensor, wherein the at least one image sensor is used to collect visible light image signals and fluorescent image signals of the part to be observed, and the image sensor may be provided at the front end of the insertion portion 130; the camera host 150 is used to obtain visible light image signals and fluorescent image signals from the at least one image sensor, so as to execute the imaging and display method 200 for the endoscope system as described above, and obtain at least two endoscopic images of different image types; the endoscopic imaging system further comprises a display 170, which is used to display at least two endoscopic images of different image types.
  • the specific structure of the endoscope system 100 and the specific steps of the imaging and display method 200 for the endoscope system have been described above and will not be repeated here.
  • the endoscope system 100 of the embodiment of the present invention can simultaneously display at least two endoscopic images of different image types and provide a stereoscopic visual effect.
  • the disclosed devices and methods can be implemented in other ways.
  • the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.
  • the various component embodiments of the present invention may be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof.
  • a microprocessor or a digital signal processor (DSP) may be used in practice to implement some or all of the functions of some modules according to the embodiments of the present invention.
  • DSP digital signal processor
  • the present invention may also be implemented as a device program (e.g., a computer program and a computer program) for executing part or all of the methods described herein.
  • a program implementing the present invention may be stored on a computer readable medium, or may be in the form of one or more signals.
  • Such a signal may be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biophysics (AREA)
  • Physics & Mathematics (AREA)
  • Optics & Photonics (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Signal Processing (AREA)
  • Endoscopes (AREA)

Abstract

L'invention concerne un procédé d'imagerie et d'affichage pour un système d'endoscope, ainsi qu'un système d'endoscope. Le procédé consiste à : acquérir un signal d'image de lumière visible et un signal d'image fluorescente d'une partie à observer ; générer au moins deux images d'endoscope de différents types d'image sur la base du signal d'image de lumière visible et du signal d'image fluorescente, les images d'endoscope comprenant au moins une première image d'endoscope d'un premier type d'image, la première image d'endoscope étant une image d'endoscope tridimensionnelle, la première image d'endoscope étant générée selon des données d'image de trajet gauche et des données d'image de trajet droit, et il existe une parallaxe entre les données d'image de trajet gauche et les données d'image de trajet droit ; et afficher simultanément les au moins deux images d'endoscope de différents types d'image dans au moins deux régions d'affichage d'une interface d'affichage, respectivement, les régions d'affichage comprenant au moins une première région d'affichage, et la première image d'endoscope étant affichée dans la première région d'affichage. La présente invention peut réaliser l'affichage simultané d'au moins deux images d'endoscope de différents types d'image, et fournit un effet visuel tridimensionnel.
PCT/CN2024/080633 2023-03-13 2024-03-07 Procédé d'imagerie et d'affichage pour système d'endoscope, et système d'endoscope Pending WO2024188159A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310239872.2 2023-03-13
CN202310239872.2A CN116158718A (zh) 2023-03-13 2023-03-13 用于内窥镜系统的成像及显示方法和内窥镜系统

Publications (1)

Publication Number Publication Date
WO2024188159A1 true WO2024188159A1 (fr) 2024-09-19

Family

ID=86411524

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/080633 Pending WO2024188159A1 (fr) 2023-03-13 2024-03-07 Procédé d'imagerie et d'affichage pour système d'endoscope, et système d'endoscope

Country Status (2)

Country Link
CN (1) CN116158718A (fr)
WO (1) WO2024188159A1 (fr)

Families Citing this family (3)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116158718A (zh) * 2023-03-13 2023-05-26 武汉迈瑞医疗技术研究院有限公司 用于内窥镜系统的成像及显示方法和内窥镜系统
CN116672000A (zh) * 2023-05-30 2023-09-01 无锡海斯凯尔医学技术有限公司 组织定位的数据处理方法、装置、电子设备
CN117281451A (zh) * 2023-11-14 2023-12-26 杭州显微智能科技有限公司 一种3d内窥镜荧光摄像系统及其成像方法

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003033324A (ja) * 1991-03-11 2003-02-04 Olympus Optical Co Ltd 内視鏡装置
CN104335580A (zh) * 2012-06-14 2015-02-04 奥林巴斯株式会社 图像处理装置和立体图像观察系统
CN107710756A (zh) * 2015-06-24 2018-02-16 索尼奥林巴斯医疗解决方案公司 医用立体观察装置、医用立体观察方法以及程序
CN112040831A (zh) * 2018-05-31 2020-12-04 松下i-PRO传感解决方案株式会社 相机设备、图像处理方法和相机系统
US20210295502A1 (en) * 2020-03-18 2021-09-23 Sony Olympus Medical Solutions Inc. Medical image processing device and medical observation system
CN116158718A (zh) * 2023-03-13 2023-05-26 武汉迈瑞医疗技术研究院有限公司 用于内窥镜系统的成像及显示方法和内窥镜系统

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003033324A (ja) * 1991-03-11 2003-02-04 Olympus Optical Co Ltd 内視鏡装置
CN104335580A (zh) * 2012-06-14 2015-02-04 奥林巴斯株式会社 图像处理装置和立体图像观察系统
CN107710756A (zh) * 2015-06-24 2018-02-16 索尼奥林巴斯医疗解决方案公司 医用立体观察装置、医用立体观察方法以及程序
CN112040831A (zh) * 2018-05-31 2020-12-04 松下i-PRO传感解决方案株式会社 相机设备、图像处理方法和相机系统
US20210295502A1 (en) * 2020-03-18 2021-09-23 Sony Olympus Medical Solutions Inc. Medical image processing device and medical observation system
CN116158718A (zh) * 2023-03-13 2023-05-26 武汉迈瑞医疗技术研究院有限公司 用于内窥镜系统的成像及显示方法和内窥镜系统

Also Published As

Publication number Publication date
CN116158718A (zh) 2023-05-26

Similar Documents

Publication Publication Date Title
WO2024188159A1 (fr) Procédé d'imagerie et d'affichage pour système d'endoscope, et système d'endoscope
US10197803B2 (en) Augmented reality glasses for medical applications and corresponding augmented reality system
US9507155B2 (en) Head-mounted display
US20100245765A1 (en) Video infrared ophthalmoscope
US20140336461A1 (en) Surgical structured light system
JP5893808B2 (ja) 立体内視鏡画像処理装置
CN108430306A (zh) 用于使用激光散斑对比度成像快速检查脉管系统和微粒流的系统和方法
WO2020045015A1 (fr) Système médical, dispositif de traitement d'informations et méthode de traitement d'informations
JP7095693B2 (ja) 医療用観察システム
JP6996518B2 (ja) 分岐光学系、撮像装置、及び撮像システム
WO2024188160A1 (fr) Procédé d'imagerie et d'affichage pour système d'endoscope et système d'endoscope
WO2018230066A1 (fr) Système médical, appareil médical et procédé de commande
WO2011092951A1 (fr) Appareil d'acquisition d'images, appareil d'observation, et système d'observation
CN105208909A (zh) 用于立体显示图像数据的方法和装置
CN111405866A (zh) 用于眼部治疗的沉浸式显示系统
JP2001104331A (ja) 医療用顔面装着型映像表示装置
Getty et al. Clinical applications for stereoscopic 3‐D displays
Hanna et al. Image display technology and image processing
WO2021256168A1 (fr) Système de traitement d'image médicale, dispositif de commande d'image chirurgicale et procédé de commande d'image chirurgicale
CN115919239A (zh) 用于3d内窥镜成像系统的成像方法和3d内窥镜成像系统
JPWO2020045014A1 (ja) 医療システム、情報処理装置及び情報処理方法
CN104887316A (zh) 基于主动立体显示技术的虚拟立体内窥镜显示方法
CN219229809U (zh) 一种电子内窥镜及3d电子内窥镜
CN116327094A (zh) 用于内窥镜系统的成像及显示方法和内窥镜系统
US20190394454A1 (en) Medical-image display control device, medical-image display device, medical-information processing system, and medical-image display control method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24769852

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE