[go: up one dir, main page]

WO2024188160A1 - Procédé d'imagerie et d'affichage pour système d'endoscope et système d'endoscope - Google Patents

Procédé d'imagerie et d'affichage pour système d'endoscope et système d'endoscope Download PDF

Info

Publication number
WO2024188160A1
WO2024188160A1 PCT/CN2024/080634 CN2024080634W WO2024188160A1 WO 2024188160 A1 WO2024188160 A1 WO 2024188160A1 CN 2024080634 W CN2024080634 W CN 2024080634W WO 2024188160 A1 WO2024188160 A1 WO 2024188160A1
Authority
WO
WIPO (PCT)
Prior art keywords
image
display
external
display area
external image
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Pending
Application number
PCT/CN2024/080634
Other languages
English (en)
Chinese (zh)
Inventor
李洋
潘维枫
林路易
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Wuhan Mindray Bio Medical Scientific Co Ltd
Shenzhen Mindray Bio Medical Electronics Co Ltd
Original Assignee
Wuhan Mindray Bio Medical Scientific Co Ltd
Shenzhen Mindray Bio Medical Electronics Co Ltd
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Wuhan Mindray Bio Medical Scientific Co Ltd, Shenzhen Mindray Bio Medical Electronics Co Ltd filed Critical Wuhan Mindray Bio Medical Scientific Co Ltd
Publication of WO2024188160A1 publication Critical patent/WO2024188160A1/fr
Anticipated expiration legal-status Critical
Pending legal-status Critical Current

Links

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
    • A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00002—Operational features of endoscopes
    • A61B1/00043—Operational features of endoscopes provided with output arrangements
    • A61B1/00045—Display arrangement
    • A61B1/0005—Display arrangement combining images e.g. side-by-side, superimposed or tiled
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/00163—Optical arrangements
    • A61B1/00193—Optical arrangements adapted for stereoscopic vision
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/043—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances for fluorescence imaging
    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/06—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor with illuminating arrangements
    • A61B1/0661—Endoscope light sources

Definitions

  • the present invention relates to the field of medical equipment, and more particularly to an imaging and display method for an endoscope system and an endoscope system.
  • Endoscopes can present the tissue morphology of a patient's internal organs and the pathological conditions in the body during minimally invasive surgery, facilitating diagnosis and implementation of surgery. They are one of the important tools for diagnosis and treatment in modern medicine.
  • Traditional endoscopes only provide two-dimensional images, and doctors can only rely on experience to judge the depth information of various organs in the human body cavity during surgery, so they are only suitable for simple operations.
  • stereo endoscopes that can observe areas in three dimensions have been developed. Compared with traditional endoscopes that can provide three-dimensional information, they can better reflect the real situation of the scene, allowing doctors to feel the visual effects of three-dimensional images during surgery, providing realistic visual experience, and helping doctors operate endoscopes more accurately.
  • doctors When doctors use an endoscopic system for examination or surgery, they sometimes need to refer to images captured by other medical devices. Since the images of other medical devices and the endoscopic images captured by the endoscopic system are displayed on different display interfaces, doctors need to frequently switch their views between different display interfaces, affecting the smooth progress of the operation.
  • the present application provides an imaging and display method for an endoscope system, the method comprising:
  • the stereoscopic endoscopic image and the external image are respectively displayed simultaneously in at least two display areas of the display interface.
  • the present application provides an endoscope system, including an endoscope, a light source, a camera host connected to the endoscope, and a display, wherein the endoscope includes an insertion portion and an operation portion, wherein the insertion portion is used to be inserted into a patient's part to be observed;
  • the light source is used to provide illumination to the part to be observed
  • the endoscope is provided with at least one image sensor, and the at least one image sensor is used to collect an endoscopic image signal of the part to be observed;
  • the camera host is used to obtain the endoscope image signal to execute the aforementioned method
  • the display is used to display the stereoscopic endoscopic image and the external image.
  • a stereoscopic endoscopic image and an external image are displayed simultaneously.
  • the stereoscopic endoscopic image can provide a more realistic visual experience, which helps to improve the accuracy of operating the endoscope system;
  • the external image can provide richer information for the user's reference, so that the user does not need to frequently switch his or her line of sight between the endoscope system and the external image source.
  • FIG1 is a schematic block diagram of an endoscope system according to an embodiment of the present invention.
  • FIG2 is a schematic flow chart showing an imaging and display method for an endoscope system according to an embodiment of the present invention
  • 3A-3C are schematic diagrams showing a display interface layout according to an embodiment of the present invention.
  • 4A and 4B are schematic diagrams showing the parallax size of a stereoscopic endoscopic image according to another embodiment of the present invention.
  • FIG. 5 is a schematic diagram showing a principle of parallax adjustment according to an embodiment of the present invention.
  • FIG. 1 shows a schematic structural block diagram of an endoscope system 100 according to an embodiment of the present invention.
  • the endoscope system 100 includes at least an endoscope and a camera host 150 connected to the endoscope.
  • the endoscope includes an insertion portion 130 and an operating portion 160.
  • the insertion portion 130 is used to be inserted into the patient's part to be observed.
  • the insertion portion 130 and the operating portion 160 can be an integral structure or a detachable structure.
  • the endoscope also includes at least one image sensor (not shown). Exemplarily, the image sensor can be arranged at the front end of the insertion portion 130 of the endoscope.
  • the camera host 150 is used to obtain an image signal from the endoscope to perform signal processing on it.
  • the endoscope system 100 also includes a light source 110, a light guide 120, a cable 140, and the light source 110 is connected to the endoscope through the light guide 120.
  • the operating portion 160 is connected to the camera host 150 through the cable 140, and is connected to the light source 110 through the light guide 120.
  • the light source 110 is used to provide an illumination light source to the part to be observed.
  • the light source 110 may include a visible light source and a special light source.
  • the visible light source is an LED light source, which can provide multiple monochromatic lights of different wavelength ranges, a combination of multiple monochromatic lights, or a wide-spectrum white light source.
  • the special light source can be a laser light source corresponding to a fluorescent agent, such as near-infrared light.
  • a fluorescent agent is injected into the part to be observed, and the fluorescent agent can generate fluorescence after absorbing the laser generated by the laser light source.
  • the insertion portion 130 of the endoscope includes a mirror tube, an image sensor and an illumination light path.
  • the front end of the mirror tube is used to be inserted into the human body and penetrate into the part to be inspected.
  • the illumination light path is connected to the light guide 120, and is used to irradiate the light generated by the light source 110 to the part to be inspected of the target object.
  • the image sensor may specifically include a first image sensor and a second image sensor, which are used to convert optical signals into electrical signals, including but not limited to CCD sensors, CMOS sensors, etc.
  • the image signal collected by the image sensor is sent to the camera host 150 for subsequent image processing after preliminary signal processing by the operating unit 160, and the preliminary signal processing includes amplification, filtering and other processing.
  • the optical axes of the first image sensor and the second image sensor can be set in parallel or at a certain angle.
  • the first image signal and the second image signal collected by the first image sensor and the second image sensor can correspond to the stereo pair image observed by the left and right eyes of a person, thereby simulating the binocular stereoscopic vision of the human eye.
  • the other end of the operation unit 160 is connected to the camera host 150 through the cable 140, and transmits the image signal to the camera host 150 for processing through the cable 140.
  • the operation unit 160 can also send the image data to the camera host 150 by wireless transmission.
  • the camera host 150 is provided with a processor, the processor acquires the image signal output by the operation unit 160, processes the image signal, and outputs the processed image data.
  • the endoscope system 100 also includes a display 170, and the camera host 150 is connected to the display 170 via a video connection line, and is used to send the endoscopic image to the display 170 for display.
  • FIG1 is merely an example of the endoscope system 100 and does not constitute a limitation on the endoscope system 100.
  • the endoscope system 100 may include more or fewer components than those shown in FIG1, or a combination of certain components, or different components.
  • the endoscope system 100 may also include a dilator, a smoke control device, input and output devices, a network access device, etc.
  • FIG2 is a schematic flow chart of an imaging and display method 200 for an endoscope system according to an embodiment of the present invention, which specifically includes the following steps:
  • step S210 an endoscopic image signal of the part to be observed is acquired
  • step S220 a first channel of image data and a second channel of image data are generated based on the endoscope image signal, wherein the first channel of image data and the second channel of image data have a parallax;
  • step S230 a stereoscopic endoscopic image is generated based on the first channel of image data and the second channel of image data;
  • step S240 an external image is acquired from an external image source that is communicatively connected to the endoscope system, wherein the external image is a two-dimensional image;
  • step S250 the stereoscopic endoscopic image and the external image are respectively displayed simultaneously in at least two display areas of the display interface.
  • the imaging and display method 200 of the endoscope system of the embodiment of the present invention can simultaneously display a stereoscopic endoscope image and an external image.
  • the stereoscopic endoscope image can provide a more realistic visual experience
  • the external image can provide richer information for the user's reference, so that the user does not need to frequently switch the line of sight between the endoscope system and the external image source.
  • the endoscope system achieves a stereoscopic imaging effect based on the binocular parallax principle.
  • the binocular parallax principle means that when the human eye observes an object, the object is projected onto the retinas of the left and right eyes respectively, forming two images with parallax.
  • the human brain can automatically calculate the depth information of the object based on the parallax, thereby forming a stereoscopic visual effect.
  • the display of the endoscope system alternately or simultaneously displays the left eye image and the right eye image with horizontal parallax, and uses the corresponding image separation technology to make the left eye image and the right eye image with horizontal parallax enter the left and right eyes of the person respectively, so that the human brain automatically fuses the stereoscopic effect and obtains the depth information. Therefore, it can be understood that the stereoscopic endoscopic image of the embodiment of the present invention refers to an image that can produce a stereoscopic visual effect for the observer, which essentially includes two images with horizontal parallax output to the left eye and the right eye.
  • Dual monitor output includes helmet-mounted monitors, dual-screen monitors, etc., so that the observer's left and right eyes can view the left and right images displayed on different monitors respectively.
  • Single monitor output mainly includes active shutter stereo display mode, passive polarization stereo display mode, etc. Active shutter stereo display mode needs to be used in conjunction with liquid crystal light valve glasses, and passive polarization stereo display mode needs to be used in conjunction with polarization glasses. Both of them realize left and right eye image separation through the cooperation of monitors and glasses.
  • the endoscopic image signal of the part to be observed is first acquired through the image sensor of the endoscope system.
  • the part to be observed includes but is not limited to the blood circulation system, the lymphatic system, and tumor tissue.
  • the image sensor for collecting the endoscopic image signal can be arranged at the front end of the endoscope insertion part.
  • the image sensor approaches the part to be observed and collects the endoscopic image signal of the part to be observed.
  • the endoscopic image signal includes a first image signal and a second image signal, and the first image signal and the second image signal can be collected by the first image sensor and the second image sensor arranged at the front end of the endoscope, respectively.
  • the first image signal may include at least one of a first visible light image signal and a first fluorescent image signal
  • the second image signal may include at least one of a second visible light signal and a second fluorescent signal.
  • the first visible light image signal and the first fluorescent image signal may be acquired by the same image sensor or by different image sensors; similarly, the second visible light image signal and the second fluorescent image signal may be acquired by the same image sensor or by different image sensors.
  • the light source of the endoscopic system may provide multiple monochromatic lights of different wavelength ranges in a time-sharing manner, or the light source may also provide a combination of multiple monochromatic lights, or a wide-spectrum white light source.
  • a fluorescent agent is injected into the part to be observed, and during the imaging process, special light illumination for exciting the fluorescent agent may be provided to the part to be observed, so that the fluorescent agent is excited to produce fluorescence.
  • a first channel of image data and a second channel of image data are generated based on the endoscopic image signal.
  • the first channel of image data and the second channel of image data are used to simulate the images observed by the left eye and the right eye of a person, respectively.
  • the first channel of image data can be generated based on the first image signal acquired by the first image sensor
  • the second channel of image data can be generated based on the second image signal acquired by the second image sensor.
  • the first channel of image data may be performed based on at least one of the first image signal and the second image signal to obtain the first image data and the second image data with parallax.
  • the first channel of image data and the second channel of image data may be a visible light image or a fluorescent image.
  • the visible light image may be a color RGB image or a grayscale image.
  • the fluorescent image may be a grayscale image or a monochrome image, or a color image obtained according to the mapping relationship between signal intensity and color value.
  • the first channel of image data and the second channel of image data may also be a fused image generated based on the visible light image and the fluorescent image, and the fused image contains both the information of the visible light image and the information of the fluorescent image.
  • the fused image may be generated by superimposing the layers of the visible light image and the fluorescent image.
  • the fluorescent image may be processed to have a certain degree of transparency, and then the fluorescent image may be superimposed on the visible light image to form a fused image.
  • each pixel on each fused image may be synthesized according to the pixel value of each pixel on the visible light image and the pixel value of the corresponding pixel in the fluorescent image. Specifically, a certain weight may be assigned to each visible light image and the fluorescent image, and the pixel value of each pixel of the two may be weighted summed to obtain the pixel value of each pixel of the fused image.
  • a stereoscopic endoscopic image is generated based on the first channel of image data and the second channel of image data, and the stereoscopic endoscopic image is displayed in the corresponding display area of the display interface.
  • the endoscope system realizes the stereoscopic imaging effect based on the binocular parallax principle, that is, the display of the endoscope system alternately or simultaneously displays the first channel of image data and the second channel of image data with parallax, and uses the corresponding image separation technology to make the two enter the left and right eyes of the person respectively, so that the human brain automatically fuses the stereoscopic effect.
  • the process of generating a stereoscopic endoscopic image based on the first channel of image data and the second channel of image data can refer to the process of combining the first channel of image data and the second channel of image data into a stereoscopic image pair according to the output method of the stereoscopic endoscopic image adopted by the endoscope system.
  • the embodiment of the present invention also obtains an external image from an external image source that is communicatively connected to the endoscope system, and displays the external image and the stereoscopic endoscopic image in different display areas of the same display interface for simultaneous viewing by the user.
  • the external image is a two-dimensional image, that is, the external image received by the left and right eyes of the user is the same external image, and there is no parallax between them.
  • the external image source includes other medical devices that are communicatively connected to the endoscope system.
  • the external image includes a medical image output by other medical devices that are communicatively connected to the endoscope system, and specifically may be a dynamic medical image or a static medical image output by other medical devices.
  • the communication connection mode between the endoscope system and other medical devices includes a wired connection or a wireless connection, which is not limited in the embodiment of the present invention.
  • Other medical devices that are communicatively connected to the endoscope system may include ultrasound devices, and the external images obtained from the ultrasound devices are ultrasound images of the ultrasound devices; the ultrasound images at least include ultrasound images, and the ultrasound images may include grayscale ultrasound images, blood flow images, and contrast images, etc.
  • Stereoscopic endoscope images can present the surface state of tissues to users, and ultrasound images can present information such as the texture structure and blood flow state of the internal tissues to users, and can also present the location of lesions, provide intraoperative guidance, etc. Displaying ultrasound images and stereoscopic endoscope images simultaneously helps to achieve more accurate surgical operations.
  • Other medical devices that are communicatively connected to the endoscope system may also include monitoring equipment, which is used to monitor the patient's vital signs in real time.
  • the external image obtained from the monitoring equipment is the monitoring screen of the monitoring equipment, which displays the monitoring data of the patient's vital sign parameters, including the real-time value and real-time waveform of vital weight parameters, etc.; the patient's vital sign parameters include electrocardiogram, respiration, pulse oxygen saturation, heart rate, blood oxygen, non-invasive blood pressure, invasive blood pressure, etc.
  • the simultaneous display of the monitoring screen and the stereoscopic endoscope image helps the doctor pay attention to the patient's vital sign status during the operation. That is, in some embodiments, the external image should be understood to also include a waveform image.
  • Ventilation equipment Since the patient is usually under anesthesia when the endoscope system is used for surgery, ventilation equipment is needed to provide ventilation support for the patient.
  • the external image obtained from the ventilation equipment is the ventilation screen of the ventilation equipment, which displays the patient's ventilation parameters, including but not limited to tidal volume, respiratory rate, positive end-expiratory pressure, inspired oxygen concentration, lung compliance, airway resistance, etc.
  • the ventilation parameters directly reflect the patient's respiratory status, and the simultaneous display of the ventilation screen and the stereoscopic endoscopic image helps to provide effective respiratory support for the patient.
  • the medical device may also include an electrocardiogram device, and the external image from the electrocardiogram device is an electrocardiogram screen, in which the patient's electrocardiogram waveform is recorded; the medical device may also include an infusion pump, and the external image from the infusion pump is an infusion screen, in which infusion parameters, etc. are recorded.
  • the endoscope system may also establish communication connections with at least two different medical devices at the same time, and obtain external images from at least two different medical devices respectively; external images obtained from different medical devices may be displayed in different display areas of the same display interface.
  • the external image may be an image received from a medical device that is communicatively coupled to the endoscope system.
  • the real-time video image outputted.
  • the stereoscopic endoscopic image and the external image can be frame-synchronized according to the time information carried by the external image and the time information carried by the stereoscopic endoscopic image, and the frame-synchronized stereoscopic endoscopic image and the external image can be displayed, that is, the external image and the stereoscopic endoscopic image at the same time can be synchronously displayed.
  • the external image and the stereoscopic endoscopic image can be time-aligned and displayed according to the timestamp of the external image and the timestamp of the stereoscopic endoscopic system.
  • the external image can be a historical image output by a medical device that is communicatively connected to the endoscope system, for example, it can be an ultrasound image collected by an ultrasound device before surgery.
  • the external image source may be a storage medium such as a USB flash drive or a mobile terminal that is communicatively connected to the endoscope system, and the external image includes a pre-stored image obtained from the storage medium.
  • the image pre-stored in the storage medium may be an image obtained after artificial image processing, specifically a video image or a static image.
  • the pre-stored image may be a tissue model related to the part to be observed, such as a three-dimensional model of the heart; the pre-stored image may also be a lesion spatial position model, which is used to present the spatial position of the lesion in the tissue.
  • tissue model or the lesion spatial position model is a stereoscopic effect obtained by image rendering, but it is essentially still a two-dimensional image, that is, the same image is output to the left and right eyes of the user, and there is no parallax like a stereoscopic endoscopic image.
  • the external image is a black and white image or a grayscale image, such as a grayscale ultrasound image, etc.
  • a black and white image or a grayscale image such as a Doppler ultrasound image or an ultrasound image marked with a specific color, etc.
  • the external image may also be a color image.
  • the polarized stereoscopic display when users view stereoscopic endoscopic images, they need to wear polarized 3D glasses and use a polarized display to observe the 3D images, so that the user's left eye observes the left image and the right eye observes the right image, and finally the human brain's binocular stereoscopic vision mechanism generates a 3D image.
  • the image of the three-dimensional endoscope system needs to pass through two more polarizers to reach the human eye through the display. Since the polarizer has a certain transmittance, the brightness of the image perceived by the human eye will drop significantly after the light passes through the polarizer.
  • the reduction in image brightness is acceptable within a certain range.
  • the embodiment of the present invention processes the external image before displaying it to increase the brightness of the external image to compensate for the need to present the stereoscopic effect of the stereoscopic endoscopic image.
  • the brightness of the external image is reduced.
  • the brightness of the external image can be increased according to a preset brightness increase ratio, or the brightness of the external image can be adaptively increased according to the initial brightness of the external image.
  • Increasing the brightness of the external image can be increasing the overall brightness of the external image, or increasing the brightness of a part of the external image. By increasing the brightness of the external image, it is possible to avoid the user having to take off the 3D glasses to view the external image, and then wearing the 3D glasses again when looking back at the stereoscopic endoscopic image.
  • the brightness of an external image it is possible to determine whether to increase the brightness of an external image based on the type of the external image. For example, for an external image from a certain specific external image source, its brightness is increased.
  • the specific external image source may be an image source that outputs a black and white image or a grayscale image, such as an ultrasound device.
  • the type of the external image may be identified, and when the external image is determined to be a black and white image or a grayscale image, its brightness is increased.
  • the current brightness of the external image may be determined, and when the current brightness of the external image is lower than a preset threshold, its brightness is increased.
  • the brightness of any external image may also be increased.
  • displaying the external image includes: performing image enhancement processing on the external image to display the enhanced external image.
  • Image enhancement can enable the user to better observe the external image and better recognize the content in the external image.
  • displaying an external image includes cropping the external image and displaying the target area image obtained after cropping.
  • the image obtained from the ultrasound device is a real-time display screen of the ultrasound device, which includes, in addition to the ultrasound image, a parameter information display bar, a toolbar and other display contents, while the user of the endoscope system is mainly concerned with the ultrasound image. Therefore, after acquiring the video stream from the ultrasound device, non-target areas such as the parameter information display bar and the toolbar can be cropped, and the ultrasound image can be retained as the target area and displayed in the display interface of the endoscope system.
  • the external image can be scaled to a preset size for display to adapt to the size of the display area used to display the external image, wherein different scaling parameters can be applied to the external image when the external image is displayed in the main display area and in the secondary display area.
  • the above-mentioned image processing such as brightness enhancement, cropping and scaling can be performed in different orders.
  • the external image can be cropped first to obtain the target area in the external image; then the target area is scaled to fit the size of the display area; and then the scaled target area is brightness enhanced.
  • the brightness can be enhanced first, then the external image with enhanced brightness can be cropped, and finally the cropped target area is scaled.
  • displaying the external image includes: identifying a region of interest in the external image according to a preset plan to display the external image and an identification of the region of interest. Lesion area in external image.
  • the area in the stereoscopic endoscopic image corresponding to the area of interest may be further marked, thereby achieving content correspondence between the two images, making it easier for doctors to observe and compare.
  • the display area of the display interface of the embodiment of the present invention includes at least a first display area and a second display area, the stereoscopic endoscopic image is displayed in the first display area, and the external image is displayed in the second display area.
  • the area of the first display area 310 is larger than the area of the second display area, that is, the first display area 310 is the main display area, and the second display area 320 is the secondary display area, and the user can mainly view the information of the stereoscopic endoscopic image.
  • FIG. 3A the area of the first display area 310 is larger than the area of the second display area, that is, the first display area 310 is the main display area, and the second display area 320 is the secondary display area, and the user can mainly view the information of the stereoscopic endoscopic image.
  • the area of the first display area 310 is smaller than the area of the second display area 320, that is, the second display area 320 is the main display area, and the first display area 310 is the secondary display area, and the user can mainly view the information of the external image.
  • the first display area 310 and the second display area 320 may also have the same area.
  • the display area includes a first display area and a second display area, and the area of the first display area is larger than the area of the second display area, that is, the first display area is defined as the main display area.
  • the user can select the image displayed in the first display area as needed, that is, the endoscope system switches the images displayed in the first display area and the second display area in response to the user's operation instructions.
  • the stereoscopic endoscope image is displayed in the first display area with a larger area, and the external image is displayed in the second display area with a smaller area; after receiving the user's operation instruction, it is switched to display the external image in the first display area with a larger area, and the stereoscopic endoscope image is displayed in the second display area with a smaller area.
  • the switchable display mode can better meet the actual clinical needs of the user.
  • the stereoscopic visual effect is generated by outputting the first channel image data and the second channel image data with parallax to the left and right eyes of the user, respectively, so that the human brain automatically fuses the stereoscopic effect.
  • FIG4A and FIG4B when the stereoscopic endoscopic image is displayed in the first display area with a larger area, it has a larger parallax ⁇ 1, and when it is displayed in the second display area with a smaller area, it has a smaller parallax ⁇ 2, so that the stereoscopic effects presented by the first display area and the second display area are inconsistent.
  • the convergence and adjustment states of the eyes are different, and the eyes will easily become tired and feel uncomfortable.
  • the first channel image data and/or the second channel image data are subjected to parallax adjustment according to the first parallax adjustment parameter so that There is a target parallax between the first channel image data and the second channel image data; and/or, when the stereoscopic endoscopic image is displayed in the second display area, the first channel image data and/or the second channel image data are parallax-adjusted according to the second parallax adjustment parameter, so that there is the above-mentioned target parallax between the first channel image data and the second channel image data.
  • the stereoscopic endoscopic image can have the same or similar parallax when displayed in different display areas, avoiding user discomfort and improving user experience.
  • the principle of parallax adjustment of the embodiment of the present invention is shown in FIG5 .
  • the parallax between the first channel of image data and the second channel of image data can be reduced.
  • the parallax between the first channel of image data and the second channel of image data can be increased by translating the second channel of image data to the right and the first channel of image data to the left.
  • the parallax adjustment may not be performed when the stereoscopic endoscopic image is displayed in the first display area, and the original parallax of the first channel of image data and the second channel of image data is the target parallax; when the stereoscopic endoscopic image is switched to be displayed in the second display area, the parallax between the first channel of image data and the second channel of image data is increased, so that the parallax between the two is adjusted to the target parallax.
  • the parallax adjustment may not be performed when the stereoscopic endoscopic image is displayed in the second display area, in which case the original parallax of the first channel image data and the second channel image data is the target parallax; when the stereoscopic endoscopic image is switched to be displayed in the first display area, the parallax between the first channel image data and the second channel image data is reduced, thereby adjusting the parallax between the two to the target parallax.
  • the parallax adjustment may be performed in both cases, and the parallax may be appropriately reduced when the stereoscopic endoscopic image is displayed in the first display area, and the parallax may be appropriately increased when the stereoscopic endoscopic image is displayed in the second display area.
  • the imaging and display method 200 for an endoscope system in an embodiment of the present invention simultaneously displays a stereoscopic endoscope image and an external image.
  • the stereoscopic endoscope image can provide a more realistic visual experience, which helps to improve the accuracy of operating the endoscope system;
  • the external image can provide richer information for the user's reference, so that the user does not need to frequently switch his or her line of sight between the endoscope system and the external image source.
  • an embodiment of the present invention further provides an endoscope system 100, including an endoscope, a light source 110, and a camera host 150 connected to the endoscope, the endoscope including an insertion portion 130 and an operation portion 160, the insertion portion 130 is used to be inserted into the patient's part to be observed; the light source 110 is used to provide illumination to the part to be observed; the endoscope is provided with at least one image sensor, the at least one image sensor is used to collect an endoscopic image signal of the part to be observed, and the image sensor can be provided at the front end of the insertion portion 130; the camera host 150 is used to obtain an endoscopic image signal from the at least one image sensor to perform the above-mentioned
  • the imaging and display method 200 for an endoscope system is used to obtain a stereoscopic endoscope image; the camera host 150 is also used to obtain an external image from an external image source.
  • the endoscope imaging system also includes a display 170 for displaying a stereoscopic endoscope image and an external image
  • the specific structure of the endoscope system 100 and the specific steps of the imaging and display method 200 for the endoscope system have been described above and will not be repeated here.
  • the endoscope system 100 of the embodiment of the present invention simultaneously displays a stereoscopic endoscope image and an external image, the stereoscopic endoscope image can provide a more realistic visual experience, and the external image can provide richer information for the user's reference, so that the user does not need to frequently switch the line of sight between the endoscope system and the external image source.
  • the disclosed devices and methods can be implemented in other ways.
  • the device embodiments described above are only schematic, for example, the division of the units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed.
  • the various component embodiments of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. It should be understood by those skilled in the art that a microprocessor or digital signal processor (DSP) can be used in practice to implement some or all of the functions of some modules according to embodiments of the present invention.
  • DSP digital signal processor
  • the present invention can also be implemented as a device program (e.g., a computer program and a computer program product) for executing part or all of the methods described herein.
  • a program implementing the present invention can be stored on a computer-readable medium, or can have the form of one or more signals. Such a signal can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Engineering & Computer Science (AREA)
  • Biomedical Technology (AREA)
  • Molecular Biology (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biophysics (AREA)
  • Physics & Mathematics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Optics & Photonics (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Signal Processing (AREA)
  • Endoscopes (AREA)
  • Instruments For Viewing The Inside Of Hollow Bodies (AREA)

Abstract

L'invention concerne un procédé d'imagerie et d'affichage pour un système d'endoscope et un système d'endoscope. Le procédé consiste à : acquérir un signal d'image d'endoscope d'une partie à observer ; générer des premières données d'image et des secondes données d'image sur la base du signal d'image d'endoscope, une parallaxe étant présente entre les premières et les secondes données d'image ; générer une image d'endoscope stéréoscopique sur la base des premières et des secondes données d'image ; acquérir une image externe à partir d'une source d'image externe qui est en liaison de communication avec un système d'endoscope, l'image externe étant une image bidimensionnelle ; et afficher respectivement l'image d'endoscope stéréoscopique et l'image externe dans au moins deux zones d'affichage d'une interface d'affichage en même temps. Dans la présente invention, une image d'endoscope stéréoscopique et une image externe sont affichées en même temps, l'image d'endoscope stéréoscopique pouvant fournir une expérience visuelle plus réaliste et l'image externe pouvant fournir des informations plus riches pour un utilisateur qui s'y réfère, de telle sorte que l'utilisateur n'a pas besoin de commuter fréquemment sa ligne de visée entre un système d'endoscope et une source d'image externe.
PCT/CN2024/080634 2023-03-13 2024-03-07 Procédé d'imagerie et d'affichage pour système d'endoscope et système d'endoscope Pending WO2024188160A1 (fr)

Applications Claiming Priority (2)

Application Number Priority Date Filing Date Title
CN202310252225.5 2023-03-13
CN202310252225.5A CN116172493A (zh) 2023-03-13 2023-03-13 用于内窥镜系统的成像及显示方法和内窥镜系统

Publications (1)

Publication Number Publication Date
WO2024188160A1 true WO2024188160A1 (fr) 2024-09-19

Family

ID=86448822

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/CN2024/080634 Pending WO2024188160A1 (fr) 2023-03-13 2024-03-07 Procédé d'imagerie et d'affichage pour système d'endoscope et système d'endoscope

Country Status (2)

Country Link
CN (1) CN116172493A (fr)
WO (1) WO2024188160A1 (fr)

Families Citing this family (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
CN116172493A (zh) * 2023-03-13 2023-05-30 武汉迈瑞医疗技术研究院有限公司 用于内窥镜系统的成像及显示方法和内窥镜系统
CN117281451A (zh) * 2023-11-14 2023-12-26 杭州显微智能科技有限公司 一种3d内窥镜荧光摄像系统及其成像方法

Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003033324A (ja) * 1991-03-11 2003-02-04 Olympus Optical Co Ltd 内視鏡装置
JP2011206425A (ja) * 2010-03-30 2011-10-20 Fujifilm Corp 画像処理装置、画像処理方法、画像処理プログラム、及び立体内視鏡装置
WO2017126156A1 (fr) * 2016-01-18 2017-07-27 オリンパス株式会社 Système d'endoscope
WO2018047465A1 (fr) * 2016-09-09 2018-03-15 オリンパス株式会社 Dispositif d'endoscope
US20180125340A1 (en) * 2015-06-24 2018-05-10 Sony Olympus Medical Solutions Inc. Medical stereoscopic observation device, medical stereoscopic observation method, and program
US20190290110A1 (en) * 2018-03-20 2019-09-26 Sony Olympus Medical Solutions Inc. Three-dimensional endoscope system
WO2021095446A1 (fr) * 2019-11-11 2021-05-20 富士フイルム株式会社 Système d'affichage d'informations et procédé d'affichage d'informations
CN116172493A (zh) * 2023-03-13 2023-05-30 武汉迈瑞医疗技术研究院有限公司 用于内窥镜系统的成像及显示方法和内窥镜系统

Patent Citations (8)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2003033324A (ja) * 1991-03-11 2003-02-04 Olympus Optical Co Ltd 内視鏡装置
JP2011206425A (ja) * 2010-03-30 2011-10-20 Fujifilm Corp 画像処理装置、画像処理方法、画像処理プログラム、及び立体内視鏡装置
US20180125340A1 (en) * 2015-06-24 2018-05-10 Sony Olympus Medical Solutions Inc. Medical stereoscopic observation device, medical stereoscopic observation method, and program
WO2017126156A1 (fr) * 2016-01-18 2017-07-27 オリンパス株式会社 Système d'endoscope
WO2018047465A1 (fr) * 2016-09-09 2018-03-15 オリンパス株式会社 Dispositif d'endoscope
US20190290110A1 (en) * 2018-03-20 2019-09-26 Sony Olympus Medical Solutions Inc. Three-dimensional endoscope system
WO2021095446A1 (fr) * 2019-11-11 2021-05-20 富士フイルム株式会社 Système d'affichage d'informations et procédé d'affichage d'informations
CN116172493A (zh) * 2023-03-13 2023-05-30 武汉迈瑞医疗技术研究院有限公司 用于内窥镜系统的成像及显示方法和内窥镜系统

Also Published As

Publication number Publication date
CN116172493A (zh) 2023-05-30

Similar Documents

Publication Publication Date Title
CN108430306B (zh) 用于使用激光散斑对比度成像快速检查脉管系统和微粒流的系统和方法
US10197803B2 (en) Augmented reality glasses for medical applications and corresponding augmented reality system
US20140336461A1 (en) Surgical structured light system
JP5893808B2 (ja) 立体内視鏡画像処理装置
CN105208909B (zh) 用于立体显示图像数据的方法和装置
WO2024188159A1 (fr) Procédé d'imagerie et d'affichage pour système d'endoscope, et système d'endoscope
WO2024188160A1 (fr) Procédé d'imagerie et d'affichage pour système d'endoscope et système d'endoscope
WO2020045015A1 (fr) Système médical, dispositif de traitement d'informations et méthode de traitement d'informations
WO2013163391A1 (fr) Système de lumière structurée chirurgicale
CN204636276U (zh) 一种穿戴型腔镜虚拟显示系统
Reiter et al. Surgical structured light for 3D minimally invasive surgical imaging
WO2011092951A1 (fr) Appareil d'acquisition d'images, appareil d'observation, et système d'observation
CN111405866A (zh) 用于眼部治疗的沉浸式显示系统
Getty et al. Clinical applications for stereoscopic 3‐D displays
Hanna et al. Image display technology and image processing
TW202017368A (zh) 智慧眼鏡、系統及其使用方法
CN104887314A (zh) 用于立体内窥镜手术导航的虚拟立体内窥镜显示方法及设备
CN115919239A (zh) 用于3d内窥镜成像系统的成像方法和3d内窥镜成像系统
CN104887316A (zh) 基于主动立体显示技术的虚拟立体内窥镜显示方法
CN219229809U (zh) 一种电子内窥镜及3d电子内窥镜
US20190394454A1 (en) Medical-image display control device, medical-image display device, medical-information processing system, and medical-image display control method
CN116327094A (zh) 用于内窥镜系统的成像及显示方法和内窥镜系统
CN111193830B (zh) 一种基于智能手机的便携式增强现实医学图像观察辅助设备
CN209611347U (zh) 基于双目视觉的口腔3d显微系统
KR20180133632A (ko) 증강현실 영상을 지원하는 내시경 영상 진단 시스템

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 24769853

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE