[go: up one dir, main page]

WO2023195103A1 - Système d'aide à l'inspection et procédé d'aide à l'inspection - Google Patents

Système d'aide à l'inspection et procédé d'aide à l'inspection Download PDF

Info

Publication number
WO2023195103A1
WO2023195103A1 PCT/JP2022/017187 JP2022017187W WO2023195103A1 WO 2023195103 A1 WO2023195103 A1 WO 2023195103A1 JP 2022017187 W JP2022017187 W JP 2022017187W WO 2023195103 A1 WO2023195103 A1 WO 2023195103A1
Authority
WO
WIPO (PCT)
Prior art keywords
observation
information
endoscope
inspection
support system
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/JP2022/017187
Other languages
English (en)
Japanese (ja)
Inventor
尚希 深津
浩正 藤田
憲輔 三宅
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Olympus Medical Systems Corp
Original Assignee
Olympus Medical Systems Corp
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Application filed by Olympus Medical Systems Corp filed Critical Olympus Medical Systems Corp
Priority to PCT/JP2022/017187 priority Critical patent/WO2023195103A1/fr
Publication of WO2023195103A1 publication Critical patent/WO2023195103A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Images

Classifications

    • A—HUMAN NECESSITIES
    • A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
    • A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
    • A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
    • A61B1/04—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor combined with photographic or television appliances
    • A61B1/045—Control thereof

Definitions

  • the present disclosure relates to an examination support system and an examination support method that support endoscopy.
  • Patent Document 1 is an endoscope system equipped with a function of detecting lesions, which determines whether the endoscopy process is an insertion process or an extraction process, and in the extraction process, the endoscope system is equipped with a function to detect lesions.
  • An endoscope system that notifies the detection of a lesion with relatively high notification power is disclosed.
  • the inspection process of colonoscopy consists of an insertion process in which the endoscope is inserted from the anus to the cecum, and an observation process in which the inside of the large intestine from the cecum to the anus is observed while withdrawing the endoscope.
  • the doctor needs to quickly and carefully insert the endoscope into the cecum in order to reduce the burden on the patient.
  • the doctor needs to carefully observe the endoscopic image while pulling out the endoscope so as not to miss any lesions.
  • the insertion process and observation process of endoscopy require different examination skills, and it is desired to develop a technology that effectively supports the examination work of doctors in the insertion process and observation process.
  • the present disclosure has been made in view of these circumstances, and its purpose is to provide a technology that supports a doctor's inspection work in the insertion process and observation process of endoscopy.
  • an inspection support system includes one or more processors having hardware.
  • the one or more processors display an insertion guide on how to insert the endoscope when the inspection process in endoscopy is an insertion process, and display an insertion guide for supporting observation when the inspection process is an observation process. Display observation support information.
  • Another aspect of the present invention is an inspection support method, which displays an insertion guide regarding how to insert an endoscope when the inspection process in endoscopy is an insertion process, and the inspection process is an observation process. display observation support information to support observation.
  • FIG. 1 is a diagram showing the configuration of an inspection support system according to an embodiment.
  • FIG. 3 is a diagram showing functional blocks of a server device.
  • FIG. 2 is a diagram showing functional blocks of an endoscopic observation device.
  • FIG. 3 is a diagram showing a first display area where an endoscopic image is displayed in the insertion process. It is a figure which shows the 2nd display area in which an endoscopic image is displayed in an observation process.
  • FIG. 3 is a diagram comparing a first display area and a second display area. It is a figure showing a flow chart of inspection support processing in an embodiment.
  • FIG. 3 is a diagram showing an example of an endoscopic image displayed in a first display area.
  • FIG. 7 is a diagram showing an example of an insertion guide displayed around the first display area.
  • FIG. 7 is a diagram showing another example of an insertion guide displayed around the first display area.
  • FIG. 7 is a diagram showing an example of an endoscopic image displayed in a second display area.
  • FIG. 7 is a diagram showing an example of observation support information displayed around the second display area.
  • FIG. 1 shows the configuration of an inspection support system 1 according to an embodiment.
  • the examination support system 1 is installed in a medical facility such as a hospital that performs endoscopy.
  • a server device 2, an image analysis device 3, an image storage device 8, an endoscope system 9, and a terminal device 10b are communicably connected via a network 4 such as a LAN (local area network). be done.
  • the endoscope system 9 is installed in an examination room and includes an endoscope observation device 5 and a terminal device 10a.
  • the server device 2, image analysis device 3, and image storage device 8 may be provided outside the medical facility, for example, as a cloud server.
  • the endoscope observation device 5 is connected to an endoscope 7 that is inserted into the patient's digestive tract.
  • the endoscope 7 has an insertion section that is inserted into a subject, an operation section provided on the proximal end side of the insertion section, and a universal cord extending from the operation section.
  • the endoscope 7 is detachably connected to the endoscope observation device 5 by a scope connector provided at the end of the universal cord.
  • the elongated insertion section has a hard distal end, a curved section formed to be freely curved, and a flexible elongated tube section in order from the distal end to the proximal end.
  • a plurality of magnetic coils are arranged at predetermined intervals along the longitudinal direction of the insertion section inside the distal end, curved section, and flexible tube section, and the magnetic coils are supplied from the endoscope observation device 5. generates a magnetic field according to the coil drive signal.
  • the endoscope 7 has a light guide for illuminating the inside of the digestive tract by transmitting the illumination light supplied from the light source section of the endoscope observation device 5, and has a distal end section that transmits illumination light that is transmitted by the light guide.
  • An illumination window for emitting illumination light to the living tissue and a photographing unit that photographs the living tissue at a predetermined period and outputs an imaging signal to the endoscopic observation device 5 are provided.
  • the imaging unit includes a solid-state image sensor (for example, a CCD image sensor or a CMOS image sensor) that converts incident light into an electrical signal.
  • the endoscopic observation device 5 generates an endoscopic image by performing image processing on the imaging signal photoelectrically converted by the solid-state image sensor of the endoscope 7, and displays the image in real time on a display device 6 provided in the examination room. indicate. In addition to normal image processing such as A/D conversion and noise removal, the endoscopic observation device 5 may have a function of performing special image processing for the purpose of highlighting and the like.
  • the imaging frame rate of the endoscope 7 is preferably 30 fps or more, and may be 60 fps.
  • the endoscopic observation device 5 generates endoscopic images at a cycle of the imaging frame rate.
  • the endoscopic observation device 5 may be configured by one or more processors having dedicated hardware, but may also be configured by one or more processors having general-purpose hardware.
  • the endoscope 7 of the embodiment is a flexible endoscope and has a forceps channel for inserting an endoscopic treatment tool. By inserting biopsy forceps into the forceps channel and manipulating the inserted biopsy forceps, the doctor can perform a biopsy and collect a portion of the diseased tissue during an endoscopy.
  • the doctor operates the endoscope 7 according to the examination procedure and observes the endoscopic image displayed on the display device 6.
  • the inspection process in endoscopy consists of an "insertion process" in the outward path from when the endoscope 7 enters the inside of the subject until it turns back at a predetermined turn-around point, and an "insertion process” on the outward path after the endoscope 7 turns back at the turn-back point. , and an "observation process” on the return trip until it is pulled out of the subject.
  • the observation process is a process of observing the inside of the large intestine while withdrawing the endoscope 7 from within the subject, and may also be referred to as a "removal process.”
  • the insertion process is a process of inserting the endoscope 7 for colon examination from the anus to the cecum, which is the turning point
  • the observation process is the process of inserting the endoscope 7 for colon examination from the anus to the cecum, which is the turning point.
  • the observation process is the process of pulling out the endoscope 7 from the cecum to the anus. This is the process of observing the inside of the large intestine.
  • the doctor operates the release switch of the endoscope 7.
  • the endoscopic observation device 5 captures an endoscopic image at the timing when the release switch is operated, and stores the captured endoscopic image together with information (image ID) for identifying the endoscopic image in the image storage device 8. Send to.
  • the endoscopic observation device 5 may assign image IDs including serial numbers to endoscopic images in the order in which they are captured. Note that the endoscopic observation device 5 may send a plurality of captured endoscopic images together to the image storage device 8 after the end of the examination.
  • the image storage device 8 records the endoscopic image transmitted from the endoscopic observation device 5 in association with the examination ID that identifies the endoscopic examination.
  • imaging means an operation in which the solid-state imaging device of the endoscope 7 converts incident light into an electrical signal.
  • imaging may include the operation from the converted electrical signal to the endoscope observation device 5 generating an endoscopic image, and may further include the operation until displaying it on the display device 6.
  • capture means an operation of acquiring an endoscopic image generated by the endoscopic observation device 5.
  • capture may include an operation of saving (recording) an acquired endoscopic image.
  • a photographed endoscopic image is captured when the doctor operates the release switch, but the photographed endoscopic image may be automatically captured regardless of the operation of the release switch. good.
  • the terminal device 10a is provided in the examination room and includes an information processing device 11a and a display device 12a.
  • the terminal device 10a may be used by a doctor, a nurse, or the like to check information regarding the biological tissue being imaged in real time during an endoscopy. Note that in the embodiment shown below, the endoscopic image taken by the endoscope 7 is displayed on the display device 6 connected to the endoscopic observation device 5, but in another embodiment, the endoscopic image is , may be displayed on the display device 12a of the terminal device 10a.
  • the terminal device 10b includes an information processing device 11b and a display device 12b, and is installed in a room other than the examination room.
  • the terminal device 10b is used by a doctor when creating a report of an endoscopy.
  • the terminal devices 10a, 10b may be configured with one or more processors having general-purpose hardware.
  • the endoscopic observation device 5 displays the endoscopic image on the display device 6 in real time, and also sends the endoscopic image to the image analysis device 3 together with the meta information of the image. Supply in real time.
  • the meta information includes at least the frame number of the image and the shooting time information.
  • the frame number may be information indicating the number of frames after the endoscope 7 starts imaging.
  • the image analysis device 3 is an electronic computer that analyzes endoscopic images, detects lesions included in the endoscopic images, and qualitatively diagnoses the detected lesions.
  • the image analysis device 3 may be a CAD (computer-aided diagnosis) system having an AI (artificial intelligence) diagnosis function.
  • the image analysis device 3 may be composed of one or more processors with dedicated hardware, but may also be composed of one or more processors with general-purpose hardware.
  • the image analysis device 3 performs machine learning using learning endoscopic images, information indicating organs and parts included in the endoscopic images, and information regarding lesion areas included in the endoscopic images as training data. Use the trained model generated by . Annotation work on endoscopic images is performed by an annotator with specialized knowledge such as a doctor, and a type of deep learning such as CNN, RNN, LSTM, etc. may be used for machine learning.
  • this trained model receives an endoscopic image, it outputs information indicating the photographed organ, information indicating the photographed region, and information regarding the photographed lesion (lesion information).
  • the lesion information output by the image analysis device 3 includes at least lesion presence/absence information indicating whether a lesion is included in the endoscopic image (a lesion is captured).
  • the lesion information includes information indicating the size of the lesion, information indicating the position of the outline of the lesion, information indicating the shape of the lesion, information indicating the depth of invasion of the lesion, and qualitative diagnosis results of the lesion.
  • the qualitative diagnosis result of the lesion may include information indicating the type of the lesion.
  • the image analysis device 3 is provided with endoscopic images from the endoscopic observation device 5 in real time together with meta information of the endoscopic images, and is provided with information indicating organs, information indicating parts, and lesions. Generate information.
  • image analysis information the information indicating the organ, the information indicating the site, and the lesion information generated by the image analysis device 3 will be collectively referred to as "image analysis information.”
  • the image analysis device 3 provides the generated image analysis information to the endoscope observation device 5. Therefore, the endoscopic observation device 5 can display information regarding the image analysis results on the display device 6 while displaying the endoscopic image.
  • the endoscope observation device 5 provides the frame number of the captured endoscopic image to the image analysis device 3 along with information indicating that the capture operation has been performed (capture operation information). do.
  • the image analysis device 3 acquires the capture operation information, it provides the server device 2 with the examination ID, a frame number, photographing time information, and image analysis information corresponding to the frame number.
  • the frame number, photographing time information, and image analysis information constitute "additional information" that expresses the characteristics and properties of the endoscopic image.
  • the image analysis device 3 acquires the capture operation information, it transmits the additional information together with the examination ID to the server device 2, and the server device 2 records the additional information in association with the examination ID.
  • the additional information may include an image ID.
  • the end examination button When the user finishes the endoscopic examination, he or she operates the end examination button on the endoscopic observation device 5.
  • the operation information of the examination end button is supplied to the server device 2 and the image analysis device 3, and the server device 2 and the image analysis device 3 recognize the end of the endoscopy.
  • the period from when the test start button is operated until the test end button is operated is the period before the endoscope is inserted into the subject, and the period when the endoscope is completely removed from the subject after observation is completed.
  • the endoscope photographs the outside of the subject (the space inside the examination room).
  • the trained model of the embodiment is configured to output information indicating that the image is an image outside the subject when an endoscopic image taken during that period is input.
  • FIG. 2 shows functional blocks of the server device 2.
  • the server device 2 includes a communication section 20, a processing section 30, and a storage device 60.
  • the communication unit 20 transmits and receives information such as data and instructions to and from the image analysis device 3, endoscope observation device 5, image storage device 8, terminal device 10a, and terminal device 10b via the network 4.
  • the processing section 30 includes an order information acquisition section 40 and an additional information acquisition section 42.
  • the storage device 60 includes an order information storage section 62 and an additional information storage section 64.
  • the server device 2 includes a computer, and the various functions shown in FIG. 2 are realized by the computer executing programs.
  • a computer includes a memory for loading a program, one or more processors for executing the loaded program, an auxiliary storage device, other LSI, and the like as hardware.
  • a processor is constituted by a plurality of electronic circuits including a semiconductor integrated circuit and an LSI, and the plurality of electronic circuits may be mounted on one chip or on a plurality of chips.
  • the functional blocks shown in FIG. 2 are realized by the cooperation of hardware and software, and therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways by only hardware, only software, or a combination thereof. It is understood.
  • the order information acquisition unit 40 acquires order information for endoscopy from the hospital information system. For example, before the start of a day's testing work at a medical facility, the order information acquisition section 40 acquires the order information for that day from the hospital information system and stores it in the order information storage section 62. Before the start of the examination, the endoscopic observation device 5 or the information processing device 11a may read order information for the examination to be performed from the order information storage unit 62 and display it on the display device.
  • the additional information acquisition unit 42 acquires the examination ID and the additional information of the endoscopic image from the image analysis device 3, and stores the additional information in the additional information storage unit 64 in association with the examination ID.
  • Additional information on the endoscopic image includes a frame number, imaging time information, and image analysis information.
  • FIG. 3 shows functional blocks of the endoscopic observation device 5.
  • the endoscopic observation device 5 has a function of controlling the endoscope 7 and displaying images taken by the endoscope 7 on the display device 6 in real time.
  • the endoscopic observation device 5 includes a receiving antenna 76, a communication section 78, and a control device 80.
  • the communication unit 78 transmits and receives information such as data and instructions to and from the server device 2, image analysis device 3, image storage device 8, and terminal devices 10a and 10b via the network 4.
  • the control device 80 includes an endoscope control section 82, a signal processing section 84, an operation information acquisition section 86, a shape information acquisition section 90, an image providing section 92, an image analysis information acquisition section 94, an endoscope image acquisition section 100, and a process. It includes a specifying section 102, an operation guide generation section 110, an observation support information generation section 112, and a display control section 120.
  • the endoscopic observation device 5 includes a computer, and various functions shown in FIG. 3 are realized by the computer executing programs.
  • a computer includes a memory for loading a program, one or more processors for executing the loaded program, an auxiliary storage device, other LSI, and the like as hardware.
  • a processor is constituted by a plurality of electronic circuits including a semiconductor integrated circuit and an LSI, and the plurality of electronic circuits may be mounted on one chip or on a plurality of chips.
  • the functional blocks shown in FIG. 3 are realized by the cooperation of hardware and software, and therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways by only hardware, only software, or a combination thereof. It is understood.
  • the endoscope control unit 82 has a light source including an LED or a lamp, and supplies the endoscope 7 with illumination light for illuminating the inside of the subject. Furthermore, the endoscope control section 82 has a drive circuit and generates a coil drive signal for driving a plurality of magnetic coils provided in the insertion section of the endoscope 7. The endoscope control unit 82 supplies coil drive signals to the plurality of magnetic coils, so that the plurality of magnetic coils of the endoscope 7 generate magnetic fields.
  • the receiving antenna 76 has a plurality of coils that three-dimensionally detects the magnetic field generated by each of the plurality of magnetic coils.
  • the receiving antenna 76 detects the magnetic field generated by each of the plurality of magnetic coils, and outputs a magnetic field detection signal corresponding to the strength of the detected magnetic field to the shape information acquisition section 90.
  • the shape information acquisition unit 90 acquires the positions of the plurality of magnetic coils within the subject based on the magnetic field detection signal output from the receiving antenna 76. Specifically, the shape information acquisition unit 90 acquires a plurality of three-dimensional coordinate values in a virtual spatial coordinate system with a predetermined position (anus, etc.) of the subject as the origin or reference point as the positions of the plurality of magnetic coils. You may do so.
  • the shape information acquisition unit 90 generates insertion shape information indicating the shape of the endoscope inserted into the subject from the three-dimensional coordinate values of the plurality of magnetic coils.
  • the shape information acquisition section 90 provides the generated insertion shape information to the process identification section 102 and the operation guide generation section 110.
  • the signal processing unit 84 performs image processing such as A/D conversion, noise removal, and optical correction (shading correction) on the imaging signal supplied from the endoscope 7 to generate an endoscopic image.
  • the signal processing unit 84 outputs the generated endoscopic image to the display control unit 120 and the endoscopic image acquisition unit 100.
  • the display control unit 120 displays the endoscopic image in a predetermined display area on the display screen of the display device 6.
  • the endoscopic image acquisition unit 100 supplies the endoscopic image to the image providing unit 92 and the operation guide generation unit 110.
  • the image providing unit 92 transmits the endoscopic image to the endoscopic observation device 5 via the communication unit 78.
  • the image analysis device 3 generates image analysis information indicating the result of analyzing the endoscopic image, and sends it back to the endoscopic observation device 5.
  • the image analysis information acquisition unit 94 acquires the image analysis information generated by the image analysis device 3 and supplies it to the process identification unit 102 and the observation support information generation unit 112.
  • the operation information acquisition unit 86 acquires operation information (capture operation information) regarding the user's operation of the release switch.
  • the image providing unit 92 adds an image ID to the endoscopic image, and sends the endoscopic image together with the image ID and meta information via the communication unit 78.
  • the data is transmitted to the storage device 8.
  • the image providing unit 92 notifies the image analysis device 3 of the frame number of the endoscopic image together with the capture operation information, and the image analysis device 3 transmits image analysis information corresponding to the notified frame number to the server device 2. Register.
  • control device 80 The endoscopy support operation performed by the control device 80 will be described below.
  • the resolution of endoscopic images has been increasing, and the screen size of the display device 6 that displays endoscopic images has become larger. If the screen size is large, the entire displayed endoscopic image may not fit within the user's field of view. need to be moved to some extent.
  • the doctor performs the insertion operation of the endoscope 7 while viewing the endoscopic image displayed on the display device 6. It is preferable to see the whole thing.
  • the doctor locally observes the endoscopic image while moving his/her line of sight so as not to miss any lesions, so the display device 6 displays high-definition endoscopic images. It is preferable that it is displayed. Therefore, the control device 80 of the embodiment has a function of displaying endoscopic images in different manners in the insertion process and the observation process.
  • the display control unit 120 displays the endoscopic image in the first display area on the display screen of the display device 6 in the insertion step. Further, in the observation step, the display control unit 120 displays the endoscopic image in a second display area larger than the first display area on the display screen of the display device 6.
  • FIG. 4 shows a first display area 130 where an endoscopic image is displayed during the insertion process
  • FIG. 5 shows a second display area 132 where an endoscopic image is displayed during the observation process.
  • FIG. 6 shows a comparison diagram comparing the first display area 130 and the second display area 132.
  • the second display area 132 in the observation process is larger than the first display area 130 and includes the first display area 130.
  • the shape of the second display area 132 and the shape of the first display area 130 are similar, and are set at screen positions having a common center point. Therefore, the user can view endoscopic images during the insertion process and observation process without significantly changing the viewing direction.
  • the display control unit 120 displays the endoscopic image in the relatively small first display area 130 so that the entire endoscopic image is within the user's visual field, so that the user can see the endoscopic image within the user's visual field. It becomes possible to concentrate on the insertion operation of the endoscope 7 without moving the line of sight toward the endoscopic image.
  • the display control unit 120 displays the endoscopic image in the relatively large second display area 132, and the user can miss a lesion by observing the high-resolution endoscopic image. It is possible to reduce the
  • FIG. 7 shows a flowchart of inspection support processing in the embodiment.
  • the endoscopy is started (S10).
  • the process identification unit 102 acquires examination status information indicating the status of the endoscopy, and determines the examination process based on the examination status information.
  • the examination status information may be at least one of information obtained by analyzing an endoscopic image, information indicating the shape of the endoscope, and information indicating the position of the endoscope.
  • the examination status information may be image analysis information provided from the image analysis information acquisition section 94 and/or insertion shape information of the endoscope 7 provided from the shape information acquisition section 90.
  • the process specifying unit 102 determines whether the inspection process is an insertion process, an observation process, or a process that is neither an insertion process nor an observation process, based on the inspection status information. Processes that are neither the insertion process nor the observation process include a preparation process before inserting the endoscope 7 into the subject, and a process after the endoscope 7 is completely removed from the subject after the observation is completed. and a termination step. In the embodiment, the process specifying unit 102 has a function of determining an inspection process based on image analysis information and/or insertion shape information.
  • the image analysis device 3 generates image analysis information including information indicating the part inside the subject (part information) for the endoscopic image taken inside the subject, and images the outside of the subject.
  • Image analysis information including information indicating that the endoscopic image is an image outside the subject is generated. Therefore, the process specifying unit 102 can determine the inspection process by referring to the site information included in the image analysis information or the information indicating that the image is an external image of the subject.
  • the endoscope 7 Immediately after the start of the examination, the endoscope 7 has not yet been inserted into the subject and exists outside the subject. At this time, the endoscope 7 is photographing the space inside the examination room, and since the image analysis information includes information indicating that the image is outside the subject, the endoscope 7 It is determined that the camera is photographing the outside of the subject and has not yet been inserted into the subject (N in S12).
  • the process specifying unit 102 determines that the endoscope 7 is inserted into the subject and the examination process is an insertion process (Y in S12). Note that in colonoscopy, the first site into which the endoscope 7 is inserted is the "rectum,” so the process identification unit 102 uses the image analysis information acquired in chronological order to identify images of the outside of the subject. When the information indicating "rectal” is no longer present and the information indicating "rectum” is included, it may be determined that the inspection process is an insertion process. In the insertion step, the display control unit 120 displays the endoscopic image in the first display area 130 (S14).
  • FIG. 8 shows an example of an endoscopic image displayed in the first display area 130.
  • the display control unit 120 displays the endoscopic image in the first display area 130 at a first resolution.
  • the first resolution of the endoscopic image in the insertion step is lower than the second resolution of the endoscopic image in the observation step.
  • the endoscopic image is displayed in the small first display area 130, so that the entire endoscopic image is within the user's visual field, and the user therefore moves his/her line of sight toward the endoscopic image. This allows the user to concentrate on the insertion operation of the endoscope 7.
  • the operation guide generation unit 110 In the insertion process, the operation guide generation unit 110 generates an insertion guide regarding how to insert the endoscope 7.
  • the operation guide generating unit 110 acquires at least one of an endoscopic image and information indicating the insertion shape of the endoscope 7, generates an insertion guide based on at least one of the endoscopic image and the insertion shape information, and displays the generated insertion guide.
  • the control unit 120 displays the insertion guide in an area different from the first display area 130 (S16).
  • FIG. 9 shows an example of an insertion guide displayed around the first display area 130.
  • the operation guide generation unit 110 estimates the direction in which the lumen exists based on the endoscopic image.
  • Various methods have been proposed in the past for estimating the direction in which a lumen exists from an endoscopic image, and the operation guide generation unit 110 may use any of the methods to estimate the direction in which a lumen exists.
  • the operation guide generation unit 110 may recognize the lumen position in the endoscopic image using an image recognition function, or may recognize the lumen position in the endoscopic image by inputting endoscopic image data into a machine-learned lumen direction estimation model.
  • the direction in which the cavity exists may also be obtained.
  • the insertion guide 140 shown in FIG. 9 is a mark indicating the direction in which the lumen exists, and in this example indicates that the lumen exists below the endoscopic image.
  • the display control unit 120 arranges the insertion guide 140 indicating the lumen direction in an area outside the first display area 130 and inside the second display area 132.
  • the boundary of the second display area 132 is shown with a broken line for reference.
  • the display control unit 120 may determine the position where the insertion guide 140 is placed depending on the direction in which the lumen exists. In the example shown in FIG. 9, the insertion guide 140 indicates that the lumen is present at the bottom of the screen, so the display control unit 120 moves the insertion guide 140 downward with respect to the first display area 130. It is placed.
  • the display control unit 120 positions the insertion guide 140 on the right side with respect to the first display area 130, and the insertion guide 140
  • the display control unit 120 arranges the insertion guide 140 in the upper left side with respect to the first display area 130. Since the direction in which the lumen exists indicates the direction in which the distal end of the endoscope 7 should be directed (curved), the user can determine the operation of the endoscope 7 to be performed based on the orientation and placement position of the insertion guide 140. can be understood intuitively.
  • FIG. 10 shows another example of the insertion guide displayed around the first display area 130.
  • the operation guide generation unit 110 derives recommended operations based on the insertion shape information of the endoscope 7.
  • the insertion guide 142 shown in FIG. 10 is a guide showing a recommended operation, and in this example shows a loop release operation to be performed when passing through the sigmoid colon.
  • the operation guide generation unit 110 displays the insertion guide 142 at a position close to the endoscopic image displayed in the first display area 130. By displaying the insertion guide 142 at a position close to the endoscopic image, the user can confirm the insertion guide 142 without significantly moving his/her line of sight.
  • the display control unit 120 preferably sets the display position of the insertion guide 142 so as not to overlap the insertion guide 140.
  • the operation guide generation unit 110 may generate a schema diagram showing the insertion status of the endoscope 7, and the display control unit 120 may display the schema diagram as an insertion guide.
  • the process specifying unit 102 refers to the site information in the image analysis information and determines whether the endoscope 7 has reached the "cecum" which is the turning point (S18). The process specifying unit 102 determines that the inspection process is not an observation process (N in S18) but an insertion process (Y in S12) until it is determined that the endoscope 7 has reached the cecum. In the insertion step, the display control unit 120 displays an endoscopic image in the first display area 130 (S14), and also displays an operation guide as necessary (S16).
  • the process specifying unit 102 refers to the site information in the image analysis information and determines that the endoscope 7 has reached the "cecum" which is the turning point, it determines that the inspection process is an observation process. Determine (Y in S18). In the observation step, the display control unit 120 displays the endoscopic image in the second display area 132 (S20).
  • FIG. 11 shows an example of an endoscopic image displayed in the second display area 132.
  • the display control unit 120 displays the endoscopic image in the second display area 132 at a second resolution higher than the first resolution. In the observation process, a high resolution endoscopic image is displayed in the second display area 132, allowing the user to find even small lesions.
  • the observation support information generation unit 112 generates observation support information for supporting observation.
  • the observation support information generation unit 112 may generate observation support information from the image analysis information provided by the image analysis information acquisition unit 94.
  • the display control unit 120 displays the observation support information in an area different from the second display area 132 (S22).
  • FIG. 12 shows an example of observation support information displayed around the second display area 132.
  • the observation support information generation unit 112 may generate the observation support information 150 using the lesion information included in the image analysis information.
  • the observation support information 150 is information regarding lesions, and specifically, a mark (in this example, A frame surrounding the lesion) is added. Note that the observation support information 150 may add a mark indicating the position of the detected lesion to the endoscopic image displayed in the second display area 132.
  • the display control unit 120 displays the observation support information 150 near the endoscopic image or superimposed on the endoscopic image. By looking at the observation support information 150, the user can recognize that the lesion is automatically detected by the image analysis device 3.
  • observation support information generation unit 112 may generate observation support information 152 indicating that a lesion has been detected, and the display control unit 120 may display the observation support information 152.
  • the observation support information generation unit 112 When the image analysis information includes lesion information, the observation support information generation unit 112 generates observation support information 152, which is an alert mark to alert the user.
  • the display control unit 120 may display the observation support information 152 in an area outside the second display area 132 while a lesion is being detected. For example, the alert mark may be displayed in a color that stands out against the endoscopic image.
  • the observation support information generating unit 112 When the observation support information generating unit 112 detects an area (unobserved area) that the user has not confirmed well, it may generate information regarding the unobserved area of the subject as observation support information. For example, if a lesion is detected by the image analysis device 3 on the outward path in the insertion process, but is not detected by the image analysis device 3 on the return path in the observation process, that is, it has not been imaged by the endoscope 7. In this case, the observation support information generation unit 112 may generate information indicating that the lesion (unobserved area) has been passed, and the display control unit 120 may display the information on the display screen as observation support information.
  • the process specifying unit 102 refers to the site information in the image analysis information and determines whether the endoscope 7 has been completely removed to the outside of the subject (S24). The process specifying unit 102 determines that the observation process is in progress (Y in S18) until the endoscope 7 is completely removed to the outside of the subject (N in S24). In the observation step, the display control unit 120 displays the endoscopic image in the second display area 132 (S20), and also displays observation support information as necessary (S22).
  • the process identification unit 102 determines that the endoscope 7 has been completely removed from the subject (S24). Y), the observation process ends.
  • the endoscope observation device 5 is equipped with the control device 80, but the terminal device 10a may be equipped with the control device 80, and the function of the control device 80 is different from that of the endoscope observation device 5 and the server device. 2. It may be distributed to the image analysis device 3 and/or the terminal device 10, and realized in cooperation with a plurality of devices.
  • the process identification unit 102 determines the inspection process based on image analysis information provided from the image analysis information acquisition unit 94.
  • the process specifying unit 102 may generate image analysis information using a learned model that the image analysis device 3 has. Further, the process specifying unit 102 performs the inspection process using a trained model that can determine whether the endoscope is inside or outside the subject and whether it has reached the cecum inside the subject. May be determined.
  • the process specifying unit 102 may determine the inspection process based on the insertion shape information of the endoscope 7 provided from the shape information acquiring unit 90. Specifically, the process specifying unit 102 determines whether the tip of the endoscope has reached the cecum based on the insertion shape of the endoscope 7 and/or the position of the tip of the endoscope 7, and therefore determines whether the tip of the endoscope has reached the cecum. It may be determined whether the inspection process is an insertion process or an observation process.
  • the display control unit 120 sets the display mode of the endoscopic image according to the inspection process determined by the process specifying unit 102, but the user may be able to forcibly change the display mode. .
  • the display control unit 120 can display a high-resolution image by manually forcibly switching the display mode.
  • the endoscopic image may be displayed in a large size.
  • the present disclosure can be used in the technical field of displaying endoscopic images.
  • Shape information acquisition unit 92... Image providing unit, 94... Image analysis information acquisition unit, 100... Endoscopic image acquisition unit, 102... Process identification unit, 110... Operation guide generation unit, 112... Observation support information generation unit, 120... Display control unit, 130... First display area, 132... Second display area, 140, 142... - Insertion guide, 150, 152...Observation support information.

Landscapes

  • Health & Medical Sciences (AREA)
  • Life Sciences & Earth Sciences (AREA)
  • Surgery (AREA)
  • Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
  • Biomedical Technology (AREA)
  • Optics & Photonics (AREA)
  • Pathology (AREA)
  • Radiology & Medical Imaging (AREA)
  • Biophysics (AREA)
  • Engineering & Computer Science (AREA)
  • Physics & Mathematics (AREA)
  • Heart & Thoracic Surgery (AREA)
  • Medical Informatics (AREA)
  • Molecular Biology (AREA)
  • Animal Behavior & Ethology (AREA)
  • General Health & Medical Sciences (AREA)
  • Public Health (AREA)
  • Veterinary Medicine (AREA)
  • Endoscopes (AREA)

Abstract

La présente invention concerne une unité (120) de commande d'affichage qui affiche un guidage d'insertion se rapportant à la manière d'insérer un endoscope lorsqu'une étape d'insertion est effectuée dans un processus d'inspection pour une inspection d'endoscope. L'unité (120) de commande d'affichage affiche des informations d'aide à l'observation pour fournir une aide à l'observation lorsqu'une étape d'observation a lieu au cours du processus d'inspection.
PCT/JP2022/017187 2022-04-06 2022-04-06 Système d'aide à l'inspection et procédé d'aide à l'inspection Ceased WO2023195103A1 (fr)

Priority Applications (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/017187 WO2023195103A1 (fr) 2022-04-06 2022-04-06 Système d'aide à l'inspection et procédé d'aide à l'inspection

Applications Claiming Priority (1)

Application Number Priority Date Filing Date Title
PCT/JP2022/017187 WO2023195103A1 (fr) 2022-04-06 2022-04-06 Système d'aide à l'inspection et procédé d'aide à l'inspection

Publications (1)

Publication Number Publication Date
WO2023195103A1 true WO2023195103A1 (fr) 2023-10-12

Family

ID=88242722

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/JP2022/017187 Ceased WO2023195103A1 (fr) 2022-04-06 2022-04-06 Système d'aide à l'inspection et procédé d'aide à l'inspection

Country Status (1)

Country Link
WO (1) WO2023195103A1 (fr)

Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06181885A (ja) * 1992-12-21 1994-07-05 Olympus Optical Co Ltd 内視鏡装置
WO2017006404A1 (fr) * 2015-07-03 2017-01-12 オリンパス株式会社 Système d'endoscope
WO2017068650A1 (fr) * 2015-10-20 2017-04-27 オリンパス株式会社 Système de support de corps d'insertion
WO2021111756A1 (fr) * 2019-12-02 2021-06-10 富士フイルム株式会社 Système d'endoscope, programme de commande et procédé d'affichage
WO2021149112A1 (fr) * 2020-01-20 2021-07-29 オリンパス株式会社 Dispositif d'aide à l'endoscopie, procédé pour l'opération du dispositif d'aide à l'endoscopie, et programme
WO2022014077A1 (fr) * 2020-07-15 2022-01-20 富士フイルム株式会社 Système d'endoscope et son procédé de fonctionnement

Patent Citations (6)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JPH06181885A (ja) * 1992-12-21 1994-07-05 Olympus Optical Co Ltd 内視鏡装置
WO2017006404A1 (fr) * 2015-07-03 2017-01-12 オリンパス株式会社 Système d'endoscope
WO2017068650A1 (fr) * 2015-10-20 2017-04-27 オリンパス株式会社 Système de support de corps d'insertion
WO2021111756A1 (fr) * 2019-12-02 2021-06-10 富士フイルム株式会社 Système d'endoscope, programme de commande et procédé d'affichage
WO2021149112A1 (fr) * 2020-01-20 2021-07-29 オリンパス株式会社 Dispositif d'aide à l'endoscopie, procédé pour l'opération du dispositif d'aide à l'endoscopie, et programme
WO2022014077A1 (fr) * 2020-07-15 2022-01-20 富士フイルム株式会社 Système d'endoscope et son procédé de fonctionnement

Similar Documents

Publication Publication Date Title
CN101912251B (zh) 内窥镜图像处理装置
JP4153963B2 (ja) 内視鏡挿入形状検出装置
JP5676058B1 (ja) 内視鏡システム及び内視鏡システムの作動方法
US12268354B2 (en) Image recording device, image recording method, and recording medium for recording time-series images of endoscopy
JP5492729B2 (ja) 内視鏡画像記録装置、及び内視鏡画像記録装置の作動方法、並びにプログラム
JPWO2014168128A1 (ja) 内視鏡システム及び内視鏡システムの作動方法
CN105283114B (zh) 胶囊型内窥镜系统
US20230419517A1 (en) Shape measurement system for endoscope and shape measurement method for endoscope
CN114980793A (zh) 内窥镜检查辅助装置、内窥镜检查辅助装置的工作方法以及程序
JPWO2014148184A1 (ja) 内視鏡システム
JP5593008B1 (ja) 画像処理装置及び画像処理方法
WO2021176664A1 (fr) Système et procédé d'aide à l'examen médical et programme
WO2022163514A1 (fr) Dispositif, procédé et programme de traitement d'image médicale
JP7441934B2 (ja) 処理装置、内視鏡システム及び処理装置の作動方法
US20250009248A1 (en) Endoscopy support system, endoscopy support method, and storage medium
US20240382067A1 (en) Medical assistance system and medical assistance method
WO2023195103A1 (fr) Système d'aide à l'inspection et procédé d'aide à l'inspection
JP4615842B2 (ja) 内視鏡システムおよび内視鏡画像処理装置
JP7314394B2 (ja) 内視鏡検査支援装置、内視鏡検査支援方法、及び内視鏡検査支援プログラム
WO2023175916A1 (fr) Système d'assistance médicale et méthode d'affichage d'image
WO2021176665A1 (fr) Système d'assistance à la chirurgie, procédé d'assistance à la chirurgie, et programme
WO2023209884A1 (fr) Système d'assistance médicale et méthode d'affichage d'image
JP2021194268A (ja) 血管観察システムおよび血管観察方法
JP7607803B2 (ja) 医療支援システム、レポート作成支援方法および情報処理装置
US20240428410A1 (en) Medical assistance system and image display method

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22936503

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22936503

Country of ref document: EP

Kind code of ref document: A1

NENP Non-entry into the national phase

Ref country code: JP