WO2017169650A1 - 医療用観察装置、映像移動補正方法及び医療用観察システム - Google Patents
医療用観察装置、映像移動補正方法及び医療用観察システム Download PDFInfo
- Publication number
- WO2017169650A1 WO2017169650A1 PCT/JP2017/009651 JP2017009651W WO2017169650A1 WO 2017169650 A1 WO2017169650 A1 WO 2017169650A1 JP 2017009651 W JP2017009651 W JP 2017009651W WO 2017169650 A1 WO2017169650 A1 WO 2017169650A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- unit
- video
- movement
- video data
- image
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Images
Classifications
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00039—Operational features of endoscopes provided with input arrangements for the user
- A61B1/00042—Operational features of endoscopes provided with input arrangements for the user for mechanical operation
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00006—Operational features of endoscopes characterised by electronic signal processing of control signals
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00002—Operational features of endoscopes
- A61B1/00004—Operational features of endoscopes characterised by electronic signal processing
- A61B1/00009—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope
- A61B1/000095—Operational features of endoscopes characterised by electronic signal processing of image signals during a use of endoscope for image enhancement
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/20—Surgical microscopes characterised by non-optical aspects
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/0004—Microscopes specially adapted for specific applications
- G02B21/0012—Surgical microscopes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B21/00—Microscopes
- G02B21/36—Microscopes arranged for photographic purposes or projection purposes or digital imaging or video purposes including associated control and data processing arrangements
- G02B21/365—Control or image processing arrangements for digital or video microscopes
-
- G—PHYSICS
- G02—OPTICS
- G02B—OPTICAL ELEMENTS, SYSTEMS OR APPARATUS
- G02B27/00—Optical systems or apparatus not provided for by any of the groups G02B1/00 - G02B26/00, G02B30/00
- G02B27/64—Imaging systems using optical elements for stabilisation of the lateral and angular position of the image
- G02B27/646—Imaging systems using optical elements for stabilisation of the lateral and angular position of the image compensating for small deviations, e.g. due to vibration or shake
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
-
- H—ELECTRICITY
- H04—ELECTRIC COMMUNICATION TECHNIQUE
- H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
- H04N7/00—Television systems
- H04N7/18—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast
- H04N7/183—Closed-circuit television [CCTV] systems, i.e. systems in which the video signal is not broadcast for receiving images from a single remote source
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00147—Holding or positioning arrangements
- A61B1/00149—Holding or positioning arrangements using articulated arms
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00147—Holding or positioning arrangements
- A61B1/0016—Holding or positioning arrangements using motor drive units
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B1/00—Instruments for performing medical examinations of the interior of cavities or tubes of the body by visual or photographical inspection, e.g. endoscopes; Illuminating arrangements therefor
- A61B1/00163—Optical arrangements
- A61B1/00188—Optical arrangements with focusing or zooming features
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B34/00—Computer-aided surgery; Manipulators or robots specially adapted for use in surgery
- A61B34/20—Surgical navigation systems; Devices for tracking or guiding surgical instruments, e.g. for frameless stereotaxis
- A61B2034/2046—Tracking techniques
- A61B2034/2048—Tracking techniques using an accelerometer or inertia sensor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
- A61B90/37—Surgical systems with images on a monitor during operation
- A61B2090/371—Surgical systems with images on a monitor during operation with simultaneous use of two cameras
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/20—Surgical microscopes characterised by non-optical aspects
- A61B90/25—Supports therefor
-
- A—HUMAN NECESSITIES
- A61—MEDICAL OR VETERINARY SCIENCE; HYGIENE
- A61B—DIAGNOSIS; SURGERY; IDENTIFICATION
- A61B90/00—Instruments, implements or accessories specially adapted for surgery or diagnosis and not covered by any of the groups A61B1/00 - A61B50/00, e.g. for luxation treatment or for protecting wound edges
- A61B90/36—Image-producing devices or illumination devices not otherwise provided for
Definitions
- the present disclosure relates to a medical observation apparatus, a video movement correction method, and a medical observation system.
- an observation device for enlarging an operation part is used.
- the observation apparatus is configured by a microscope unit supported by an arm unit (support unit) (for example, Patent Document 1).
- the microscope unit of the observation apparatus described in Patent Document 1 is an optical type, and the operator looks directly into the microscope unit from the eyepiece unit provided in the microscope unit and observes the surgical unit.
- an observation apparatus including an optical microscope unit is also referred to as an optical observation apparatus.
- such an optical observation apparatus has been the mainstream as an observation apparatus.
- the observation device has an image sensor and can be used for electronic imaging of the surgical site.
- formula microscope part is developed (for example, patent document 2).
- an observation apparatus equipped with an electronic imaging microscope unit hereinafter also referred to as an electronic imaging observation apparatus
- an image of the surgical part imaged by the microscope unit is displayed on a display device installed in the operating room, Surgery is performed while observing the surgical site through the display device.
- the microscope portion and the support portion should not obstruct the view of the operator who observes the display device as much as possible. Is required. Accordingly, there is a great demand for downsizing the microscope unit and the support unit of the electronic imaging observation apparatus.
- the support portion when the support portion is reduced in size and diameter, the rigidity is reduced, and the support portion is easily elastically deformed.
- the observation apparatus when the user moves the microscope unit to a position where a desired visual field can be obtained, it is assumed that the user directly holds the microscope unit by hand and moves the microscope unit. At this time, if the rigidity of the support portion is relatively small, the support portion is elastically deformed relatively large as the user moves the microscope portion. Therefore, when the user moves the microscope unit to a desired position and releases his / her hand, the microscope unit may move unintentionally due to elastic restoration of the support unit, and the phenomenon that the field of view may move may occur. .
- a phenomenon in which the field of view on the display screen image moves due to the movement of the microscope unit due to the elastic restoration of the support unit, and the image on the display screen deviates from the desired image. This will be referred to as video shift.
- the image shift occurs, it is necessary to fine-tune the position of the microscope unit. For this reason, the image shift increases the mental and physical burden on the user, hinders the smooth execution of the operation, and increases the operation time.
- the apparent size of the shift amount in the above-described video shift (the shift amount on the screen) is more than that when a small display device is used. It will be big. Further, when the video is further enlarged and displayed by the electronic zoom function in addition to the optical zoom function, the shift amount in the video shift becomes larger. As described above, in the electronic imaging type observation apparatus, there is a concern that a slight movement of the microscope unit may cause a large image shift, particularly when shooting at a high resolution.
- the microscope unit may vibrate due to, for example, walking of a medical staff who moves in the operating room.
- Such vibrations cause unintentional image shaking (image shaking) in magnified observation, which means that work cannot be performed until the shaking has subsided, increasing the mental burden on the surgeon and increasing the efficiency of the operation. It was a cause of lowering.
- the present disclosure proposes a new and improved medical observation apparatus, image movement correction method, and medical observation system that can further improve the safety of medical practices such as surgery and examination.
- a support unit that supports the image capturing unit, and the captured image data
- the determination unit for determining whether or not the visual field movement instruction is input and whether or not the video movement is generated with respect to the video displayed on the display device based on the captured video data.
- correction that generates display video data, which is video data displayed on a display device, by correcting the video movement based on the captured video data when it is determined that the video movement has occurred.
- a medical observation device for an image displayed on a display device based on the image capturing unit that acquires captured image data that is image data obtained by capturing an observation target.
- captured image data that is image data obtained by capturing an observation target is acquired, and whether or not the processor inputs a visual field movement instruction for an image displayed on a display device based on the captured image data Determining whether or not video movement has occurred with respect to the video displayed on the display device based on the captured video data, and determining that there is no input of the visual field movement instruction and the video movement has occurred.
- a correction unit that generates display video data, which is video data displayed on a display device, by correcting the video movement based on the captured video data.
- the medical observation device includes: a medical observation device that images an observation target; and the display device that displays an image of the observation target captured by the medical observation device.
- An imaging unit that acquires captured video data that is video data of an observation target, a support unit that supports the imaging unit, and an input of a visual field movement instruction for a video displayed on a display device based on the captured video data
- the determination unit for determining presence / absence and presence / absence of occurrence of image movement with respect to the image displayed on the display device based on the captured image data, and the determination unit does not input the visual field movement instruction and the image movement is performed.
- display video data that is video data displayed on a display device is generated based on the captured video data by correcting the video movement.
- a correction unit, the medical viewing system are provided.
- a medical observation apparatus having an imaging unit supported by a support unit
- a visual field movement instruction that is, no intentional visual field movement is performed by a user
- an image When the movement has occurred, a process for correcting the video movement is executed. Therefore, for example, even when the support portion is relatively thin and its rigidity is small, it is possible to obtain a more stable image in which image movement unintended by the user is suppressed. Therefore, the operation can be performed more smoothly and safely.
- FIG. 1 is a diagram illustrating a configuration of an observation system according to the first embodiment.
- an observation system 1 includes a microscope unit 110, an observation device 10 that photographs a patient's surgical part with the microscope unit 110, and a surgical unit imaged by the observation device 10. And a display device 20 for displaying the video.
- the observation system 1 is a medical observation system for observing an observation target part (operation target part (surgical part) or inspection target part) that is a part of a patient's body when performing a medical action such as surgery or examination. It is. At the time of surgery or examination, the surgeon observes the target site while referring to the video imaged by the observation device 10 and displayed on the display device 20, and performs various treatments on the target site as necessary. .
- an operation part surgical part
- inspection target part that is a part of a patient's body when performing a medical action such as surgery or examination. It is.
- the surgeon observes the target site while referring to the video imaged by the observation device 10 and displayed on the display device 20, and performs various treatments on the target site as necessary.
- the site to be observed is also referred to
- the display device 20 displays an image of the patient's surgical site taken by the observation device 10 under the control of the control device 140 described later.
- the display device 20 is installed in a place that can be visually recognized by an operator in the operating room, such as a wall surface of the operating room.
- the type of the display device 20 is not particularly limited. Examples of the display device 20 include various known displays such as a CRT (Cathode Ray Tube) display device, a liquid crystal display device, a plasma display device, and an EL (Electro-Luminescence) display device.
- An apparatus may be used.
- the display device 20 does not necessarily have to be installed in the operating room, and is mounted on a device worn by the surgeon such as a head mounted display (HMD) or a glasses-type wearable device. May be.
- HMD head mounted display
- a glasses-type wearable device May be.
- the imaging unit 111 of the microscope unit 110 of the observation apparatus 10 is configured as a stereo camera or configured to be capable of high-resolution imaging
- 3D display is correspondingly performed.
- a display device 20 capable of displaying at a high resolution is possible.
- the observation apparatus 10 includes a microscope unit 110 for magnifying and observing a surgical site of a patient, a support unit 120 (arm unit 120) that holds the microscope unit 110, and one end of the support unit 120 connected to each other.
- movement of the observation apparatus 10 and the observation system 1 are provided.
- the base unit 130 supports the microscope unit 110 and the support unit 120.
- the base part 130 includes a pedestal 131 having a plate shape and a plurality of casters 132 provided on the lower surface of the pedestal 131.
- One end of the support unit 120 is connected to the upper surface of the gantry 131, and the microscope unit 110 is connected to the other end (tip) of the support unit 120 extending from the gantry 131.
- the observation apparatus 10 is configured to be in contact with the floor surface via the caster 132 and to be movable on the floor surface by the caster 132.
- a direction perpendicular to the floor on which the observation apparatus 10 is installed is defined as a z-axis direction.
- the z-axis direction is also referred to as the vertical direction or the vertical direction.
- Two directions orthogonal to the z-axis direction are defined as an x-axis direction and a y-axis direction, respectively.
- a direction parallel to the xy plane is also referred to as a horizontal direction.
- the microscope unit 110 is configured by a microscope body for magnifying and observing a patient's surgical site.
- the optical axis direction of the microscope unit 110 substantially coincides with the z-axis direction.
- the microscope unit 110 has a configuration corresponding to an electronic imaging type microscope unit, and includes a cylindrical part 112 having a substantially cylindrical shape, and an imaging unit 111 provided in the cylindrical part 112.
- the imaging unit 111 includes an optical system including optical elements such as an objective lens, a zoom lens, and a focus lens, and a mirror, and an imaging element that captures an image of a surgical site that is an observation target using light that has passed through the optical system. Is composed of.
- a cover glass for protecting the imaging unit 111 is provided on the opening surface at the lower end of the cylindrical portion 112.
- a light source is also provided inside the cylindrical portion 112, and illumination light is emitted from the light source to the subject through the cover glass during photographing. Reflected light (observation light) from the subject of the illumination light is incident on the imaging unit 111 through the cover glass, whereby a signal (video signal) related to the image of the subject is acquired by the imaging unit 111.
- imaging unit 111 configurations used in various known electronic imaging microscope units may be applied, and thus detailed description thereof is omitted here.
- various known imaging elements such as a CCD (Charge Coupled Device) sensor and a CMOS (Complementary Metal-Oxide-Semiconductor) sensor may be applied as the imaging element of the imaging unit 111.
- the imaging unit 111 may be configured as a so-called stereo camera including a pair of imaging elements corresponding to 3D display.
- Various known configurations can also be applied to the optical system of the imaging unit 111.
- the imaging unit 111 can be equipped with various functions that are generally provided in an electronic imaging type microscope unit, such as an AF (Auto Focus) function and an optical zoom function.
- AF Auto Focus
- the imaging unit 111 is preferably configured to be capable of shooting at a high resolution, such as 4K or 8K. Since the imaging unit 111 is configured to be capable of shooting at a high resolution, it is possible to display an image on the display device 20 having a large screen of, for example, 50 inches or more while ensuring a predetermined resolution. Visibility of the person is improved. In addition, since the imaging unit 111 is configured to be capable of shooting at high resolution, it is possible to ensure a predetermined resolution even when an image is appropriately enlarged and displayed by the electronic zoom function. For example, the Full HD image quality can be ensured even when the image is enlarged by the electronic zoom function by 2 times for 4K, 4 times for 8K, and 8 times for 16K.
- the microscope unit 110 is not required to have an optical zoom function up to that point, and the optical system of the microscope unit 110 can be simplified. Therefore, the microscope unit 110 can be configured more compactly. .
- the microscope unit 110 is preferably smaller, but by configuring the imaging unit 111 to be capable of imaging at high resolution, such a microscope can be used. The effect which can make the part 110 smaller can also be show
- the magnification in the electronic zoom function is 1 ⁇ ( ⁇ 1), 1.5 ⁇ ( ⁇ 1.5), 2 It can be adjusted to any one of double ( ⁇ 2), four times ( ⁇ 4) and eight times ( ⁇ 8).
- the imaging unit 111 is capable of shooting at a resolution that can maintain a predetermined resolution (for example, Full HD image quality) even when the magnification in the electronic zoom function is 2 times or more, for example. Can be configured.
- the video signal acquired by the microscope unit 110 that is, video data is transmitted to the control device 140, and the control device 140 relates to the video data, for example, gamma correction processing, white balance adjustment processing, and electronic zoom function.
- Enlargement processing for example, magnification is 1 ⁇ ( ⁇ 1), 1.5 ⁇ ( ⁇ 1.5), 2 ⁇ ( ⁇ 2), 4 ⁇ ( ⁇ 4) or 8 ⁇ ( ⁇ 8)
- image processing such as correction processing is performed.
- various types of image processing that are generally performed to display video on the display device 20 may be performed.
- Various settings such as white balance and electronic zoom magnification in the image processing may be appropriately set by the operator.
- control device 140 detects the occurrence of an unintended video movement, and when the video movement is detected, the image is corrected so as to correct the video movement. Process. The details of this video movement correction processing will be described later in (1-2. Functional configuration of video movement correction system).
- the video data on which the above image processing has been performed is transmitted to a display device 20 provided in the operating room, and an image of the surgical site is transmitted to the display device 20 by using, for example, an optical zoom function and / or an electronic zoom function.
- the image is enlarged and displayed as appropriate.
- the communication between the control device 140 and the display device 20 may be realized by various known wired or wireless methods.
- the microscope unit 110 may be provided with a processing circuit for performing the above image processing.
- video data after image processing is appropriately performed in the processing circuit mounted on the microscope unit 110 can be transmitted from the microscope unit 110 to the display device 20.
- communication between the microscope unit 110 and the display device 20 may be realized by various known wired or wireless methods.
- the microscope unit 110 is provided with various switches for controlling the operation of the microscope unit 110.
- the microscope unit 110 includes a zoom switch 151 (zoom SW 151) and a focus switch 152 (focus SW 152) for adjusting imaging conditions of the microscope unit 110, and an operation for changing the operation mode of the support unit 120.
- a mode change switch 153 (operation mode change SW 153) is provided.
- the surgeon can adjust the magnification and the focal length of the microscope unit 110 by operating the zoom SW 151 and the focus SW 152, respectively. Further, the surgeon can switch the operation mode of the support unit 120 to either the fixed mode or the free mode by operating the operation mode change SW 153.
- the fixed mode is an operation mode in which the position and posture of the microscope unit 110 are fixed by restricting rotation of each rotation shaft provided in the support unit 120 by a brake.
- the free mode is an operation mode in which each rotation shaft provided in the support portion 120 can freely rotate when the brake is released.
- the position and posture of the microscope unit 110 can be adjusted by a direct operation by an operator.
- the direct operation means an operation in which an operator holds the microscope unit 110 by hand and moves the microscope unit 110 directly.
- the operation mode of the support unit 120 is in the free mode while the operator presses the operation mode change SW 153, and the operation mode of the support unit 120 is fixed while the operator releases the operation mode change SW 153. It becomes a mode.
- switches are not necessarily provided in the microscope unit 110.
- a mechanism for accepting an operation input having a function equivalent to those of the switches may be provided in the observation apparatus 10, and the specific configuration of the mechanism is not limited.
- these switches may be provided in other parts of the observation apparatus 10.
- commands corresponding to these switches may be remotely input to the observation apparatus 10 using an input device such as a remote controller or a foot switch.
- the cylindrical portion 112 of the microscope unit 110 is illustrated as a simple cylindrical member in FIG. 1, but the cylindrical portion 112 has a gripping portion to be gripped by an operator.
- the gripping part can be realized by forming a structure such as a handle gripped by the operator on the outer periphery of the cylindrical part 112.
- the gripping portion can be realized by forming the shape of the cylindrical portion 112 into a shape that can be easily gripped by an operator. For example, as described above, in the free mode, an operation in which the operator moves the microscope unit 110 while directly holding the tubular unit 112 with a hand can be assumed.
- the shape of the cylindrical portion 112 and the arrangement position of the operation mode change SW 153 are the same as those in the free mode. It can be appropriately determined in consideration of the operability of the operator. Similarly, the arrangement positions of the zoom SW 151 and the focus SW 152 may be appropriately determined in consideration of the operator's operability.
- the control device 140 may be, for example, a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor), or a control board on which these processors and a storage element such as a memory are mounted. Each function in the control device 140 is realized by a processor constituting the control device 140 executing arithmetic processing according to a predetermined program.
- a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor)
- a control board on which these processors and a storage element such as a memory are mounted.
- Each function in the control device 140 is realized by a processor constituting the control device 140 executing arithmetic processing according to a predetermined program.
- the control device 140 controls the driving of the brakes provided in the respective rotation shafts of the support unit 120 in accordance with the operator's operation input via the operation mode change SW 153, so that the support unit 120 described above is controlled. It has a function of switching operation modes.
- the control device 140 appropriately drives the optical system of the imaging unit 111 of the microscope unit 110 in accordance with an operation input from the operator via the zoom SW 151 and the focus SW 152, and the magnification and focal length of the microscope unit 110. It has a function to adjust.
- the control device 140 has a function of performing various image processing on the video data acquired by the microscope unit 110 and causing the display device 20 to display a video based on the processed video data. At this time, in the first embodiment, the control device 140 performs video movement correction processing as necessary, and displays a video in which unintended video movement is suppressed on the display device 20.
- the control device 140 is provided as a different configuration from the microscope unit 110, the support unit 120, and the base unit 130, and is connected to the base unit 130 by a cable, but the first embodiment is applied. It is not limited to examples.
- a processor, a control board, or the like that realizes the same function as that of the control device 140 may be disposed in the base unit 130.
- the control device 140 and the microscope unit 110 may be integrally configured by incorporating a processor, a control board, and the like that realize the same function as the control device 140 into the microscope unit 110.
- a processor, a control board, etc. are each arrange
- the support unit 120 moves the microscope unit 110 three-dimensionally, and supports the microscope unit 110 after movement in a fixed manner in its position and posture.
- the support portion 120 is configured as a balance arm having six degrees of freedom.
- the first embodiment is not limited to such an example, and the support unit 120 only needs to be configured to be able to move the microscope unit 110 appropriately according to the application, and has other different numbers of degrees of freedom. May be configured.
- the support unit 120 includes six rotation axes (first axis O 1 , second axis O 2 , third axis O 2 , fourth axis O 4 , fifth axis O 5, and sixth axis corresponding to six degrees of freedom. O 6 ) is provided.
- members constituting each rotation shaft are collectively referred to as a rotation shaft portion.
- the rotating shaft portion can be configured by a bearing, a shaft that is rotatably inserted into the bearing, a brake that restricts rotation of the rotating shaft, and the like.
- a parallelogram link mechanism 240 described later can also be regarded as one of the rotating shaft portions.
- the support portion 120 includes a first rotation shaft portion 210, a second rotation shaft portion 220, a third rotation shaft portion 230, a fourth rotation shaft portion 240, a fifth rotation shaft portion 250, and a sixth rotation shaft corresponding to each rotation shaft.
- a counterweight 280 for balancing the moments of the microscope unit 110 and the support unit 120 as a whole.
- the fourth rotation shaft portion 240 corresponds to the parallelogram link mechanism 240.
- the side on which the microscope unit 110 is provided is also referred to as the distal end side or the distal end portion, and the side closer to the base unit 130 is the proximal end side or the proximal end portion.
- the first rotating shaft portion 210 has a substantially cylindrical shape, and is arranged at the proximal end portion of the cylindrical portion 112 of the microscope portion 110 so that the central axis thereof substantially coincides with the central axis of the cylindrical portion 112 of the microscope portion 110. Connected.
- the first rotating shaft part 210 supports the microscope part 110 in a rotatable manner with the direction substantially coincident with the optical axis of the microscope part 110 as a rotating axis direction (first axis O 1 direction).
- the first axis O 1 is provided as a rotation axis substantially parallel to the z axis.
- a part of the imaging unit 111 of the microscope unit 110 is stored in a cylindrical casing that forms the first rotating shaft unit 210. That is, the microscope unit 110 and the first rotating shaft unit 210 are configured as an integral member. However, this embodiment is not limited to such an example, and the first rotating shaft part 210 and the microscope part 110 may be configured as separate members.
- the first rotating shaft portion 210 is connected to the tip of a first arm portion 271 that extends in a direction substantially perpendicular to the first axis O 1 . Further, the proximal end of the first arm portion 271, a first arm portion 271 rotatably stretching direction substantially parallel to the direction of the first arm portion 271 as a rotation axis direction (second axis O 2 direction) A second rotating shaft portion 220 to be supported is provided.
- the second axis O 2 is a rotation axis that is substantially perpendicular to the first axis O 1, and is provided as a rotation axis that is substantially parallel to the y axis in the example shown in FIG.
- the position of the microscope unit 110 in the x-axis direction is adjusted by the rotation of the microscope unit 110 and the first arm unit 271 about the second axis O 2 by the second rotation shaft unit 220.
- the second rotating shaft part 220 is connected to the tip of a second arm part 272 that extends in a direction substantially perpendicular to the first axis O 1 and the second axis O 2 . Further, the base end side of the second arm portion 272 is bent in an approximately L shape, and a direction substantially parallel to the extending direction of the portion corresponding to the long side of the second arm portion 272 is located at a position corresponding to the bent short side.
- the rotation axis direction (third axis O 3 direction) as the third rotating shaft 230 a second arm portion 272 is rotatably supported is provided.
- the third axis O 3 is a rotation axis substantially perpendicular to the first axis O 1 and the second axis O 2, and is provided as a rotation axis substantially parallel to the x axis in the example shown in FIG.
- the microscope section 110 of the third shaft O 3 as the rotation axis, since the first arm portion 271 and the second arm portion 272 is pivoted, adjusted y-axis direction position of the microscope 110 Will be.
- the support unit 120 is configured to control the posture of the microscope unit 110 by controlling the rotation about the second axis O 2 and the third axis O 3, respectively. That is, the second rotation shaft part 220 and the third rotation shaft part 230 may be rotation shaft parts that define the posture of the microscope unit 110.
- the distal end of the upper side of the parallelogram link mechanism 240 is connected to the proximal end side of the third rotating shaft part 230.
- the parallelogram link mechanism 240 includes four arms (arms 241, 242, 243, 244) arranged in a parallelogram shape, and four joints provided at positions corresponding to substantially the apexes of the parallelogram. (Joint portions 245, 246, 247, 248).
- the tip end of an arm 241 that extends in a direction substantially parallel to the third axis O 3 is connected to the third rotating shaft portion 230.
- a joint portion 245 is provided near the distal end of the arm 241, and a joint portion 246 is provided near the proximal end.
- the joints 245 and 246 are connected to the ends of the arms 242 and 243, respectively, so as to be rotatable about rotation axes (fourth axis O 4 ) that are substantially perpendicular to the extending direction of the arm 241 and substantially parallel to each other.
- joint portions 247 and 248 are provided at the base ends of the arms 242 and 243, respectively.
- the joint portions 247 and 248 are respectively connected to the distal end and the proximal end of the arm 244 so as to be rotatable about the fourth axis O 4 and substantially parallel to the arm 241.
- the four joint portions constituting the parallelogram link mechanism 240 have rotation axes (fourth axis O 4 ) in substantially the same direction that are substantially parallel to each other, and are interlocked with each other around the fourth axis O 4.
- the fourth axis O 4 is provided as a rotation axis that is substantially parallel to the y axis. That is, the parallelogram link mechanism 240 is configured to have a plurality of joint portions that are arranged at different positions and rotate in conjunction with each other on the rotation shaft in the same direction, and transmits the operation at one end to the other end. Act as a mechanism.
- the configuration on the tip side of the parallelogram link mechanism 240 (that is, the microscope unit 110, the first rotating shaft unit 210, the second rotating shaft unit 220, the third rotating shaft unit). 230, the first arm portion 271 and the second arm portion 272) are transmitted to the base end side of the parallelogram link mechanism 240.
- a parallelogram linkage 240 pivotally A fifth rotating shaft portion 250 to be supported.
- the fifth axis O 5 is a rotation axis substantially parallel to the fourth axis O 4, and is provided as a rotation axis substantially parallel to the y axis in the example shown in FIG.
- a distal end of a third arm portion 273 extending in the z-axis direction is connected to the fifth rotating shaft portion 250, and the microscope portion 110, the first arm portion 271, the second arm portion 272, and the parallelogram link mechanism.
- 240 is configured to be rotatable with respect to the third arm portion 273 with the fifth axis O 5 as a rotation axis via the fifth rotation shaft portion 250.
- the third arm portion 273 has a substantially L-shaped shape, and the base end side thereof is bent so as to be substantially parallel to the floor surface.
- a sixth rotation shaft capable of rotating the third arm portion 273 around a rotation axis (sixth axis O 6 ) orthogonal to the fifth axis O 5.
- Unit 260 is connected.
- the sixth axis O 6 is provided as a rotation axis that is substantially parallel to the z axis.
- the sixth rotation shaft portion 260 is configured integrally with a fourth arm portion 274 that extends in the vertical direction. That is, the distal end of the fourth arm portion 274 is connected to a surface substantially parallel to the floor surface of the base end of the third arm portion 273. Further, the base end of the fourth arm portion 274 is connected to the upper surface of the gantry 131 of the base portion 130.
- the microscope unit 110, the first arm unit 271, the second arm unit 272, the parallelogram link mechanism 240, and the third arm unit 273 pass the sixth axis O 6 through the sixth rotation shaft unit 260. It rotates with respect to the base part 130 as a rotation axis.
- the arm 244 constituting the lower side of the parallelogram link mechanism 240 is formed to be longer than the arm 241 constituting the upper side, and the third rotating shaft portion 230 of the parallelogram link mechanism 240 of the arm 242 is formed. An end located diagonally to the connected portion extends to the outside of the parallelogram link mechanism 240. A counterweight 280 is provided at the end of the extended arm 244.
- the counterweight 280 is arranged on the tip side from itself (that is, the microscope unit 110, the first rotating shaft unit 210, the second rotating shaft unit 220, the third rotating shaft unit 230, the first arm unit 271, The mass of the second arm portion 272 and the parallelogram link mechanism 240) so that the rotational moment generated around the fourth axis O 4 and the rotational moment generated around the fifth axis O 5 can be offset.
- the placement position has been adjusted.
- the arrangement position of the fifth rotation shaft portion 250 is adjusted so that the center of gravity of each component arranged on the tip side of the fifth rotation shaft portion 250 is located on the fifth axis O 5 . Furthermore, the arrangement position of the sixth rotation shaft portion 260 is adjusted so that the center of gravity of each component arranged on the tip side of the sixth rotation shaft portion 260 is located on the sixth axis O 6 .
- the support unit 120 can be used as the microscope unit 110 and the support unit.
- the whole 120 can be configured as a balance arm in which moments are balanced.
- the support unit 120 is configured as a balance arm, when the surgeon tries to move the microscope unit 110 by direct operation, the microscope unit 110 is moved with a smaller external force as if it is under zero gravity. It can be moved. Accordingly, the operability of the operator can be improved.
- Brakes that restrict the rotation of the first rotation shaft portion 210 to the sixth rotation shaft portion 260 are provided on the first rotation shaft portion 210 to the sixth rotation shaft portion 260 of the support portion 120, respectively. Since the parallelogram link mechanism 240 has four joint portions (joint portions 245 to 248) that rotate in conjunction with each other, the brake for the parallelogram link mechanism 240 is applied to at least one of these four joint portions. What is necessary is just to be provided.
- the driving of these brakes is controlled by the control device 140. Under the control of the control device 140, these brakes are released all at once, so that the operation mode of the support portion 120 shifts to the free mode. Similarly, these brakes are driven simultaneously under the control of the control device 140, so that the operation mode of the support portion 120 shifts to the fixed mode.
- the various brakes used for a general balance arm may be applied, and the specific mechanism is not limited.
- these brakes may be mechanically driven or may be electromagnetic brakes that are electrically driven.
- observation system 1 and the observation apparatus 10 may be configured as follows.
- an actuator is mounted on at least one of the rotating shafts of the support unit 120, and driving of the actuator is controlled by the control device 140, thereby controlling the position and orientation of the support unit 120, that is, a microscope.
- the position and orientation of the unit 110 may be controlled.
- the support unit 120 may be remotely operated in accordance with an operation input performed by the surgeon via various input devices such as a remote controller and a foot switch.
- the input device may be a pressure-sensitive sensor provided in a partial region of the outer peripheral surface of the microscope unit 110. For example, when the surgeon touches the pressure sensor, the driving of the actuator may be controlled so that the microscope unit 110 moves in the touched direction.
- the observation system 1 is instructed to move the microscope unit 110 to the observation apparatus 10.
- An issuing navigation device may be provided.
- the control device 140 drives the actuator, and the position and orientation of the microscope unit 110 can be controlled.
- said input device or a navigation apparatus the various well-known things used when moving a microscope part in a general observation apparatus may be used.
- the observation system 1 there is a request to slightly move the visual field of the image on the screen of the display device 20, or the visual field is changed to any one without changing the vertical and horizontal directions of the video on the screen of the display device 20.
- the microscope unit 110 By controlling the movement of the microscope unit 110 by the actuator as described above, the microscope unit 110 can be moved more accurately to a position where a desired field of view can be obtained.
- the movement of the microscope unit 110 by a direct operation is referred to as manual movement of the microscope unit 110, and the movement of the microscope unit 110 by driving an actuator is an automatic microscope.
- the movement of the unit 110 may be called.
- FIG. 2 is a diagram showing a state of surgery using the observation system 1 shown in FIG.
- FIG. 2 schematically shows that the surgeon 401 is performing an operation on the surgical part 407 of the patient 405 on the patient bed 403 using the observation system 1.
- the observation apparatus 10 is illustrated in a simplified manner.
- the microscope unit 110 and the support unit 120 of the observation apparatus 10 may be positioned between the operator 401 and the display device 20. Therefore, the microscope unit 110 and the support unit 120 are required to be small in size so as not to obstruct the view of the operator 401 as much as possible.
- the support unit 120 can be configured so that the configuration of the support unit 120, particularly the vicinity of the microscope unit 110, is as small as possible. Thereby, the field of view of the surgeon 401 who sees the display device 20 becomes better, and the surgery can be performed more smoothly.
- the diameter of the thinnest part in the support part 120 is 60 mm or less.
- the diameter of the narrowest part may be more preferably 30 mm or less, still more preferably 20 mm or less, and particularly preferably 10 mm or less. As the diameter of the thinnest portion becomes smaller, the support portion 120 is configured to be smaller, and the field of view of the operator 401 can be further ensured.
- the thinnest part in the support part 120 is, for example, the inside of the arm parts (the first arm part 271, the second arm part 272, the third arm part 273, and the fourth arm part 274) constituting the support part 120.
- the second rotation shaft portion 220 and the third rotation shaft portion corresponding to the rotation axes (the second axis O 2 and the third axis O) that are relatively close to the microscope unit 110 (for example, the attitude of the microscope unit 110 that can define the attitude of the microscope unit 110) 230 may be an arm portion arranged near (230).
- the second arm portion 272 located between the second rotation shaft portion 220 and the third rotation shaft portion 230 can correspond to the thinnest portion of the support portion 120.
- the support portion 120 when the support portion 120 is reduced in size and diameter, the rigidity is reduced.
- the observation device 10 there is a mode in which the surgeon 401 moves the microscope unit 110 by a direct operation to a position where a desired field of view can be obtained while viewing the video displayed on the display screen in the free mode. is assumed.
- the rigidity of the support part 120 is relatively small, the support part 120 is elastically deformed comparatively greatly as the operator 401 moves the microscope part 110. Therefore, when the surgeon moves the microscope unit 110 to a desired position and releases his / her hand, the support unit 120 elastically restores to cause the microscope unit 110 to move unintentionally, resulting in image shift.
- the observation device 10 may be configured to be capable of shooting at a high resolution as described above, thereby displaying an image on the display device 20 having a larger screen while ensuring a predetermined resolution.
- the video can be appropriately enlarged and displayed by the electronic zoom function.
- the shift amount in the video shift becomes more prominent.
- the microscope unit 110 may be shaken by, for example, vibration of the floor accompanying walking of a medical staff.
- the microscope unit 110 may shake due to inertia when the microscope unit 110 is stopped. Such unintentional vibration of the microscope unit 110 causes image shaking.
- a video movement correction system for correcting unintended video movement is incorporated into the observation system 1 described above.
- the video movement correction system unintentional video movement that occurs during the surgery is appropriately corrected, so that safer surgery can be realized.
- FIG. 3 is a block diagram illustrating an example of a functional configuration of the video movement correction system according to the first embodiment.
- the video movement correction system 2 according to the first embodiment includes an imaging unit 310, a display unit 320, a visual field movement instruction acquisition unit 330, and a control unit 340 as its functions.
- the imaging unit 310 corresponds to the imaging unit 111 shown in FIG.
- the imaging unit 310 acquires video data obtained by capturing an observation target during surgery as needed, and transmits the acquired video data to the storage unit 341 of the control unit 340 described later.
- the display unit 320 includes the display device 20 shown in FIG.
- the display unit 320 displays the video imaged by the imaging unit 310 based on the video data appropriately subjected to image processing by the image processing unit 344 of the control unit 340 described later.
- the video data acquired by the imaging unit 310 is also referred to as captured video data
- the video data displayed by the display unit 320 that is, various types of image processing for the captured video data. Is referred to as display video data).
- the visual field movement instruction acquisition unit 330 acquires an explicit visual field movement instruction (hereinafter also simply referred to as a visual field movement instruction) for the video displayed on the display unit 320.
- the visual field movement instruction may mean an instruction to move the microscope unit 110 illustrated in FIG. 1 (that is, an instruction to move the imaging unit 111).
- the explicit visual field movement instruction is an intentional visual field movement instruction by the operator.
- the visual field movement instruction acquisition unit 330 is, for example, the operation mode illustrated in FIG. 1. It can be configured by a change SW 153.
- the visual field movement instruction acquisition unit 330 can acquire a visual field movement instruction when an operation for changing the operation mode of the support unit 120 to the free mode is input to the operation mode change SW 153. While the operation mode change SW 153 is operated and the operation mode is in the free mode, it can be considered that an instruction for moving the microscope unit 110 is input by a direct operation by the operator. It is.
- the visual field movement instruction acquisition unit 330 acquires the visual field movement instruction
- the visual field movement instruction acquisition unit 330 transmits information indicating that the visual field movement instruction has been acquired to the determination unit 342 of the control unit 340 described later.
- the visual field movement instruction may be acquired by other methods.
- the visual field movement instruction acquiring unit 330 may be configured by various input devices (not shown in FIG. 1) (the above-described remote controller, foot switch, pressure sensor, etc.), and a microscope unit for these input devices of the surgeon.
- a visual field movement instruction may be acquired.
- the observation system 1 includes a navigation device that issues an instruction for moving the microscope unit 110
- the visual field movement instruction acquisition unit 330 may be configured by a communication device or the like that receives an instruction from the navigation device.
- a visual field movement instruction may be acquired.
- the control unit 340 includes a control board on which a processor such as a CPU and a storage element such as a memory are mounted. Note that the control unit 340 may be configured by the control device 140 illustrated in FIG. 1, or may be configured by a device separate from the control device 140.
- the control unit 340 comprehensively controls various processes in the video movement correction system 2.
- the control unit 340 includes a storage unit 341, a determination unit 342, and a correction unit 343 as its functions. These functions are realized by the processor constituting the control unit 340 operating according to a predetermined program.
- the storage unit 341 includes a storage element such as a memory.
- the storage unit 341 stores the captured video data acquired by the imaging unit 310.
- the storage unit 341 temporarily stores imaged video data for a predetermined time necessary for performing determination processing of presence / absence of video movement by the determination unit 342 described later and video movement correction processing by the correction unit 343. Is possible.
- the storage unit 341 stores the captured video data for the predetermined time including the latest captured video data while updating the captured video data as needed.
- the storage unit 341 provides the stored captured video data to the determination unit 342 and the correction unit 343 as appropriate, as necessary.
- the determining unit 342 determines whether or not a visual field movement instruction is input and whether or not a video movement has occurred. Specifically, the determination unit 342 determines whether or not a visual field movement instruction is input based on information indicating that the visual field movement instruction transmitted from the visual field movement instruction acquisition unit 330 has been acquired. Further, the determination unit 342 determines whether or not a video movement has occurred by analyzing the captured video data stored in the storage unit 341. For example, the determination unit 342 can determine whether video movement has occurred by calculating a difference between the latest captured video data and past captured video data (for example, captured video data immediately before the captured video data). .
- the discriminating unit 342 provides the correction unit 343 with information on whether or not a visual field movement instruction has been input and whether or not a video movement has occurred.
- the correction unit 343 performs a video movement correction process when the determination unit 342 determines that there is no input of a visual field movement instruction and that a video movement has occurred. This is because the above case is considered to be a case where a video movement unintended by the surgeon has occurred. For example, an unintended video shift occurs when an image movement occurs even though an operation for changing the operation mode of the support unit 120 to the free mode is not input to the operation mode change SW 153. It is expected that In addition, for example, although an operation input for moving the microscope unit 110 is not performed on various input devices or an instruction for moving the microscope unit 110 is not input from the navigation device, When video movement has occurred, it is expected that unintended video shaking has occurred.
- the video correction process is a process of correcting video movement and generating display video data based on captured video data.
- the correction unit 343 functions as the image processing unit 344 and generates display video data in which video movement is corrected by appropriately performing image processing on the captured video data.
- FIG. 4 is a diagram for explaining a generation process of display video data in which video movement is corrected by the image processing unit 344.
- FIG. 4 schematically shows a captured image 409 of the surgical unit 407 captured by the imaging unit 310.
- the image processing unit 344 cuts out a predetermined area 411 in the captured video 409, and performs various image processing (for example, gamma correction, white balance) on the video data in the cut-out area 411.
- Display image data is generated by performing adjustments of (2), enlargement and inter-pixel correction related to the electronic zoom function, and the like.
- various types of image processing that are generally performed to display video may be performed.
- the image processing unit 344 analyzes the captured video data stored in the storage unit 341 to calculate the moving amount and moving direction of the video in moving the video, and the calculated moving amount and moving direction are calculated. Accordingly, by appropriately adjusting the cutout position of the region 411 so as to cancel the video movement, display video data in which the video movement is corrected can be generated.
- the correction unit 343, that is, the image processing unit 344 performs only normal image processing on the captured video data without generating video movement correction processing, and generates display video data. Also in this case, the image processing unit 344 cuts out the predetermined area 411 in the captured video 409 and generates display video data, but the image processing unit 344 only cuts out the predetermined area 411 specified in advance. The process of adjusting the position of the area 411 according to the moving amount and moving direction of the video movement is not performed. For example, the predetermined region 411 when the image movement correction is not performed can be set as a region having a shape similar to the captured image 409 and having the same position as the center of the captured image 409.
- the image processing unit 344 cuts out a predetermined area 411 from the captured video 409, and performs various types of image processing on the captured video data corresponding to the area 411, thereby displaying the display video data. At this time, if an unintended video movement has not occurred, the area 411 at a predetermined position is cut out. If an unintended video movement has occurred, the area 411 is canceled so as to cancel the video movement. Adjust the cutout position as appropriate. As a result, even when an unintended video movement occurs, display video data in which the video movement is corrected is generated.
- the image processing unit 344 transmits the generated display video data to the display unit 320, and causes the display unit 320 to display a video based on the display video data.
- display video data in which the video movement is corrected is generated, so that a more stable video in which video movement is suppressed for the surgeon. Will be provided. Therefore, the operation can be performed more smoothly and safely.
- FIG. 5 is a flowchart illustrating an example of a processing procedure of the video movement correction method according to the first embodiment.
- Each process shown in FIG. 5 corresponds to a process executed by the control unit 340 of the video movement correction system 2 shown in FIG. Since details of each of these processes have already been described when the functional configuration of the video movement correction system 2 is described, in the following description of the processing procedure of the video movement correction method, a detailed description of each process will be given. Will be omitted.
- step S101 captured video data acquired by the imaging unit 310 is stored (step S101).
- the processing in step S101 corresponds to the processing executed by the storage unit 341 shown in FIG.
- step S103 a visual field movement instruction is fetched (step S103).
- the processing in step S103 corresponds to the processing in which information indicating that the visual field movement instruction acquired by the visual field movement instruction acquiring unit 330 illustrated in FIG. 3 is input is transmitted to the control unit 340.
- step S105 it is determined whether or not a visual field movement instruction has been input.
- the process in step S105 corresponds to the process executed by the determination unit 342 shown in FIG.
- step S105 If it is determined in step S105 that a visual field movement instruction has been input, the process proceeds to step S107.
- a predetermined area of the captured video (for example, a predetermined area centered on the same position as the center of the captured video) is determined based on the stored captured video data in step S107 without performing the video movement correction process.
- the video data is cut out, and various video processes are performed on the captured video data corresponding to the cut out area, thereby generating display video data. Note that the processing in step S107 corresponds to the processing executed by the image processing unit 344 shown in FIG.
- step S105 determines whether the visual field movement instruction has been input. If it is determined in step S105 that the visual field movement instruction has not been input, the process proceeds to step S109.
- step S109 video movement is detected based on the stored captured video data.
- step S111 the presence / absence of video movement is determined based on the detection result. The processing in step S109 and step S111 corresponds to the processing executed by the determination unit 342 shown in FIG.
- step S111 If it is determined in step S111 that no video movement has occurred, it is naturally not necessary to perform a video movement correction process. Therefore, similarly to the case where it is determined that the visual field movement instruction is input, the process proceeds to step S107, and the display video data is generated without performing the video movement correction process.
- step S111 determines whether a video movement has occurred, it is considered that a video movement unintended by the surgeon has occurred. Therefore, in this case, the process proceeds to step S113, and the position of the area to be cut out from the picked-up video is adjusted so as to cancel the video movement based on the stored picked-up video data, and this cut-out area is supported.
- Display video data is generated by performing various types of image processing on the captured video data. Note that the processing in step S113 corresponds to the processing executed by the correction unit 343 shown in FIG. 3, that is, the image processing unit 344.
- step S115 the processing executed by the image processing unit 344 shown in FIG.
- FIG. 6 is a block diagram illustrating an example of a functional configuration of the video movement correction system according to the second embodiment.
- the video movement correction system 3 includes a display unit 320, a visual field movement instruction acquisition unit 330, and a microscope unit 350 as its functions.
- the microscope unit 350 includes an imaging unit 310, a control unit 340a, and a motion detection unit 360 as functions thereof.
- the functions of the imaging unit 310, the display unit 320, and the visual field movement instruction acquisition unit 330 are substantially the same as those in the first embodiment. That is, the imaging unit 310 acquires video data obtained by capturing an observation target during surgery as needed, and provides it to an image processing unit 344a of the control unit 340a described later.
- the display part 320 displays the video image
- the visual field movement instruction acquisition unit 330 acquires a visual field movement instruction and provides it to the determination unit 342a of the control unit 340a described later.
- the microscope unit 350 corresponds to the microscope unit 110 shown in FIG.
- the control unit 340a includes a processor such as a CPU or a control board on which a program and a storage element such as a memory are mounted.
- the motion detection unit 360 is configured by a sensor that can detect the movement of the microscope unit 350 (more specifically, the movement of the imaging unit 310) such as an acceleration sensor or a gyro sensor.
- the microscope unit 350 includes the imaging unit 310, a processor or a control board corresponding to the control unit 340a, a sensor corresponding to the motion detection unit 360, Can be configured.
- functions of the control unit 340a and the motion detection unit 360 will be described in more detail.
- the motion detection unit 360 detects the motion of the imaging unit 310 and acquires motion information representing the motion of the imaging unit 310.
- the types of sensors constituting the motion detection unit 360 are not limited, and various sensors that are generally used when detecting the motion of an object, such as the acceleration sensor or the gyro sensor described above, may be used. However, as will be described later, in the second embodiment, based on the motion information acquired by the motion detection unit 360, a process for determining whether or not video movement has occurred and a video movement correction process are performed. Therefore, it is preferable that the motion detection unit 360 is configured to detect at least the motion of the imaging unit 310 in the in-plane direction parallel to the light receiving surface of the imaging element 311 of the imaging unit 310.
- the sensor constituting the motion detection unit 360 be installed relatively close to the imaging unit 310 so that the vibration of the imaging unit 310 can be detected with higher accuracy. Furthermore, in order to reduce the size of the microscope unit 350, it is preferable that the sensor constituting the motion detection unit 360 is also relatively small.
- the motion detection unit 360 provides the acquired motion information to the determination unit 342a and the correction unit 343a of the control unit 340a described later.
- the control unit 340a comprehensively controls various processes in the video movement correction system 3.
- the control unit 340a includes a determination unit 342a and a correction unit 343a as its functions. These functions are realized by the processor constituting the control unit 340a operating according to a predetermined program.
- the control unit 340a may be configured by a processor, a control board, or the like, but these may be the same device as the control device 140 illustrated in FIG. 1 or a part of the control device 140.
- the control device 140 may be a separate device.
- the determination unit 342a determines whether or not a visual field movement instruction is input and whether or not a video movement has occurred. Specifically, the determination unit 342a determines whether or not a visual field movement instruction is input based on information indicating that the visual field movement instruction provided from the visual field movement instruction acquisition unit 330 has been acquired. The process for determining whether or not the visual field movement instruction is input is the same as in the first embodiment.
- the method for determining whether or not video movement has occurred is different from that in the first embodiment.
- the determination unit 342a determines whether or not video movement has occurred based on the motion information provided from the motion detection unit 360. For example, the determination unit 342a determines that a video movement has occurred when the motion detection unit 360 detects the movement of the imaging unit 310 in the in-plane direction parallel to the light receiving surface of the image sensor 311.
- the discriminating unit 342a provides the correction unit 343a with information on the presence / absence of the visual field movement instruction and the discrimination result on whether or not the video movement has occurred.
- the correction unit 343a performs a video movement correction process when the determination unit 342a determines that there is no input of a visual field movement instruction and that a video movement has occurred. This is because, as in the first embodiment, the above case is considered to be a case where a video movement that is not intended by the operator occurs.
- the correction unit 343a performs display image data by appropriately performing image processing on the drive control unit 345 that drives the optical system 312 of the imaging unit 310 and the captured video data. It functions as the image processing unit 344a to be generated.
- the drive control unit 345 calculates the moving amount and moving direction of the image in moving the image based on the motion information provided from the motion detecting unit 360, and moves the image according to the calculated moving amount and moving direction.
- the amount of movement of the lens included in the optical system 312 that guides the observation light to the image sensor 311 so as to cancel is calculated.
- the drive control unit 345 corrects the image movement by appropriately moving the lens according to the calculated movement amount.
- a technique related to a so-called lens shift type camera shake prevention function mounted on a digital camera or the like may be applied to the lens movement processing for canceling the image movement by the drive control unit 345.
- the image processing unit 344a performs various types of image processing (for example, gamma correction, white balance adjustment, enlargement and inter-pixel correction related to the electronic zoom function) on the captured video data acquired by the imaging unit 310.
- image processing various types of image processing that are generally performed to display video may be performed.
- the captured video data acquired by the image processing unit 344a is corrected for video movement. Yes. That is, the image processing unit 344a can generate display video data in which video movement is corrected.
- the image processing unit 344a transmits the generated display video data to the display unit 320, and causes the display unit 320 to display a video based on the display video data.
- display video data in which the video movement is corrected is generated, so that a more stable video in which video movement is suppressed for the surgeon. Will be provided. Therefore, the operation can be performed more smoothly and safely.
- the correction unit 343a corrects image movement using a technique related to a so-called lens shift method, but the second embodiment is not limited to this example.
- the correction unit 343a may correct the image movement using another method.
- the correction unit 343a can correct the image movement using various techniques related to a so-called optical correction method used in a camera shake prevention function mounted on a digital camera or the like.
- the optical correction method refers to the image sensor 311 by moving the position of the image sensor 311 or the optical system 312 (for example, a lens) of the image sensor 310 based on the detected movement of the image sensor 310.
- the lens shift method described above is one of optical correction methods.
- an optical correction method in addition to the lens shift method described above, an image sensor shift method, a lens unit swing method, and the like are known.
- the correcting unit 343a may correct the image movement according to these various methods.
- FIG. 7 is a flowchart illustrating an example of a processing procedure of the video movement correction method according to the second embodiment.
- Each process shown in FIG. 7 corresponds to a process executed by the control unit 340a of the video movement correction system 3 shown in FIG. Details of each of these processes have already been described when the functional configuration of the video movement correction system 3 is described. Therefore, in the following description of the processing procedure of the video movement correction method, a detailed description of each process will be given. Will be omitted.
- step S201 motion information of the imaging unit 310 is captured (step S201).
- the process in step S201 corresponds to the process in which the motion information of the imaging unit 310 acquired by the motion detection unit 360 illustrated in FIG. 6 is provided to the control unit 340a.
- step S203 a visual field movement instruction is fetched (step S203).
- the processing in step S203 corresponds to the processing in which information indicating that the visual field movement instruction acquired by the visual field movement instruction acquisition unit 330 illustrated in FIG. 6 is input is transmitted to the control unit 340a.
- step S205 it is determined whether or not a visual field movement instruction has been input (step S205).
- the processing in step S205 corresponds to the processing executed by the determination unit 342a shown in FIG.
- step S205 If it is determined in step S205 that a visual field movement instruction has been input, the process proceeds to step S207.
- display image data is generated by performing various types of image processing on the captured video data acquired by the imaging unit 310 in step S207 without performing video movement correction processing. Note that the processing in step S207 corresponds to the processing executed by the image processing unit 344a shown in FIG.
- step S205 if it is determined in step S205 that the visual field movement instruction has not been input, the process proceeds to step S209.
- step S209 based on the motion information obtained in step S201, the movement of the imaging unit 310 that causes video movement (specifically, for example, imaging in an in-plane direction parallel to the light receiving surface of the imaging element 311). Motion of the unit 310) is detected.
- step S211 the presence / absence of video movement is determined based on the detection result.
- the processing in step S209 and step S211 corresponds to the processing executed by the determination unit 342a shown in FIG.
- step S211 If it is determined in step S211 that no video movement has occurred, it is naturally not necessary to perform a video movement correction process. Accordingly, similarly to the case where it is determined that the visual field movement instruction is input, the process proceeds to step S207, and the display video data is generated without performing the video movement correction process.
- step S211 determines whether an image movement has occurred, it is considered that an image movement unintended by the surgeon has occurred. Therefore, in this case, the process proceeds to step S213, and the position of the optical system 312 of the imaging unit 310 is moved so as to cancel the image movement.
- step S213 corresponds to the processing executed by the drive control unit 345 of the correction unit 343a shown in FIG.
- step S213 When the processing in step S213 ends, the process proceeds to step S207, and various video processes are performed on the captured video data acquired by the imaging unit 310, thereby generating display video data.
- the captured video data acquired by the imaging unit 310 has been corrected for video movement.
- step S207 display video data for which video movement has been corrected is generated. Will be.
- step S207 When the processing in step S207 is completed, the generated display video data is transmitted to the display unit 320 (step S215). Then, a video based on the display video data is displayed on the display unit 320. Note that the processing in step S215 corresponds to the processing executed by the image processing unit 344a shown in FIG.
- each configuration in each embodiment described above may be combined with each other as much as possible.
- a determination process for determining whether or not a video movement has occurred according to the first embodiment that is, a method for determining by comparing the latest captured video data and past captured video data
- the determination processing of presence / absence of occurrence of video movement according to the second embodiment that is, a determination method based on motion information of the imaging unit 310) may be performed together.
- the determination process can be performed with higher accuracy.
- the present technology is not limited to such an example.
- An imaging unit that acquires captured video data that is video data of an observation target; A support unit for supporting the imaging unit; A discriminating unit that discriminates whether or not a visual field movement instruction is input to the video displayed on the display device based on the captured video data, and whether or not video movement occurs on the video displayed on the display device based on the captured video data.
- a medical observation apparatus comprising: (2) The determining unit determines whether or not there is an input of the visual field movement instruction according to the presence or absence of an operation input for moving the imaging unit via the input device; The medical observation apparatus according to (1) above. (3) The discriminating unit discriminates whether or not the visual field movement instruction is input according to whether or not the instruction is input from a navigation device that issues an instruction to move the imaging unit to the medical observation apparatus. , The medical observation apparatus according to (1) or (2).
- the determination unit determines whether or not the video movement has occurred by comparing the latest captured video data with the captured video data acquired in the past.
- the medical observation apparatus according to any one of (1) to (3).
- a motion detector that acquires motion information representing the motion of the imaging unit; The determination unit determines whether or not the video movement has occurred based on the motion information;
- the medical observation apparatus according to any one of (1) to (4).
- the correction unit corrects the video movement by adjusting a cutout position of the predetermined area when cutting out a predetermined area of the video related to the captured video data and generating the display video data;
- the medical observation apparatus according to any one of (1) to (5).
- a motion detector that acquires motion information representing the motion of the imaging unit;
- the correction unit corrects the image movement by moving a position of an image sensor or an optical system constituting the image pickup unit based on the motion information.
- the medical observation apparatus according to any one of (1) to (5).
- the display video data generated by the correction unit is magnified twice or more by the electronic zoom function and displayed on the display device.
- the medical observation apparatus according to any one of (1) to (7).
- the diameter of the thinnest part in the support part is 30 mm or less, The medical observation apparatus according to any one of (1) to (8).
- Capturing captured video data which is video data of an observation target;
- the processor determines whether or not a visual field movement instruction is input to the video displayed on the display device based on the captured video data, and whether or not video movement occurs on the video displayed on the display device based on the captured video data
- To do Display video data that is video data to be displayed on a display device based on the captured video data when the visual field movement instruction is not input and it is determined that the video movement has occurred.
- a method for correcting image movement including: (11) A medical observation device for photographing an observation object; A display device for displaying an image of the observation object photographed by the medical observation device;
- the medical observation apparatus includes: An imaging unit that acquires captured video data that is video data of an observation target; A support unit for supporting the imaging unit; A discriminating unit that discriminates whether or not a visual field movement instruction is input to the video displayed on the display device based on the captured video data, and whether or not video movement occurs on the video displayed on the display device based on the captured video data.
Landscapes
- Health & Medical Sciences (AREA)
- Life Sciences & Earth Sciences (AREA)
- Engineering & Computer Science (AREA)
- Surgery (AREA)
- Physics & Mathematics (AREA)
- Optics & Photonics (AREA)
- General Health & Medical Sciences (AREA)
- Veterinary Medicine (AREA)
- Pathology (AREA)
- Biomedical Technology (AREA)
- Heart & Thoracic Surgery (AREA)
- Medical Informatics (AREA)
- Molecular Biology (AREA)
- Animal Behavior & Ethology (AREA)
- Nuclear Medicine, Radiotherapy & Molecular Imaging (AREA)
- Public Health (AREA)
- Signal Processing (AREA)
- Biophysics (AREA)
- Radiology & Medical Imaging (AREA)
- General Physics & Mathematics (AREA)
- Multimedia (AREA)
- Chemical & Material Sciences (AREA)
- Analytical Chemistry (AREA)
- Oral & Maxillofacial Surgery (AREA)
- Mechanical Engineering (AREA)
- Computer Vision & Pattern Recognition (AREA)
- Microscoopes, Condenser (AREA)
- Endoscopes (AREA)
Abstract
Description
1.第1の実施形態
1-1.観察システムの構成
1-2.映像移動補正システムの機能構成
1-3.映像移動補正方法
2.第2の実施形態
2-1.映像移動補正システムの機能構成
2-2.映像移動補正方法
3.補足
(1-1.観察システムの構成)
図1を参照して、本開示の第1の実施形態に係る観察システムの構成について説明するとともに、当該観察システムを構成する観察装置の構成について説明する。図1は、第1の実施形態に係る観察システムの構成を示す図である。
表示装置20は、後述する制御装置140からの制御により、観察装置10によって撮影された患者の術部の映像を表示する。表示装置20は、例えば手術室の壁面等、手術室内において術者によって視認され得る場所に設置される。表示装置20の種類は特に限定されず、表示装置20としては、例えばCRT(Cathode Ray Tube)ディスプレイ装置、液晶ディスプレイ装置、プラズマディスプレイ装置、EL(Electro-Luminescence)ディスプレイ装置等、各種の公知の表示装置が用いられてよい。また、表示装置20は、必ずしも手術室内に設置されなくてもよく、ヘッドマウントディスプレイ(HMD:Head Mounted Display)や眼鏡型のウェアラブルデバイスのように、術者が身に付けて使用するデバイスに搭載されてもよい。
観察装置10は、患者の術部を拡大観察するための顕微鏡部110と、顕微鏡部110を保持する支持部120(アーム部120)と、支持部120の一端が接続され顕微鏡部110及び支持部120を支持するベース部130と、観察装置10及び観察システム1の動作を制御する制御装置140と、を備える。
ベース部130は、顕微鏡部110及び支持部120を支持する。ベース部130は板状の形状を有する架台131と、架台131の下面に設けられる複数のキャスター132と、を有する。架台131の上面に支持部120の一端が接続され、架台131から延伸される支持部120の他端(先端)に顕微鏡部110が接続される。また、観察装置10は、キャスター132を介して床面と接地し、当該キャスター132によって床面上を移動可能に構成されている。
顕微鏡部110は、患者の術部を拡大観察するための顕微鏡鏡体によって構成される。図示する例では、顕微鏡部110の光軸方向は、z軸方向と略一致している。顕微鏡部110は、電子撮像式の顕微鏡部に対応する構成を有しており、略円筒形状を有する筒状部112と、筒状部112内に設けられる撮像部111と、から構成される。撮像部111は、対物レンズ、ズームレンズ及びフォーカスレンズ等のレンズやミラー等の光学素子からなる光学系と、当該光学系を通過した光により観察対象である術部の像を撮影する撮像素子と、から構成される。
制御装置140は、例えばCPU(Central Processing Unit)やDSP(Digital Signal Pocessor)等のプロセッサ、又はこれらのプロセッサとメモリ等の記憶素子がともに搭載された制御基板等であり得る。制御装置140を構成するプロセッサが所定のプログラムに従って演算処理を実行することにより、制御装置140における各機能が実現される。
支持部120は、顕微鏡部110を3次元的に移動させるとともに、移動後の顕微鏡部110を、その位置及び姿勢を固定的に支持する。第1の実施形態では、支持部120は、6自由度を有するバランスアームとして構成されている。ただし、第1の実施形態はかかる例に限定されず、支持部120は、用途に応じて顕微鏡部110を適宜移動し得るように構成されればよく、他の異なる数の自由度を有するように構成されてもよい。
図3を参照して、上記映像移動補正システムの構成について詳細に説明する。図3は、第1の実施形態に係る映像移動補正システムの機能構成の一例を示すブロック図である。図3を参照すると、第1の実施形態に係る映像移動補正システム2は、その機能として、撮像部310と、表示部320と、視野移動指示取得部330と、制御部340と、を有する。
図5を参照して、第1の実施形態に係る映像移動補正方法の処理手順について説明する。図5は、第1の実施形態に係る映像移動補正方法の処理手順の一例を示すフロー図である。なお、図5に示す各処理は、図3に示す映像移動補正システム2の制御部340によって実行される処理に対応している。これらの各処理の詳細については、映像移動補正システム2の機能構成について説明する際に既に説明しているため、以下の映像移動補正方法の処理手順についての説明では、各処理についての詳細な説明は割愛する。
本開示の第2の実施形態について説明する。なお、第2の実施形態は、上述した第1の実施形態において、映像移動の発生の有無の判別処理の方法、及び映像移動補正処理の方法が異なるものに対応する。これら以外は第1の実施形態と同様であるため、以下の第2の実施形態についての説明では、第1の実施形態と相違する事項について主に説明することとし、重複する事項についてはその詳細な説明を省略する。
第2の実施形態に係る映像移動補正システムが適用され得る観察システムの構成は、図1を参照して説明した第1の実施形態に係る観察システム1と同様であるため、その説明を省略する。ここでは、図6を参照して、第2の実施形態に係る映像移動補正システムの構成について説明する。図6は、第2の実施形態に係る映像移動補正システムの機能構成の一例を示すブロック図である。
図7を参照して、第2の実施形態に係る映像移動補正方法の処理手順について説明する。図7は、第2の実施形態に係る映像移動補正方法の処理手順の一例を示すフロー図である。なお、図7に示す各処理は、図6に示す映像移動補正システム3の制御部340aによって実行される処理に対応している。これらの各処理の詳細については、映像移動補正システム3の機能構成について説明する際に既に説明しているため、以下の映像移動補正方法の処理手順についての説明では、各処理についての詳細な説明は割愛する。
以上、添付図面を参照しながら本開示の好適な実施形態について詳細に説明したが、本開示の技術的範囲はかかる例に限定されない。本開示の技術分野における通常の知識を有する者であれば、特許請求の範囲に記載された技術的思想の範疇内において、各種の変更例または修正例に想到し得ることは明らかであり、これらについても、当然に本開示の技術的範囲に属するものと了解される。
(1)
観察対象を撮影した映像データである撮像映像データを取得する撮像部と、
前記撮像部を支持する支持部と、
前記撮像映像データに基づいて表示装置に表示される映像に対する視野移動指示の入力の有無、及び前記撮像映像データに基づいて表示装置に表示される映像に対する映像移動の発生の有無を判別する判別部と、
前記判別部によって、前記視野移動指示の入力がなく、かつ前記映像移動が発生していると判別された場合に、前記撮像映像データに基づいて、表示装置に表示される映像データである表示映像データを、前記映像移動を補正して生成する補正部と、
を備える、医療用観察装置。
(2)
前記判別部は、入力装置を介した前記撮像部を移動させる旨の操作入力の有無に応じて、前記視野移動指示の入力の有無を判別する、
前記(1)に記載の医療用観察装置。
(3)
前記判別部は、前記撮像部を移動させる旨の指示を前記医療用観察装置に対して発行するナビゲーション装置からの前記指示の入力の有無に応じて、前記視野移動指示の入力の有無を判別する、
前記(1)又は(2)に記載の医療用観察装置。
(4)
前記判別部は、最新の前記撮像映像データと、過去に取得された前記撮像映像データとを比較することにより、前記映像移動の発生の有無を判別する、
前記(1)~(3)のいずれか1項に記載の医療用観察装置。
(5)
前記撮像部の動きを表す動き情報を取得する動き検出部、を更に備え、
前記判別部は、前記動き情報に基づいて、前記映像移動の発生の有無を判別する、
前記(1)~(4)のいずれか1項に記載の医療用観察装置。
(6)
前記補正部は、前記撮像映像データに係る映像の所定の領域を切り出して前記表示映像データを生成する際に、前記所定の領域の切り出し位置を調整することにより、前記映像移動を補正する、
前記(1)~(5)のいずれか1項に記載の医療用観察装置。
(7)
前記撮像部の動きを表す動き情報を取得する動き検出部、を更に備え、
前記補正部は、前記動き情報に基づいて、前記撮像部を構成する撮像素子又は光学系の位置を移動させることにより、前記映像移動を補正する、
前記(1)~(5)のいずれか1項に記載の医療用観察装置。
(8)
前記補正部によって生成される表示映像データは、電子ズーム機能により2倍以上に拡大されて前記表示装置に表示される、
前記(1)~(7)のいずれか1項に記載の医療用観察装置。
(9)
前記支持部において最も細い部分の直径が、30mm以下である、
前記(1)~(8)のいずれか1項に記載の医療用観察装置。
(10)
観察対象を撮影した映像データである撮像映像データを取り込むことと、
プロセッサが、前記撮像映像データに基づいて表示装置に表示される映像に対する視野移動指示の入力の有無、及び前記撮像映像データに基づいて表示装置に表示される映像に対する映像移動の発生の有無を判別することと、
前記視野移動指示の入力がなく、かつ前記映像移動が発生していると判別された場合に、前記撮像映像データに基づいて、表示装置に表示される映像データである表示映像データを、前記映像移動を補正して生成する補正部と、
を含む、映像移動補正方法。
(11)
観察対象を撮影する医療用観察装置と、
前記医療用観察装置によって撮影された前記観察対象の映像を表示する表示装置と、
を備え、
前記医療用観察装置は、
観察対象を撮影した映像データである撮像映像データを取得する撮像部と、
前記撮像部を支持する支持部と、
前記撮像映像データに基づいて表示装置に表示される映像に対する視野移動指示の入力の有無、及び前記撮像映像データに基づいて表示装置に表示される映像に対する映像移動の発生の有無を判別する判別部と、
前記判別部によって、前記視野移動指示の入力がなく、かつ前記映像移動が発生していると判別された場合に、前記撮像映像データに基づいて、表示装置に表示される映像データである表示映像データを、前記映像移動を補正して生成する補正部と、
を有する、
医療用観察システム。
2、3 映像移動補正システム
10 顕微鏡装置
20 表示装置
110 顕微鏡部
111、310 撮像部
112 筒状部
120 支持部(アーム部)
130 ベース部
131 架台
132 キャスター
140 制御装置
151 ズームSW
152 フォーカスSW
153 動作モード変更SW
210 第1回転軸部
220 第2回転軸部
230 第3回転軸部
240 第4回転軸部(平行四辺形リンク機構)
250 第5回転軸部
260 第6回転軸部
241、242、243、244 アーム
245、246、247、248 関節部
271 第1アーム部
272 第2アーム部
273 第3アーム部
274 第4アーム部
320 表示部
330 視野移動指示取得部
340、340a 制御部
341 記憶部
342、342a 判別部
343、343a 補正部
344、344a 画像処理部
345 駆動制御部
Claims (11)
- 観察対象を撮影した映像データである撮像映像データを取得する撮像部と、
前記撮像部を支持する支持部と、
前記撮像映像データに基づいて表示装置に表示される映像に対する視野移動指示の入力の有無、及び前記撮像映像データに基づいて表示装置に表示される映像に対する映像移動の発生の有無を判別する判別部と、
前記判別部によって、前記視野移動指示の入力がなく、かつ前記映像移動が発生していると判別された場合に、前記撮像映像データに基づいて、表示装置に表示される映像データである表示映像データを、前記映像移動を補正して生成する補正部と、
を備える、医療用観察装置。 - 前記判別部は、入力装置を介した前記撮像部を移動させる旨の操作入力の有無に応じて、前記視野移動指示の入力の有無を判別する、
請求項1に記載の医療用観察装置。 - 前記判別部は、前記撮像部を移動させる旨の指示を前記医療用観察装置に対して発行するナビゲーション装置からの前記指示の入力の有無に応じて、前記視野移動指示の入力の有無を判別する、
請求項1に記載の医療用観察装置。 - 前記判別部は、最新の前記撮像映像データと、過去に取得された前記撮像映像データとを比較することにより、前記映像移動の発生の有無を判別する、
請求項1に記載の医療用観察装置。 - 前記撮像部の動きを表す動き情報を取得する動き検出部、を更に備え、
前記判別部は、前記動き情報に基づいて、前記映像移動の発生の有無を判別する、
請求項1に記載の医療用観察装置。 - 前記補正部は、前記撮像映像データに係る映像の所定の領域を切り出して前記表示映像データを生成する際に、前記所定の領域の切り出し位置を調整することにより、前記映像移動を補正する、
請求項1に記載の医療用観察装置。 - 前記撮像部の動きを表す動き情報を取得する動き検出部、を更に備え、
前記補正部は、前記動き情報に基づいて、前記撮像部を構成する撮像素子又は光学系の位置を移動させることにより、前記映像移動を補正する、
請求項1に記載の医療用観察装置。 - 前記補正部によって生成される表示映像データは、電子ズーム機能により2倍以上に拡大されて前記表示装置に表示される、
請求項1に記載の医療用観察装置。 - 前記支持部において最も細い部分の直径が、30mm以下である、
請求項1に記載の医療用観察装置。 - 観察対象を撮影した映像データである撮像映像データを取り込むことと、
プロセッサが、前記撮像映像データに基づいて表示装置に表示される映像に対する視野移動指示の入力の有無、及び前記撮像映像データに基づいて表示装置に表示される映像に対する映像移動の発生の有無を判別することと、
前記視野移動指示の入力がなく、かつ前記映像移動が発生していると判別された場合に、前記撮像映像データに基づいて、表示装置に表示される映像データである表示映像データを、前記映像移動を補正して生成する補正部と、
を含む、映像移動補正方法。 - 観察対象を撮影する医療用観察装置と、
前記医療用観察装置によって撮影された前記観察対象の映像を表示する表示装置と、
を備え、
前記医療用観察装置は、
観察対象を撮影した映像データである撮像映像データを取得する撮像部と、
前記撮像部を支持する支持部と、
前記撮像映像データに基づいて表示装置に表示される映像に対する視野移動指示の入力の有無、及び前記撮像映像データに基づいて表示装置に表示される映像に対する映像移動の発生の有無を判別する判別部と、
前記判別部によって、前記視野移動指示の入力がなく、かつ前記映像移動が発生していると判別された場合に、前記撮像映像データに基づいて、表示装置に表示される映像データである表示映像データを、前記映像移動を補正して生成する補正部と、
を有する、
医療用観察システム。
Priority Applications (4)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| CN201780018687.3A CN109152519A (zh) | 2016-03-31 | 2017-03-10 | 医疗观察装置、图像移动校正方法、及医疗观察系统 |
| JP2018508927A JPWO2017169650A1 (ja) | 2016-03-31 | 2017-03-10 | 医療用観察装置、映像移動補正方法及び医療用観察システム |
| US16/087,417 US11422346B2 (en) | 2016-03-31 | 2017-03-10 | Medical observation device, image movement correcting method, and medical observation system |
| EP17774194.9A EP3437547A4 (en) | 2016-03-31 | 2017-03-10 | MEDICAL OBSERVATION DEVICE, IMAGING CORRECTION SYSTEM AND MEDICAL OBSERVATION SYSTEM |
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| JP2016070022 | 2016-03-31 | ||
| JP2016-070022 | 2016-03-31 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2017169650A1 true WO2017169650A1 (ja) | 2017-10-05 |
Family
ID=59964167
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/JP2017/009651 Ceased WO2017169650A1 (ja) | 2016-03-31 | 2017-03-10 | 医療用観察装置、映像移動補正方法及び医療用観察システム |
Country Status (5)
| Country | Link |
|---|---|
| US (1) | US11422346B2 (ja) |
| EP (1) | EP3437547A4 (ja) |
| JP (1) | JPWO2017169650A1 (ja) |
| CN (1) | CN109152519A (ja) |
| WO (1) | WO2017169650A1 (ja) |
Cited By (3)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2020048729A (ja) * | 2018-09-26 | 2020-04-02 | ソニー・オリンパスメディカルソリューションズ株式会社 | 医療用保持装置、および医療用観察装置 |
| CN111223556A (zh) * | 2018-11-23 | 2020-06-02 | 西门子医疗有限公司 | 集成医学图像可视化和探索 |
| WO2022080008A1 (ja) * | 2020-10-15 | 2022-04-21 | ソニー・オリンパスメディカルソリューションズ株式会社 | 医療用画像処理装置及び医療用観察システム |
Families Citing this family (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| CN113392675B (zh) * | 2020-03-12 | 2023-04-07 | 平湖莱顿光学仪器制造有限公司 | 一种呈现显微视频信息的方法与设备 |
| CN111694476B (zh) * | 2020-05-15 | 2022-07-08 | 平安科技(深圳)有限公司 | 平移浏览方法、装置、计算机系统及可读存储介质 |
| EP4057047A1 (en) * | 2021-03-09 | 2022-09-14 | Leica Instruments (Singapore) Pte. Ltd. | Microscope system and corresponding system, method and computer program |
| DE102021126658B8 (de) | 2021-10-14 | 2023-01-05 | Carl Zeiss Meditec Ag | Medizinisches Visualisierungsystem und Verfahren zur Video-Stabilisierung bei einem solchen System |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005198700A (ja) * | 2004-01-13 | 2005-07-28 | Olympus Corp | 手術用顕微鏡システム |
| US20080278781A1 (en) * | 2007-05-10 | 2008-11-13 | Leica Microsystems (Schweiz) Ag | Optical Device With Vibration Compensation |
| US20090020666A1 (en) * | 2007-07-20 | 2009-01-22 | Carl Zeiss Surgical Gmbh | Stand arrangement and stand for a medico-optical instrument |
| JP2012239644A (ja) * | 2011-05-19 | 2012-12-10 | Olympus Corp | 画像処理装置、内視鏡装置、画像処理方法 |
Family Cites Families (16)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JPH0549599A (ja) | 1991-08-23 | 1993-03-02 | Olympus Optical Co Ltd | 電子内視鏡装置 |
| JP4832679B2 (ja) * | 2001-09-11 | 2011-12-07 | オリンパス株式会社 | 顕微鏡システム |
| JP4436623B2 (ja) | 2003-06-19 | 2010-03-24 | オリンパス株式会社 | 手術用顕微鏡システム |
| US7783133B2 (en) * | 2006-12-28 | 2010-08-24 | Microvision, Inc. | Rotation compensation and image stabilization system |
| US8620473B2 (en) | 2007-06-13 | 2013-12-31 | Intuitive Surgical Operations, Inc. | Medical robotic system with coupled control modes |
| JP5683882B2 (ja) * | 2009-10-15 | 2015-03-11 | オリンパス株式会社 | 画像処理装置、画像処理方法および画像処理プログラム |
| US10531869B2 (en) * | 2009-12-16 | 2020-01-14 | Boston Scientific Scimed, Inc. | Tissue retractor for minimally invasive surgery |
| JP2012055498A (ja) * | 2010-09-09 | 2012-03-22 | Olympus Corp | 画像処理装置、内視鏡装置、画像処理プログラム及び画像処理方法 |
| JP5657375B2 (ja) * | 2010-12-24 | 2015-01-21 | オリンパス株式会社 | 内視鏡装置及びプログラム |
| JP5864880B2 (ja) * | 2011-04-07 | 2016-02-17 | オリンパス株式会社 | 内視鏡装置及び内視鏡装置の作動方法 |
| CN104837432B (zh) | 2012-12-11 | 2017-05-17 | 奥林巴斯株式会社 | 内窥镜装置和控制内窥镜装置的方法 |
| CN104869937B (zh) | 2013-02-25 | 2017-10-10 | 奥林巴斯株式会社 | 医疗用机械手和医疗用机械手控制方法 |
| US10238457B2 (en) * | 2013-09-13 | 2019-03-26 | Vanderbilt University | System and method for endoscopic deployment of robotic concentric tube manipulators for performing surgery |
| CN105555222B (zh) | 2013-09-24 | 2018-08-17 | 索尼奥林巴斯医疗解决方案公司 | 医用机械臂装置、医用机械臂控制系统、医用机械臂控制方法、及程序 |
| JP6150130B2 (ja) | 2014-01-30 | 2017-06-21 | ソニー株式会社 | 内視鏡システム、内視鏡画像処理装置、画像処理方法、およびプログラム |
| JP6519144B2 (ja) * | 2014-11-06 | 2019-05-29 | ソニー株式会社 | 内視鏡システム、画像処理装置、画像処理方法、およびプログラム |
-
2017
- 2017-03-10 EP EP17774194.9A patent/EP3437547A4/en not_active Withdrawn
- 2017-03-10 CN CN201780018687.3A patent/CN109152519A/zh active Pending
- 2017-03-10 JP JP2018508927A patent/JPWO2017169650A1/ja active Pending
- 2017-03-10 WO PCT/JP2017/009651 patent/WO2017169650A1/ja not_active Ceased
- 2017-03-10 US US16/087,417 patent/US11422346B2/en active Active
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2005198700A (ja) * | 2004-01-13 | 2005-07-28 | Olympus Corp | 手術用顕微鏡システム |
| US20080278781A1 (en) * | 2007-05-10 | 2008-11-13 | Leica Microsystems (Schweiz) Ag | Optical Device With Vibration Compensation |
| US20090020666A1 (en) * | 2007-07-20 | 2009-01-22 | Carl Zeiss Surgical Gmbh | Stand arrangement and stand for a medico-optical instrument |
| JP2012239644A (ja) * | 2011-05-19 | 2012-12-10 | Olympus Corp | 画像処理装置、内視鏡装置、画像処理方法 |
Non-Patent Citations (1)
| Title |
|---|
| See also references of EP3437547A4 * |
Cited By (8)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| JP2020048729A (ja) * | 2018-09-26 | 2020-04-02 | ソニー・オリンパスメディカルソリューションズ株式会社 | 医療用保持装置、および医療用観察装置 |
| JP7208752B2 (ja) | 2018-09-26 | 2023-01-19 | ソニー・オリンパスメディカルソリューションズ株式会社 | 医療用観察システム、医療用観察装置、および制御方法 |
| US11648082B2 (en) | 2018-09-26 | 2023-05-16 | Sony Olympus Medical Solutions Inc. | Medical holding device, and medical observation device |
| CN111223556A (zh) * | 2018-11-23 | 2020-06-02 | 西门子医疗有限公司 | 集成医学图像可视化和探索 |
| CN111223556B (zh) * | 2018-11-23 | 2023-11-03 | 西门子医疗有限公司 | 集成医学图像可视化和探索 |
| WO2022080008A1 (ja) * | 2020-10-15 | 2022-04-21 | ソニー・オリンパスメディカルソリューションズ株式会社 | 医療用画像処理装置及び医療用観察システム |
| JPWO2022080008A1 (ja) * | 2020-10-15 | 2022-04-21 | ||
| US12295543B2 (en) | 2020-10-15 | 2025-05-13 | Sony Olympus Medical Solutions Inc. | Medical image processing device and medical observation system |
Also Published As
| Publication number | Publication date |
|---|---|
| JPWO2017169650A1 (ja) | 2019-02-14 |
| EP3437547A1 (en) | 2019-02-06 |
| EP3437547A4 (en) | 2019-04-03 |
| CN109152519A (zh) | 2019-01-04 |
| US20190107699A1 (en) | 2019-04-11 |
| US11422346B2 (en) | 2022-08-23 |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| WO2017169650A1 (ja) | 医療用観察装置、映像移動補正方法及び医療用観察システム | |
| JP6970780B2 (ja) | 医療用観察装置、手術用観察装置及び医療用観察システム | |
| US11278369B2 (en) | Control device, control method, and surgical system | |
| JP6552592B2 (ja) | 手術用顕微鏡装置及び手術用顕微鏡システム | |
| JP6935409B2 (ja) | 医療用観察装置及び医療用観察システム | |
| EP3530228B1 (en) | Microscope device and control method | |
| US11510751B2 (en) | Medical observation apparatus | |
| JP2016093225A (ja) | 医療用観察装置、医療用ビデオ顕微鏡装置及び医療用ビデオ顕微鏡システム | |
| JP6816721B2 (ja) | 医療用支持アーム装置及び医療用システム | |
| JP6965338B2 (ja) | 医療用観察装置、制御装置、及び観察視野補正方法 | |
| US11648082B2 (en) | Medical holding device, and medical observation device | |
| JP6704255B2 (ja) | 医療用観察装置、医療用観察システム及び画揺れ補正方法 | |
| US20200093545A1 (en) | Control apparatus, control method, and medical observation system | |
| JPWO2016181730A1 (ja) | 手術用顕微鏡装置及び手術用顕微鏡システム | |
| JP6656237B2 (ja) | 手術用顕微鏡装置及び手術用顕微鏡システム |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| WWE | Wipo information: entry into national phase |
Ref document number: 2018508927 Country of ref document: JP |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| WWE | Wipo information: entry into national phase |
Ref document number: 2017774194 Country of ref document: EP |
|
| ENP | Entry into the national phase |
Ref document number: 2017774194 Country of ref document: EP Effective date: 20181031 |
|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 17774194 Country of ref document: EP Kind code of ref document: A1 |