WO2020009127A1 - Medical observation system, medical observation device, and medical observation device driving method - Google Patents

Abstract

A medical observation system (3) is provided with: an imaging unit (303) that captures images of an affected area; a detection unit (311) that extracts movement of a treatment tool held in the vicinity of the affected area, on the basis of the images of the affected area sequentially captured by the imaging unit, and that detects movement of the affected area, on the basis of the result of the extraction; and a control unit (301) that controls a process for observation of the affected area in accordance with the result of detection of the movement of the affected area.

Description

Medical observation system, medical observation device, and method of driving medical observation device

The present disclosure relates to a medical observation system, a medical observation device, and a driving method of the medical observation device.

In recent years, due to the development of surgical techniques and surgical instruments, surgery (so-called microsurgery) for performing various treatments while observing an affected part with a medical observation device such as a surgical microscope or an endoscope is frequently performed. It is becoming. In addition, some of such medical observation devices are not limited to devices capable of optically observing an affected part, and an image of the affected part captured by an imaging device (camera) is electronically displayed on a display device such as a monitor. A device for displaying an image has also been proposed. With the development of such technology, it has become possible to observe more objects. For example, Patent Literature 1 discloses an example of a technique that enables observation of a blood flow.

JP 2017-170064 A

By the way, in a situation in which a procedure is performed while observing an affected part with a medical observation device, the affected part to be observed may exhibit a motion such as vibration with pulsation or the like regardless of the presence or absence of the procedure. . As a specific example, when observing a blood vessel or its vicinity, such as observing an aneurysm, a situation where the affected part vibrates due to pulsation may be assumed. In such a situation, for example, it may be difficult to accurately observe the affected part because the affected part shows a motion such as vibration. As a specific example, when observing the blood flow after clipping the aneurysm, accurate observation of the aneurysm is difficult because the aneurysm vibrates due to pulsation or the like. There are cases.

Therefore, the present disclosure proposes a technique that enables the observation of the affected part in a more suitable mode even in a situation where the affected part can move regardless of the presence or absence of a procedure.

According to the present disclosure, an imaging unit that captures an image of an affected part, and the movement of the treatment tool held near the affected part is extracted based on the image of the affected part sequentially captured by the imaging unit, and the result of the extraction is obtained. A medical observation system is provided, comprising: a detection unit that detects movement of the affected part based on the detection result; and a control unit that controls processing related to observation of the affected part in accordance with a detection result of the movement of the affected part.

Further, according to the present disclosure, the detection of extracting the movement of the treatment tool held near the affected part based on the image of the affected part sequentially captured by the imaging unit and detecting the movement of the affected part based on the extraction result A medical observation apparatus is provided, comprising: a control unit that controls a process related to observation of the diseased part according to a detection result of the movement of the diseased part.

Further, according to the present disclosure, the computer extracts the movement of the treatment tool held in the vicinity of the affected part based on the image of the affected part sequentially captured by the imaging unit, and calculates the movement of the affected part based on the result of the extraction. A method for driving a medical observation apparatus is provided, which includes: detecting, and controlling processing related to observation of the diseased part according to a detection result of the movement of the diseased part.

According to the present disclosure, as described above, a technique is provided that enables observation of an affected part in a more suitable manner even in a situation where the affected part can move irrespective of the presence or absence of a procedure.

Note that the above effects are not necessarily limited, and any of the effects described in the present specification or other effects that can be grasped from the present specification, together with or instead of the above effects. May be played.

1 is a diagram illustrating an example of a configuration of a system including a video microscope apparatus for surgery to which the technology according to the present disclosure can be applied. It is an explanatory view for explaining an example of a technique. It is explanatory drawing for demonstrating an example of the situation where an affected part moves with a beat. FIG. 1 is an explanatory diagram for describing an example of a configuration of a medical observation system according to an embodiment of the present disclosure. It is explanatory drawing for demonstrating the basic idea of the technical characteristic of the medical observation system which concerns on the same embodiment. It is a block diagram showing an example of the functional composition of the medical observation system concerning the embodiment. It is a flow chart which showed an example of a flow of a series of processings of a medical observation system concerning the embodiment. FIG. 4 is an explanatory diagram for describing an example of control according to the first embodiment. FIG. 11 is an explanatory diagram for describing an example of control according to a second embodiment. FIG. 11 is an explanatory diagram for describing another example of the control according to the second embodiment. FIG. 14 is an explanatory diagram for describing an example of control according to a third embodiment. 1 is a functional block diagram illustrating a configuration example of a hardware configuration of an information processing device configuring a medical observation system according to an embodiment of the present disclosure. 1 is a diagram illustrating an example of a schematic configuration of an endoscopic surgery system according to an application example of an embodiment of the present disclosure. FIG. 14 is a block diagram illustrating an example of a functional configuration of a camera head and a CCU illustrated in FIG. 13.

Hereinafter, preferred embodiments of the present disclosure will be described in detail with reference to the accompanying drawings. In the specification and the drawings, components having substantially the same functional configuration are denoted by the same reference numerals, and redundant description is omitted.

The description will be made in the following order.
1. 1. Configuration example of medical observation system 2. Examination on observation using medical observation system Technical features 3.1. System configuration example 3.2. Basic philosophy 3.3. Functional configuration 3.4. Processing 3.5. Embodiment 4 FIG. 4. Example of hardware configuration Application example 6. Conclusion

<< 1. Configuration example of medical observation system >>
First, an example of a system including a so-called surgical video microscope apparatus will be described with reference to FIG. 1 as an example of a schematic configuration of a medical observation system to which the technology according to an embodiment of the present disclosure can be applied.

For example, FIG. 1 is a diagram illustrating an example of a configuration of a system including a surgical video microscope apparatus to which the technology according to the present disclosure can be applied, and schematically illustrates a state of a surgical operation using the surgical video microscope apparatus. Represents. Specifically, referring to FIG. 1, a doctor who is a practitioner (user) 820 uses a surgical instrument 821 such as a scalpel, forceps, forceps, or the like to perform a treatment (patient) on a treatment table 830. A state in which an operation is performed on the 840 is illustrated. In the following description, the term “treatment” is a general term for various medical treatments, such as surgery and examination, performed by a doctor who is a user 820 on a patient who is a treatment target 840. In addition, in the example illustrated in FIG. 1, the state of the operation is illustrated as an example of the operation, but the operation using the surgical video microscope apparatus 810 is not limited to the operation, and may be various other operations. .

手術 A surgical video microscope apparatus 810 is provided beside the treatment table 830. The surgical video microscope apparatus 810 includes a base 811 serving as a base, an arm 812 extending from the base 811, and an imaging unit 815 connected to a tip of the arm 812 as a tip unit. The arm 812 includes a plurality of joints 813a, 813b, 813c, a plurality of links 814a, 814b connected by the joints 813a, 813b, and an imaging unit 815 provided at the tip of the arm 812. In the example shown in FIG. 1, for simplicity, the arm 812 has three joints 813a to 813c and two links 814a and 814b, but actually, the position of the arm 812 and the imaging unit 815 and The number and shape of the joints 813a to 813c and the links 814a and 814b, the directions of the drive axes of the joints 813a to 813c, and the like are appropriately set so as to realize the desired degree of freedom in consideration of the degree of freedom of the posture. Good.

The joints 813a to 813c have a function of rotatably connecting the links 814a and 814b to each other, and the rotation of the joints 813a to 813c controls the driving of the arm 812. Here, in the following description, the position of each component of the surgical video microscope device 810 means a position (coordinate) in a space defined for drive control, and the posture of each component is , The direction (angle) with respect to an arbitrary axis in a space defined for drive control. In the following description, the driving (or driving control) of the arm 812 refers to the driving (or driving control) of the joints 813a to 813c and the driving (or driving control) of the joints 813a to 813c. Means that the position and posture of each component of the arm 812 is changed (change is controlled).

The imaging unit 815 is connected to the tip of the arm 812 as a tip unit. The imaging unit 815 is a unit that acquires an image of an imaging target, and is, for example, a camera that can capture a moving image or a still image. As shown in FIG. 1, the posture of the arm unit 812 and the imaging unit 815 is operated by the video microscope for surgery 810 such that the imaging unit 815 provided at the tip of the arm unit 812 captures an image of the treatment site of the operation target 840. And position are controlled. The configuration of the imaging unit 815 connected as a tip unit to the tip of the arm 812 is not particularly limited. For example, the imaging unit 815 is configured as a microscope that acquires an enlarged image of an imaging target. Further, the imaging unit 815 may be configured to be detachable from the arm unit 812. With such a configuration, for example, the imaging unit 815 corresponding to the intended use may be appropriately connected to the tip of the arm 812 as a tip unit. Note that, as the imaging unit 815, for example, an imaging device to which the branch optical system according to the above-described embodiment is applied can be applied. That is, in this application example, the imaging unit 815 or the surgical video microscope device 810 including the imaging unit 815 can correspond to an example of a “medical observation device”. In this description, the description has been made focusing on the case where the imaging unit 815 is applied as the distal end unit. However, the distal end unit connected to the distal end of the arm 812 is not necessarily limited to the imaging unit 815.

表示 A display device 850 such as a monitor or a display is provided at a position facing the user 820. The image of the treatment site captured by the imaging unit 815 is displayed on the display screen of the display device 850 as an electronic image. The user 820 performs various treatments while viewing the electronic image of the treatment site displayed on the display screen of the display device 850.

With the above configuration, it is possible to perform an operation while capturing an image of a treatment site by the video microscope for operation 810.

In the above description, an example of the surgical video microscope device has been described, but a portion corresponding to the surgical video microscope device may be configured as a so-called optical microscope device. In this case, an optical microscope unit may be connected as a tip unit connected to the tip of the arm 812. Further, the microscope unit may include an imaging device.

As described above, as an example of a schematic configuration of the medical observation system according to an embodiment of the present disclosure, a case where the medical observation system is configured as a microscope imaging system including a microscope unit with reference to FIG. An example has been described.

<<< 2. Study on observation using medical observation system >>
Subsequently, in a case where a medical observation system is used to observe a diseased part (for example, an observation target such as an aneurysm) in a patient's body, a technical technique for performing the observation is assumed. The problem will be described below with a specific example.

For example, FIG. 2 is an explanatory diagram for describing an example of a procedure, and shows an outline of an example of an unruptured cerebral aneurysm clipping operation. In unruptured cerebral aneurysm clipping, clipping a part of the blood vessel using a titanium clip or the like to prevent the occurrence of a failure due to rupture of the aneurysm (for example, cerebral aneurysm) Of blood (ie, block an aneurysm). For example, the example illustrated in FIG. 2 illustrates an example in which an aneurysm generated in a part of the blood vessel M101 is closed by clipping using the clip M111. In FIG. 2, the upper diagram shows the state before clipping, and the lower diagram shows the state before clipping. In FIG. 2, reference numeral M103 indicates a dome of an aneurysm. Reference numeral M105 indicates the neck of the aneurysm. That is, in the example shown in FIG. 2, the clip M111 is applied to the neck M105 of the aneurysm to prevent blood flowing through the blood vessel M101 from flowing into the aneurysm.

In the above clipping operation, after the clip, a measure to prevent the rupture of the aneurysm may be taken by crushing the aneurysm by puncture or the like and coagulating blood to form a thrombus. In order to prevent rupture of the aneurysm by coagulating blood and forming a thrombus, for example, the flow of blood into the aneurysm can be suppressed or the blood flow can be reduced to a degree that causes blood to coagulate and form a thrombus. It is important to make the blood flow minute so that the speed decreases.

On the other hand, in a situation where the above-mentioned clipping operation is applied, accurate observation is sometimes difficult due to a motion such as a vibrating diseased part (eg, an aneurysm) of a blood vessel or the like accompanying pulsation. is there. For example, FIG. 3 is an explanatory diagram for describing an example of a situation where an affected part moves with a beat. The example illustrated in FIG. 3 illustrates an example in which the clip M111 is applied to the neck M105 of the aneurysm to close the aneurysm, as in the example illustrated in FIG. In this case, the pulsation causes the blood vessel M101 to vibrate, and the vibration may be manifested as, for example, the movement of the aneurysm (the movement of the dome M103) or the movement of the clip M111 applied to the aneurysm. .

Further, in a situation where the aneurysm is vibrating due to pulsation or the like, the state of blood flow in the aneurysm changes due to the vibration, and the inflow of blood into the aneurysm is blocked by clipping. Regardless, it may be observed as not being occluded (ie, a false positive). In such a situation, for example, measures such as additional clipping are taken, which may lead to an increase in time and cost. An increase in the duration of the procedure may, for example, cause an increase in the burden on the patient. In addition, a situation in which the state of the affected part is difficult to observe due to vibration or the like may cause an increase in the burden on the doctor.

確認 In addition, for confirmation of occlusion of an aneurysm by the clipping operation, for example, a method using a phosphor such as ICG may be applied. Specifically, in this method, a fluorescent substance such as ICG is injected into blood by intravenous injection, and light having a wavelength to be excited by the fluorescent substance is irradiated, and then the excitation light is spectrally detected, whereby an aneurysm is detected. The presence or absence of blood inflow into the blood is checked. When ICG is used as a phosphor, for example, near-infrared light having a wavelength of about 800 nm is irradiated, and excitation light having a wavelength of about 830 nm is spectrally detected by using a filter or the like. Thus, the above confirmation is performed.

On the other hand, when a phosphor such as ICG is injected into blood, it takes time until it is discharged (washed out). Therefore, for example, when the aneurysm is not sufficiently occluded by the clip and blood flows into the aneurysm, the occlusion of the blood into the aneurysm is prevented by performing the occlusion again by adding a clip or the like. Even so, since the fluorescent substance such as ICG that has flowed into the reservoir together with the blood emits fluorescence, it may be difficult to determine whether the blockage is sufficient again.

手法 Another example of a method of observing a blood flow is a method called LSCI (Laser Speckle Contrast Imaging). In the LSCI, the presence or absence of blood flow is detected by irradiating a laser light to a scattered substance such as red blood cells in blood and observing the scattered light. Due to such characteristics, even in LSCI, observation may be difficult due to movement of the affected part due to pulsation or the like. Further, in the LSCI, for example, it is difficult to obtain an image of visible light in a situation in which an image is obtained by irradiating near-infrared light. Therefore, for example, even if the affected part is clipped, In some cases, it may be difficult to confirm the position of the clip.

In view of the situation as described above, in the present disclosure, regardless of the presence or absence of the intervention, for example, even in a situation where the affected part can move due to the influence of pulsation or the like, the observation of the affected part in a more suitable mode We propose technologies that can be realized.

<< 3. Technical Features >>
Hereinafter, the technical features of the medical observation system according to an embodiment of the present disclosure will be described.

<3.1. Example of system configuration>
First, an example of a configuration of a medical observation system according to an embodiment of the present disclosure will be described. For example, FIG. 4 is an explanatory diagram for describing an example of a configuration of a medical observation system according to an embodiment of the present disclosure, and illustrates an example of a system configuration when LSCI is applied. That is, in the example shown in FIG. 4, in a situation where the above-described clipping operation is applied to an aneurysm as an observation target, a situation where an affected part (aneurysm) is observed by irradiating infrared light or visible light is considered. It has an assumed configuration. In the following description, the medical observation system shown in FIG. 4 is also referred to as “medical observation system 2” for convenience.

In the example shown in FIG. 4, the medical observation system 2 includes a control unit 201, an imaging unit 203, a sensor driver 205, an input unit 207, and an output unit 209. The input unit 207 is an input interface for the medical observation system 2. The user can input various information and input instructions to the medical observation system 2 via the input unit 7. The output unit 209 corresponds to the display device 850 in the example shown in FIG.

The imaging unit 203 includes, for example, an imaging optical system 211, a branching optical system 213, imaging elements 215 and 217, an RGB laser 219, an IR (for example, near infrared ray) laser 223, and a vibration sensor 227. .

Each of the RGB laser 219 and the IR laser 223 corresponds to a light source device for irradiating a diseased part with light having a predetermined wavelength.

The RGB laser 219 is a light source that emits visible light, and includes, for example, red (Red; wavelength around 650 nm), green (Green; wavelength around 530 nm), and blue (Blue; wavelength around 450 nm) lasers. However, an LED light source may be used, or a laser, an LED, or a configuration in which a phosphor is excited by a laser to emit white light may be used. This is used, for example, as a light source when a bright field image of the affected part is acquired. Specifically, the visible light emitted from the RGB laser 219 is transmitted through a transmission cable 221 configured to be able to guide the light using an optical fiber or the like, and is applied to the affected part. As a result, the bright field image of the affected part is collected by the imaging optical system 211 described later.

The IR laser 223 is a light source that emits infrared light (IR light), and is used, for example, as a light source when performing fluorescence observation or the like. Specifically, the infrared light emitted from the IR laser 223 is transmitted via a transmission cable 225 configured to be able to guide the light using an optical fiber or the like, and irradiated to the affected part. As a result, the fluorescent substance such as ICG injected into blood or the like is excited by the irradiation of the infrared light, and the excitation light emitted from the fluorescent substance is collected by the imaging optical system 211 described later.

The imaging optical system 211 schematically shows an optical system for acquiring an image of an affected part to be observed. The imaging optical system 211 can correspond to, for example, an endoscope or a microscope. The imaging optical system 211 forms an image of the incident light on one of the imaging devices 215 and 217 located at a subsequent stage via a branching optical system 213 described later. As a result, the image of the affected part to be observed is captured by the imaging elements 215 and 217. Note that the imaging optical system 211 may include a plurality of optical systems such as lenses.

The branch optical system 213 separates light in a part of the wavelength band and light in another wavelength band from the incident light, and forms an image on each of the image sensors 215 and 217 different from each other. As a specific example, the branching optical system 213 includes a dichroic filter or the like, and transmits light in a part of the wavelength band of the incident light and reflects light in another wavelength band, thereby making these light components. Separates light. For example, in the example illustrated in FIG. 4, light transmitted through the branch optical system 213 is guided to the image sensor 215, and light reflected by the branch optical system 213 is guided to the image sensor 217. In the following description, for the sake of convenience, light belonging to the visible light wavelength band is guided to the image sensor 215, and the image sensor 217 has a longer wavelength than visible light, such as infrared light or fluorescent light emitted by ICG. It is assumed that light (for example, light belonging to the near infrared wavelength band) is guided. However, the configuration of the branch optical system 213 is not necessarily limited to the above example as long as the incident light can be separated into a plurality of lights. That is, it may be appropriately changed according to the wavelength of light to be observed, the observation method, the configuration of the imaging unit 203, and the like.

The image sensor 215 is an image sensor that is provided downstream of the branch optical system 213 and forms an image of light belonging to the visible light wavelength band separated by the branch optical system 213. As the image sensor 215, for example, an image sensor such as a CCD or a CMOS having an RGB color filter can be applied.

The image sensor 217 is provided at a stage subsequent to the branch optical system 213, and forms an image of light having a longer wavelength than visible light (for example, light belonging to a near infrared wavelength band) separated by the branch optical system 213. It is. As the image sensor 217, an image sensor having higher sensitivity may be applied. As a specific example, as the imaging element 217, an imaging element such as a CCD or a CMOS without a color filter or the like may be applied.

The vibration sensor 227 is a sensor that detects a movement (for example, vibration or the like) of the imaging unit 203. The vibration sensor 227 may include, for example, an acceleration sensor or an angular velocity sensor, and may detect movement of the housing of the imaging unit 203 (for example, acceleration or angular velocity acting on the housing). The vibration sensor 227 notifies the sensor driver 205 of the detection result of the movement of the imaging unit 203.

The vibration sensor 229 is a sensor that detects the movement of the predetermined part M107 of the patient (in other words, the movement of the affected part). As a specific example, when a cerebral aneurysm is to be observed, the vibration sensor 229 is configured to detect a part of the patient's head or the like so that the movement of the part M107 can be detected. May be installed. The vibration sensor 229 notifies the sensor driver 205 of the detection result of the movement of the predetermined part M107 of the patient.

(4) The sensor driver 205 controls the operation of various sensors and acquires information corresponding to the detection results of various states from the sensors. As a specific example, the sensor driver 205 controls the operation of the vibration sensor 227 and acquires information corresponding to the detection result of the movement (for example, vibration) of the imaging unit 203 from the vibration sensor 229. As another example, the sensor driver 205 controls the operation of the vibration sensor 227, and acquires information corresponding to the detection result of the movement (for example, vibration) of the imaging unit 203 from the vibration sensor 229. The sensor driver 205 may execute the control of the operation of various sensors and the acquisition of information from the sensors based on the control by the control unit 201. Further, the sensor driver 205 may output information obtained from various sensors to the control unit 201.

The control unit 201 may control the operation of various light sources such as the RGB laser 219 and the IR laser 223 according to the observation target and the observation method. Further, the control unit 201 may control an operation related to imaging of an image by at least one of the imaging elements 215 and 217. At this time, the control unit 201 may control the imaging conditions of the image (for example, shutter speed, aperture, gain, and the like). In addition, the control unit 201 may acquire an image corresponding to an imaging result of at least one of the imaging elements 215 and 217, and cause the output unit 209 to present the image. At this time, the control unit 201 may perform predetermined image processing on the acquired image. Further, the control unit 201 may control the operation of each unit according to the detection results of various states. As a specific example, the control unit 201 may acquire information from the sensor driver 205 based on the detection result of the movement of the imaging unit 203 by the vibration sensor 227, and execute so-called camera shake correction based on the information. In this case, the control unit 201 cuts out a part of the image corresponding to the imaging result of the imaging elements 215 and 217 in accordance with the movement (that is, blurring) of the imaging unit 203, thereby May be corrected. Further, the control unit 201 may execute the above-described various processes according to an instruction from the user input via the input unit 207.

4 Note that the example described with reference to FIG. 4 is merely an example, and does not necessarily limit the configuration of the medical observation system according to an embodiment of the present disclosure. That is, as long as the basic idea of the medical observation system according to the present embodiment, which will be described later, is not deviated, a part of the configuration may be appropriately changed depending on the observation target and the observation method.

As described above, an example of the configuration of the medical observation system according to an embodiment of the present disclosure has been described with reference to FIG.

<3.2. Basic Thought>
Next, the basic concept of the technical features of the medical observation system according to an embodiment of the present disclosure will be described.

As described above, in a situation in which the above-described clipping operation is applied, accurate observation may be difficult due to movement such as vibration of an affected part (eg, an aneurysm) of a blood vessel or the like. . Therefore, in the medical observation system according to the present embodiment, the movement of the affected part is detected based on an image captured by an imaging unit or the like such as an endoscope device or a microscope device, and various types of detection are performed using the detection result. By executing the processing, observation in a more suitable mode can be realized. As a specific example, by controlling the imaging conditions (for example, shutter speed or the like) of the imaging device in accordance with the movement of the affected part such as an aneurysm, an image that allows a doctor to make more accurate determinations is generated. Good. As another example, in a situation where the movement of the affected part is larger, a warning can be issued to notify the doctor that accurate determination is difficult. An example of such a process using the detection result of the movement of the affected part will be described later in detail.

In addition, in the medical observation system according to the present embodiment, the detection result of the movement of the treatment tool such as a clip held near the affected part is used to detect the movement of the affected part. Specifically, in the medical observation system according to the present embodiment, the movement of the treatment tool is extracted by extracting a characteristic portion of the treatment tool such as a clip from the images sequentially captured by the imaging unit or the like. Then, the movement of the affected part is detected based on the extraction result of the movement of the treatment tool.

For example, FIG. 5 is an explanatory diagram for explaining the basic concept of the technical features of the medical observation system according to the present embodiment, and the movement of the treatment tool is extracted from the sequentially captured images to extract the affected part. An example in which motion is detected is shown. In the example illustrated in FIG. 5, as in the example described with reference to FIG. 2, the clip M111 is applied to the neck M105 of the aneurysm to prevent blood flowing through the blood vessel M101 from flowing into the aneurysm. . At this time, similarly to the example described with reference to FIG. 3, the blood vessel M101 vibrates due to the pulsation, and the vibration is applied to, for example, the movement of the aneurysm (the movement of the dome M103) or the aneurysm. This will become apparent as the movement of the clip M111. At this time, if the dome M103 of the aneurysm vibrates due to the pulsation of the blood vessel M101 or the like, the clip M111 applied to the neck M105 of the aneurysm also vibrates in conjunction with the vibration of the aneurysm. . Therefore, in the medical observation system according to the present embodiment, the movement (for example, vibration) of the aneurysm is detected by extracting the movement of the clip M111 and using the extraction result of the movement of the clip M111.

As a specific example, in the example illustrated in FIG. 5, the light emitting unit M113 is provided in a part of the clip M111, and the medical observation system according to the present embodiment includes the light emitting unit M113 in the image sequentially captured. , The motion of the clip M111 is extracted. Then, the medical observation system detects the movement of the dome M103 of the aneurysm (that is, the affected part) on which the clip M111 is applied, based on the extraction result of the movement of the clip M111. With such a configuration, for example, even when it is difficult to directly view the affected part according to the observation environment of the affected part or the observation method of the affected part, the movement of the affected part is detected, and the movement of the affected part is determined according to the result of the detection. Can be executed.

Note that the example shown in FIG. 5 is merely an example. If it is possible to extract the movement of the treatment tool such as the clip M111 and detect the movement of the affected part based on the result of the extraction of the movement, the configuration or the The method is not particularly limited. For example, without being limited to the method of extracting the luminous body as described above, by extracting a portion having predetermined characteristics such as a shape and a color from a sequentially captured image in a target treatment tool, It is possible to extract the movement of the treatment tool. As a specific example, at least a part of the treatment tool is provided with a part having a color different from the observation target, so that the part of the color in the captured image can be extracted as a part corresponding to the treatment tool. It is possible. Further, as another example, at least a part of the treatment tool is provided with a portion having a characteristic shape, and a portion where the shape is detected in the captured image is extracted as a portion corresponding to the treatment tool. It is possible.

Also, a portion serving as an index for extracting a treatment tool from an image, such as the light emitter, may be configured to be detachable from the treatment tool. By adopting such a configuration, for example, it is also possible to remove only the portion serving as the index after the treatment such as clipping is completed.

The configuration and method for extracting the movement of the treatment tool may be appropriately changed according to the assumed observation environment and observation method. As a specific example, when assuming fluorescence observation using a phosphor excited by near-infrared light such as ICG, at least a part of a treatment tool such as a clip is emitted by the near-infrared light. Those made of a material that excites light may be used. As another example, as a treatment tool such as a clip, at least a part of which is coated with a paint that emits and emits light by the infrared light may be used. Thereby, even when it is difficult to acquire a bright-field image, the movement of the treatment tool is extracted from the acquired image, and based on the result of the extraction, the movement of the affected part where the treatment tool is held in the vicinity is extracted. It becomes possible to detect.

Especially, in the medical field, the observation method may be selectively changed depending on the observation target, and a situation in which the observation environment changes depending on the observation method may be assumed. Therefore, when observing a part of the object to be observed (for example, blood flow), it is difficult to observe another part (for example, a position where an aneurysm or a clip is applied), and as a result, accurate observation or diagnosis is performed. Can be assumed. Even in such a case, according to the medical observation system according to the embodiment of the present disclosure, it is possible to appropriately change the configuration and method for extracting the movement of the treatment tool according to the observation environment and the observation method. During the observation of the observation target, it is possible to observe another portion where the treatment tool is held in the vicinity.

The basic concept of the technical features of the medical observation system according to an embodiment of the present disclosure has been described above with reference to FIG.

<3.3. Functional Configuration>
Subsequently, an example of a functional configuration of the medical observation system according to an embodiment of the present disclosure will be described, particularly focusing on an example of a functional configuration of a control unit that controls the operation of each configuration of the medical observation system. . For example, FIG. 6 is a block diagram illustrating an example of a functional configuration of a medical observation system according to an embodiment of the present disclosure. Specifically, FIG. 6 illustrates the configuration of the medical observation system according to the present embodiment, in particular, by extracting the movement of the treatment tool from the sequentially captured images, the affected part where the treatment tool is held nearby. The figure focuses on a part that detects movement and executes various processes according to the result of the detection. In the following description, the medical observation system shown in FIG. 6 is also referred to as “medical observation system 3” for convenience.

As shown in FIG. 6, the medical observation system 3 includes a control unit 301, an imaging unit 303, a detection unit 305, and an output unit 307. The imaging unit 303 may correspond to, for example, the imaging unit 203 (and eventually the imaging elements 215 and 217) illustrated in FIG. Further, the detection unit 305 may correspond to the sensor driver 205 shown in FIG. Further, the output unit 307 may correspond to the output unit 209 illustrated in FIG. Therefore, detailed description of the imaging unit 303, the detection unit 305, and the output unit 307 is omitted.

The control unit 301 may correspond to the control unit 201 shown in FIG. As shown in FIG. 6, the control unit 301 includes an image analysis unit 309, a vibration detection unit 311, an imaging control unit 313, an image processing unit 315, and an output control unit 317.

The image analysis unit 309 acquires images sequentially captured by the imaging unit 303 and performs image analysis on the images to obtain a predetermined object (for example, a predetermined treatment such as a clip or the like) captured in the images. Tool). As a specific example, the image analysis unit 309 may calculate a feature amount of the acquired image, and extract a portion having a predetermined feature from the image as a portion corresponding to a target object. Of course, the above is an example, and the method is not particularly limited as long as a predetermined object (a predetermined treatment tool) can be extracted from the images sequentially captured by the imaging unit 303. In the following description, the images sequentially captured by the imaging unit 303 are also simply referred to as “captured images” for convenience.

(4) The image analysis unit 309 may extract a predetermined affected part (for example, an affected part to be observed) from the captured image. In this case, the image analysis unit 309 may extract, from the captured image, a part having the characteristics of the target affected part as a part corresponding to the affected part. Then, the image analysis unit 309 outputs the captured image and the analysis result of the captured image (that is, the extraction result of the object captured in the image) to the vibration detection unit 311.

The vibration detection unit 311 acquires the captured image and the analysis result of the captured image from the image analysis unit 309. The vibration detection unit 311 extracts a movement of a predetermined object (for example, vibration of a treatment tool) captured in the captured image based on the analysis result of the captured image. Further, the vibration detection unit 311 may detect the movement of the affected part in which the object is held in the vicinity based on the extraction result of the movement of the predetermined object. As a specific example, the vibration detection unit 311 may detect the movement of the aneurysm in which the clip is applied to a neck or the like by extracting the movement of the clip used in the clipping operation. At this time, the vibration detection unit 311 may detect the movement of the aneurysm to which the clip has been applied in consideration of the position where the clip is applied, the orientation of the clip, and the like.

In addition, the vibration detection unit 311 detects the vibration of the predetermined object such as a treatment tool or the like, and detects the movement of the affected part based on the extraction result of the movement of the object. The detection result of the movement of the imaging unit 303 or the detection result of the movement of the part of the patient) may be used. As a specific example, the vibration detection unit 311 uses the detection result of the movement of the imaging unit 303 by the detection unit 305 to correct image blur (eg, camera shake) due to the movement of the imaging unit 303. Thus, the motion of a predetermined object such as a clip may be extracted. Similarly, the vibration detection unit 311 uses the detection result of the movement of the part of the patient by the detection unit 305 to correct the blur of the image caused by the movement of the part, and then moves the predetermined object such as a clip. May be extracted.

The vibration detection unit 311 outputs the acquired captured image to at least one of the image processing unit 315 and the output control unit 317. In addition, the vibration detection unit 311 outputs information regarding the detection result of the motion of the affected part based on the analysis result of the captured image to, for example, at least one of the imaging control unit 313, the image processing unit 315, and the output control unit 317. May be.

(4) The imaging control unit 313 controls the operation of the imaging unit 303. As a specific example, the imaging control unit 313 determines various conditions (for example, imaging conditions such as shutter speed, aperture, and white balance, etc.) set via a predetermined input unit (not shown). The operation of the image capturing section 303 for capturing an image may be controlled.

撮 像 Alternatively, the imaging control unit 313 may obtain information on the detection result of the motion of the affected part from the vibration detection unit 311 and control the operation of the imaging unit 303 based on the information. For example, the imaging control unit 313 may control imaging conditions related to imaging of an image by the imaging unit 303, such as a shutter speed, an aperture, and a gain, according to the detected magnitude of movement of the affected part. As a specific example, when the movement of the affected part is detected, the imaging control unit 313 increases the amount of light captured by opening the aperture more as the movement of the affected part is larger, and then increases the shutter speed. May be controlled so that As another example, when the movement of the affected part is detected, the imaging control unit 313 increases the gain to improve the sensitivity of the imaging device, and then controls the shutter speed to increase. Is also good.

The image processing unit 315 performs various image processing on the captured image. As a specific example, the image processing unit 315 may correct the brightness, contrast, color tone, and the like of the captured image. In addition, the image processing unit 315 may generate an enlarged image of the affected part by cutting out a part of the captured image and enlarging (ie, performing digital zoom processing). Further, the image processing unit 315 may perform image processing on the captured image based on an instruction input via a predetermined input unit (not shown).

The image processing unit 315 may acquire information on the detection result of the movement of the affected part from the vibration detection unit 311 and perform image processing on the captured image based on the information. As a specific example, the image processing unit 315 corrects the blurring (for example, subject blurring) of the affected part that has become apparent in the captured image based on the detection result of the movement of the affected part (that is, the blurring of the affected part is suppressed). May be generated.

As described above, the image processing unit 315 performs image processing on the captured image, and outputs the captured image after the image processing to the output control unit 317.

The output control unit 317 presents the information by causing the output unit 307 to output various information. For example, the output control unit 317 may acquire a captured image and cause the output unit 307 to output the captured image. Further, the output control unit 317 obtains the image-processed captured image (hereinafter, also referred to as “image-processed image”) from the image processing unit 315, and outputs the image-processed image to the output unit 307. You may let it. As another example, the output control unit 317 may present display information indicating a basin of interest, notification information such as a message or a warning, and the like, superimposed on an image.

In addition, the output control unit 317 may present a screen (in other words, an image) on which a plurality of types of information are presented, and output the screen to the output unit 307 to present the plurality of types of information. . As a specific example, the output control unit 317 may generate a screen on which the captured image and the image after the image processing are presented, and cause the output unit 307 to output the screen. At this time, the output control unit 317 may generate a screen in which the captured image and the image after the image processing are displayed side by side. Further, as another example, the output control unit 317 generates a so-called PIP (Picture @ In @ Picture) image in which one of the captured image and the image after the image processing is superimposed on the other. You may. As described above, the output control unit 317 may present the captured image and the image after the image processing in association with each other. In this case, the presentation mode of these images (in other words, a method of associating these images) Is not particularly limited.

The output control unit 317 may acquire information on the detection result of the movement of the affected part from the vibration detection unit 311 and control output of various types of information to the output unit 307 based on the information. As a specific example, the output control unit 317 may display a warning on the output unit 307 when the movement of the affected part is detected and the magnitude of the movement of the affected part is equal to or larger than a threshold. In addition, the output control unit 317 may selectively switch information to be displayed on the output unit 307 (for example, notification information such as a warning or a message) according to the magnitude of the movement of the affected part.

The functional configuration described above is merely an example, and the functional configuration of the medical observation system is not necessarily limited to the example illustrated in FIG. 6 as long as the operation of each configuration described above can be realized. As a specific example, at least one of the imaging unit 303, the detection unit 305, and the output unit 307 and the control unit 301 may be integrally configured. Further, as another example, some functions of the control unit 301 may be provided outside the control unit 301. Further, at least a part of the function of the control unit 301 may be realized by a plurality of devices operating in cooperation with each other. Further, within the range not departing from the basic concept of the technical features of the medical observation system according to the present embodiment described above, a part of the configuration of the medical observation system may be changed, and other configurations may be used. It may be added separately.

Note that an apparatus including a configuration corresponding to the control unit 301 illustrated in FIG. 6 corresponds to an example of a “medical observation apparatus”. Further, the vibration detection unit 311 corresponds to an example of a “detection unit” that detects the movement of the affected part. Further, a configuration that executes or controls various processes (especially, processes related to observation of an affected part) according to the detection result of the movement of the affected part, such as an imaging control unit 313, an image processing unit 315, and an output control unit 317, is described as “ Control unit ”.

As described above, with reference to FIG. 6, an example of a functional configuration of the medical observation system according to an embodiment of the present disclosure, in particular, an example of a functional configuration of a control unit that controls the operation of each configuration of the medical observation system It was explained focusing on.

<3.4. Processing>
Subsequently, regarding an example of a flow of a series of processes of the medical observation system according to an embodiment of the present disclosure, in particular, the control unit 301 illustrated in FIG. 6 detects movement of the affected part and responds to the result of the detection. A description will be given focusing on the flow of processing for controlling various operations. For example, FIG. 7 is a flowchart illustrating an example of a flow of a series of processes of the medical observation system according to an embodiment of the present disclosure.

First, the control unit 301 (image analysis unit 309) acquires an image of the affected part (that is, a captured image) sequentially captured by the imaging unit 303 (S101), and performs image analysis on the image to obtain the image. A predetermined treatment tool (for example, a clip or the like) imaged therein is specified (S103). That is, the control unit 301 extracts a predetermined treatment tool from the captured image based on the image analysis of the captured image.

Next, the control unit 301 (vibration detection unit 311) extracts the movement of the treatment tool based on the extraction result of the predetermined treatment tool from the captured image, so that the affected part in which the treatment tool is held in the vicinity is extracted. The movement (for example, vibration) is detected (S105).

The control unit 301 controls various processes related to the observation of the affected part according to the detection result of the movement (vibration or the like) of the affected part (S107). For example, the control unit 301 (imaging control unit 313) may control the operation of the imaging unit 303 for capturing an image of the affected part according to the detection result of the movement of the affected part. As another example, the control unit 301 (image processing unit 315) may perform predetermined image processing on the captured image based on the detection result of the movement of the affected part. Further, as another example, the control unit 301 (output control unit 317) may present various types of information via the output unit 307 according to the detection result of the movement of the affected part.

As described above, the control unit 301 sequentially executes the processes indicated by reference numerals S101 to S107 unless the end of a series of processes is instructed (S109, NO). Then, when the control unit 301 is instructed to end the series of processing (S109, YES), the control unit 301 ends the execution of the processing indicated by reference numerals S101 to S107.

As described above, with reference to FIG. 7, regarding an example of a series of processing flows of the medical observation system according to an embodiment of the present disclosure, in particular, the control unit 301 illustrated in FIG. The description has focused on the flow of processing for controlling various operations in accordance with the result of detection.

<3.5. Example>
Subsequently, as an example, an example of control of a process related to observation of an affected part according to a detection result of movement of the affected part by the medical observation system according to an embodiment of the present disclosure will be described.

(Example 1: Example of image processing)
First, as a first embodiment, an example in which image processing is performed on a captured image based on a detection result of movement of an affected part will be described. For example, FIG. 8 is an explanatory diagram for describing an example of control according to the first embodiment, and illustrates an example of a case where image processing is performed on a captured image based on a detection result of movement of an affected part. More specifically, FIG. 8 illustrates image processing of a captured image of an affected part (aneurysm), assuming a situation in which the affected part is observed by clipping as in an unruptured cerebral aneurysm clipping operation. An example of the case of applying is shown.

よ う As described above, accurate observation may be difficult due to movement such as vibration of an affected part (for example, an aneurysm) of a blood vessel or the like accompanying pulsation. Therefore, for example, by using the detection result of the movement of the affected part, the movement of the affected part in the image (for example, subject shake) is corrected, and the image in which the movement of the affected part is suppressed (the presentation of the affected part in the image is further reduced). By generating an image whose position is fixed, an effect of enabling the affected part to be observed more accurately can be expected. In this case, for example, based on the detection result of the movement of the affected part, the position of cutting out a part of the image including the affected part from the captured image is controlled so that the movement of the affected part is canceled. It is possible to generate an image in which is suppressed.

In the case where the image processing is performed on the captured image, the image before the image processing and the image after the image processing may be presented in association with each other. For example, in the example shown in FIG. 8, a corrected image V105 in which blurring of an affected part in an image is suppressed (corrected) by image processing, and a captured image V103 before the image processing is performed (in other words, a so-called live image) Are presented in a screen V101 in which both are displayed side by side. As described above, by presenting the corrected image V105 in which the blurring of the affected part in the image is suppressed, it is possible to more accurately observe a more detailed movement or change of the affected part. Note that the image presented as the screen V101 may be selectively switched. For example, as shown in FIG. 8, a screen in which both the captured image V103 and the corrected image V105 are presented and a screen in which only one of the captured image V103 and the corrected image V105 is presented are selectively switched. Is also good. As another example, a so-called PIP image in which a part of one of the captured image V103 and the corrected image V105 is superimposed on the other image may be presented as the screen V101.

(Example 2: Example of information presentation)
Next, as a second embodiment, an example in which various types of information are presented according to the detection result of the movement of the affected part will be described.

For example, in a situation where the vibration of the affected part is larger, it may be difficult to obtain an accurate result even if the affected part is observed or measured. Therefore, assuming such a situation, when the movement of the affected part is larger (for example, when the magnitude of the movement of the affected part is equal to or larger than the threshold), a warning is presented to suppress the observation of the affected part and the execution of measurement. May be prompted.

For example, FIG. 9 is an explanatory diagram for describing an example of control according to the second embodiment, and shows an example of a case where information is presented according to a detection result of a movement of an affected part. Specifically, in the example illustrated in FIG. 9, the magnitude of the motion of the affected part (aneurysm) is equal to or larger than the threshold, and a warning that prompts suppression of the measurement is displayed on the screen V111 on which the captured image is presented. It is presented as display information V113.

As another example, the operation related to the presentation of the information may be controlled according to the magnitude of the movement of the affected part. For example, FIG. 10 is an explanatory diagram for describing another example of the control according to the second embodiment, and illustrates another example in which information is presented according to the detection result of the movement of the affected part. In the example illustrated in FIG. 10, when the movement of the affected part is small (for example, when the magnitude of the movement of the affected part is less than the threshold), information on the procedure being performed (for example, information for supporting the procedure of the operator) Is presented. Specifically, the size of the affected part (for example, an aneurysm) (for example, the size of the dome of the aneurysm indicated by reference numeral V125) is measured based on image analysis or the like, and the measurement result of the size is displayed as display information V115. , On the screen V121 on which the captured image is presented. The size of the affected part can be calculated based on, for example, information on the size of the affected part extracted from the captured image in the captured image and imaging conditions (for example, focal length) of the captured image.

(Embodiment 3: An example of control of an operation related to imaging of an image)
Third Embodiment Next, as a third embodiment, an example in which an operation related to imaging of an image by an imaging unit is controlled based on a detection result of movement of an affected part will be described.

Specifically, in a situation where the affected part to be observed shows movement such as vibration, so-called subject shake may occur. For example, such subject blur becomes more evident as the exposure time becomes longer (that is, as the shutter speed becomes slower). Therefore, for example, by controlling the exposure time to be shorter (i.e., to make the shutter speed faster) as the movement of the affected part is larger, it is possible to further reduce the influence of subject shake. . On the other hand, as the exposure time becomes shorter, the amount of light tends to decrease. Therefore, for example, when the exposure time is shortened, for example, the amount of light taken in by opening the aperture may be increased. As another example, when the exposure time is shortened, a decrease in the brightness of a captured image due to a decrease in the amount of captured light is interpolated by increasing the gain (in other words, improving the imaging sensitivity). Is also good.

For example, FIG. 11 is an explanatory diagram for describing an example of control according to the third embodiment, and illustrates an example in which an operation related to imaging of an image by an imaging unit is controlled in accordance with a detection result of movement of an affected part. ing. In the example shown in FIG. 11, the horizontal axis represents the magnitude of the vibration of the affected part (ie, the magnitude of the movement of the affected part). Further, the example illustrated in FIG. 11 illustrates an example of the relationship between the magnitude of the vibration of the affected part, the shutter speed, and the light amount. That is, in the example shown in FIG. 11, in the graph showing the relationship between the magnitude of the vibration of the affected part and the shutter speed, the vertical axis indicates the speed of the shutter speed. In addition, in the graph showing the relationship between the magnitude of the vibration of the affected part and the magnitude of the light amount (in other words, the opening amount of the aperture), the vertical axis indicates the magnitude of the light amount.

That is, in the example shown in FIG. 11, as described above, as the vibration of the affected part is larger, the shutter speed is controlled to be higher, and the light amount is larger (that is, the aperture is more opened). ) Is controlled. Further, as another example, a gain (in other words, imaging sensitivity) applied to an imaging result may be controlled instead of the amount of light to be captured. In this case, control for increasing the gain may be applied instead of control for increasing the light amount. That is, the shutter speed may be controlled to increase as the vibration of the affected part increases, and the gain may be increased (that is, the imaging sensitivity may be increased). Further, both the control of the amount of light to be taken in and the control of the gain may be performed. Alternatively, the amount of light emitted from the light source may be controlled to be large while the aperture and the gain are constant. With the above-described control, the conditions related to the imaging of the image of the affected part by the imaging unit are controlled in accordance with the magnitude of the movement of the affected part, so that it becomes difficult to observe the affected part with the vibration of the affected part. It is possible to further suppress the occurrence of a situation.

<< 4. Example of hardware configuration >>
Subsequently, with reference to FIG. 12, in the medical observation system according to the present embodiment, an information processing apparatus that executes various processes (for example, the control unit 201 illustrated in FIG. 4, the control unit 310 illustrated in FIG. 5, and the like). An example of the hardware configuration will be described in detail. FIG. 12 is a functional block diagram illustrating a configuration example of a hardware configuration of an information processing device included in the medical observation system according to an embodiment of the present disclosure.

The information processing apparatus 900 configuring the medical observation system according to the present embodiment mainly includes a CPU 901, a read only memory (ROM) 903, and a random access memory (RAM) 905. The information processing device 900 further includes a host bus 907, a bridge 909, an external bus 911, an interface 913, an input device 915, an output device 917, a storage device 919, a drive 921, and a connection port 923. And a communication device 925.

The CPU 901 functions as an arithmetic processing device and a control device, and controls the entire operation or a part of the operation in the information processing device 900 in accordance with various programs recorded in the ROM 903, the RAM 905, the storage device 919, or the removable recording medium 927. The ROM 903 stores programs used by the CPU 901 and operation parameters. The RAM 905 temporarily stores a program used by the CPU 901, parameters that appropriately change in execution of the program, and the like. These are interconnected by a host bus 907 constituted by an internal bus such as a CPU bus. The components of the control unit 301 shown in FIG. 6, that is, the image analysis unit 309, the vibration detection unit 311, the imaging control unit 313, the image processing unit 315, and the output control unit 317 can be realized by the CPU 901.

The host bus 907 is connected to an external bus 911 such as a PCI (Peripheral Component Interconnect / Interface) bus via a bridge 909. The input device 915, the output device 917, the storage device 919, the drive 921, the connection port 923, and the communication device 925 are connected to the external bus 911 via the interface 913.

The input device 915 is an operation unit operated by the user, such as a mouse, a keyboard, a touch panel, a button, a switch, a lever, and a pedal. The input device 915 may be, for example, a remote control unit (so-called remote controller) using infrared rays or other radio waves, or an externally connected device such as a mobile phone or a PDA corresponding to the operation of the information processing device 900. 929. Further, the input device 915 includes, for example, an input control circuit that generates an input signal based on information input by a user using the above-described operation means and outputs the input signal to the CPU 901. By operating the input device 915, the user of the information processing device 900 can input various data to the information processing device 900 and instruct a processing operation.

The output device 917 is a device that can visually or audibly notify the user of the acquired information. Examples of such a device include a liquid crystal display device, an organic EL (Electro Luminescence) display device, a CRT (Cathode Ray Tube) display device, a plasma display device, a display device such as a lamp, an audio output device such as a speaker and headphones, There are printer devices and the like. The output device 917 outputs, for example, results obtained by various processes performed by the information processing device 900. Specifically, the display device displays results obtained by various processes performed by the information processing device 900 as text or images. On the other hand, the audio output device converts an audio signal including reproduced audio data, acoustic data, and the like into an analog signal and outputs the analog signal. Note that the output unit 307 illustrated in FIG. 6 can be realized by the output device 917.

The storage device 919 is a data storage device configured as an example of a storage unit of the information processing device 900. The storage device 919 is configured by, for example, a magnetic storage device such as an HDD (Hard Disk Drive), a semiconductor storage device, an optical storage device, or a magneto-optical storage device. The storage device 919 stores programs executed by the CPU 901 and various data.

The drive 921 is a reader / writer for a recording medium, and is built in or externally attached to the information processing apparatus 900. The drive 921 reads information recorded on a removable recording medium 927 such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory, and outputs the information to the RAM 905. The drive 921 can also write data on a removable recording medium 927 such as a mounted magnetic disk, optical disk, magneto-optical disk, or semiconductor memory. The removable recording medium 927 is, for example, a DVD medium, an HD-DVD medium, or a Blu-ray (registered trademark) medium. Further, the removable recording medium 927 may be a compact flash (registered trademark) (CF: CompactFlash (registered trademark)), a flash memory, an SD memory card (Secure Digital memory card), or the like. Further, the removable recording medium 927 may be, for example, an IC card (Integrated Circuit card) on which a non-contact type IC chip is mounted, an electronic device, or the like.

The connection port 923 is a port for directly connecting to the information processing device 900. Examples of the connection port 923 include a USB (Universal Serial Bus) port, an IEEE 1394 port, and a SCSI (Small Computer System Interface) port. Other examples of the connection port 923 include an RS-232C port, an optical audio terminal, and an HDMI (registered trademark) (High-Definition Multimedia Interface) port. By connecting the external connection device 929 to the connection port 923, the information processing apparatus 900 obtains various data directly from the external connection device 929 or provides various data to the external connection device 929.

The communication device 925 is, for example, a communication interface including a communication device for connecting to a communication network (network) 931. The communication device 925 is, for example, a communication card for a wired or wireless LAN (Local Area Network), Bluetooth (registered trademark), or WUSB (Wireless USB). The communication device 925 may be a router for optical communication, a router for ADSL (Asymmetric Digital Subscriber Line), a modem for various kinds of communication, or the like. The communication device 925 can transmit and receive signals and the like to and from the Internet and other communication devices in accordance with a predetermined protocol such as TCP / IP. Further, the communication network 931 connected to the communication device 925 is configured by a network or the like connected by wire or wirelessly, and may be, for example, the Internet, a home LAN, infrared communication, radio wave communication, satellite communication, or the like. .

As described above, an example of the hardware configuration capable of realizing the function of the information processing device 900 included in the medical observation system according to the embodiment of the present disclosure has been described. Each of the above components may be configured using a general-purpose member, or may be configured by hardware specialized for the function of each component. Therefore, it is possible to appropriately change the hardware configuration to be used according to the technical level at the time of implementing the present embodiment. Although not shown in FIG. 12, various components corresponding to the information processing device 900 included in the medical observation system are naturally provided.

Note that a computer program for realizing each function of the information processing device 900 included in the medical observation system according to the present embodiment as described above can be created and mounted on a personal computer or the like. Further, a computer-readable recording medium in which such a computer program is stored can be provided. The recording medium is, for example, a magnetic disk, an optical disk, a magneto-optical disk, a flash memory, or the like. Further, the above-described computer program may be distributed, for example, via a network without using a recording medium. The number of computers that execute the computer program is not particularly limited. For example, a plurality of computers (for example, a plurality of servers) may execute the computer program in cooperation with each other.

<< 5. Application examples >>
Subsequently, as an application example of the medical observation system according to an embodiment of the present disclosure, with reference to FIGS. 13 and 14, the medical observation system was configured as an endoscopic surgery system including a microscope unit. An example of the case will be described.

FIGS. 13 and 14 are explanatory diagrams for describing an application example of the medical observation system according to an embodiment of the present disclosure, and show an example of a schematic configuration of an endoscopic surgery system.

For example, FIG. 13 is a diagram illustrating an example of a schematic configuration of an endoscopic surgery system to which the technology according to the present disclosure can be applied. FIG. 13 illustrates a state in which an operator (doctor) 167 performs an operation on a patient 171 on a patient bed 169 using the endoscopic operation system 100. As shown in the figure, the endoscope operation system 100 includes an endoscope 101, other surgical instruments 117, a support arm device 127 for supporting the endoscope 101, and various devices for endoscopic surgery. And a cart 137 on which is mounted.

In endoscopic surgery, instead of cutting the abdominal wall and opening the abdomen, a plurality of tubular opening instruments called trocars 125a to 125d are punctured into the abdominal wall. Then, the lens barrel 103 of the endoscope 101 and other surgical instruments 117 are inserted into the body cavity of the patient 171 from the trocars 125a to 125d. In the illustrated example, an insufflation tube 119, an energy treatment device 121, and forceps 123 are inserted into the body cavity of the patient 171 as other operation tools 117. The energy treatment tool 121 is a treatment tool that performs incision and exfoliation of tissue, sealing of blood vessels, and the like by high-frequency current and ultrasonic vibration. However, the illustrated surgical tool 117 is merely an example, and various surgical tools that are generally used in an endoscopic operation, such as a set, a retractor, and the like, may be used as the surgical tool 117.

画像 The image of the operative site in the body cavity of the patient 171 taken by the endoscope 101 is displayed on the display device 141. The operator 167 performs a procedure such as excision of the affected part using the energy treatment tool 121 and the forceps 123 while viewing the image of the operated part displayed on the display device 141 in real time. Although not shown, the insufflation tube 119, the energy treatment tool 121, and the forceps 123 are supported by the surgeon 167 or an assistant during the operation.

(Support arm device)
The support arm device 127 includes an arm 131 extending from the base 129. In the illustrated example, the arm unit 131 includes joints 133a, 133b, and 133c, and links 135a and 135b, and is driven by the control of the arm control device 145. The endoscope 101 is supported by the arm 131, and its position and posture are controlled. Thereby, stable fixing of the position of the endoscope 101 can be realized.

(Endoscope)
The endoscope 101 includes a lens barrel 103 in which a region of a predetermined length from the distal end is inserted into a body cavity of the patient 171, and a camera head 105 connected to a proximal end of the lens barrel 103. In the illustrated example, the endoscope 101 is configured as a so-called rigid scope having a hard barrel 103. However, the endoscope 101 is configured as a so-called flexible scope having a flexible barrel 103. Is also good. Note that the camera head 105 or the endoscope 101 including the camera head 105 corresponds to an example of a “medical observation device”.

開口 At the tip of the lens barrel 103, an opening in which the objective lens is fitted is provided. A light source device 143 is connected to the endoscope 101, and light generated by the light source device 143 is guided to a tip of the lens barrel by a light guide extending inside the lens barrel 103, and an objective is provided. The light is irradiated toward an observation target (in other words, an imaging target) in the body cavity of the patient 171 via the lens. In addition, the endoscope 101 may be a direct view, a perspective view, or a side view.

(4) An optical system and an image sensor are provided inside the camera head 105, and the reflected light (observation light) from the observation target is condensed on the image sensor by the optical system. The observation light is photoelectrically converted by the imaging element, and an electric signal corresponding to the observation light, that is, an image signal corresponding to the observation image is generated. The image signal is transmitted to a camera control unit (CCU) 139 as RAW data. The camera head 105 has a function of adjusting the magnification and the focal length by appropriately driving the optical system.

Note that the camera head 105 may be provided with a plurality of image sensors in order to support, for example, stereoscopic viewing (3D display). In this case, a plurality of relay optical systems are provided inside the lens barrel 103 in order to guide observation light to each of the plurality of imaging elements.

(Various devices mounted on cart)
The CCU 139 is configured by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and the like, and controls the operations of the endoscope 101 and the display device 141 as a whole. Specifically, the CCU 139 performs various types of image processing for displaying an image based on the image signal, such as a development process (demosaicing process), on the image signal received from the camera head 105. The CCU 139 provides the image signal subjected to the image processing to the display device 141. In addition, the CCU 139 transmits a control signal to the camera head 105 and controls its driving. The control signal may include information on imaging conditions such as a magnification and a focal length.

The display device 141 displays an image based on an image signal on which image processing has been performed by the CCU 139 under the control of the CCU 139. When the endoscope 101 supports high-resolution imaging such as 4K (3840 horizontal pixels × 2160 vertical pixels) or 8K (7680 horizontal pixels × 4320 vertical pixels), and / or 3D display In the case where the display device 141 is compatible, a display device that can display a high-resolution image and / or a device that can display a 3D image can be used. In the case of shooting at a high resolution such as 4K or 8K, the use of the display device 141 having a size of 55 inches or more can provide a more immersive feeling. Further, a plurality of display devices 141 having different resolutions and sizes may be provided depending on the application.

The light source device 143 includes a light source such as an LED (light emitting diode), for example, and supplies the endoscope 101 with irradiation light when imaging the operation site.

The arm control device 145 is configured by a processor such as a CPU, for example, and operates according to a predetermined program to control the driving of the arm 131 of the support arm device 127 according to a predetermined control method.

The input device 147 is an input interface to the endoscopic surgery system 100. The user can input various information and input instructions to the endoscopic surgery system 100 via the input device 147. For example, the user inputs, via the input device 147, various types of information related to surgery, such as physical information of a patient and information about a surgical procedure. Further, for example, the user issues an instruction to drive the arm unit 131 via the input device 147 or an instruction to change imaging conditions (such as the type of irradiation light, magnification, and focal length) of the endoscope 101. , An instruction to drive the energy treatment tool 121 is input.

The type of the input device 147 is not limited, and the input device 147 may be various known input devices. As the input device 147, for example, a mouse, a keyboard, a touch panel, a switch, a foot switch 157, and / or a lever can be applied. When a touch panel is used as the input device 147, the touch panel may be provided on the display surface of the display device 141.

Alternatively, the input device 147 may be a sensor provided on a device worn by a user, such as a glasses-type wearable device or an HMD (Head Mounted Display). In this case, the user's movement detected by these sensors And various inputs are performed according to the line of sight. Further, the input device 147 includes a camera capable of detecting the movement of the user, and various inputs are performed in accordance with the user's gestures and eyes, which are detected from the video captured by the camera. Further, the input device 147 includes a microphone capable of collecting a user's voice, and various inputs are performed by voice via the microphone. As described above, since the input device 147 is configured to be capable of inputting various kinds of information in a non-contact manner, a user (for example, an operator 167) belonging to a clean area can operate a device belonging to a dirty area in a non-contact manner. Becomes possible. In addition, since the user can operate the device without releasing his / her hand from the surgical tool, the convenience for the user is improved.

The treatment instrument control device 149 controls the driving of the energy treatment instrument 121 for cauterizing, incising a tissue, or sealing a blood vessel. The insufflation device 151 supplies gas through the insufflation tube 119 through the insufflation tube 119 to inflate the body cavity of the patient 171 for the purpose of securing the visual field by the endoscope 101 and securing the working space of the operator. Send. The recorder 153 is a device that can record various types of information related to surgery. The printer 155 is a device that can print various types of information related to surgery in various formats such as text, images, and graphs.

Hereinafter, a particularly characteristic configuration of the endoscopic surgery system 100 will be described in more detail.

(Support arm device)
The support arm device 127 includes a base 129 as a base, and an arm 131 extending from the base 129. In the illustrated example, the arm 131 includes a plurality of joints 133a, 133b, and 133c, and a plurality of links 135a and 135b connected by the joints 133b. , The configuration of the arm section 131 is simplified. Actually, the shapes, numbers and arrangements of the joints 133a to 133c and the links 135a and 135b, the directions of the rotation axes of the joints 133a to 133c, and the like are appropriately set so that the arm 131 has a desired degree of freedom. obtain. For example, the arm part 131 can be preferably configured to have six or more degrees of freedom. Accordingly, the endoscope 101 can be freely moved within the movable range of the arm 131, so that the lens barrel 103 of the endoscope 101 can be inserted into the body cavity of the patient 171 from a desired direction. Will be possible.

The joints 133a to 133c are provided with actuators, and the joints 133a to 133c are configured to be rotatable around a predetermined rotation axis by driving the actuators. When the drive of the actuator is controlled by the arm control device 145, the rotation angles of the joints 133a to 133c are controlled, and the drive of the arm 131 is controlled. Thereby, control of the position and orientation of the endoscope 101 can be realized. At this time, the arm control device 145 can control the driving of the arm unit 131 by various known control methods such as force control or position control.

For example, when the surgeon 167 appropriately performs an operation input via the input device 147 (including the foot switch 157), the drive of the arm unit 131 is appropriately controlled by the arm control device 145 in accordance with the operation input. The position and orientation of the endoscope 101 may be controlled. With this control, after the endoscope 101 at the tip of the arm 131 is moved from an arbitrary position to an arbitrary position, it can be fixedly supported at the position after the movement. Note that the arm 131 may be operated by a so-called master slave method. In this case, the arm 131 can be remotely controlled by the user via the input device 147 installed at a location away from the operating room.

When force control is applied, the arm control device 145 receives the external force from the user, and controls the actuators of the joints 133a to 133c so that the arm 131 moves smoothly in accordance with the external force. Driving, so-called power assist control, may be performed. Thus, when the user moves the arm 131 while directly touching the arm 131, the arm 131 can be moved with a relatively light force. Therefore, the endoscope 101 can be moved more intuitively and with a simpler operation, and the convenience for the user can be improved.

Here, generally, in the endoscopic operation, the endoscope 101 is supported by a doctor called a scopist. On the other hand, by using the support arm device 127, the position of the endoscope 101 can be fixed more reliably without manual operation, so that an image of the operation site can be stably obtained. Thus, the operation can be performed smoothly.

Note that the arm control device 145 is not necessarily provided in the cart 137. Further, the arm control device 145 need not necessarily be one device. For example, the arm control device 145 may be provided in each of the joint portions 133a to 133c of the arm portion 131 of the support arm device 127, and the plurality of arm control devices 145 cooperate with each other to drive the arm portion 131. Control may be realized.

(Light source device)
The light source device 143 supplies the endoscope 101 with irradiation light when capturing an image of an operation part. The light source device 143 includes, for example, a white light source including an LED, a laser light source, or a combination thereof. At this time, when a white light source is formed by combining the RGB laser light sources, the output intensity and output timing of each color (each wavelength) can be controlled with high accuracy. Can be adjusted. In this case, the laser light from each of the RGB laser light sources is irradiated to the observation target in a time-division manner, and the driving of the image pickup device of the camera head 105 is controlled in synchronization with the irradiation timing, so that each of the RGB laser light sources is controlled. It is also possible to capture the image obtained in a time sharing manner. According to this method, a color image can be obtained without providing a color filter in the image sensor.

The driving of the light source device 143 may be controlled so as to change the intensity of the output light every predetermined time. By controlling the driving of the image pickup device of the camera head 105 in synchronization with the timing of the change of the light intensity, an image is acquired in a time-division manner, and the image is synthesized, so that a high dynamic image without a so-called blackout or overexposure is obtained. An image of the range can be generated.

The light source device 143 may be configured to be able to supply light in a predetermined wavelength band corresponding to special light observation. In the special light observation, for example, by utilizing the wavelength dependence of light absorption in body tissue, by irradiating light in a narrower band compared to irradiation light (ie, white light) during normal observation, the surface of the mucous membrane is exposed. A so-called narrow-band light observation (Narrow Band Imaging) for photographing a predetermined tissue such as a blood vessel with high contrast is performed. Alternatively, in the special light observation, a fluorescence observation for obtaining an image by fluorescence generated by irradiating the excitation light may be performed. In fluorescence observation, a body tissue is irradiated with excitation light to observe fluorescence from the body tissue (autofluorescence observation), or a reagent such as indocyanine green (ICG) is locally injected into the body tissue and the body tissue is irradiated with the reagent. Irradiation with excitation light corresponding to the fluorescence wavelength of the reagent to obtain a fluorescence image may be performed. The light source device 143 can be configured to be able to supply narrowband light and / or excitation light corresponding to such special light observation.

(Camera head and CCU)
The functions of the camera head 105 and the CCU 139 of the endoscope 101 will be described in more detail with reference to FIG. FIG. 14 is a block diagram illustrating an example of a functional configuration of the camera head 105 and the CCU 139 illustrated in FIG.

参照 Referring to FIG. 14, the camera head 105 has, as its functions, a lens unit 107, an imaging unit 109, a driving unit 111, a communication unit 113, and a camera head control unit 115. The CCU 139 has a communication unit 159, an image processing unit 161, and a control unit 163 as its functions. The camera head 105 and the CCU 139 are communicably connected by a transmission cable 165.

First, the functional configuration of the camera head 105 will be described. The lens unit 107 is an optical system provided at a connection with the lens barrel 103. Observation light taken in from the tip of the lens barrel 103 is guided to the camera head 105 and enters the lens unit 107. The lens unit 107 is configured by combining a plurality of lenses including a zoom lens and a focus lens. The optical characteristics of the lens unit 107 are adjusted so that the observation light is focused on the light receiving surface of the imaging element of the imaging unit 109. Further, the zoom lens and the focus lens are configured such that their positions on the optical axis are movable for adjusting the magnification and the focus of the captured image.

The imaging unit 109 is configured by an imaging element, and is arranged at a stage subsequent to the lens unit 107. The observation light that has passed through the lens unit 107 is collected on the light receiving surface of the image sensor, and an image signal corresponding to the observation image is generated by photoelectric conversion. The image signal generated by the imaging unit 109 is provided to the communication unit 113.

The imaging device constituting the imaging unit 109 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) type image sensor or a CCD (Charge Coupled Device) type image sensor, and is capable of taking a color image having a Bayer array. Although one is used, it may be for single-pane monochrome photography. A plurality of black and white image sensors may be used. Note that, as the image pickup device, an image pickup device capable of capturing a high-resolution image of, for example, 4K or more may be used. By obtaining the image of the operation part with high resolution, the operator 167 can grasp the state of the operation part in more detail, and can perform the operation more smoothly.

The image pickup device constituting the image pickup unit 109 may be configured to have a pair of image pickup devices for acquiring right-eye and left-eye image signals corresponding to 3D display. By performing the 3D display, the operator 167 can more accurately grasp the depth of the living tissue in the operative part. When the image pickup unit 109 is configured as a multi-plate type, a plurality of lens units 107 are provided corresponding to the respective image pickup devices.

撮 像 In addition, the imaging unit 109 does not necessarily need to be provided in the camera head 105. For example, the imaging unit 109 may be provided inside the lens barrel 103 immediately after the objective lens.

The drive unit 111 is configured by an actuator, and moves the zoom lens and the focus lens of the lens unit 107 by a predetermined distance along the optical axis under the control of the camera head control unit 115. Thereby, the magnification and the focus of the image captured by the imaging unit 109 can be appropriately adjusted.

The communication unit 113 includes a communication device for transmitting and receiving various information to and from the CCU 139. The communication unit 113 transmits the image signal obtained from the imaging unit 109 as RAW data to the CCU 139 via the transmission cable 165. At this time, it is preferable that the image signal be transmitted by optical communication in order to display the captured image of the operation site with as little delay as possible. At the time of the operation, the operator 167 performs the operation while observing the state of the affected part with the captured image, so that a moving image of the operation part is displayed in real time as much as possible for safer and more reliable operation. Is required. When optical communication is performed, the communication unit 113 includes a photoelectric conversion module that converts an electric signal into an optical signal. The image signal is converted into an optical signal by the photoelectric conversion module, and then transmitted to the CCU 139 via the transmission cable 165.

(4) The communication unit 113 receives a control signal for controlling the driving of the camera head 105 from the CCU 139. The control signal includes, for example, information for specifying a frame rate of a captured image, information for specifying imaging conditions (shutter speed, aperture, gain, and the like) at the time of shooting, and / or magnification and focus of the captured image. And information on the imaging condition, such as information indicating that is designated. The communication unit 113 provides the received control signal to the camera head control unit 115. Note that the control signal from the CCU 139 may also be transmitted by optical communication. In this case, the communication unit 113 is provided with a photoelectric conversion module that converts an optical signal into an electric signal, and the control signal is provided to the camera head control unit 115 after being converted into an electric signal by the photoelectric conversion module.

Note that the above-described imaging conditions such as the frame rate, the exposure value, the magnification, and the focus are automatically set by the control unit 163 of the CCU 139 based on the acquired image signals. That is, the CCU 139 and the endoscope 101 realize a so-called AE (Auto Exposure) function, an AF (Auto Focus) function, and an AWB (Auto White Balance) function.

The camera head control unit 115 controls the driving of the camera head 105 based on the control signal from the CCU 139 received via the communication unit 113. For example, the camera head control unit 115 controls the driving of the imaging element of the imaging unit 109 based on the information for specifying the frame rate of the captured image and / or the information for specifying the shutter speed and aperture during imaging. I do. In addition, for example, the camera head control unit 115 appropriately moves the zoom lens and the focus lens of the lens unit 107 via the driving unit 111 based on information for designating the magnification and the focus of the captured image. The camera head control unit 115 may further have a function of storing information for identifying the lens barrel 103 and the camera head 105.

By arranging the components such as the lens unit 107 and the imaging unit 109 in a hermetically sealed structure having high airtightness and waterproofness, the camera head 105 can have resistance to autoclave sterilization.

Next, the functional configuration of the CCU 139 will be described. The communication unit 159 is configured by a communication device for transmitting and receiving various information to and from the camera head 105. The communication unit 159 receives an image signal transmitted from the camera head 105 via the transmission cable 165. At this time, as described above, the image signal can be suitably transmitted by optical communication. In this case, the communication unit 159 is provided with a photoelectric conversion module that converts an optical signal into an electric signal corresponding to the optical communication. The communication unit 159 provides the image signal converted to the electric signal to the image processing unit 161.

(4) The communication unit 159 transmits a control signal for controlling the driving of the camera head 105 to the camera head 105. The control signal may also be transmitted by optical communication.

The image processing unit 161 performs various types of image processing on an image signal that is RAW data transmitted from the camera head 105. The image processing includes, for example, development processing, high image quality processing (band enhancement processing, super-resolution processing, NR (Noise reduction) processing, and / or camera shake correction processing, etc.), and / or enlargement processing (electronic zoom processing). And various known signal processing. The image processing unit 161 performs a detection process on the image signal for performing AE, AF, and AWB.

The image processing unit 161 is configured by a processor such as a CPU and a GPU, and the above-described image processing and detection processing can be performed by the processor operating according to a predetermined program. When the image processing unit 161 is configured by a plurality of GPUs, the image processing unit 161 appropriately divides information related to the image signal and performs image processing in parallel by the plurality of GPUs.

The control unit 163 performs various controls related to the imaging of the operation site by the endoscope 101 and the display of the captured image. For example, the control unit 163 generates a control signal for controlling driving of the camera head 105. At this time, when the imaging condition is input by the user, the control unit 163 generates a control signal based on the input by the user. Alternatively, when the AE function, the AF function, and the AWB function are mounted on the endoscope 101, the control unit 163 determines the optimal exposure condition, the focal length, and the like in accordance with the result of the detection processing by the image processing unit 161. The white balance is appropriately calculated and a control signal is generated.

(4) The control unit 163 causes the display device 141 to display an image of the surgical site based on the image signal on which the image processing has been performed by the image processing unit 161. At this time, the control unit 163 recognizes various objects in the operative image using various image recognition techniques. For example, the control unit 163 detects a surgical tool such as forceps, a specific living body site, a bleeding, a mist at the time of using the energy treatment tool 121, and the like by detecting an edge shape, a color, and the like of an object included in the surgical image. Can be recognized. When displaying the image of the surgical site on the display device 141, the control unit 163 superimposes and displays various types of surgery support information on the image of the surgical site using the recognition result. By superimposing the operation support information and presenting it to the operator 167, the operation can be performed more safely and reliably.

The transmission cable 165 connecting the camera head 105 and the CCU 139 is an electric signal cable corresponding to electric signal communication, an optical fiber corresponding to optical communication, or a composite cable thereof.

Here, in the illustrated example, the communication is performed by wire using the transmission cable 165, but the communication between the camera head 105 and the CCU 139 may be performed wirelessly. When the communication between the two is performed wirelessly, the transmission cable 165 does not need to be laid in the operating room, and the situation in which the movement of the medical staff in the operating room is hindered by the transmission cable 165 can be solved.

As described above, an example of the endoscopic surgery system 100 to which the technology according to the present disclosure can be applied has been described. Although the endoscopic surgery system 100 has been described here as an example, a system to which the technology according to the present disclosure can be applied is not limited to such an example. For example, the technology according to the present disclosure may be applied to an inspection flexible endoscope system or a microscopic surgery system.

The technique according to the present disclosure described above can be applied without departing from the basic idea of the medical observation system according to an embodiment of the present disclosure without being limited to the above. As a specific example, the present invention is not limited to a system to which the endoscope or the operation microscope described above is applied, and a system in which an image of an affected part can be observed by capturing an image of the affected part by an imaging device of a desired form. The technology according to the present disclosure described above can be appropriately applied.

観 察 Needless to say, the method of observing the affected area and the applied technique are not particularly limited. For example, as an observation method (treatment method) of setting an aneurysm as an affected part to be observed, not only the above-described clipping operation but also a method using a stent and a method using a flow diverter are known. Further, a treatment tool to be used may be different depending on an observation method and an applied technique. Even in such a case, for example, if the treatment tool is held in the vicinity of the affected part, the above-described technology according to the present disclosure may be applied to extract the movement of the treatment tool from the sequentially captured image of the affected part. Thus, the movement of the affected part can be detected.

As described above, as an application example of the medical observation system according to an embodiment of the present disclosure, an example in which the medical observation system is configured as an endoscopic surgery system has been described with reference to FIGS. .

<< 6. Conclusion >>
As described above, the medical observation system according to an embodiment of the present disclosure includes an imaging unit, a detection unit, and a control unit. The imaging unit captures an image of the affected part. The detection unit extracts the movement of the treatment tool held near the affected part based on the image of the affected part sequentially captured by the imaging unit, and detects the movement of the affected part based on the result of the extraction. The control unit controls a process related to observation of the affected part according to the detection result of the movement of the affected part. With the above-described configuration, for example, even in a situation where the affected part can move regardless of the presence or absence of the procedure, such as a case where the affected part such as a blood vessel vibrates with pulsation, the observation of the affected part can be improved. This can be realized in a suitable manner.

Although the preferred embodiments of the present disclosure have been described above in detail with reference to the accompanying drawings, the technical scope of the present disclosure is not limited to such examples. It is apparent that a person having ordinary knowledge in the technical field of the present disclosure can arrive at various changes or modifications within the scope of the technical idea described in the claims. It is understood that also belongs to the technical scope of the present disclosure.

効果 In addition, the effects described in this specification are merely illustrative or exemplary, and are not restrictive. That is, the technology according to the present disclosure can exhibit other effects that are obvious to those skilled in the art from the description in the present specification, in addition to or instead of the above effects.

The following configuration also belongs to the technical scope of the present disclosure.
(1)
An imaging unit that captures an image of the affected part;
A detection unit that extracts the movement of the treatment tool held in the vicinity of the affected part based on the image of the affected part sequentially captured by the imaging unit, and detects the movement of the affected part based on the result of the extraction;
A control unit that controls a process related to observation of the affected part according to a detection result of the movement of the affected part,
A medical observation system comprising:
(2)
An endoscope unit including a lens barrel inserted into the body cavity of the patient,
The imaging unit captures an image of the affected part obtained by the endoscope unit,
The medical observation system according to (1).
(3)
With a microscope unit to obtain an enlarged image of the affected area,
The imaging unit captures the enlarged image acquired by the microscope unit,
The medical observation system according to (1).
(4)
A detection unit that extracts the movement of the treatment tool held in the vicinity of the affected part based on the image of the affected part sequentially captured by the imaging unit, and detects the movement of the affected part based on the result of the extraction.
A control unit that controls a process related to observation of the affected part according to a detection result of the movement of the affected part,
A medical observation device comprising:
(5)
The medical observation device according to (4), wherein the control unit performs image processing on a captured image of the affected part based on a detection result of movement of the affected part.
(6)
The medical observation device according to (5), wherein the control unit corrects blurring of the captured image of the affected part based on a detection result of the movement of the affected part.
(7)
The control unit controls the image of the affected part before the image processing is performed, and the image of the affected part after the image processing is performed, so that the image is presented to the output unit in association with the image processing. The medical observation device according to (5) or (6).
(8)
The medical observation device according to any one of (4) to (7), wherein the control unit causes the output unit to present display information based on a detection result of the movement of the affected part.
(9)
The medical observation device according to (8), wherein the control unit controls the display information to be presented on an output unit according to the detected magnitude of the movement of the affected part.
(10)
The medical observation device according to (9), wherein the control unit causes the output unit to present a warning as the display information when the detected magnitude of movement of the affected part exceeds a threshold value.
(11)
The medical observation according to (9) or (10), wherein the control unit causes the output unit to present information regarding a procedure as the display information when the detected motion magnitude threshold value is equal to or smaller than the threshold value. apparatus.
(12)
The medical observation device according to any one of (4) to (11), wherein the control unit controls conditions relating to observation of the diseased part based on a detection result of movement of the diseased part.
(13)
The medical observation apparatus according to (12), wherein the control unit controls at least one of a shutter speed, an aperture, and a gain of the imaging unit according to the detected magnitude of the movement of the affected part. .
(14)
When the control unit exceeds the detected threshold value of the movement of the affected part, the control to increase the shutter speed, the control to increase the aperture, and the control to increase the gain, The medical observation device according to (13), which performs at least one of the above.
(15)
The detection unit according to any one of (4) to (14), wherein the detection unit extracts a movement of the treatment tool by detecting a light emitting body held by the treatment tool from the sequentially captured images. 4. The medical observation device according to claim 1.
(16)
The detection unit according to any one of (4) to (14), wherein the detecting unit detects at least a part of the treatment tool having a predetermined characteristic from the sequentially captured images to extract a movement of the treatment tool. The medical observation device according to claim 1.
(17)
The medical observation device according to any one of (4) to (16), wherein the affected part is a blood vessel.
(18)
The medical observation device according to (17), wherein the affected part is an aneurysm.
(19)
Computer
Extracting the movement of the treatment tool held near the affected part based on the image of the affected part sequentially captured by the imaging unit, and detecting the movement of the affected part based on the result of the extraction;
According to the detection result of the movement of the affected part, to control the processing related to the observation of the affected part,
A method for driving a medical observation device, comprising:
(20)
On the computer,
Extracting the movement of the treatment tool held near the affected part based on the image of the affected part sequentially captured by the imaging unit, and detecting the movement of the affected part based on the result of the extraction;
According to the detection result of the movement of the affected part, to control the processing related to the observation of the affected part,
To run the program.

2, 3 Medical observation system 165, 221, 225 Transmission cable 201 Control unit 203 Imaging unit 205 Sensor driver 207 Input unit 209 Output unit 211 Imaging optical system 213 Branch optical system 215, 217 Image sensor 219 RGB laser 223 IR laser 227, 229 Vibration sensor 301 Control unit 303 Imaging unit 305 Detection unit 307 Output unit 309 Image analysis unit 311 Vibration detection unit 313 Imaging control unit 315 Image processing unit 317 Output control unit

Claims (19)

An imaging unit that captures an image of the affected part;
A detection unit that extracts the movement of the treatment tool held in the vicinity of the affected part based on the image of the affected part sequentially captured by the imaging unit, and detects the movement of the affected part based on the result of the extraction;
A control unit that controls a process related to observation of the affected part according to a detection result of the movement of the affected part,
A medical observation system comprising: An endoscope unit including a lens barrel inserted into the body cavity of the patient,
The imaging unit captures an image of the affected part obtained by the endoscope unit,
The medical observation system according to claim 1. With a microscope unit to obtain an enlarged image of the affected area,
The imaging unit captures the enlarged image acquired by the microscope unit,
The medical observation system according to claim 1. A detection unit that extracts the movement of the treatment tool held in the vicinity of the affected part based on the image of the affected part sequentially captured by the imaging unit, and detects the movement of the affected part based on the result of the extraction.
A control unit that controls a process related to observation of the affected part according to a detection result of the movement of the affected part,
A medical observation device comprising: 5. The medical observation apparatus according to claim 4, wherein the control unit performs image processing on the captured image of the affected part based on a detection result of the movement of the affected part. 6. The medical observation apparatus according to claim 5, wherein the control unit corrects blurring of the captured image of the affected part based on a detection result of movement of the affected part. The control unit controls the image of the affected part before the image processing is performed, and the image of the affected part after the image processing is performed, so that the image is presented to the output unit in association with the image processing. The medical observation device according to claim 5. The medical observation device according to claim 4, wherein the control unit causes the output unit to present display information based on the detection result of the movement of the affected part. The medical observation device according to claim 8, wherein the control unit controls the display information to be presented on the output unit according to the detected magnitude of the movement of the affected part. The medical observation device according to claim 9, wherein the control unit causes the output unit to present a warning as the display information when the detected magnitude of the motion of the affected part exceeds a threshold value. 10. The medical observation apparatus according to claim 9, wherein the control unit causes the output unit to present information on a procedure as the display information when the detected motion magnitude threshold value is equal to or less than the threshold value. 10. 5. The medical observation apparatus according to claim 4, wherein the control unit controls a condition regarding observation of the diseased part based on a detection result of movement of the diseased part. 13. The medical observation device according to claim 12, wherein the control unit controls at least one of a shutter speed, an aperture, and a gain of the imaging unit according to the detected magnitude of the movement of the affected part. When the control unit exceeds the detected threshold value of the movement of the affected part, the control to increase the shutter speed, the control to increase the aperture, and the control to increase the gain, 14. The medical observation device according to claim 13, which performs at least one of them. 5. The medical observation apparatus according to claim 4, wherein the detection unit extracts a movement of the treatment tool by detecting a luminous body held by the treatment tool from the sequentially captured images. 5. The medical observation according to claim 4, wherein the detection unit extracts at least a part of the treatment tool having a predetermined characteristic from the sequentially captured images to detect a movement of the treatment tool. apparatus. The medical observation device according to claim 4, wherein the affected part is a blood vessel. The medical observation device according to claim 17, wherein the affected part is an aneurysm. Computer
Extracting the movement of the treatment tool held near the affected part based on the image of the affected part sequentially captured by the imaging unit, and detecting the movement of the affected part based on the result of the extraction;
According to the detection result of the movement of the affected part, to control the processing related to the observation of the affected part,
A method for driving a medical observation device, comprising:

Images