WO2018167969A1 - Imaging device - Google Patents

Abstract

An imaging device 1 visualizes an image of a lymphatic vessel as a near-infrared image by fluorescence of indocyanine green, and visualizes an image of a vein as a reverse image of near-infrared image luminance absorbed by blood. In the present invention, lighting of first and second light sources and acquisition of an image by a first imaging element are executed in synchronized fashion using a synchronous clock, whereby a lymphatic vessel and a vein are visualized at the same time in the same field of view, and the images are synthesized with a visible image of a subject and displayed. A lymphatic vessel and a vein to be anastomosed can thereby be visualized at the same time in the same field of view.

Description

Imaging device

The present invention relates to an imaging apparatus for simultaneously displaying a lymph vessel image and a vein image in a subject.

Treatment of swelling caused by lymphedema of the extremities includes indirect methods that improve lymphatic flow in addition to drainage, which is a direct method of sucking fat that causes swelling. A lymphatic venule anastomosis has been adopted.

In this lymphatic venule anastomosis, in order to visualize the vein at the time of surgery, a method of directly irradiating the vein with near infrared light and directly observing the light absorbed by blood (hemoglobin) is adopted. (See Patent Document 1).

On the other hand, as a method for visualizing lymphatic vessels, a technique called near-infrared fluorescence imaging is used for angiography in surgery. In this near-infrared fluorescence imaging, indocyanine green (ICG), which is a fluorescent dye, is injected into a vein or lymph vessel by an injector or the like. When indocyanine green is irradiated with near-infrared light having a wavelength of about 600 to 850 nm (nanometer) as excitation light, indocyanine green emits near-infrared fluorescence having a wavelength of about 750 to 900 nm. This fluorescence is photographed by an imaging device capable of detecting near infrared light, and the image is displayed on a display unit such as a liquid crystal display panel. According to this near-infrared fluorescence imaging, it is possible to observe blood vessels, lymph vessels, and the like existing at a depth of about 20 mm from the body surface.

Patent Document 2 discloses an intensity distribution image of near-infrared fluorescence obtained by irradiating an indocyanine green excitation light to a living organ to which indocyanine green is administered, and before indocyanine green administration. Compared with the cancer lesion distribution image obtained by applying X-ray, nuclear magnetic resonance or ultrasound to the subject's organs, it is detected by the intensity distribution image of near-infrared fluorescence, but the cancer lesion distribution image Discloses a data collection method for collecting data of a region that is not detected as secondary lesion region data of cancer.

JP 2011-139731 A International Publication No. 2009/139466

In the lymphatic venule anastomosis, when the anastomotic vein becomes 1.5 mm or more, the blood flows backward into the lymphatic vessel, so a thin vein having a diameter of about 0.6 mm to 1.0 mm is selected. When performing microscopic surgery for anastomosing such venules and lymphatic vessels (about 0.3 mm in diameter), the field of view can be reduced to several to several tens of times by using an eyepiece such as a so-called surgical microscope. Enlarging the surgery. At this time, since the vein to be anastomosed close to the lymphatic vessel is selected, marking with a vascular tape or the like is required to prevent misidentification between the vein and the vein before surgery. In order to eliminate such a necessity, it is preferable that the anastomosing vein and lymphatic vessel are simultaneously visible in the same visual field.

The present invention has been made to solve the above-described problems, and an object of the present invention is to provide an imaging apparatus that can simultaneously visualize a vein to be anastomosed and a lymphatic vessel within the same visual field.

The invention according to claim 1 is an imaging apparatus for simultaneously displaying an image of a lymphatic vessel and an image of a vein in a subject, and excites a fluorescent dye administered to the lymphatic vessel of the subject A first light source that irradiates light of a first wavelength toward the subject, and a light of a second wavelength that is absorbed in blood in the vein of the subject is directed toward the subject. A second light source for irradiating, a first image sensor for photographing the subject at a predetermined frame rate, and the first light source and the second light source are synchronized with a frame rate for photographing by the first image sensor. The light source controller that alternately turns on and the luminance of the image of the subject imaged by the first image sensor when the second light source is turned on are reversed, and the first light source is turned on. Taken by the first image sensor when Coloring at least one of a lymphatic region in an image and a vein region in an image captured by the first image sensor and inverted in luminance when the second light source is turned on, And an image processing unit to be displayed on the display unit.

According to a second aspect of the present invention, in the first aspect of the present invention, the image processing unit is configured to display an image of a subject photographed by the first image sensor when the first light source is turned on. Luminance of the image of the subject imaged by the first image sensor when the lymph vessel region is colored in a first color and displayed on the display unit and the second light source is turned on And the vein region in the image whose luminance is inverted is colored in a second color different from the first color and then displayed on the display unit.

The invention according to claim 3 is the invention according to claim 2, further comprising: a third light source that irradiates visible light toward the subject; and a second imaging element that photographs the subject. The image processing unit includes a visible image captured by the second image sensor together with the lymphatic vessel region and the vein region in the subject image captured by the first image sensor. And display it.

According to a fourth aspect of the present invention, in the third aspect of the present invention, the fluorescence from the fluorescent dye and the light having the second wavelength are incident on the first imaging element, and visible light is incident on the second. A dichroic prism that is incident on the image sensor is provided.

The invention according to claim 5 is the invention according to claim 4, wherein the fluorescence from the fluorescent dye excited by the light of the first wavelength and the fluorescence between the dichroic prism and the first image sensor. A first band-pass filter that transmits light of the second wavelength and does not transmit the light of the first wavelength and the fluorescence of the fluorescent dye excited by the light of the second wavelength; and the dichroic A second band-pass filter that transmits visible light but does not transmit light other than visible light is disposed between the prism and the second image sensor.

According to the first aspect of the present invention, it is possible to simultaneously visualize a vein to be anastomosed and a lymphatic vessel within the same visual field.

According to the second aspect of the present invention, the vein and lymphatic vessel to be anastomosed can be colored in different colors and simultaneously visualized in the same visual field, thereby improving the visibility of the vein and lymphatic vessel. It becomes possible to make it.

According to the third aspect of the present invention, it is possible to visually recognize veins and lymph vessels and the image of the subject at the same time in a synthesized state, thereby improving the efficiency of the operation.

According to the fourth aspect of the present invention, the dichroic prism allows the fluorescence from the fluorescent dye and the light of the second wavelength to be efficiently incident on the first image sensor, and the second image is efficiently captured with the visible light. It becomes possible to make it enter into an element.

According to the fifth aspect of the present invention, the fluorescence from the fluorescent dye excited by the first wavelength light and the second wavelength light by the first band pass filter is prevented from entering the first image sensor. At the same time, the second band pass filter can prevent light other than visible light from entering the second image sensor.

1 is a perspective view showing an imaging apparatus 1 according to the present invention together with a display apparatus 2. FIG. 2 is a schematic diagram of an illumination / photographing unit 12. FIG. 2 is a schematic diagram showing an internal structure of an illumination / photographing unit 12. FIG. It is a block diagram which shows the main control systems of the imaging device 1 which concerns on this invention. It is explanatory drawing which shows each process of the imaging operation in the camera 21 and the image process part 73 with a block. 6 is a graph showing a synchronous clock applied to the first and second light sources 22, 23 and the first image sensor 61.

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a perspective view showing an imaging apparatus 1 according to the present invention together with a display apparatus 2.

The display device 2 has a configuration in which a display unit 52 including a large liquid crystal display device is supported by a support mechanism 51.

The imaging apparatus 1 according to the present invention is for simultaneously displaying an image of a lymph vessel and a vein image of a subject on the display unit 52 together with a visible image of the subject. That is, the imaging apparatus 1 according to the present invention visualizes a lymph vessel image as a near-infrared image by indocyanine green fluorescence, and visualizes a vein image as an inverted image of the brightness of the near-infrared image absorbed by blood. These images are combined with the visible image of the subject and displayed.

The imaging apparatus 1 according to the present invention includes a carriage 11 having four wheels 13, an arm mechanism 30 disposed near the front of the carriage 11 in the traveling direction on the upper surface of the carriage 11, and the arm mechanism 30. An illumination / photographing unit 12 disposed via a sub arm 41 and a monitor 15 are provided. A handle 14 used when moving the carriage 11 is attached to the rear of the carriage 11 in the traveling direction. Further, on the upper surface of the carriage 11, a recess 16 for mounting an operation unit 74 (described later) used for remote operation of the imaging apparatus 1 is formed.

The arm mechanism 30 described above is disposed on the front side in the traveling direction of the carriage 11. The arm mechanism 30 includes a first arm member 31 connected to a support portion 37 disposed on a support column 36 erected on the front side in the traveling direction of the carriage 11 by a hinge portion 33. The first arm member 31 can swing with respect to the carriage 11 through the support column 36 and the support portion 37 by the action of the hinge portion 33. The monitor 15 described above is attached to the support column 36.

The second arm member 32 is connected to the upper end of the first arm member 31 by a hinge portion 34. The second arm member 32 can swing with respect to the first arm member 31 by the action of the hinge portion 34. For this reason, the first arm member 31 and the second arm member 32 are a connecting portion between the first arm member 31 and the second arm member 32 as shown in FIG. It is possible to take a photographing posture opened by a predetermined angle around a certain hinge portion 34 and a standby posture in which the first arm member 31 and the second arm member 32 are close to each other.

A support portion 43 is connected to the lower end of the second arm member 32 by a hinge portion 35. The support portion 43 can swing with respect to the second arm member 32 by the action of the hinge portion 35. A rotating shaft 42 is supported on the support portion 43. Then, the sub arm 41 that supports the illumination / photographing unit 12 rotates around the rotation shaft 42 disposed at the tip of the second arm member 32. For this reason, the illumination / photographing unit 12 rotates the sub arm 41 so that the position of the carriage 11 in the traveling direction relative to the arm mechanism 30 for taking the photographing posture or the standby posture shown in FIG. It moves between the position of the rear side in the traveling direction of the carriage 11 with respect to the arm mechanism 30 which is the posture when moving the carriage 11.

FIG. 2 is a schematic diagram of the illumination / photographing unit 12.

The illumination / imaging unit 12 is provided on a camera 21 including a first image sensor 61 capable of detecting near infrared rays and a second image sensor 62 capable of detecting visible light, which will be described later, and an outer peripheral part of the camera 21. The four first light sources 22, the four second light sources 23, and the four third light sources 24 are provided.

The first light source 22 irradiates the subject with light having a first wavelength that excites indocyanine green as a fluorescent dye administered to the subject's lymphatic vessel. This first wavelength is, for example, 780 nm. Indocyanine green administered to a subject's lymphatic vessel emits near-infrared light having a wavelength peak of about 820 nm when irradiated with near-infrared light of 780 nm. Note that the wavelength of the first light source 22 is not limited to 780 nm, and may be any wavelength of about 650 nm to 850 nm that can excite indocyanine green.

The second light source 23 irradiates the subject with light of the second wavelength absorbed in the blood in the subject's veins. This second wavelength is, for example, 850 nm. This near infrared light of 850 nm is absorbed by hemoglobin in the blood. The wavelength of the second light source 23 may be about 700 nm to 900 nm, for example.

The third light source 24 irradiates the subject with white light that is visible light.

In this embodiment, the illumination / photographing unit 12 in which the first light source 22, the second light source 23, the third light source 24, and the camera 21 are integrated is used. The light source 23, the third light source 24, and the camera 21 may be provided individually.

FIG. 3 is a schematic diagram showing the internal structure of the illumination / photographing unit 12.

As described above, the illumination / imaging unit 12 includes the camera 21, the first light source 22, the second light source 23, and the third light source 24. The camera 21 includes a lens system 66, a first image sensor 61 that can detect near infrared rays, a second image sensor 62 that can detect visible light, and a first image sensor 61 that transmits near infrared light. And a dichroic prism 65 that reflects visible light and enters the second image sensor 62. The first image sensor 61 and the second image sensor 62 are composed of CMOS or CCD.

Between the dichroic prism 65 and the first image sensor 61, the fluorescence from indocyanine green excited by the near-infrared light having the first wavelength and the near-infrared light having the second wavelength are transmitted. A first band-pass filter 63 that does not transmit fluorescence from near-infrared light having a first wavelength and indocyanine green excited by light having a second wavelength is disposed. A second band pass filter 64 that transmits visible light but does not transmit light other than visible light is disposed between the dichroic prism 65 and the second imaging element 62.

FIG. 4 is a block diagram showing a main control system of the imaging apparatus 1 according to the present invention.

The imaging apparatus 1 includes a CPU that executes logical operations, a ROM that stores an operation program necessary for controlling the apparatus, a RAM that temporarily stores data during control, and the like, and controls the entire apparatus. The unit 70 is provided. The control unit 70 is connected to the illumination / photographing unit 12 including the camera 21, the first light source 22, the second light source 23, and the third light source 24. The control unit 70 is connected to the monitor 15. The control unit 70 is connected to an operation unit 74 through which various types of information are input by an operator (surgeon). Further, the control unit 70 is also connected to the display device 2 including the display unit 52 described above.

The control unit 70 turns on and off the camera control unit 71 for controlling the camera 21 including the first image sensor 61 and the second image sensor 62, and the first light source 22, the second light source 23, and the third light source 24. And a light source control unit 72 for controlling the image processing unit 73 and an image processing unit 73 for executing various types of image processing including luminance inversion processing, gamma correction processing, coloring processing, and composition processing, which will be described later.

Next, an imaging operation in the above-described imaging apparatus 1 when performing surgery or the like on the subject will be described. FIG. 5 is an explanatory diagram showing the steps of the imaging operation in the camera 21 and the image processing unit 73 in blocks.

In the case of performing surgery or the like, in order to simultaneously display the image of the lymph vessel and the image of the vein in the subject together with the visible image of the subject on the display unit 52 by the imaging apparatus 1 according to the present invention, First, the operator operates the handle 14 to move the carriage 11 and moves the imaging apparatus 1 to a place where surgery or the like is performed. Then, indocyanine green is injected by injection near the suture vein in the subject, and indocyanine green is allowed to flow into the lymphatic vessel.

In this state, the first imaging element 61 captures an image of a lymph vessel and a vein of the subject, and the second imaging element 62 captures a visible image of the subject, and displays these images on the display device 2. Is displayed on the display unit 52 and the monitor 15.

At this time, the third light source 24 is always turned on, and visible light is irradiated toward the subject. Then, the visible light reflected by the subject passes through the lens system 66 in the camera 21 and is then reflected by the dichroic prism 65 to transmit the visible light but not the light other than the visible light. Then, the light enters the second image sensor 62. The second image sensor 62 acquires the visible image at a frame rate of 60 FPS (Frame Per Second) under the control of the camera control unit 71. As the second image sensor 62, a color image sensor capable of acquiring a visible image as an RGB signal is used. The image signal acquired by the second image sensor 62 is subjected to gamma correction in the image processing unit 73 to create a visible image.

On the other hand, the lymphatic vessel image and the vein image of the subject are taken by the first image sensor 61. The first image sensor 61 also acquires an image at a frame rate of 60 FPS. At this time, the first image sensor 61 is controlled by the camera control unit 71 according to the timing of the synchronous clock A and the synchronous clock B that synchronize the image of the lymphatic vessel and the image of the vein with the frame rate at the time of imaging. It is obtained alternately every frame, that is, at a frame rate of 30 FPS. Correspondingly, the first light source 22 and the second light source 23 are alternately turned on in synchronization with the frame rate of shooting by the first image sensor 61 under the control of the light source controller 72.

FIG. 6 is a graph showing a synchronous clock applied to the first and second light sources 22 and 23 and the first image sensor 61. In this graph, symbol A indicates a synchronous clock for turning on the first light source 22, and symbol B indicates a synchronous clock for lighting the second light source 23.

As described above, the first image sensor 61 alternately acquires images of lymphatic vessels and veins at a frame rate of 60 FPS. For this reason, the 1st light source 22 and the 2nd light source 23 light in the state corresponding to the frame rate of 30 FPS by turns. The turn-on and turn-off times of the first light source 22 and the second light source 23 at this time are both 16.6 msec as shown in FIG.

Referring to FIG. 5 again, in the state where the first light source 22 is turned on in response to the synchronous clock A shown in FIG. 6, indocyanine green in the lymph vessel of the subject is excited and generates fluorescence. This fluorescence is, for example, near infrared light having a wavelength of 820 nm. The fluorescence passes through the lens system 66 in the camera 21, then passes through the dichroic prism 65, and further passes through the first band pass filter 63 and then enters the first image sensor 61. The first image sensor 61 acquires a fluorescent image from indocyanine green corresponding to the synchronous clock A shown in FIG. 6 under the control of the camera control unit 71. The first imaging element 61 is a monochrome imaging element that acquires a fluorescence image and a near-infrared image with a second wavelength described later as a monochrome image.

The image processing unit 73 performs gamma correction on the fluorescence image acquired by the first image sensor 61 at a frame rate of 30 FPS. Then, the image processing unit 73 performs a coloring process for coloring the lymphatic region in the fluorescent image to the first color. This first color is, for example, green.

On the other hand, in a state where the second light source 23 is turned on in response to the synchronous clock B shown in FIG. 6, near-infrared light having a wavelength of 850 nm emitted from the second light source 23 is hemoglobin in a blood vessel in the vein. Absorbed and reflected in areas other than veins. The near-infrared light passes through the lens system 66 in the camera 21, then passes through the dichroic prism 65, and further passes through the first band pass filter 63 and then enters the first image sensor 61. The first image sensor 61 acquires a near-infrared image corresponding to the synchronous clock B shown in FIG. 6 under the control of the camera control unit 71.

The near-infrared image acquired by the first image sensor 61 at a frame rate of 30 FPS is an image in which near-infrared light is absorbed in the vein region and the vein region is darkened. For the near-infrared image, the image processing unit 73 executes luminance inversion processing for inverting the luminance of the pixel value. At this time, if each pixel of the first image sensor 61 has a 1-Kbyte pixel value from 0 to 1023, for example, the output pixel value after the inversion processing is OUT (x, y). When the input pixel value before the inversion process is IN (x, y), the image processing unit 73 performs an operation represented by the following expression.
OUT (x, y) = 1023-IN (x, y)

This pixel value inversion process makes it possible to visualize the vein area of the subject as a high-luminance image. Then, the image processing unit 73 performs gamma correction on the image after the inversion process. Thereafter, the image processing unit 73 performs a coloring process for coloring the vein region in the image after the reversal process into a second color different from the first color. This second color is a color other than green and easily visible.

The image processing unit 73 combines an image in which the lymph vessel region is colored in the first color and an image in which the vein region is colored in the second color, so that the vein and lymph vessel to be anastomosed can be seen in the same field of view. A colored near-infrared image that can be visualized simultaneously can be obtained. Then, the colored near-infrared image and the visible image acquired by the second image sensor 62 are synthesized by the image processing unit 73, thereby synthesizing the vein and lymph vessel and the subject image. In this state, it can be displayed on the display unit 52 and the monitor 15 in the display device 2 at the same time. Thereby, the operator can easily visually recognize the veins and lymph vessels and the image of the subject.

Note that between the dichroic prism 65 and the first image sensor 61, fluorescence from indocyanine green excited by near-infrared light having the first wavelength and near-infrared light having the second wavelength are transmitted. However, a first band-pass filter 63 that does not transmit near-infrared light having the first wavelength and indocyanine green excited by light having the second wavelength is disposed.

As described above, the first wavelength is 780 nm and the second wavelength is 850 nm. The peak of the wavelength of fluorescence from indocyanine green irradiated with excitation light of 780 nm is 820 nm. When indocyanine green is irradiated with near-infrared light having a second wavelength of 850 nm, fluorescence having a peak at 870 nm is generated from indocyanine green. Therefore, near-infrared light having a first wavelength that is not used for visualization of lymphatic vessels and blood vessels and fluorescence from indocyanine green excited by light having the second wavelength are prevented from entering the first image sensor 61. Therefore, the first band pass filter 63 is used that allows only near-infrared light of 800 nm to 860 nm to pass therethrough.

As described above, according to the imaging apparatus 1 described above, by using the synchronous clock, the lighting of the first and second light sources 22 and 23 and the acquisition of the image by the first image sensor 61 are executed in synchronization. The lymphatic vessel and vein can be visualized simultaneously in the same visual field. For this reason, an operator (operator) can visually observe lymphatic vessels and blood vessels within the same visual field, and can perform surgery in a concentrated manner. At this time, it is not necessary to prepare dedicated devices for lymphatic vessels and veins.

Also, according to the imaging apparatus 1 described above, the lymphatic vessels and veins can be displayed in different colors, so that both can be easily identified. Further, vein marking using a vascular tape or the like is not required.

In the above-described embodiment, the lymph vessel region and the vein region are colored with different colors, but this coloring may be performed only on one of the lymph vessel and the vein. In this case, the non-colored region of the lymph vessel region and the vein region is displayed in white.

In the above-described embodiment, the visible image is displayed together with the lymphatic vessels and veins by irradiating the subject with visible light. However, the visible image display is omitted and the visible image is visually observed. You may make it observe directly.

Furthermore, in the above-described embodiment, indocyanine green is used as the fluorescent dye, but other fluorescent dyes such as 5-aminolevulinic acid (5-ALA / 5-Aminolevulinic Acid) may be used.

DESCRIPTION OF SYMBOLS 1 Imaging apparatus 2 Display apparatus 11 Car 12 Illumination / imaging | photography part 15 Monitor 21 Camera 22 1st light source 23 2nd light source 24 3rd light source 30 Arm mechanism 52 Display part 61 1st image pick-up element 62 2nd image pick-up element 63 1st band pass Filter 64 Second band pass filter 65 Dichroic prism 66 Lens system 70 Control unit 71 Camera control unit 72 Light source control unit 73 Image processing unit 74 Operation unit

Claims (5)

An imaging device for simultaneously displaying a lymphatic vessel image and a vein image in a subject,
A first light source that irradiates the subject with light having a first wavelength that excites a fluorescent dye administered to the subject's lymphatic vessel;
A second light source that irradiates the subject with light of a second wavelength that is absorbed in blood in the vein of the subject;
A first image sensor that images the subject at a predetermined frame rate;
A light source controller that alternately turns on the first light source and the second light source in synchronization with a frame rate of photographing by the first image sensor;
When the second light source is turned on, the luminance of the image of the subject photographed by the first image sensor is reversed, and when the first light source is turned on, the first image sensor After coloring at least one of the lymphatic region in the photographed image and the vein region in the image that has been photographed by the first image sensor and inverted in luminance when the second light source is turned on, these An image processing unit for displaying the image on the display unit;
An imaging apparatus comprising: The imaging apparatus according to claim 1, wherein
The image processing unit
While displaying the region of the lymphatic vessel in the image of the subject imaged by the first image sensor when the first light source is turned on in the first color, and displaying on the display unit,
The luminance of the image of the subject imaged by the first image sensor when the second light source is turned on is inverted, and the vein region in the inverted image is the first color. An imaging device that displays a color on a second color different from that on the display unit. The imaging apparatus according to claim 2, wherein
A third light source that emits visible light toward the subject;
A second image sensor for imaging the subject;
Further comprising
The image processing unit synthesizes a visible image captured by the second image sensor together with the lymphatic region and the vein region in the subject image captured by the first image sensor on the display unit. And display the imaging device. The imaging device according to claim 3.
An imaging apparatus comprising: a dichroic prism that causes fluorescence from the fluorescent dye and light having the second wavelength to enter the first imaging element and causes visible light to enter the second imaging element. The imaging device according to claim 4.
Between the dichroic prism and the first imaging element, the fluorescence from the fluorescent dye excited by the light of the first wavelength and the light of the second wavelength are transmitted, and the light of the first wavelength and A first bandpass filter that does not transmit fluorescence from the fluorescent dye excited by the light of the second wavelength;
An imaging apparatus in which a second band-pass filter that transmits visible light but does not transmit light other than visible light is disposed between the dichroic prism and the second imaging element.

Images