JP5349899B2 - Imaging system and program - Google Patents

Abstract

<P>PROBLEM TO BE SOLVED: To provide an image in a desired band and an image for observation by a simple structure. <P>SOLUTION: The image capturing system includes an image capturing part having a plurality of first light receiving elements for receiving the light in a first wavelength range and a plurality of second light receiving elements for receiving the light in a second wavelength range; a control part for generating light of different spectra at different times from a subject in the first wavelength range and in the second wavelength range; and an image generating part for generating images from the light of a first spectrum from the subject received by the first light receiving elements at prescribed timing and from the light of a second spectrum from the subject received by the second light receiving elements at prescribed timing. <P>COPYRIGHT: (C)2009,JPO&amp;INPIT

Description

  The present invention relates to an imaging system, an imaging method, and a program. The present invention particularly relates to an imaging system and an imaging method for capturing an image, and a program for the imaging system.

There is known an electronic endoscope system capable of imaging a tissue to be observed below the surface of a living tissue without using a rotating filter (see, for example, Patent Document 1). Also, an imaging device is known that irradiates an observation part with illumination light and picks up an optical image based on re-radiant light emitted from the observation part by irradiation of the illumination light (see, for example, Patent Document 2). .
JP 2007-54113 A JP 2002-345733 A

  Since the technique of Patent Document 1 has a plurality of B pixels that receive light in different wavelength regions, a visible light image cannot be captured with high sensitivity. Further, in the invention of Patent Document 2, a narrow band image cannot be obtained. As described above, the technique described in the above-mentioned patent document has a problem that it is not possible to provide an image of a desired band and an observation image with a simple configuration.

  In order to solve the above problems, according to a first aspect of the present invention, in an imaging system, a plurality of first light receiving elements that receive light in a first wavelength region and a plurality that receive light in a second wavelength region. An imaging unit having the second light receiving element, a control unit for generating light of different spectrum from the subject at different timings in each of the first wavelength region and the second wavelength region, and a plurality of first light receiving elements at predetermined timings The first spectrum light from the subject received by the light, the second spectrum light from the subject received by the plurality of second light receiving elements at a predetermined timing, and the plurality of first light receiving elements received at a timing other than the predetermined timing Third spectrum light from the subject and fourth from the subject received by the plurality of second light receiving elements at timings other than the predetermined timing. It is generating a first image from a combination comprising at least one of the light spectrum, and an image generating unit that generates a second image from a combination different from the combination.

  The control unit generates light of the first partial wavelength region included in the first wavelength region and light of the second partial wavelength region included in the second wavelength region from the subject at a predetermined timing, and timings other than the predetermined timing The light of the third partial wavelength region included in the first wavelength region and the light of the fourth partial wavelength region included in the second wavelength region are generated from the subject. The light of the first partial wavelength region from the subject received by one light receiving element, the light of the third partial wavelength region from the subject received by a plurality of first light receiving elements at a timing other than a predetermined timing, and a plurality of lights at a predetermined timing Light in the second partial wavelength region from the subject received by the second light receiving element and a plurality of second light receiving elements received at a timing other than a predetermined timing Generating a first image from a combination comprising at least one of the light of the fourth partial wavelength region from Utsushitai may generate the second image from a combination different from the combination.

  A light emitting unit that emits light that causes the light of the first partial wavelength region, the second partial wavelength region, the third partial wavelength region, and the fourth partial wavelength region to be generated from the subject, and the control unit, at a predetermined timing, Light that causes light from the subject to generate light in the first partial wavelength region and light in the second partial wavelength region is emitted from the light emitting unit, and at a timing other than a predetermined timing, the light in the third partial wavelength region and the light in the fourth partial wavelength region Light that generates light from the subject may be emitted from the light emitting unit.

  The light emitting unit emits light in the first partial wavelength region, the second partial wavelength region, the third partial wavelength region, and the fourth partial wavelength region, and the control unit emits light in the first partial wavelength region at a predetermined timing. The light of the second partial wavelength region is emitted from the light emitting unit toward the subject, and the light of the third partial wavelength region and the light of the fourth partial wavelength region are directed to the subject from the light emitting unit at a timing other than a predetermined timing. At a predetermined timing, the plurality of first light receiving elements receive light in the first partial wavelength region reflected from the subject, and the plurality of second light receiving elements receive light in the second partial wavelength region reflected from the subject. At a timing other than the predetermined timing, the plurality of first light receiving elements receive light in the third partial wavelength region reflected from the subject, and the plurality of second light receiving elements reflect the fourth partial wavelength region reflected from the subject. The light may be received.

  The image generation unit may generate a composite image obtained by combining the first image and the second image. In addition, the image generation unit is configured to receive light in the first partial wavelength region received by the plurality of first light receiving elements at a predetermined timing, and in the third partial wavelength region received by the plurality of first light receiving elements at a timing other than the predetermined timing. From the light, the light in the second partial wavelength region received by the plurality of second light receiving elements at a predetermined timing, and the light in the fourth partial wavelength region received by the plurality of second light receiving elements at a timing other than the predetermined timing A second image indicating the image may be generated.

  The image generation unit may output a composite image in which the first image is emphasized and superimposed on the second image. You may further provide the output part which matches and outputs the 1st image and 2nd image which the image generation part produced | generated.

  The image generator receives the amount of light in the first partial wavelength region received by each of the plurality of first light receiving elements at a predetermined timing and the first light receiving element received by each of the plurality of first light receiving elements at a timing other than the predetermined timing. The total value of the light reception amounts of the light in the three partial wavelength regions and the light reception amounts of the light in the second partial wavelength region received by each of the plurality of second light receiving elements at a predetermined timing The second image may be generated based on the total amount of light received in the fourth partial wavelength region received by each of the second light receiving elements.

  The plurality of first light receiving elements and the plurality of second light receiving elements receive the light reflected by the object existing inside the substance, and the plurality of first light receiving elements are in the second wavelength region. The controller receives light in the first wavelength region, which is a shorter wavelength region, and the control unit transmits light in the first partial wavelength region and the second partial wavelength region, which are shorter than the third partial wavelength region, at a predetermined timing. The light is emitted from the light emitting unit, and the image generation unit is configured to detect an object existing at a shallower position from the surface of the substance based on at least light in the first partial wavelength region received by the plurality of first light receiving elements at a predetermined timing. A first image showing an image may be generated.

  The control unit emits light of the first partial wavelength region that is shorter than the third partial wavelength region and light of the second partial wavelength region that is longer than the fourth partial wavelength region at a predetermined timing. The image generation unit emits light from the first partial wavelength region received by the plurality of first light receiving elements at a predetermined timing and light of the second partial wavelength region received by the plurality of second light receiving elements at the predetermined timing. Based on at least, a first image may be generated that shows an image of an object that is at a shallower position from the surface of the substance and a deeper position from the surface of the substance.

  The plurality of first light receiving elements and the plurality of second light receiving elements receive the light reflected by the object existing inside the substance, and the plurality of first light receiving elements are in the second wavelength region. The control unit receives light of the first wavelength region that is a longer wavelength region, and the control unit receives light of the first partial wavelength region and the second partial wavelength region that are longer than the third partial wavelength region at a predetermined timing. The light is emitted from the light emitting unit, and the image generating unit is configured to detect an object existing at a deeper position from the surface of the substance based at least on light in the first partial wavelength region received by the plurality of first light receiving elements at a predetermined timing. A first image including the image may be generated.

  A first spectral filter unit that passes light in the first partial wavelength region and light in the third partial wavelength region; and a second spectral filter unit that passes light in the second partial wavelength region and light in the fourth partial wavelength region. The plurality of first light receiving elements receive light from the subject that has passed through the first spectral filter unit, and the plurality of second light receiving elements receive light from the subject that has passed through the second spectral filter unit. Good.

  The light-emitting unit has a plurality of light-emitting elements that respectively emit light of different spectra, and the control unit controls the light emission intensity of each of the plurality of light-emitting elements, thereby at a predetermined timing and timing other than the predetermined timing. Light emission may be controlled.

  The light emitting unit further includes an irradiation light filter that passes light in the first partial wavelength region and light in the second partial wavelength region, and the light emitting unit includes the first partial wavelength region, the second partial wavelength region, the third partial wavelength region, and the The controller may emit light in a wavelength region including at least one of the four partial wavelength regions, and the control unit may irradiate the subject with light from the light emitting unit through the irradiation light filter unit at a predetermined timing. .

  The image generation unit includes light in a first partial wavelength region received by a plurality of first light receiving elements at a plurality of timings including a predetermined timing, and a second partial wavelength region received by a plurality of second light receiving elements at a plurality of timings. Based on a plurality of images of light, a motion specifying unit that specifies the movement of an object on the image during a plurality of timings, and light in a first wavelength region received by the plurality of first light receiving elements at a predetermined timing, Based on the light in the second wavelength region received by the plurality of second light receiving elements at the predetermined timing and the movement, based on the light in the first partial wavelength region and the light in the second partial wavelength region at a timing other than the predetermined timing A corrected image generation unit that generates a corrected image that is an image of the subject.

  The image generation unit includes the light in the third partial wavelength region received by the plurality of first light receiving elements at a timing other than the predetermined timing, and the fourth partial wavelength received by the plurality of second light receiving elements at a timing other than the predetermined timing. A subject image generation unit that generates a second image based on the image of the light of the region and the corrected image may be further included.

  The imaging unit further includes a plurality of third light receiving elements that receive light in the third wavelength region, and the light emitting unit includes the first partial wavelength region, the second partial wavelength region, the third partial wavelength region, and the fourth partial wavelength. And the light of the fifth partial wavelength region and the sixth partial wavelength region included in the third wavelength region, and the control unit emits the light of the first partial wavelength region and the second partial wavelength region at a predetermined timing. The light and the light in the fifth partial wavelength region are emitted to the light emitting unit toward the subject, and the light in the third partial wavelength region, the light in the fourth partial wavelength region, and the sixth partial wavelength are at timings other than the predetermined timing. The light of the region is emitted to the light emitting portion toward the subject, and at a predetermined timing, the plurality of first light receiving elements receive the light of the first partial wavelength region reflected from the subject, and the plurality of second light receiving elements are received from the subject. Reflected second partial wavelength region The third light receiving element receives light in the fifth partial wavelength region reflected from the subject, and at a timing other than a predetermined timing, the plurality of first light receiving elements reflect the third partial wavelength region reflected from the subject. The plurality of second light receiving elements receive light in the fourth partial wavelength region reflected from the subject, the third light receiving element receives light in the sixth partial wavelength region reflected from the subject, and an image generating unit Are light in the first partial wavelength region received by the plurality of first light receiving elements at a predetermined timing, light in the third partial wavelength region received by the plurality of first light receiving elements at timings other than the predetermined timing, and predetermined timing Light in the second partial wavelength region received by the plurality of second light receiving elements in FIG. 4, light in the fourth partial wavelength region received by the plurality of second light receiving elements at a timing other than the predetermined timing, and A combination including at least one of the light in the fifth partial wavelength region received by the plurality of third light receiving elements at the timing of and the light in the sixth partial wavelength region received by the plurality of third light receiving elements at a timing other than the predetermined timing The first image may be generated from the combination, and the second image may be generated from a combination different from the combination.

  The first wavelength region may be a blue wavelength region, the second wavelength region may be a red wavelength region, and the third wavelength region may be a green wavelength region.

  According to a second aspect of the present invention, there is provided an imaging method comprising: a plurality of first light receiving elements that receive light in the first wavelength region; and a plurality of second light receiving elements that receive light in the second wavelength region. A control step for generating light of different spectrum from the subject at different timings in each of the first wavelength region and the second wavelength region, and a first from the subjects received by the plurality of first light receiving elements at a predetermined timing Spectrum light, second spectrum light from a subject received by a plurality of second light receiving elements at a predetermined timing, and third spectrum light from a subject received by a plurality of first light receiving elements at a timing other than the predetermined timing , And at least the fourth spectrum light from the subject received by the plurality of second light receiving elements at a timing other than the predetermined timing One generates a first image from a combination comprising, and an image generation step of generating a second image from a combination different from the combination.

  According to a third aspect of the present invention, there is provided a program for an imaging system, wherein the imaging system includes a plurality of first light receiving elements that receive light in the first wavelength region and a plurality of light that receives light in the second wavelength region. In each of the imaging unit having the second light receiving element, the first wavelength region, and the second wavelength region, a control unit that generates light of different spectra from the subject at different timings, and the plurality of first light receiving elements received light at a predetermined timing First spectrum light from a subject, second spectrum light from a subject received by a plurality of second light receiving elements at a predetermined timing, from a subject received by a plurality of first light receiving elements at a timing other than the predetermined timing Fourth light from the third spectrum light and the subject received by the plurality of second light receiving elements at timings other than the predetermined timing Generating a first image from a combination comprising at least one of the light spectrum, to function as an image generating unit that generates a second image from a combination different from the combination.

  It should be noted that the above summary of the invention does not enumerate all the necessary features of the present invention. In addition, a sub-combination of these feature groups can also be an invention.

  Hereinafter, the present invention will be described through embodiments of the invention, but the following embodiments do not limit the invention according to the claims. In addition, not all the combinations of features described in the embodiments are essential for the solving means of the invention.

  FIG. 1 shows an example of the configuration of an imaging system 10 according to this embodiment together with a specimen 20. The imaging system 10 includes an endoscope 100, an image generation unit 140, an output unit 180, a control unit 105, a light irradiation unit 150, and an ICG injection unit 190. In FIG. 1, the A part shows the distal end part 102 of the endoscope 100 in an enlarged manner.

  The ICG injection unit 190 injects indocyanine green (ICG), which is a luminescent substance, into the specimen 20 that is an example of a subject. In addition, although ICG is illustrated as a luminescent substance in this embodiment, you may use fluorescent substances other than ICG as a luminescent substance.

  ICG is excited by infrared rays having a wavelength of 750 nm, for example, and emits a broad spectrum of fluorescence centering on 810 nm. When the specimen 20 is a living body, the ICG injection unit 190 injects ICG into the blood vessel of the living body by intravenous injection. The imaging system 10 images a blood vessel in a living body with luminescence light from the ICG. Note that the luminescence light is an example of light in a specific wavelength region, and includes fluorescence and phosphorescence. Note that luminescence light, which is an example of light from a subject, includes luminescence light by chemiluminescence, triboluminescence, and thermoluminescence in addition to photoluminescence by light such as excitation light.

  The ICG injection unit 190 injects ICG into the specimen 20 so that the ICG concentration in the living body is maintained substantially constant under the control of the control unit 105, for example. The specimen 20 may be a living body such as a human body, and is an imaging target of an image processed by the imaging system 10. An object such as a blood vessel exists inside the specimen 20.

  The endoscope 100 includes an imaging unit 110, a light guide 120, and a forceps port 130. The distal end portion 102 of the endoscope 100 has an objective lens 112 as a part of the imaging unit 110. Further, the distal end portion 102 has an emission port 124 as a part of the light guide 120. The distal end portion 102 of the endoscope 100 has a nozzle 138.

  A forceps 135 is inserted into the forceps port 130, and the forceps port 130 guides the forceps 135 to the distal end portion 102. Note that the forceps 135 may have various tip shapes. In addition to the forceps, various treatment tools for treating a living body may be inserted into the forceps port 130. The nozzle 138 delivers water or air.

  The light irradiation unit 150 generates light irradiated from the distal end portion 102 of the endoscope 100. The light generated by the light irradiation unit 150 irradiates the sample 20 with infrared rays as an example of excitation light that is light in a wavelength region that excites a luminescent substance contained in the sample 20 to emit light in a specific wavelength region. Includes irradiation light. The irradiation light includes, for example, component light of R component, G component, and B component.

  The light guide 120 is formed of an optical fiber, for example. The light guide 120 guides the light generated by the light irradiation unit 150 to the distal end portion 102 of the endoscope 100. The light guide 120 can include an emission port 124 provided in the distal end portion 102. The light generated by the light irradiation unit 150 is irradiated to the specimen 20 from the emission port 124.

  The imaging unit 110 receives at least one of the light emitted from the luminescent material and the reflected light reflected by the object. The image generation unit 140 generates an image by processing the light reception data acquired from the imaging unit 110. The output unit 180 outputs the image generated by the image generation unit 140.

  The control unit 105 includes an imaging control unit 160 and a light emission control unit 170. The imaging control unit 160 controls imaging by the imaging unit 110. In addition, the light emission control unit 170 controls the light irradiation unit 150 under the control of the imaging control unit 160. For example, when the imaging unit 110 captures each component light of an infrared ray, an R component, a G component, and a B component in a time division manner, the light emission control unit 170 synchronizes the irradiation timing of each component light with the imaging timing. The light irradiated on the specimen 20 from the light irradiation unit 150 is controlled so as to cause the light irradiation to occur.

  FIG. 2 shows an example of the configuration of the imaging unit 110. The imaging unit 110 includes an objective lens 112, an imaging device 210, a spectral filter unit 220, and a light receiving side excitation light cut filter unit 230. The imaging device 210 includes a plurality of first light receiving elements 251 including a first light receiving element 251a, a plurality of second light receiving elements 252 including a second light receiving element 252a and a second light receiving element 252b, and a plurality of elements including a third light receiving element 253a. A third light receiving element 253 is included.

  Hereinafter, functions and operations of components included in the imaging unit 110 will be described. In the following description, in order to prevent the description from becoming complicated, the plurality of first light receiving elements 251 are collectively referred to as first light receiving elements 251 and the plurality of second light receiving elements 252 are collectively referred to as second light receiving elements. The plurality of third light receiving elements 253 may be collectively referred to as a third light receiving element 253. In addition, the plurality of first light receiving elements 251, the plurality of second light receiving elements 252, and the plurality of third light receiving elements 253 may be collectively referred to simply as light receiving elements.

  The first light receiving element 251, the second light receiving element 252, and the third light receiving element 253 receive light from the subject supplied through the objective lens 112. Specifically, the first light receiving element 251 receives light in the first wavelength region. The second light receiving element 252 receives light in the second wavelength region. The third light receiving element 253 receives light in a third wavelength region different from the second wavelength region.

  The first wavelength region, the second wavelength region, and the third wavelength region are different wavelength regions, and include a wavelength region that does not include other wavelength regions. The first light receiving element 251, the second light receiving element 252, and the third light receiving element 253 are two-dimensionally arranged in a predetermined pattern.

  The spectral filter unit 220 includes a plurality of filter elements that pass one of the light in the first wavelength region, the light in the second wavelength region, and the light in the third wavelength region. Each filter element is two-dimensionally arranged according to the respective light receiving elements of the first light receiving element 251, the second light receiving element 252, and the third light receiving element 253. The individual light receiving elements receive the light that has passed through the individual filter elements. As described above, the first light receiving element 251, the second light receiving element 252, and the third light receiving element 253 receive light in different wavelength regions.

  The light receiving side excitation light cut filter unit 230 is provided at least between the subject and the second light receiving element 252 and the third light receiving element 253, and cuts light in the wavelength region of the excitation light. The second light receiving element 252 and the third light receiving element 253 receive the reflected light from the subject through the excitation light cut filter. For this reason, the second light receiving element 252 and the third light receiving element 253 do not receive the reflected light that the excitation light reflects from the subject.

  The light-receiving-side excitation light cut filter unit 230 may cut light in the wavelength region of excitation light and light in a specific wavelength region. In this case, the second light receiving element 252 and the third light receiving element 253 do not receive the luminescence light from the subject.

  The light receiving side excitation light cut filter unit 230 may be provided between the subject and the second light receiving element 252. In this case, the light receiving side excitation light cut filter unit 230 transmits the luminescence light.

  The light receiving side excitation light cut filter unit 230 is two-dimensionally according to the respective light receiving elements of the first light receiving element 251, the second light receiving element 252, and the third light receiving element 253, similarly to the spectral filter unit 220. An array of filter elements may be included. The filter element that supplies light to the first light receiving element 251 cuts the light in the wavelength region of the excitation light and allows the light in the first wavelength region and the specific wavelength region to pass. On the other hand, the filter element that supplies light to the second light receiving element 252 cuts the light in the wavelength region of the excitation light and the light in the specific wavelength region, and passes at least the light in the second wavelength region. The filter element that supplies light to the third light receiving element 253 cuts light in the wavelength region of the excitation light and light in the specific wavelength region, and passes at least light in the third wavelength region.

  The image generation unit 140 determines a pixel value of one pixel based at least on the amount of light received by the first light receiving element 251a, the second light receiving element 252a, the second light receiving element 252b, and the third light receiving element 253a. That is, one pixel element is formed by a two-dimensional array structure of the first light receiving element 251a, the second light receiving element 252a, the second light receiving element 252b, and the third light receiving element 253a, and the pixel element array is two-dimensionally arranged. As a result, a plurality of pixel elements are formed. The light receiving elements are not limited to the arrangement structure shown in the figure, and may be arranged in various arrangement structures.

  FIG. 3 shows an example of the spectral sensitivity characteristic of the light receiving element and the spectral reflectance of the biological surface layer, which is an example of a subject to be imaged. Lines 330, 310, and 320 indicate spectral sensitivity distributions of the first light receiving element 251, the second light receiving element 252, and the third light receiving element 253, respectively. A line 340 shows an example of the spectral reflectance of the stomach mucosa to be imaged.

  As an example, the first light receiving element 251 is sensitive to light having a wavelength in the vicinity of 650 nm where other light receiving elements are not substantially sensitive. The second light receiving element 252 is sensitive to light having a wavelength in the vicinity of 450 nm where other light receiving elements are not substantially sensitive. The third light receiving element 253 is sensitive to light having a wavelength in the vicinity of 550 nm where other light receiving elements are not substantially sensitive.

  In addition, the 1st light receiving element 251 can receive the light of the infrared region (for example, 810 nm) which is an example of a specific wavelength area | region with the characteristic of the light reception side excitation light cut filter part 230 and the spectral filter part 220. FIG. However, here, an example of the operation of the imaging system 10 when imaging using light in the visible light region will be described. An operation example in the case of imaging using light in the infrared region (for example, 810 nm) will be described with reference to FIGS.

  As described above, the first light receiving element 251, the second light receiving element 252, and the third light receiving element 253 receive R component light, B component light, and G component light, respectively. In addition, the 1st light receiving element 251, the 2nd light receiving element 252, and the 3rd light receiving element 253 may be image pick-up elements, such as CCD and CMOS, as an example. The first light receiving element 251 and the second light receiving element 252 are combined according to the combination of the spectral transmittance of the light receiving side excitation light cut filter unit 230, the spectral transmittance of the filter element included in the spectral filter unit 220, and the spectral sensitivity of the imaging device itself. , And the third light receiving element 253 have spectral sensitivity characteristics indicated by a line 330, a line 310, and a line 320, respectively.

  The filter elements that supply the light received by the first light receiving element 251 (including the filter elements included in the spectral filter unit 220 and the light receiving side excitation light cut filter unit 230) are the light in the first partial wavelength region (R1) and It passes light in the third partial wavelength region (R2). In addition, the said filter element functions as a 1st spectral filter part in this invention. Further, the filter elements that supply the light received by the second light receiving element 252 (including the filter elements included in the spectral filter unit 220 and the light receiving side excitation light cut filter unit 230) include the light in the second partial wavelength region (B1) and It passes light in the fourth partial wavelength region (B2). In addition, the said filter element functions as a 2nd spectral filter part in this invention. Further, the filter elements that supply the light received by the third light receiving element 253 (including the filter elements included in the spectral filter unit 220 and the light receiving side excitation light cut filter unit 230) are the light in the fifth partial wavelength region (G1) and It passes light in the sixth partial wavelength region (G2).

  As an example, the first partial wavelength region (R1) may be 600 to 630 nm, and the third partial wavelength region (R2) may be 570 to 600 nm. The second partial wavelength region (B1) may be 440 to 470 nm, and the fourth partial wavelength region (B2) may be 470 to 500 nm. The fifth partial wavelength region (G1) may be 500 to 530 nm, and the sixth partial wavelength region (R2) may be 530 to 570 nm.

  FIG. 4 shows an example of the configuration of the light irradiation unit 150. The light irradiation unit 150 includes a light emitting unit 410 and a light source side filter unit 420. The light emitting unit 410 emits light in a wavelength region including the first wavelength region, the second wavelength region, the third wavelength region, the fifth partial wavelength region, and the sixth partial wavelength region. In the present embodiment, the light emitting unit 410 may be a xenon lamp as an example.

  FIG. 5 shows an example of the configuration of the light source side filter unit 420. FIG. 5 shows a structure when viewed in a direction in which light is guided from the light emitting unit 410 to the light source side filter unit 420. The light source side filter unit 420 includes an irradiation light filter unit 520 and an irradiation light filter unit 510. Note that the light emission control unit 170 rotates the light source side filter unit 420 around a central axis of the light source side filter unit 420 in a plane substantially perpendicular to the direction in which the light emitted from the light emitting unit 410 travels.

  The irradiation light filter unit 510 cuts the light of the third partial wavelength region, the light of the fourth partial wavelength region, and the light of the sixth partial wavelength region, the light of the first partial wavelength region, the light of the second partial wavelength region, And passes light in the fifth partial wavelength region. In addition, the irradiation light filter unit 520 cuts the light in the first partial wavelength region, the light in the second partial wavelength region, and the light in the fifth partial wavelength region, and the light in the third partial wavelength region and the fourth partial wavelength. The light in the region and the light in the sixth partial wavelength region are passed. The light from the light emitting unit 410 is guided to a position shifted from the central axis of the light source side filter unit 420.

  Therefore, at the timing when the light from the light emitting unit 410 is guided to the irradiation light filter unit 510, among the light from the light emitting unit 410, the light in the third partial wavelength region, the light in the fourth partial wavelength region, and the sixth The light in the partial wavelength region is cut by the irradiation light filter unit 510, and the light in the first partial wavelength region, the light in the second partial wavelength region, and the light in the fifth partial wavelength region pass through the irradiation light filter unit 510. Therefore, at this timing, the subject is irradiated with light in the first partial wavelength region, light in the second partial wavelength region, and light in the fifth partial wavelength region.

  On the other hand, at the timing when the light from the light emitting unit 410 is guided to the irradiation light filter unit 520, among the light from the light emitting unit 410, the light in the first partial wavelength region, the light in the second partial wavelength region, and the fifth The light in the partial wavelength region is cut by the irradiation light filter unit 510, and the light in the third partial wavelength region, the light in the fourth partial wavelength region, and the light in the sixth partial wavelength region pass through the irradiation light filter unit 520. Therefore, at this timing, the subject is irradiated with light in the third partial wavelength region, light in the fourth partial wavelength region, and light in the sixth partial wavelength region.

  The imaging unit 110 is controlled by the imaging control unit 160 at a timing when the first partial wavelength region light, the second partial wavelength region light, and the fifth partial wavelength region light, which are visible light, are irradiated. Then, the reflected light reflected by the specimen 20 is received. Then, the image generation unit 140 generates a first visible light image based on the amount of received light received by the imaging unit 110. The first visible light image may be an example of the first image in the present invention.

  In addition, the imaging unit 110 is controlled by the imaging control unit 160 at the timing when the light of the third partial wavelength region, the light of the fourth partial wavelength region, and the light of the sixth partial wavelength region, which are visible light, are irradiated. Then, the reflected light reflected by the specimen 20 is received. Then, the image generation unit 140 receives the first partial wavelength in addition to the amount of light received by the imaging unit 110 in the third partial wavelength region, the fourth partial wavelength region, and the sixth partial wavelength region. A second visible light image is generated based on the received light amounts of the region light, the second partial wavelength region light, and the reflected light in the fifth partial wavelength region. The second visible light image may be an example of the second image in the present invention.

  FIG. 6 shows an example of the imaging timing by the imaging unit 110 and an image generated by the image generation unit 140. The imaging control unit 160 causes the imaging unit 110 to capture images at times t600, t601, t602, t603,. In addition, the light emission control unit 170 causes the light emitted from the light emitting unit 410 to irradiate the subject through the irradiation light filter unit 510 at the first timing including the times t600 and t602 by the timing control by the imaging control unit 160. As described above, the light emission control unit 170 transmits the light from the light emitting unit 410 through the irradiation light filter unit 510, thereby allowing the light in the first partial wavelength region, the light in the second partial wavelength region, and the fifth partial wavelength region. The light is emitted from the light emitting unit 410 toward the subject. In the present invention, the light in the first partial wavelength region and the light in the second partial wavelength region that are generated from the subject are the light in the first partial wavelength region and the second partial wavelength region that are generated from the subject due to reflection by the subject. Light.

  Then, at the first timing, the imaging control unit 160 irradiates the subject with light in the wavelength region including the first partial wavelength region, the second partial wavelength region, and the fifth wavelength region, and reflects the first reflected from the subject. The light in the partial wavelength region is received by the first light receiving element 251, the light in the second partial wavelength region reflected from the subject is received by the second light receiving element 252, and the light in the third partial wavelength region reflected from the subject is third. The light receiving element 253 receives light. As described above, the imaging control unit 160 causes the first light receiving element 251 to receive the light in the first partial wavelength region from the subject at the first timing, and the second light receiving light in the second partial wavelength region from the subject. The element 252 receives light, and the third light receiving element 253 receives light in the third partial wavelength region from the subject.

  Further, at the second timing including time t <b> 602, the light emission control unit 170 causes the light emitted from the light emitting unit 410 to irradiate the subject through the irradiation light filter unit 520 by the timing control by the imaging control unit 160. As described above, the light emission control unit 170 passes the light from the light emitting unit 410 through the irradiation light filter unit 520 at a timing other than the predetermined timing, whereby the light in the third partial wavelength region and the light in the fourth partial wavelength region are transmitted. Light and light in the sixth partial wavelength region are emitted from the light emitting unit 410 toward the subject. In the present invention, the light generated in the third partial wavelength region and the light generated in the fourth partial wavelength region from the subject means the light in the third partial wavelength region and the fourth partial wavelength region generated from the subject due to reflection by the subject. Light.

  Then, at the second timing, the imaging control unit 160 causes the first light receiving element 251 to receive the light in the third partial wavelength region reflected from the subject, and the light in the fourth partial wavelength region reflected from the subject is the second. The third light receiving element 253 receives the light of the fifth partial wavelength region reflected by the light receiving element 252 and reflected from the subject.

  In this way, the control unit 105 applies the light in the first partial wavelength region included in the first wavelength region, the light in the second partial wavelength region included in the second wavelength region, and the third wavelength region at a predetermined timing. The included light in the fifth partial wavelength region is generated from the subject. In addition, the control unit 105 performs the light of the third partial wavelength region included in the first wavelength region, the light of the fourth partial wavelength region included in the second wavelength region, and the third wavelength region at timings other than the predetermined timing. The light of the sixth partial wavelength region included in is generated from the subject.

  As described above, the control unit 105 controls the wavelength regions of light received by the first light receiving element 251, the second light receiving element 252, and the third light receiving element 253 at respective timings. Then, as will be described below, the image generation unit 140 generates an image of the subject based on the amount of light received by the light receiving element at each timing.

  The image generation unit 140 generates the first visible light image 620a based on the amount of light received by the light receiving element at the timing represented by time t600. Further, the image generation unit 140 generates the second visible light image 620b based on the amount of light received by the light receiving element at the timing represented by time t600 and the amount of light received by the light receiving element at the timing represented by time t601. Generate.

  Specifically, the image generation unit 140 determines the first light receiving amount (SR1) received by the first light receiving element 251 and the light receiving amount (SB1) received by the second light receiving element 252 at the timing represented by time t600. A visible light image 620a is generated. Further, the light emission control unit 170 emits light in the first partial wavelength region, which is a wavelength region longer than the third partial wavelength region, from the light emitting unit 410 toward the subject at a timing represented by time t600. The first light receiving element 251 can receive light in the first wavelength region, which is a longer wavelength region than the second wavelength region and the third wavelength region. Therefore, when the first light receiving element 251 receives the light reflected by the object existing inside the substance, the first light receiving element 251 has the first partial wavelength region, Among the two partial wavelength regions, the third partial wavelength region, the fourth partial wavelength region, the fifth partial wavelength region, and the sixth partial wavelength region, light in the longest wavelength region can be received.

  For this reason, the first light receiving element 251 can receive the reflected light from the object existing at a deeper position from the surface of the substance. Therefore, the image generation unit 140 is based on at least the light in the first partial wavelength region received by the first light receiving element 251 at a predetermined timing, and an image of an object that exists at a deeper position from the surface of the substance (for example, a blood vessel image). A first visible light image 620a including 626a) may be generated.

  In addition, the light emission control unit 170 transmits light of the second partial wavelength region and light of the second partial wavelength region, which are shorter than the fourth partial wavelength region, from the light emitting unit 410 to the subject at the timing represented by time t600. The light is emitted. The second light receiving element 252 can receive light in the second wavelength region, which is a shorter wavelength region than the first wavelength region and the third wavelength region. Therefore, when the second light receiving element 252 receives the light reflected by the object existing inside the substance, the second light receiving element 252 has the first partial wavelength region, Among the two partial wavelength regions, the third partial wavelength region, the fourth partial wavelength region, the fifth partial wavelength region, and the sixth partial wavelength region, light in the shortest wavelength region can be received.

  For this reason, the first light receiving element 251 cannot receive the reflected light from the object existing at the deep layer position, but can receive the reflected light from the object existing at a shallower position from the surface of the substance. . Therefore, the image generation unit 140 is based on at least the light in the second partial wavelength region received by the second light receiving element 252 at a predetermined timing, and the image of the object existing at a shallow position from the surface of the substance (for example, the blood vessel image 622a). And a first visible light image 620a that includes a blood vessel image 624a).

  As described above, the image generation unit 140 converts the light in the first partial wavelength region received by the first light receiving element 251 at the predetermined timing and the light in the second partial wavelength region received by the second light receiving element 252 at the predetermined timing. Based at least on, a first image can be generated that shows an image of an object that is at a shallower position from the surface of the material and a deeper position from the surface of the material. In the above, the first light receiving element 251 receives light in a wavelength region longer than the wavelength region received by the other light receiving elements, and the second light receiving element 252 has a wavelength region shorter than the wavelength region received by the other light receiving elements. Although the case where light is received has been described, the second light receiving element 252 receives light in a wavelength range longer than the wavelength range received by the other light receiving elements, and the first light receiving element 251 receives the wavelength range received by the other light receiving elements. It goes without saying that light in a shorter wavelength region may be received.

  In addition, the image generation unit 140 receives the received light amount (SR2) received by the first light receiving element 251 at the timing represented by time t602, the received light amount (SB2) received by the second light receiving element 252 and the third light receiving element 253. The second visible light image 620a is generated by the amount of received light (SG2) and the above-described SR1, SB1, and SG1.

  Specifically, the image generation unit 140 calculates the received light amount SR of the R component in each pixel by adding SR1, SR2, SG1, SG2, SB1, and SB2 with a predetermined weight for each pixel. Similarly, the image generation unit 140 adds SR1, SR2, SG1, SG2, SB1, and SB2 with a predetermined weight for each pixel, thereby receiving the G component received light amount SG and the B component received light amount in each pixel. SB is calculated. Then, the image generation unit 140 generates the second visible light image 620b based on the calculated SR, SG, and SB.

  Note that the image generation unit 140 may calculate SR, SG, and SB according to SR = SR1 + SR2, SB = SB1 + SB2, and SG = SG1 + SG2. As described above, the image generation unit 140 is configured so that each of the first light receiving elements 251 receives a light reception amount of the first partial wavelength region received by each of the first light receiving elements 251 at a predetermined timing and a timing other than the predetermined timing. The total value of the received light amount of the third partial wavelength region and the received light amount of the second partial wavelength region at a predetermined timing and at a timing other than the predetermined amount of light received by the second light receiving element 252 at a predetermined timing. The second visible light image may be generated based on the total amount of received light of the fourth partial wavelength region received by each of the second light receiving elements 252.

  As described above, the image generation unit 140 receives the light in the first partial wavelength region received by the first light receiving element 251 at a predetermined timing, and the third part received by the first light receiving element 251 at a timing other than the predetermined timing. From the light in the wavelength region, the light in the second partial wavelength region received by the second light receiving element 252 at a predetermined timing, and the light in the fourth partial wavelength region received by the second light receiving element 252 at a timing other than the predetermined timing, A second image indicating the image of the subject is generated.

  Further, the image generation unit 140 may generate a composite image obtained by combining the first visible light image 620a and the second visible light image 620b. At this time, the image generation unit 140 may output a composite image in which the first visible light image is emphasized and superimposed on the second visible light image. For example, the image generation unit 140 may generate a composite image by superimposing the enhanced image obtained by enhancing the pixel value of the first visible light image 620a and the second visible light image 620b. Further, the output unit 180 may output the first visible light image and the second visible light image generated by the image generation unit 140 in association with each other.

  As described above, the image generation unit 140 receives the light in the first partial wavelength region received by the first light receiving element 251 at a predetermined timing, and the third part received by the first light receiving element 251 at a timing other than the predetermined timing. Light in the wavelength region, light in the second partial wavelength region received by the second light receiving element 252 at a predetermined timing, light in the fourth partial wavelength region received by the second light receiving element 252 at a timing other than the predetermined timing, and predetermined From the combination including at least one of the light of the fifth partial wavelength region received by the third light receiving element 253 at the timing of the second light and the light of the sixth partial wavelength region received by the third light receiving element 253 at a timing other than the predetermined timing. One visible light image is generated, and a second visible light image is generated from a combination different from the combination.

  According to the imaging system 10 of the present embodiment, it is possible to provide a first visible light image in which surface blood vessels and deep blood vessels are emphasized, and a second visible light image 620b for surface layer observation. For this reason, when the imaging system 10 is actually applied, for example, when a doctor performs an operation or the like while viewing an image displayed by the output unit 180, the doctor recognizes an internal blood vessel that is difficult to see by surface observation. There are cases where it is possible. In addition, the doctor can obtain an advantage of performing an operation or the like while referring to an image in which blood vessels are emphasized.

  In addition, according to the imaging system 10, light in a plurality of partial wavelength regions can be separately received for each wavelength region received by the light receiving element, and thus the reflectance of light by the subject in each partial wavelength region is calculated. can do. For example, according to the imaging system 10, the reflection of light by the subject in each partial wavelength region based on the light emission amount and the light reception amount of each partial wavelength region indicated by B1, B2, G1, G2, R2, and R1 in FIG. The rate can be calculated.

  Further, according to the imaging system 10, the reflectance in each partial wavelength region can be calculated for each pixel. Further, according to the imaging system 10, by comparing the spectral reflectance of the gastric mucosa as shown by the line 340 in FIG. 3 with the reflectance of each pixel, there is an abnormal part in the gastric mucosa. In some cases, the position can be specified.

  Further, according to the imaging system 10, the absorptance of each partial wavelength region can be calculated from the reflectance of each partial wavelength region. In some cases, the component contained in the specimen 20 can be calculated based on the absorption rate. For example, there are cases where the ratio of blood components can be calculated by irradiating light from the light irradiation unit 150 with light in a wavelength region having different absorption rates among various blood components such as oxyhemoglobin and reduced hemoglobin. Thereby, the blood oxygen concentration or the like may be calculated.

  Further, the control unit 105 may cause the light irradiation unit 150 to emit light in a narrow-band partial wavelength region having a reflectance characteristic in the spectral reflectance of the subject according to the subject. According to the imaging system 10 of the present embodiment, the wavelength region of light received by the light receiving element having a fixed spectral sensitivity characteristic can be controlled by controlling the wavelength region irradiated by the light irradiation unit 150. For this reason, the endoscope 100 may be reduced in size.

  FIG. 7 shows an example of a block configuration of the image generation unit 140. In FIG. 6, the first visible light image 620a is assumed to have substantially no factors that cause temporal changes in the image such as the movement of the distal end portion 102 of the endoscope 100 and the movement of the specimen 20 for the sake of simplicity. The process of combining the first visible light image 620b with the first visible light image has been described. In this synthesis processing, when there is a movement of the distal end portion 102 of the endoscope 100, a movement of the sample 20, or the like, the positional displacement of the object is between the first visible light image 620a and the first visible light image 620b. May occur.

  In this figure, the operation and function of the image generation unit 140 for correcting the influence on the visible light image due to the above-described movement and the configuration of the image generation unit 140 will be described. The image generation unit 140 includes a motion identification unit 910, a corrected image generation unit 920, and a subject image generation unit 930.

  The movement specifying unit 910 specifies the movement of the object in the image based on the visible light images at a plurality of timings. Here, the movement of the object includes a movement that causes a temporal change in the image, such as a movement of the specimen 20 itself, a movement of the distal end portion 102 of the endoscope 100, and a temporal change in the zoom value of the imaging unit 110. In addition, the movement of the distal end portion 102 of the endoscope 100 is a time change in the position of the distal end portion 102 that causes a time change in the imaging position of the imaging unit 110 and a direction of the distal end portion 102 that causes a time change in the imaging direction of the imaging unit 110. Including time changes.

  Here, the movement specifying unit 910 specifies the movement of the object based on the visible light images at time t600 and time t601. For example, the movement specifying unit 910 may specify the movement of the object by matching the objects extracted from the plurality of visible light images.

  Based on the movement, the corrected image generation unit 920 corrects the image signal indicating the visible light image at the timing of time t601 and generates an image signal indicating the visible light image to be obtained at the timing of time t602. Thereby, the corrected image generation unit 920 can generate a subject image at the time t602.

  FIG. 8 is a diagram for explaining the generation of a visible light image whose motion is corrected. The visible light image 820a is an image generated by image signals from the first light receiving element 251 and the second light receiving element 252 at time t600. The visible light image 820c is an image generated by image signals from the first light receiving element 251 and the second light receiving element 252 at time t602.

  Here, the movement specifying unit 910 specifies the movement based on the image contents of the visible light image 820a and the visible light image 820c. Specifically, the movement identifying unit 910 extracts an object indicating the same subject from the visible light image 820a and the visible light image 820c. In the example of this figure, the movement identification unit 910 extracts an object 850a and an object 850c from the visible light image 820a and the visible light image 820c, respectively.

  The movement identifying unit 910 calculates the difference between the positions of the object 850a and the object 850c. In the example of this figure, for the sake of simplicity, it is assumed that the difference in the position occurs in the y direction on the image, and the movement specifying unit 910 calculates the position difference Δy between the position of the object 850a and the position of the object 850c. calculate. Then, the corrected image generation unit 920 shifts the visible light image 830b by shifting the image 821a in the y direction by an amount Δy / 2 corresponding to the calculated positional difference Δy and each timing at time t600, time t601, and time t602. Generate.

In the above description, the process of specifying the motion using the visible light image 820 has been described, but the motion can also be specified using the image of each color component in the same manner. Here, it may be determined based on the contrast of the captured image which wavelength of the image is used by the motion specifying unit 910 to specify the motion. For example, the motion specifying unit 910 may specify the motion by using an image with higher contrast more preferentially. When an image of a fine structure can be used as an object for motion identification, such as when the image of the fine structure of the surface is clear, the movement is more accurately performed using an image of a B signal (for example, an image of S _B1 ). There are cases where it can be specified. In addition, when the image of the concavo-convex structure can be used as an object for motion identification, such as when the image of the larger concavo-convex structure on the surface is clear, the movement is performed using the image of the G signal (eg, S _G1 ). It may be possible to specify more accurately.

  Further, the corrected image generation unit 920 may vary the amount of motion correction for the visible light image for each image region. For example, if the imaging direction of the imaging unit 110 is perpendicular to the subject surface and the distal end portion 102 of the endoscope 100 is moved horizontally to the subject surface, the amount of movement of the object is considered to be equal in any image region. Can do. On the other hand, for example, when the imaging direction of the imaging unit 110 is not perpendicular to the subject surface, the amount of motion in the image region in which the region far from the tip 102 is imaged is greater than that in the image region in which the region near the tip 102 is imaged. The amount of movement may be small.

  In order for the corrected image generation unit 920 to calculate the amount of motion correction for the visible light image for each image region, if the positional relationship between the subject surface and the imaging unit 110 is known or can be estimated, the positional relationship and the image region Based on the position, the amount of motion correction can be calculated. Note that the corrected image generation unit 920 controls the endoscope 100 that causes temporal changes in the image, such as a control value for controlling the position / orientation of the distal end portion 102 and a control value for controlling the zoom value of the imaging unit 110. And the amount of motion correction for the visible light image may be calculated based on the control value.

  In addition, the movement specifying unit 910 may calculate the movement of the object for each image area. The corrected image generation unit 920 may calculate a motion correction amount for the image of each image region based on the motion of the object for each image region.

  In addition, when specifying the motion for each image region, the motion specifying unit 910 may determine for each image region which image of which wavelength is used to specify the motion. For example, the motion identification unit 910 calculates the contrast of each image for each image region. Then, for each image region, the motion specifying unit 910 selects an image having a wavelength with a higher contrast calculated in preference to an image having another wavelength, and specifies the motion of the object using the selected plurality of images. You can do it.

  In the above description, the movement is specified using the visible light image 820a and the visible light image 820c. However, the movement specifying unit 910 uses the visible light image 820a and light of the same wavelength region obtained before time t600. A visible light image may be used to identify the movement.

  When it is acceptable to delay the display of the visible light image to some extent, the motion identifying unit 910 includes a plurality of timings including times before and after the time t601 that is a target time for generating a visible light image whose motion is corrected. The movement may be specified from the image obtained at the timing. By using an image at a later timing, it may be possible to further increase the accuracy of motion identification. Note that the motion specifying unit 910 may specify a motion using a visible light image (or an image of each color component) captured at three or more timings.

  The subject image generation unit 930 may synthesize the visible light image 830b and the visible light image captured at time t601 to generate a second visible light image as an example of the subject image in the present invention. Thereby, the 2nd visible light image by which the motion was corrected can be obtained using the visible light image imaged at different timing.

  In the above description, the operation of the image generation unit 140 in the case of correcting the motion in order to generate the second visible light image at time t601 has been described. In addition, the corrected image generation unit 920 can also generate a first visible light image in which the movement is corrected based on the movement specified by the movement specifying unit 910. The subject image generation unit 930 may output an image obtained by synthesizing the first visible light image whose motion is corrected and the second visible light image whose motion is corrected to the output unit 180.

  In the above description, the case where various visible light images are generated at time t601 has been described. However, when various visible light images are generated at time t602, visible light images are generated by similar processing. Can do. For example, the motion specifying unit 910 may specify the motion using the visible light images (or the images of the respective color components) obtained at time t601 and time t603, respectively. The corrected image generation unit 920 may correct various visible light images obtained at time t601 according to the movement.

  As described above with reference to FIGS. 7 and 8, the movement identifying unit 910 receives the light in the first partial wavelength region received by the first light receiving element 251 at a plurality of timings including a predetermined timing, and the plurality of timings. The movement of the object on the image during a plurality of timings is specified based on the plurality of images by the light in the second partial wavelength region received by the second light receiving element 252. Then, the corrected image generation unit 920 receives the light in the first wavelength region received by the first light receiving element 251 at a predetermined timing, the light in the second wavelength region received by the second light receiving element 252 at the first timing, and the movement. Based on the above, a corrected image that is an image of a subject by light in the first partial wavelength region and light in the second partial wavelength region at a timing other than the predetermined timing is generated. The subject image generation unit 930 receives the light in the third partial wavelength region received by the plurality of first light receiving elements at a timing other than the predetermined timing, and the plurality of second light receiving elements received at a timing other than the predetermined timing. The second image may be generated based on the image of the light in the fourth partial wavelength region and the corrected image.

  FIG. 9 shows an example of the spectral sensitivity characteristics of the light receiving element and the configuration of the light source side filter unit 420. In the following description, differences from the spectral sensitivity characteristics and the function of the light source side filter unit 420 shown in FIGS. 4 and 5 will be described. In the example of this figure, the spectral sensitivity characteristic in the wavelength region including the wavelength region longer than the wavelength region shown in FIG. 3 is shown. As shown in the figure, the first light receiving element 251 can receive light in an infrared region (for example, 810 nm) which is an example of the specific wavelength region.

  In addition, the light source side filter unit 420 in this drawing includes irradiation light filter units 710, 720, and 730. The irradiation light filter unit 710 includes light in the third partial wavelength region (R2), light in the fourth partial wavelength region (B2), light in the sixth partial wavelength region (G2), light in the seventh partial wavelength region (R3), The light of the 8th partial wavelength region (B3) and the light of the 9th partial wavelength region (G3) are cut, and the excitation light, the light of the second partial wavelength region (B1), and the fifth partial wavelength region (G1) Through light. The irradiation light filter unit 720 includes excitation light, light in the second partial wavelength region (B1), light in the fifth partial wavelength region (G1), light in the seventh partial wavelength region (R3), and eighth partial wavelength region. The light of (B3) and the light of the ninth partial wavelength region (G3) are cut, the light of the third partial wavelength region (R2), the light of the fourth partial wavelength region (B2), and the sixth partial wavelength region The light of (G2) passes. The irradiation light filter unit 730 includes excitation light, light in the second partial wavelength region (B1), light in the third partial wavelength region (R2), light in the fourth partial wavelength region (B2), and fifth partial wavelength region. The light of (G1), the light of the sixth partial wavelength region (G2) is cut, the light of the seventh partial wavelength region (R3), the light of the eighth partial wavelength region (B3), and the ninth partial wavelength region ( G3) light passes through.

  Therefore, at the timing when the light from the light emitting unit 410 is guided to the irradiation light filter unit 730, among the light from the light emitting unit 410, the excitation light, the light in the second partial wavelength region, and the light in the fifth partial wavelength region Passes through the irradiation light filter unit 710 and is irradiated onto the subject. In addition, at the timing when the light from the light emitting unit 410 is guided to the irradiation light filter unit 720, among the light from the light emitting unit 410, the light in the third partial wavelength region, the light in the fourth partial wavelength region, and the sixth The light of the partial wavelength region passes through the irradiation light filter unit 720, and the subject is irradiated with the light of the third partial wavelength region, the light of the fourth partial wavelength region, and the light of the sixth partial wavelength region. At the timing when the light from the light emitting unit 410 is guided to the irradiation light filter unit 730, the light of the seventh partial wavelength region (R3) and the eighth partial wavelength region (B3) of the light from the light emitting unit 410. And the light of the ninth partial wavelength region (G3) pass through the irradiation light filter unit 730, and the light of the seventh partial wavelength region (R3), the light of the eighth partial wavelength region (B3), and the ninth The subject is irradiated with light in the partial wavelength region (G3).

  FIG. 10 shows an example of the imaging timing by the imaging unit 110 and an image generated by the image generation unit 140. The imaging control unit 160 causes the imaging unit 110 to capture images at times t800, t801, t802, t803,. Also, the light emission control unit 170 causes the light emitted from the light emitting unit 410 to irradiate the subject through the irradiation light filter unit 710 at the first timing including the times t800 and t803 by the timing control by the imaging control unit 160. Accordingly, the subject is irradiated with excitation light, light in the second partial wavelength region, and light in the fifth partial wavelength region.

  Then, the image generation unit 140 generates a luminescence light image 820a at time t800 based on the amount of luminescence light received by the first light receiving element 251. Since the excitation light reaches deeper into the substance than light in other partial wavelength regions, the luminescence light image 820a includes blood vessel images 822a and 824a at a shallow position from the material surface and a blood vessel image 826a at a deep position.

  On the other hand, the light emission control unit 170 causes the light emitted from the light emitting unit 410 to irradiate the subject through the irradiation light filter unit 720 at time t801 by the timing control by the imaging control unit 160. As a result, the subject is irradiated with light in the third partial wavelength region, light in the fourth partial wavelength region, and light in the sixth partial wavelength region.

  In addition, the light emission control unit 170 causes the light emitted from the light emitting unit 410 to irradiate the subject through the irradiation light filter unit 730 at time t <b> 802 by timing control by the imaging control unit 160. Thereby, the light of the seventh partial wavelength region (R3), the light of the eighth partial wavelength region (B3), and the light of the ninth partial wavelength region (G3) are irradiated to the subject. Then, the image generation unit 140 receives the first visible light image based on the received light amount (SB1) received by the second light receiving element 252 at time t800 and the received light amount (SR2) received by the first light receiving element 251 at time t801. 820b is generated. Similar to the first visible light image 620b, the first visible light image 820b is an image of an object existing at a deep position from the surface of the substance (for example, a blood vessel image 826b) and an image of an object existing at a shallow position from the surface of the substance. (For example, blood vessel image 822b and blood vessel image 624b).

  In addition, the image generation unit 140 receives the amount of light received by the light receiving elements other than the first light receiving element 251 at the timing represented by time t800 (SB1 and SG1), and each light receiving element receives light at the timing represented by time t801. Based on the received light amount (SB2, SG2, and SR2) and the received light amount (SB3, SG3, and SR3) received by each light receiving element at the timing represented by time t803, the second visible light image 820c is generated. Note that the image generation unit 140 can generate the second visible light image 820c in the same manner as the method described with reference to FIG.

  For example, the image generation unit 140 adds the SB1, SG1, SB2, SG2, SR2, SB3, SG3, and SR3 with a predetermined weight to calculate the SR, SG, and SG, thereby calculating the second visible light image 820c. Can be generated. The second visible light image 820c includes an image (for example, blood vessel images 822c and 824c) of an object existing at a relatively shallow position.

  Thus, according to the imaging system 10 of the present embodiment, the luminescence light image 820a can be obtained from the luminescence light in the infrared region generated from the specimen 20 by the excitation light in the infrared region. Excitation light having a wavelength longer than that of visible light is less likely to be absorbed by the substance than visible light, so that it penetrates deeper into the substance than visible light and generates luminescence light in the specimen 20. In addition, since the luminescence light has a longer wavelength than the excitation light, it easily reaches the material surface. For this reason, according to the imaging system 10, a blood vessel image having a wider range of depth than visible light can be obtained at one time.

  9 and 10, the corrected image generation unit 920 also uses the first visible light whose movement is corrected by the same process as the process described with reference to FIGS. 7 and 8. Images and luminescent light images can be generated. The subject image generation unit 930 generates a second visible light image by combining the visible light image whose motion is corrected, or generates an image by combining the second visible light image and the luminescence light image. be able to.

  In the above description, the embodiment of the present invention has been described by taking as an example the case where the control unit 105 generates light in different wavelength regions from the subject at different timings. In order to obtain an image using light in different wavelength regions, the main wavelength component in the spectrum of light from the subject only needs to be in a predetermined wavelength region, even if it has a certain spectral intensity in other wavelength regions. Good. For example, light from a subject at a predetermined timing may have spectral intensity in the third partial wavelength region and the fourth partial wavelength region in addition to the first partial wavelength region and the second partial wavelength region. If light from a subject at a predetermined timing has a spectral intensity mainly in the first partial wavelength region and the second partial wavelength region, an image of the first partial wavelength region and the second partial wavelength region is substantially generated. can do.

  As described above, the control unit 105 may generate light of different spectra from the subject at different timings in each of the first wavelength region and the second wavelength region. For example, the control unit 105 can increase the ratio of the spectral intensity of the third partial wavelength region to the spectral intensity of the first partial wavelength region in the light from the subject at a timing other than the predetermined timing than the predetermined timing. May be irradiated to the light irradiation unit 150. More specifically, the light irradiation unit 150 emits light whose spectral intensity in the first partial wavelength region is larger than the spectral intensity in the third partial wavelength region at a predetermined timing, and at a timing other than the predetermined timing, The light irradiation unit 150 may be irradiated with light whose spectral intensity in the third partial wavelength region is greater than that in the first partial wavelength region.

  Similarly, the control unit 105 can increase the ratio of the spectral intensity of the fourth partial wavelength region to the spectral intensity of the second partial wavelength region in the subject light at a timing other than the predetermined timing. The light irradiation unit 150 may be irradiated. More specifically, the light irradiation unit 150 emits light whose spectral intensity in the second partial wavelength region is larger than the spectral intensity in the fourth partial wavelength region at a predetermined timing, and at a timing other than the predetermined timing, The light irradiation unit 150 may be irradiated with light whose spectral intensity in the fourth partial wavelength region is larger than that in the second partial wavelength region.

  Then, the image generation unit 140 receives the first spectrum light from the subject received by the first light receiving element 251 at a predetermined timing, the second spectrum light from the subject received by the second light receiving element 252 at the predetermined timing, At least the third spectrum light from the subject received by the first light receiving element 251 at a timing other than the predetermined timing and the fourth spectrum light from the subject received by the second light receiving element 252 at a timing other than the predetermined timing. The first image may be generated from a combination including one. The image generation unit 140 may generate the second image from a combination different from the combination.

  4, 5, 9, and the like, as the operation of the light irradiation unit 150, the spectrum of the irradiation light is temporally controlled by rotating the light source side filter unit 420 with the light from the light emitting unit 410. The operation to be explained. As another example of the light irradiation unit 150, the light irradiation unit 150 may not include the light source side filter unit 420. Specifically, the light emitting unit 410 may include a plurality of light emitting elements that respectively emit light having different spectra. Then, the control unit 105 may control light emission at a timing other than the first predetermined timing and the predetermined timing.

  For example, the light emitting unit 410 includes a light emitting element that emits light in a red wavelength region, a light emitting element that emits light in a blue wavelength region, a light emitting element that emits light in a green wavelength region, and a wavelength region of excitation light. A light emitting element that emits light may be included. Examples of light emitting elements that emit light in the visible light region include semiconductor elements such as LEDs. Moreover, as a light emitting element which emits excitation light, semiconductor elements, such as a semiconductor laser, can be illustrated. The light emitting element may be a phosphor that emits luminescence light such as fluorescence when excited.

  The control unit 105 can control the spectrum of light emitted to the subject by controlling the emission intensity of each of the plurality of light emitting elements at each timing. Note that “controlling the light emission intensity of each of the plurality of light emitting elements” includes controlling the combination of the light emitting elements to emit light at each timing. The light-emitting element may include a filter that transmits light in a specific wavelength region and a light emitter. If the spectrum of the light after the illuminant emits light and passes through the filter is different, these light-emitting elements can be regarded as a plurality of light-emitting elements that respectively emit light having different spectra in the present invention.

  Note that the light emitting element may be provided at the distal end portion 102 of the endoscope 100. Note that the light emitting element may be a light emitting element that emits light by electrical excitation or a light emitting element that emits light by optical excitation. When the light emitting element is a light emitting element that emits light by light excitation, the light irradiation unit 150 includes an excitation unit that emits excitation light for exciting the light emitting element, and the light emitting element. Here, the light-emitting element may emit light having a different spectrum depending on the wavelength of the excitation light. In this case, the control unit 105 can control the spectrum of the irradiation light by controlling the wavelength of the excitation light emitted from the light emitting unit at each timing. Further, the spectrum of light emitted from each light emitting element by the excitation light may be different among the plurality of light emitting elements. Further, the light that has passed through the light emitting element among the excitation light may be irradiated to the subject as irradiation light.

  FIG. 11 shows an example of a hardware configuration of a computer 1500 that functions as the imaging system 10. The imaging system 10 according to this embodiment includes a CPU peripheral unit including a CPU 1505, a RAM 1520, a graphic controller 1575, and a display device 1580 connected to each other by a host controller 1582, and an input / output controller 1584 to the host controller 1582. Input / output unit having communication interface 1530, hard disk drive 1540, and CD-ROM drive 1560 connected, and legacy input / output having ROM 1510, flexible disk drive 1550, and input / output chip 1570 connected to input / output controller 1584 A part.

  The host controller 1582 connects the RAM 1520 to the CPU 1505 and the graphic controller 1575 that access the RAM 1520 at a high transfer rate. The CPU 1505 operates based on programs stored in the ROM 1510 and the RAM 1520 to control each unit. The graphic controller 1575 acquires image data generated by the CPU 1505 or the like on a frame buffer provided in the RAM 1520 and displays the image data on the display device 1580. Alternatively, the graphic controller 1575 may include a frame buffer that stores image data generated by the CPU 1505 or the like.

  The input / output controller 1584 connects the host controller 1582 to the communication interface 1530, the hard disk drive 1540, and the CD-ROM drive 1560, which are relatively high-speed input / output devices. The communication interface 1530 communicates with other devices via a network. The hard disk drive 1540 stores programs and data used by the CPU 1505 in the imaging system 10. The CD-ROM drive 1560 reads a program or data from the CD-ROM 1595 and provides it to the hard disk drive 1540 via the RAM 1520.

  The input / output controller 1584 is connected to the ROM 1510, the flexible disk drive 1550, and the relatively low-speed input / output device of the input / output chip 1570. The ROM 1510 stores a boot program that the imaging system 10 executes at startup, a program that depends on the hardware of the imaging system 10, and the like. The flexible disk drive 1550 reads a program or data from the flexible disk 1590 and provides it to the hard disk drive 1540 via the RAM 1520. The input / output chip 1570 connects various input / output devices via a flexible disk drive 1550 such as a parallel port, a serial port, a keyboard port, and a mouse port.

  A communication program provided to the hard disk drive 1540 via the RAM 1520 is stored in a recording medium such as the flexible disk 1590, the CD-ROM 1595, or an IC card and provided by the user. The communication program is read from the recording medium, installed in the hard disk drive 1540 in the imaging system 10 via the RAM 1520, and executed by the CPU 1505. The communication program installed and executed in the imaging system 10 works on the CPU 1505 and the like, and the imaging system 10 is described with reference to FIGS. 1 to 10, the imaging unit 110, the image generation unit 140, the output unit 180, the control unit 105, And function as the light irradiation unit 150 and the like.

  As mentioned above, although this invention was demonstrated using embodiment, the technical scope of this invention is not limited to the range as described in the said embodiment. It will be apparent to those skilled in the art that various modifications or improvements can be added to the above-described embodiment. It is apparent from the scope of the claims that the embodiments added with such changes or improvements can be included in the technical scope of the present invention.

1 is a diagram illustrating an example of a configuration of an imaging system 10 according to an embodiment together with a sample 20. FIG. 2 is a diagram illustrating an example of a configuration of an imaging unit 110. FIG. It is a figure which shows an example of the spectral reflectance characteristic of the biological surface layer which is an example of the spectral sensitivity characteristic of a light receiving element, and imaging | photography object. 2 is a diagram illustrating an example of a configuration of a light irradiation unit 150. FIG. 5 is a diagram illustrating an example of a configuration of a light source side filter unit 420. FIG. 3 is a diagram illustrating an example of an imaging timing by an imaging unit 110 and an image generated by an image generation unit 140. FIG. 3 is a diagram illustrating an example of a block configuration of an image generation unit 140. FIG. It is a figure explaining the production | generation of the visible light image by which the motion was correct | amended. It is a figure which shows an example of the structure of the spectral sensitivity characteristic of a light receiving element, and the light source side filter part 420. FIG. 3 is a diagram illustrating an example of an imaging timing by an imaging unit 110 and an image generated by an image generation unit 140. FIG. FIG. 25 is a diagram illustrating an example of a hardware configuration of a computer 1500 that functions as the imaging system 10.

Explanation of symbols

DESCRIPTION OF SYMBOLS 10 Imaging system 20 Specimen 100 Endoscope 102 Tip part 105 Control part 110 Imaging part 112 Objective lens 120 Light guide 124 Output port 130 Forceps port 135 Forceps 138 Nozzle 140 Image generation part 150 Light irradiation part 160 Imaging control part 170 Light emission control part 180 output unit 190 ICG injection unit 210 imaging device 220 spectral filter unit 230 light receiving side excitation light cut filter unit 251 first light receiving element 252 second light receiving element 253 third light receiving element 410 light emitting unit 420 light source side filter unit 510 irradiation light filter unit 520 Irradiation light filter unit 626 Blood vessel image 710 Irradiation light filter unit 720 Irradiation light filter unit 730 Irradiation light filter unit 1505 CPU
1510 ROM
1520 RAM
1530 Communication interface 1540 Hard disk drive 1550 Flexible disk drive 1560 CD-ROM drive 1570 Input / output chip 1575 Graphic controller 1580 Display device 1582 Host controller 1584 Input / output controller 1590 Flexible disk 1595 CD-ROM

Claims (17)

A plurality of first light receiving elements for receiving light in the first wavelength region; a plurality of second light receiving elements for receiving light in the second wavelength region; and a plurality of third light receiving elements for receiving light in the third wavelength region. The light of the first partial wavelength region included in the first wavelength region, the light of the second partial wavelength region included in the second wavelength region, and the fifth partial wavelength included in the third wavelength region at a predetermined timing. Imaging with the light of the region, and at a timing other than the predetermined timing, the light of the third partial wavelength region included in the first wavelength region, the light of the fourth partial wavelength region included in the second wavelength region, and the first An imaging unit for imaging with light in a sixth partial wavelength region included in the three wavelength region;
At the predetermined timing, the light of the first partial wavelength region, the light of the second partial wavelength region, and the light of the fifth partial wavelength region are generated from the subject, and at the timing other than the predetermined timing, the third A control unit for generating light of a partial wavelength region, light of the fourth partial wavelength region, and light of the sixth partial wavelength region from a subject;
The light of the first partial wavelength region from the subject received by the plurality of first light receiving elements at the predetermined timing, the first from the subject received by the plurality of first light receiving elements at a timing other than the predetermined timing. Light in three partial wavelength regions, light in the second partial wavelength region from a subject received by the plurality of second light receiving elements at the predetermined timing, and the plurality of second light receiving elements at timings other than the predetermined timing Light in the fourth partial wavelength region from the received subject, light in the fifth partial wavelength region from the subject received by the plurality of third light receiving elements at the predetermined timing, and timing other than the predetermined timing A combination including at least one of light in the sixth partial wavelength region from a subject received by the plurality of third light receiving elements. From generates a first image, and an image generating unit that generates a second image from a combination different from the combination,
The first wavelength region, the second wavelength region, and the third wavelength region are each wavelength regions of blue, red, and green,
The first partial wavelength region is a wavelength region shorter than the third partial wavelength region or a wavelength region longer than the third partial wavelength region,
The image generation unit is configured to receive light in the first partial wavelength region received by the plurality of first light receiving elements at the predetermined timing, and the first light receiving elements received by the plurality of first light receiving elements at timings other than the predetermined timing. Light in three partial wavelength regions, light in the second partial wavelength region received by the plurality of second light receiving elements at the predetermined timing, and the plurality of second light receiving elements received at timings other than the predetermined timing Light in the fourth partial wavelength region, light in the fifth partial wavelength region received by the plurality of third light receiving elements at the predetermined timing, and light received by the plurality of third light receiving elements at timings other than the predetermined timing <br/> imaging system from the light of the sixth partial wavelength region to generate the second image showing an image of an object by visible light. Light that generates light from the subject from the first partial wavelength region, the second partial wavelength region, the third partial wavelength region, the fourth partial wavelength region, the fifth partial wavelength region, and the sixth partial wavelength region. A light emitting unit that emits
The control unit causes the light emitting unit to emit light that causes light of the first partial wavelength region, light of the second partial wavelength region, and light of the fifth partial wavelength region from a subject at the predetermined timing. And at a timing other than the predetermined timing, the light emitting unit emits light that causes light of the third partial wavelength region, light of the fourth partial wavelength region, and light of the sixth partial wavelength region from a subject. Item 2. The imaging system according to Item 1. The light emitting unit emits light in the first partial wavelength region, the second partial wavelength region, the third partial wavelength region, the fourth partial wavelength region, the fifth partial wavelength region, and the sixth partial wavelength region. Flash
The control unit causes the light of the first partial wavelength region, the light of the second partial wavelength region, and the light of the fifth partial wavelength region to emit light from the light emitting unit toward a subject at the predetermined timing, and At a timing other than a predetermined timing, the light of the third partial wavelength region, the light of the fourth partial wavelength region and the light of the sixth partial wavelength region are emitted from the light emitting unit toward the subject,
At the predetermined timing, the plurality of first light receiving elements receive light in the first partial wavelength region reflected from the subject, and the plurality of second light receiving elements reflect light in the second partial wavelength region reflected from the subject. The plurality of third light receiving elements receive the light in the fifth partial wavelength region reflected from the subject, and at a timing other than the predetermined timing, the plurality of first light receiving elements are reflected from the subject. The plurality of second light receiving elements receive light in the fourth partial wavelength region reflected from the subject, and the plurality of third light receiving elements reflect the sixth partial light reflected from the subject. The imaging system according to claim 2, which receives light in a partial wavelength region. The imaging system according to claim 2, wherein the image generation unit generates a composite image obtained by combining the first image and the second image. The imaging system according to claim 4, wherein the image generation unit outputs a composite image obtained by emphasizing the first image and overlaying the second image. The imaging system according to claim 4, further comprising an output unit that outputs the first image and the second image generated by the image generation unit in association with each other. The image generation unit is configured to receive the received light amount of the first partial wavelength region received by each of the plurality of first light receiving elements at the predetermined timing and the plurality of first light receiving elements at a timing other than the predetermined timing. And the total amount of light received in the third partial wavelength region received by each of the plurality of second light receiving elements and the amount of light received in the second partial wavelength region received by each of the plurality of second light receiving elements at the predetermined timing. A total value of the received light amounts of the fourth partial wavelength regions received by each of the plurality of second light receiving elements at timings other than the predetermined timing, and the plurality of third light receiving elements at the predetermined timing The received light amount of the fifth partial wavelength region received by each of the plurality of third light receiving elements at a timing other than the predetermined timing. Each is based on the sum of the amount of received light of the sixth partial wavelength region received, the imaging system according to any one of claims 4 to 6 to generate the second image. The plurality of first light-receiving elements, the plurality of second light-receiving elements, and the plurality of third light-receiving elements receive light reflected by an object existing in the substance of light emitted from the light emitting unit,
The first wavelength region is a wavelength region shorter than any of the second wavelength region and the third wavelength region,
The first partial wavelength region is a wavelength region shorter than the third partial wavelength region,
The image generation unit shows an image of an object existing at a shallower position from the surface of the substance based on at least light in the first partial wavelength region received by the plurality of first light receiving elements at the predetermined timing. The imaging system according to claim 7, wherein the first image is generated. The second wavelength region is a wavelength region longer than any of the first wavelength region and the third wavelength region,
The first partial wavelength region is a wavelength region shorter than the third partial wavelength region, the second partial wavelength region is a wavelength region longer than the fourth partial wavelength region,
The image generation unit includes the light in the first partial wavelength region received by the plurality of first light receiving elements at the predetermined timing and the second partial wavelength received by the plurality of second light receiving elements at the predetermined timing. The imaging system according to claim 8, wherein the first image indicating an image of an object existing at a shallower position from the surface of the substance and a deeper position from the surface of the substance is generated based on at least light of a region. The plurality of first light-receiving elements, the plurality of second light-receiving elements, and the plurality of third light-receiving elements receive light reflected by an object existing in the substance of light emitted from the light emitting unit,
The first wavelength region is a wavelength region longer than any of the second wavelength region and the third wavelength region,
The first partial wavelength region is a wavelength region longer than the third partial wavelength region,
The image generation unit includes an image of an object existing at a deeper position from the surface of the substance based at least on the light in the first partial wavelength region received by the plurality of first light receiving elements at the predetermined timing. The imaging system according to claim 7, wherein the first image is generated. A first spectral filter section that passes the light of the first partial wavelength region and the light of the third partial wavelength region;
A second spectral filter section that passes the light of the second partial wavelength region and the light of the fourth partial wavelength region;
A third spectral filter section that passes through the light in the fifth partial wavelength region and the light in the sixth partial wavelength region; and
The plurality of first light receiving elements receive light from the subject that has passed through the first spectral filter unit, and the plurality of second light receiving elements receive light from the subject that has passed through the second spectral filter unit. The imaging system according to claim 2, wherein the plurality of third light receiving elements receive light from a subject that has passed through the third spectral filter unit. The light emitting unit has a plurality of light emitting elements that respectively emit light of different spectra,
The control unit controls the light emission of the light at the predetermined timing and a timing other than the predetermined timing by controlling the light emission intensity of each of the plurality of light emitting elements. The imaging system according to item. An irradiation light filter that passes through the light of the first partial wavelength region, the light of the second partial wavelength region, and the light of the fifth partial wavelength region;
The light emitting unit includes at least one of a first partial wavelength region, the second partial wavelength region, and the fifth partial wavelength region, the third partial wavelength region, the fourth partial wavelength region, and the sixth partial wavelength region. And emits light in a wavelength region including the wavelength region of
The imaging system according to any one of claims 2 to 11, wherein the control unit irradiates a subject with light from the light emitting unit through the irradiation light filter unit at the predetermined timing. The image generation unit
Light in the first partial wavelength region received by the plurality of first light receiving elements at a plurality of timings including the predetermined timing, and the second partial wavelength region received by the plurality of second light receiving elements at the plurality of timings And the movement of the object on the image during the plurality of timings based on the plurality of images of the light of the fifth partial wavelength region received by the plurality of third light receiving elements at the plurality of timings A motion identification unit to perform,
The light in the first wavelength region received by the plurality of first light receiving elements at the predetermined timing, the light in the second wavelength region received by the plurality of second light receiving elements at the predetermined timing, and the predetermined timing The light of the first partial wavelength region at the timing other than the predetermined timing based on the light of the fifth partial wavelength region received by the plurality of third light receiving elements and the movement, and the second partial wavelength 4. The imaging system according to claim 1, further comprising: a correction image generation unit configured to generate a correction image that is an image of a subject by light in a region and light in the fifth partial wavelength region. The image generation unit
Light in the third partial wavelength region received by the plurality of first light receiving elements at a timing other than the predetermined timing, and the fourth partial wavelength received by the plurality of second light receiving elements at a timing other than the predetermined timing The second image is generated based on the light of the region and the image of the light of the sixth partial wavelength region received by the plurality of third light receiving elements at timings other than the predetermined timing and the corrected image. The imaging system according to claim 14, further comprising a subject image generation unit. The imaging system according to claim 3, wherein the first wavelength region is a blue wavelength region, the second wavelength region is a red wavelength region, and the third wavelength region is a green wavelength region.   The program for functioning a computer as an imaging system as described in any one of Claim 1 to 16.

Images