WO2016185697A1 - Infrared imaging device and signal correction method using infrared imaging device - Google Patents

Abstract

[Problem] To provide an infrared imaging device and a signal correction method using the infrared imaging device, wherein the pixel signals of reference pixels of an infrared image sensor are used to correct pixel signals of effective pixels, thereby reducing band-shaped noise. [Solution] An infrared imaging device (1) comprising: an optical system (2); an infrared image sensor (3) equipped with an effective pixel part obtained by positioning effective pixels in a matrix, and a reference pixel part obtained by positioning reference pixels in relation to each row of effective pixels, the effective region and reference region being provided to a single detection region; a shielding part (9) that is positioned a distance away from the reference pixels, and that blocks infrared light from entering the reference region from the optical system (2); and a signal correction unit (6), wherein the signal correction unit (6) calculates the reference level value of the pixel signal of the reference pixels, and the band level value that indicates the average level of the pixel signal of reference pixels that belong to each row of reference pixels, and corrects, for each effective pixel, the pixel signal for the effective pixels according to the difference between the reference level value and the band level value corresponding to the row of pixels to which the effective pixel belongs.

Description

Infrared imaging device and signal correction method using infrared imaging device

The present invention relates to an infrared imaging device that captures an infrared image and a signal correction method using the infrared imaging device, and more particularly to an infrared imaging device that corrects a pixel signal of an infrared image and a signal correction method using the infrared imaging device.

Conventionally, various correction techniques for pixel signals detected by a solid-state imaging device have been proposed in various imaging devices using the solid-state imaging device.

For example, Patent Documents 1 and 2 relate to an uncooled infrared image sensor. An image signal of an effective pixel that detects infrared light that has passed through an optical system is used as a pixel signal of a reference pixel that does not increase in temperature due to infrared light that has passed through the optical system. The technique which corrects using is proposed.

Patent document 3 relates to a photoelectric conversion type image sensor and proposes a technique for correcting an output signal of an effective pixel unit including a photoelectric conversion unit using an output signal of an optical black unit in which the photoelectric conversion unit is shielded from light. Yes.

Japanese Patent Laid-Open No. 2004-245692 No. 2011/013197 JP 2012-015587 A

Here, in an infrared image sensor mounted on an imaging device that captures an infrared image, a signal line (data line) corresponding to each pixel column increases or decreases in voltage due to individual differences such as elements connected to each pixel column. As a result, the pixel signal varies in units of columns. There is a need for a technique for reducing streak noise due to such pixel signal column-by-column variations.

However, in Patent Document 1, in order to cancel noise caused by Joule heat of a bias current for accumulating infrared signals, a bias current passed through an effective pixel is set in units of horizontal address lines in accordance with an electrical signal of a reference pixel. Since the offset control is performed by the method, the streak noise cannot be reduced.

Further, Patent Document 2 calculates an average value of electrical signals of reference pixels (thermal insensitive pixels) for each pixel column in order to reduce vertical streak noise due to noise components existing in a signal line, and detects infrared rays. A technique for subtracting an average value calculated from a pixel signal is disclosed. In addition, Patent Document 2 discloses a reference pixel by attaching a specific structure to the thermoelectric conversion unit, such as a support structure unit that supports the thermoelectric conversion unit above the recess, or an infrared reflection film on the upper part of the thermoelectric conversion unit. Provided. In this case, the structures of the reference pixel and the effective pixel are different, and the thermal characteristics of the effective pixel and the reference pixel cannot be made sufficiently close. For this reason, it is difficult to sufficiently reduce the streak noise.

Further, Patent Document 3 discloses an output signal of an optical black unit in which a photoelectric conversion unit is shielded in order to reduce vertical streak noise caused by variations in characteristics of elements constituting an AD conversion (Analog-to-Digital conversion) circuit. Is calculated for each column (vertical line) and added to or subtracted from the output signal of the effective pixel unit including the photoelectric conversion element. However, details of the difference between the optical black unit and the effective pixel unit are disclosed. Not disclosed.

The present invention has been made in view of the above circumstances, and in an infrared imaging device including an infrared image sensor including an effective pixel and a reference pixel, the pixel signal of the effective pixel is accurately corrected using the pixel signal of the reference pixel. An object of the present invention is to propose an infrared imaging device that reduces streak noise caused by variations in pixel signals in units of columns, and a signal correction method using such an infrared imaging device.

An infrared imaging device according to the present invention includes an optical system, an effective pixel portion in which effective pixels as infrared detection elements are arranged in a matrix in an effective region where infrared rays from the optical system are incident, and infrared rays from the optical system are shielded And a reference pixel unit in which one or more rows of reference pixels as infrared detection elements are arranged for each pixel column of the effective pixel unit, and the effective region and the reference region are provided in one detection region. , An infrared image sensor located on the imaging plane of the optical system, a shielding part that is located away from the reference pixel and shields the infrared rays from the optical system from entering the reference region, and a pixel signal detected by the infrared image sensor And a signal correction unit that performs correction processing, and the signal correction unit calculates a reference level value that represents the reference level of the pixel signal of the reference pixel, and a reference level calculator that corresponds to each pixel column of the effective pixels. For each pixel column of pixels, a muscle level calculation unit that calculates an average level of the pixel signal of the reference pixel belonging to the pixel column of the reference pixel and a pixel column to which the effective pixel belongs correspond to each effective pixel And a muscle level correction unit that performs a muscle level correction process for correcting a pixel signal of an effective pixel according to a difference between the muscle level value and the reference level value.

A signal correction method using an infrared imaging device according to the present invention includes an optical system, an effective pixel portion in which effective pixels as infrared detection elements are arranged in a matrix in an effective region where infrared rays from the optical system are incident, and an optical system. And a reference pixel unit in which one or more rows of reference pixels as infrared detection elements are arranged for each pixel column of the effective pixel unit in the reference region where the infrared ray is shielded, and one effective region and one reference region are detected. An infrared image sensor provided in a region and positioned on the imaging plane of the optical system, a shielding portion that is spaced apart from the reference pixel and shields incident infrared rays from the optical system to the reference region, and an infrared image sensor A signal correction method using an infrared imaging device including a signal correction unit that performs correction processing on a detected pixel signal, wherein the signal correction unit calculates a reference level value that represents a reference level of a pixel signal of a reference pixel. And a streak level for calculating a streak level value representing an average level of pixel signals of reference pixels belonging to the pixel column of the reference pixel for each pixel column of the reference pixel corresponding to each pixel column of the effective pixel. A streak level correcting step for performing a streak level correcting process for correcting the pixel signal of the effective pixel in accordance with the difference between the streak level value corresponding to the pixel column to which the effective pixel belongs and the reference level value for each effective pixel. And executing.

The above “pixel column” means a set of pixels connected to one signal line for reading out pixel signals.

The “shielding portion that shields the incidence of infrared rays from the optical system to the reference region” is located between the optical system and the infrared image sensor, and directly enters the infrared image sensor through the optical system. It means a shielding part that shields infrared rays. Arbitrary shapes, structures, and materials may be employed as the shielding portion as long as infrared rays incident on the infrared image sensor from the optical system can be shielded and a desired reference region can be secured.

Also, the phrase “the shielding portion is“ positioned away from the reference pixel ”” means that the shielding portion is located at a distance from the reference pixel, and the reference pixel and the shielding portion are not in direct contact.

The above-mentioned “stitch level correction” is a variation in pixel signal column units caused by a voltage rise or voltage drop due to variations in the elements connected to the pixel column in the infrared image sensor. It means a correction that reduces.

In the infrared imaging device according to the present invention, it is preferable that the reference pixel unit includes a plurality of rows of reference pixels with respect to each pixel column of the effective pixel unit.

In the infrared imaging device according to the present invention, the reference pixel unit may include a first reference pixel unit located on one end side in the column direction of the effective pixel unit.

The above "column direction" means the direction in which the pixel column extends.

When the infrared imaging device according to the present invention includes the first reference pixel unit, the streak level correction unit includes a streak level value and a reference level value corresponding to the pixel column to which the valid pixel belongs with respect to the pixel signal of each valid pixel. The muscle level can be corrected by adding or subtracting the difference between the two.

When the infrared imaging device according to the present invention includes the first reference pixel unit, the reference pixel unit is positioned opposite to the first reference pixel unit with the effective pixel unit sandwiched between the other end side in the column direction. You may further provide a pixel part.

When the infrared imaging device according to the present invention includes the first reference pixel unit and the second reference pixel unit, the muscle level calculation unit has a muscle level value for the first reference pixel unit for each pixel column of the effective pixels. A first muscle level value and a second muscle level value that is a muscle level value for the second reference pixel portion are respectively calculated, and the muscle level correction portion corresponds to the pixel column to which the effective pixel belongs for each effective pixel. Using the 1 streak level value and the 2nd streak level value, the shading correction amount for the effective pixel is interpolated and calculated, and the pixel signal of each effective pixel is corrected using the shading correction amount corresponding to the effective pixel. The shading correction may be performed by the following.

The above-mentioned “shading correction” is caused by non-uniformity of incident infrared rays generated on the detection surface of the infrared image sensor, such as a decrease in the amount of infrared rays at the periphery of the image circle due to the optical system, or by applying current to the circuit board. It means correction for reducing non-uniformity of infrared rays at each pixel position, which is caused by non-uniformity of infrared rays generated from a substrate, non-uniformity of external heat from a lens, a camera body, or the like.

When the infrared imaging device according to the present invention includes the first reference pixel unit and the second reference pixel unit, the third reference pixel unit located on one end side in the row direction of the effective pixel unit, and the row direction of the effective pixel unit On the other end side, a fourth reference pixel unit located opposite to the third reference pixel unit with the effective pixel unit interposed therebetween is provided, and the muscle level calculation unit outputs a third reference pixel unit for each pixel row of the effective pixel unit. A third muscle level value representing the average level of the pixel signal of each reference pixel belonging to the pixel and a fourth muscle level value representing the average level of the pixel signal of each reference pixel belonging to the fourth reference pixel portion, respectively. The level correction unit further interpolates the shading correction amount for the effective pixel unit with respect to each effective pixel by further using the third stripe level value and the fourth stripe level value corresponding to the pixel row to which the effective pixel belongs. Calculate the pixel signal of each effective pixel It may be carried out shading correction by correction using the shading correction amount corresponding to.

The above “row direction” means a direction in which a pixel row extends. The “pixel row” means a set of pixels connected to one scanning line for inputting a scanning signal.

In the present invention, the reference level value can be a value representing the average level of the pixel signals of the reference pixels belonging to each of the pixel columns of the reference pixels.

In the present invention, the standard level value can be an average value of the streak level values of the pixel columns of the respective reference pixels.

In the present invention, the stripe level value can be an average value of pixel signals of reference pixels belonging to the pixel column of the reference pixels.

In the infrared imaging apparatus according to the present invention, it is preferable that the apparatus main body is provided, and the shielding portion is thermally coupled to the apparatus main body.

The above “thermally coupled” means that the shielding unit and the apparatus main body of the infrared imaging device are directly or indirectly connected to each other and can exchange heat with each other. For example, a part of the shielding part and the apparatus main body may be brought into direct contact to enable heat exchange between the shielding part and the apparatus main body. The heat exchange between the shielding unit and the apparatus main body may be enabled via the.

When the shielding part is thermally coupled to the apparatus main body, the shielding part may include a plate-like member extending from the apparatus main body toward the optical axis.

When the shielding part is thermally coupled to the apparatus main body, the shielding part thermally couples the plate-like member having an opening, the plate-like member and the infrared device main body, and supports the plate-like member. May be provided.

According to the infrared imaging device of the present invention and the signal correction method using the infrared imaging device of the present invention, in the infrared imaging device including the infrared image sensor including the effective pixel and the reference pixel, the effective pixel is obtained using the pixel signal of the reference pixel. The pixel noise can be accurately corrected, and the streak noise caused by the variation of the pixel signal in units of columns can be suitably reduced.

1 is a schematic block diagram showing the configuration of an infrared imaging device according to an embodiment of the present invention. Schematic which shows the image sensor of 1st Embodiment. Cutaway end view showing the AA cross section of FIG. 2A The flowchart which shows the process by the signal correction part of 1st Embodiment. The figure for demonstrating the muscle level correction process of 1st Embodiment. The figure which shows the infrared image before the signal correction process by the 1st Embodiment, and the infrared image after a signal correction process Schematic which shows the image sensor of 2nd Embodiment. The flowchart which shows the process by the signal correction part of 2nd Embodiment. The figure for demonstrating the signal correction process of 2nd Embodiment. The figure which shows the infrared image before the signal correction process by the 2nd Embodiment, and the infrared image after a signal correction process Schematic which shows the image sensor of 3rd Embodiment. The figure for demonstrating the signal correction process of 3rd Embodiment. The figure which shows the infrared image before the signal correction process by the 3rd Embodiment, and the infrared image after a signal correction process Schematic which shows the internal structure of the infrared imaging device of 4th Embodiment Schematic which shows the internal structure of the infrared imaging device of the modification of 4th Embodiment

Hereinafter, embodiments of the present invention will be described with reference to the drawings. FIG. 1 is a schematic block diagram showing a configuration of an infrared imaging device 1 according to an embodiment of the present invention.

First, an infrared imaging apparatus 1 according to the present embodiment includes an optical system 2 for infrared imaging, and an image sensor 3 (which detects an infrared ray that has passed through the optical system 2 by positioning a detection surface on the imaging surface of the optical system 2). Infrared image sensor), shielding unit 9 that partially shields the incidence of infrared rays from the optical system 2 to the image sensor 3, and known analog signal processing including amplification processing on the pixel signal detected by the image sensor 3 An analog signal processing circuit 4 that performs analog signal processing, an AD conversion circuit 5 that performs analog-to-digital conversion on an analog signal-processed pixel signal, and a digital signal that has undergone AD conversion processing A signal correction unit 6 that is a digital signal processor (Digital Signal Processor) that performs various signal correction processes including a signal correction process according to an embodiment of the present invention on an infrared image, and a signal correction unit A storage unit 8 for storing various data used, and an output unit 7 for outputting an infrared image corrected by the signal correcting section 6 by. In addition, the infrared imaging device 1 includes a device main body (not shown in FIG. 1), and the above-described units are arranged in the imaging device main body. The infrared imaging device 1 also includes a control unit and a control mechanism (not shown) that perform control such as imaging of an infrared image.

The storage unit 8 stores various types of information used in the signal correction unit 6, infrared images subjected to various digital signal processing, and the like as necessary, and a volatile memory such as a DRAM (Dynamic Random Access Memory) and a flash. A non-volatile memory such as a memory is included. The output unit 7 outputs an infrared image subjected to various digital signal processing including the signal correction processing according to the embodiment of the present invention to the external storage unit (not shown) and the display unit by wireless or wired communication. Here, the external storage unit (not shown) is composed of various storage media such as a hard disk. For example, the external storage unit may be configured as a memory card type auxiliary storage device. The display unit (not shown) includes a known display such as a liquid crystal display, and displays the output infrared image. An external storage unit (not shown) stores the infrared image acquired from the output unit 7. Here, the storage unit 8 and the signal correction unit 6 are mounted on a single integrated circuit chip. However, the present invention is not limited to this, and the storage unit 8 and the signal correction unit 6 may be mounted on different integrated circuit chips.

The image sensor 3 is composed of an image sensor that is a solid-state imaging device in which a plurality of infrared detection elements are arranged in a matrix. FIG. 2A shows a schematic view of the image sensor 3 according to the first embodiment, and FIG. 2B shows a cut end view showing the AA cross section of FIG. 2A.

The image sensor 3 includes an effective pixel area 31 in which effective pixels as infrared detection elements are arranged in a matrix in an effective area AR in which infrared rays having passed through the optical system 2 are incident, and a reference area in which infrared rays having passed through the optical system 2 are not incident. The BR includes a reference pixel unit 32 in which one or more rows of reference pixels serving as infrared detection elements are arranged for each pixel column of the effective pixel unit 31. The effective pixel unit 31 detects an image signal corresponding to the infrared image, and the reference pixel unit 32 detects an image signal used for signal correction processing of the infrared image.

In each figure, the image sensor 3 schematically shows only its detection area. The image sensor 3 has one detection region, and a plurality of infrared detection elements having the same structure (common structure) are provided in the one detection region. Note that when the distance between the shielding unit 9 and the detection surface is increased, the outline of the region corresponding to the shielding unit 9 may be blurred in the detection region. In this case, a test photographing is performed so that an image of a reference subject having a uniform temperature appears in the entire imaging region, and a pixel signal of the infrared detection element in the detection region is obtained. Of the detection areas, an area where the infrared detection element can be regarded as representing the amount of infrared rays corresponding to the reference subject is an effective area, and an area where the infrared detection element can be regarded as having no contribution of the infrared amount of the reference subject is defined as a reference area. Good.

Here, the effective pixel or the reference pixel is an infrared detection element (infrared detector) capable of detecting infrared rays (wavelength 0.7 μm to 1 mm), and in particular, infrared detection capable of detecting far infrared rays (wavelength 8 to 15 μm). It is an element. For example, a bolometer-type or SOI (Silicon-on-Insulator) diode-type infrared detection element can be used as the infrared detection element used as the effective pixel or the reference pixel.

In the reference region BR where no infrared light is incident from the optical system 2, there is no variation in pixel signals (output level value of the reference pixel) caused by direct incidence of infrared light from the optical system 2, so that the reference pixels in the reference region BR The pixel signal in the unit 32 represents a pixel signal unrelated to the incident infrared ray from the optical system 2. In other words, the pixel signal of the reference pixel unit 32 is a pixel signal corresponding to the incident infrared ray irrelevant to the infrared ray from the optical system 2, that is, the incident infrared ray from the apparatus main body that changes in temperature according to the environmental temperature outside the apparatus, and It represents a pixel signal corresponding to the sum of incident infrared rays from internal elements such as the image sensor 3 itself.

Further, in each embodiment of the present invention, the effective region and the reference region are provided in one detection region, and are separated from the reference pixel, instead of a method in which the presence or absence of the heat-sensitive portion for detecting infrared rays or the structure of the heat-sensitive portion is different. The infrared detecting element having the same structure is used as an effective pixel having infrared incident or a reference pixel having no infrared incident by differentiating the presence or absence of infrared incident depending on the shielding portion 9 positioned at the same position. By using the reference pixel unit 32 including such an effective pixel and a reference pixel, the output level difference between the effective pixel and the reference pixel due to the difference in the thermal characteristics due to the difference in the structure of the thermal part and the structure of the thermal part. (Difference between the pixel signal of the effective pixel and the pixel signal of the reference pixel) does not occur. Therefore, it is possible to accurately calculate values used for signal correction such as an average level value and a muscle level value, which will be described later, using the reference pixels.

Here, the effective pixel unit 31 is provided in the range corresponding to the effective region AR, and the reference pixel unit 32 is provided in the range corresponding to the reference region BR. The reference pixel unit 32 is included in the reference region BR, and has an arbitrary position, shape, and range as long as it has one or more rows of reference pixels connected to the signal lines of each pixel column of the effective pixel unit 31. It can be a number. For example, the effective pixel unit 31 and the reference pixel unit 32 do not have to be directly adjacent as long as they are provided at a sufficiently close distance.

Further, in the first embodiment, the image sensor 3 is a rectangular image sensor, and the reference pixel unit 32 is one end side in the column direction in which the pixel column of the effective pixel unit 31 extends (lower end in FIG. 2A). The reference region BR is provided, and the reference pixel portion 32 is provided in one end side (lower end portion in FIG. 2A) in the column direction within the reference region BR. Thereby, while ensuring the effective pixel part 31, the reference pixel required for a stripe level correction process can be ensured with respect to an effective pixel by the reference pixel part 32 (1st reference pixel part 32A). The position information of the effective pixel unit 31 and the reference pixel unit 32 is stored in the storage unit 8 and is referred to by the signal correction unit 6 as appropriate.

Here, in the reference pixel section 32, a plurality of reference pixels are arranged for each pixel column of the effective pixel section. In other words, for each pixel column of the effective pixel unit, a pixel column of a reference pixel including a plurality (a plurality of rows) of reference pixels connected to a signal line of the pixel column of the effective pixel unit is provided. In such a case, the influence of the individual difference of the reference pixel in each column is suppressed and the pixel signal of the reference pixel is more accurately detected than when only one row is arranged for each pixel column. Parameters used for signal correction processing such as level values can be calculated, which is advantageous for improving the accuracy of signal correction processing. For example, the reference pixel unit 32 preferably includes four or more rows of reference pixels for each pixel column. Further, from the viewpoint of securing the effective pixel portion 31, the ratio of the reference pixel portion 32 in the entire detection surface of the image sensor 3 is preferably 20% or less, more preferably 10% or less, and 5% or less. More preferably.

As shown in FIGS. 2A and 2B, the shielding unit 9 is spaced from the reference pixel and is positioned between the detection surface of the image sensor 3 and the optical system 2. By shielding a part of the infrared rays incident on the image sensor 3, an effective area AR where the infrared rays are incident on the image sensor 3 and a reference area BR which is an area where the infrared rays are not incident are provided. In the following drawings, each component is shown in a different scale from the actual one for the sake of explanation.

Here, as an example, the shielding portion 9 is provided on the lower end side of the image sensor 3. As shown in FIG. 2B, the shielding part 9 extends in a direction approaching the optical axis from the support part 9B erected on the substrate BP, and receives infrared rays from the optical system 2 at the tip part on the optical axis side. The shield 9A is configured to be shielded. Here, the shield 9A is a plate-like member. The shield 9A is located between the detection surface of the image sensor 3 and the optical system 2, and may be configured with an arbitrary shape, structure, and material as long as a desired reference region BR can be formed. For example, the shield 9A preferably employs a material or structure that reduces 90% or more of the incident infrared light, and more preferably uses a material or structure that reduces 95% or more. Further, as the material of the shield 9A of the shield part 9, a material such as aluminum (aluminum alloy) or copper can be adopted. Further, the support portion 9B may be configured as one member with the shield 9A, may be configured as a separate member, and may have any configuration that can support the shield 9A at a desired position.

Referring back to FIG. As shown in FIG. 1, the signal correction unit 6 includes a reference level calculation unit 61 that calculates a reference level value indicating a reference level of a pixel signal of a reference pixel, and pixels of reference pixels corresponding to each pixel column of effective pixels. A streak level calculation unit 62 that calculates a streak level value representing an average level of pixel signals of reference pixels belonging to the pixel column of the reference pixel for each column, and a streak corresponding to the pixel column to which the valid pixel belongs for each valid pixel. A streak level correction unit 63 that performs a streak level correction process for correcting the pixel signal of the effective pixel according to the difference between the level value and the reference level value is provided.

FIG. 3 is a flowchart showing a processing flow of the signal correction unit 6 of the first embodiment, FIG. 4 is a diagram for explaining the signal correction processing of the first embodiment, and FIG. FIG. 3 is a diagram illustrating an infrared image I before signal correction processing and a corrected image Ip that is an infrared image after signal correction processing according to the first embodiment.

First, the infrared image I before the signal correction processing is shown on the upper side of FIG. As shown in the infrared image I on the upper side of FIG. 5, in the pixel column connected to the signal line (data line), when the pixel signal varies in a column unit, the pixel signal passes through the optical system 2 in a certain column. Since this becomes relatively large with respect to the amount of incident infrared light, white streak noise occurs, and in some other column, the pixel signal becomes relatively small with respect to the amount of incident infrared light that has passed through the optical system 2. Therefore, black streak noise is generated. The signal correction unit 6 performs the streak level correction to reduce the variation of the pixel signal in the column unit, thereby reducing the streak noise that appears in the infrared image on the upper side of FIG.

Hereinafter, the processing flow of the signal correction unit will be described in detail with reference to FIG. 3 and FIG. In the description of the image sensor 3 in each of the following embodiments, the row direction that is the direction in which the pixel column of the effective pixel portion 31 extends is the x direction, and the column direction that is the direction in which the pixel row extends is the y direction. The position of each effective pixel 31 is expressed in the xy coordinate system. The x coordinate of the endmost column on the one end side in the row direction is X1, the x coordinate of the endmost column on the other end side is X2, the y coordinate of the endmost row on the one end side in the column direction is Y1, and the x coordinate on the other end side in the column direction is Let y2 be the y coordinate of the endmost row. Note that X1 <X2 and Y1 <Y2. Further, it is assumed that an image signal representing the infrared image I detected by the image sensor 3 is stored in the storage unit 8 prior to the processing of the signal correction unit 6.

When acquiring the pixel signal representing the infrared image I from the storage unit 8, the muscle level calculation unit 62 first selects the pixel signal corresponding to the reference pixel unit 32A and the effective pixel unit 31 with reference to the storage unit 8, The streak level value Za (x) representing the average level of the pixel signals of the reference pixels belonging to the pixel row of the reference pixels corresponding to the coordinate x (hereinafter referred to as the x row) is calculated using the following equation (1) (S01). ). Here, Expression (1) is calculated as a numerical integration of the range of the reference pixel unit 32 in the Y direction. The muscle level calculation unit 62 calculates the muscle level value Za (x) for each x column belonging to the section (X1 ≦ x ≦ X2) as shown in FIG.
Za (x) = ∫Z (x, y) dy / ∫dy… (1)
In formula (1),
Z (x, y): Pixel signal of a reference pixel at coordinates (x, y) (see A in FIG. 4)
Za (x): Muscle level value in the x column.

As described above, when the streak level value Za (x) is an average value of pixel signals of reference pixels belonging to the x column, the streak level value Za (x) can be calculated easily and accurately. . Note that the present invention is not limited to the above example, and a value representing the average level of reference pixels in various x columns may be adopted as the muscle level value. For example, the median value of the pixel signal of the reference pixel in the corresponding pixel column can be used as the muscle level value.

Next, the standard level calculation unit 61 calculates the addition average value of the pixel signals of all the reference pixels belonging to the reference pixel unit 32 as the standard level value Zave (S02). Here, the reference level calculation unit 61 uses the muscle level value calculated by Equation (1) and uses Equation (2) to calculate a value obtained by adding and averaging the muscle level values of each column of the reference pixel unit 32. Calculated as level value Zave. Equation (2) is calculated as a numerical integration of the interval (X1 ≦ x ≦ X2).
Zave = ∫Za (x) dx / ∫dx (2)
In formula (2),
Za (x): Muscle level value in column x Zave: Reference level value.

As described above, it is preferable that the standard level value is a value representing the average level of the pixel signals of the reference pixels respectively belonging to each of the pixel columns of the reference pixel unit 32. In this case, the reference level value that is the reference value of the plurality of reference pixels belonging to the reference pixel unit 32 can be calculated easily and accurately.

Further, as described above, when the reference level value is an average value of pixel signals of a plurality of reference pixels belonging to the reference pixel unit 32 (herein, all reference pixels belonging to the reference pixel unit 32), The effect can be enhanced. When the value obtained by adding and averaging the muscle level values of each column of the reference pixel unit 32 is calculated as the standard level value Zave using the equation (2) using the muscle level value calculated by the equation (1), The reference level value can be calculated more easily by using the muscle level value of Equation (1).

In the above description, the muscle level value is first calculated as the average value of the pixel signals of the reference pixels for each muscle, and then the average value of the muscle level value is calculated as the reference level value. Is not limited. The average value may be calculated by adding the pixel signals of the entire reference region and dividing this by the total number of reference pixels. In addition, as a reference level value, a reference value of the pixel signal of the reference pixel of the reference pixel unit 32 can be adopted. For example, a representative value of a pixel signal of a reference pixel belonging to the reference pixel unit 32 such as an average value or a median value of all reference pixels of the reference pixel unit 32 may be used as the standard level value.

Next, for each effective pixel, the muscle level correction unit 63 determines the difference L (x) between the muscle level value Za (x) corresponding to the pixel column to which the effective pixel belongs and the reference level value Zave (hereinafter referred to as a muscle level difference). The pixel signal of the effective pixel is corrected (S03). Here, the muscle level correction unit 63 uses the equation (4) to correct the muscle level difference L (x) by subtracting it from the pixel signal orig_image (x, y) of the effective pixel at the coordinates (x, y). The streak level correction is performed by performing offset correction for the subsequent pixel signal new_image1 (x, y) on the pixel signals of all effective pixels.
L (x) = Za (x)-Zave (3)
new_image1 (x, y) = orig_image (x, y)-L (x) (4)
In equations (3) and (4),
Za (x): Muscle level value in column x Zave: Reference level value L (x): Difference in muscle level in column x orig_image (x, y): Pixel signal new_image1 (x, y) of coordinate (x, y) y): A pixel signal after correction of the streak level of the effective pixel at the coordinates (x, y).

Note that the lower part of FIG. 4 shows the muscle level difference L (x) calculated for each x pixel column in the section (X1 ≦ x ≦ X2). The horizontal axis represents the x-axis, and the vertical axis represents the muscle level difference L (x) indicating the difference from the muscle level value Z (x) from the reference level value Zave. Since the reference pixel indicates a pixel signal not caused by the incident infrared ray from the optical system 2, the streak level value Z (x) is a pixel signal for each column in a state where the infrared ray amount of the incident infrared ray from the optical system 2 is zero. It can be considered that represents. Further, it is considered that the streak noise appears more prominently as the absolute value of the streak level difference L (x) increases.

According to the present embodiment, the reference level calculation unit 61 calculates a reference level value indicating the reference level of the pixel signal of the reference pixel, and the muscle level calculation unit 62 calculates the pixel of the reference pixel corresponding to each pixel column of the effective pixels. For each column, a streak level value indicating the average level of the pixel signal of the reference pixel belonging to the pixel column of the reference pixel is calculated, and the streak level correcting unit 63 performs, for each effective pixel, the streak corresponding to the pixel column to which the effective pixel belongs. In accordance with the difference between the level value and the reference level value, the pixel signal of the effective pixel is corrected to perform the streak level correction process. Therefore, the variation of the pixel signal in the column unit is reduced, and the streak noise is preferably reduced. be able to.

More specifically, the muscle level correction unit 63 adds / subtracts the difference between the muscle level value corresponding to the pixel column to which the effective pixel belongs and the reference level value to / from the pixel signal of each effective pixel. Correction is being performed. For this reason, from the pixel signal of each effective pixel, the pixel signal that contributes to the streak noise of each column to which the effective pixel belongs is preferably reduced, and the corrected image Ip with accurate line level correction can be obtained. it can.

In particular, in the present embodiment, the reference pixel unit 32 and the effective pixel unit 31 are provided in one detection region, and the reference pixel unit 32 receives incident infrared rays from the optical system 2 by the shielding unit 9 that is positioned away from the reference pixel. Shielded. This makes it possible to use the infrared detection element having the same structure as an effective pixel and a reference pixel by using the image sensor 3 in which the infrared detection element having the same structure (common structure) is provided in one detection region. For this reason, it is possible to detect an infrared ray not related to the incident infrared ray from the optical system 2 using the reference pixel of the reference pixel unit 32 without causing a difference in incident infrared ray amount due to a difference in structure between the reference pixel and the effective pixel. it can. Therefore, the signal correction process can be performed with very high accuracy. Further, in the infrared imaging device 1 equipped with the infrared image sensor having a lower pixel signal output level than the photoelectric conversion type image sensor, the above effect is greatly beneficial for improving the image quality of the infrared image. Further, according to the present embodiment, incident infrared rays from the optical system 2 are shielded by the shielding portion 9 provided in a portion other than the sensor (for example, a substrate or an infrared imaging device main body), and the infrared detection element is used as a reference pixel. For this reason, there is no difference in the amount of incident infrared rays due to the difference in the structure of the reference pixel and the effective pixel compared to the method of providing the reference pixel by mounting the light shielding structure on the sensor chip described in References 2 and 3. The signal correction processing can be performed with high accuracy. Based on Documents 1 to 3, the configuration in which the incident infrared rays from the optical system 2 are shielded by the shielding portion 9 provided in a portion other than the sensor as described above and the infrared detection element is used as a reference pixel can be easily recalled. However, it is considered that a sufficient correction effect cannot be obtained according to the prior art that does not have such a configuration.

FIG. 5 shows a corrected image Ip which is an infrared image after the signal correction processing of the present embodiment. It can be seen that the streak noise is suitably removed in the corrected image Ip.

The reference level calculation process (process corresponding to S02 in FIG. 3) is not limited to the above example as long as it is performed prior to the muscle level correction process (process corresponding to S03 in FIG. 3). The muscle level calculation processing (processing corresponding to S01 in FIG. 3) may be performed prior to the muscle level correction processing (processing corresponding to S03 in FIG. 3). For example, the reference level calculation process and the muscle level calculation process may be performed in a reversed order or may be performed in parallel.

It should be noted that a corrected image that is a corrected infrared image subjected to signal correction processing is stored in the storage unit 8 and is appropriately output by the output unit 7 to an external storage unit and a display unit (not shown). The corrected image may be appropriately subjected to other necessary correction processing before or after the signal correction processing by the digital signal processing device of the infrared imaging device 1.

In the present embodiment, the signal correction unit 6 performs a reference level calculation process and a muscle level calculation process as shown in S01 to S02, respectively, every time an infrared image is captured, and the reference level value Zave and each x The muscle level value Za (x) of the column is calculated (updated) for each photographing. For this reason, it is possible to accurately perform the muscle level correction process in response to a variation in the variation of the pixel signal that changes nonlinearly in time series.

When shooting a plurality of infrared images such as moving images, the signal correction unit 6 calculates the reference level value Zave calculated by the processes shown in S01 to S02 and the streak level value Za (x) of each x column. Such a correction parameter may be stored in the storage unit 8, and the stored correction parameter may be used for the streak level correction processing of a plurality of infrared images. In this case, it is preferable that the signal correction unit 6 appropriately updates the correction parameters at intervals of time by the processes shown in S01 to S02. The update of the muscle level difference L (x) using Expression (3) may be performed at an arbitrary timing as long as it is performed prior to the muscle level correction process using Expression (4). The time interval for updating the correction parameter can be any time interval required, and can be a constant time interval or a different time interval depending on the specifications and requirements of the apparatus. For example, if the amount of infrared rays due to external factors unrelated to the subject such as the substrate temperature (especially the sensor temperature) and the housing temperature of the infrared imaging device fluctuates frequently, set the above time interval short and external If the amount of infrared rays due to the factor does not vary much, it is conceivable to set the time interval longer. As an example, the time interval can be 0.1 seconds or more and 60 seconds or less, and can be 1 second or more and 10 seconds or less.

Hereinafter, the processing of the image sensor 3 and the signal correction unit 6 and the shielding unit 9 according to the second embodiment will be described with reference to FIGS. In each embodiment, the same components as those in the first embodiment are denoted by the same reference numerals, and description thereof is omitted. FIG. 6 is a schematic diagram illustrating the image sensor 3 according to the second embodiment, FIG. 7 is a flowchart illustrating processing performed by the signal correction unit 6 according to the second embodiment, and FIG. 8 illustrates the second embodiment. FIG. 9 is a diagram showing an infrared image I before the signal correction processing and a post-correction image Ip which is an infrared image after the signal correction processing according to the second embodiment.

In the second embodiment, the shielding unit 9 is configured to provide the reference region BR at both ends of the image sensor 3 in the column direction, and the image sensor 3 includes the first reference pixel unit 32A in the reference region BR. And the second reference pixel unit 32B, and the point that the signal correction unit 6 performs signal correction processing using the image signals of the first reference pixel unit 32A and the second reference pixel unit 32B is different from the first embodiment. . Hereinafter, the processing of the image sensor 3 and the signal correction unit 6 will be described with a focus on differences from the first embodiment, and description of other common parts will be omitted.

In the second embodiment, the shielding unit 9 is an example in which the same shielding unit as that in the first embodiment is provided not only on the lower end side of the image sensor 3 but also on the upper end side of the image sensor 3. The shielding unit 9 is located between the optical system 2 and the image sensor 3, partially shields the infrared rays that have passed through the optical system 2, and sets the reference region BR outside the both end portions in the column direction of the image sensor 3. A shield to be provided is provided. Here, the shields are two plate-like members that are respectively supported by support portions (not shown) standing from a substrate (not shown) and extend from the support portions toward the optical axis. The image sensor 3 includes a first reference pixel unit 32A and a second reference pixel unit 32B having a shape corresponding to the reference region BR. The image sensor 3 includes an effective pixel unit 31 in an effective area AR that is an area obtained by removing the reference area BR from the detection surface included in the imaging area IR.

The reference pixel unit 32 includes a first reference pixel unit 32A on one end side in the column direction, and is positioned opposite to the first reference pixel unit 32A with the effective pixel unit 31 sandwiched on the other end side in the column direction. 2 reference pixel section 32B is provided.

The signal correction unit 6 of the second embodiment has both a streak noise correction function and a shading correction function, and further performs shading correction using the pixel signals of the first reference pixel unit 32A and the second reference pixel unit 32B. Is. The processing of the signal correction unit 6 of the second embodiment will be described in detail using FIG.

The muscle level calculation unit 62 calculates the muscle level value Za (x) for each of the x columns belonging to the section (X1 ≦ x ≦ X2) with respect to the first reference pixel unit 32A using the above equation (1). The calculated muscle level value Za (x) is set as the muscle level value Za2 (x) x (first muscle level value) of the first reference pixel unit 32A. Similarly, the muscle level calculation unit 62 uses the equation (1) to calculate the muscle level value Za (x) for each x column belonging to the section (X1 ≦ x ≦ X2) with respect to the second reference pixel unit 32B. The calculated muscle level value Za (x) is set as the muscle level value Za1 (x) (second muscle level value) of the first reference pixel unit 32B (S11).

In addition, as in the first embodiment, the reference level calculation unit 61 inserts the streak level value Za2 (x) of the first reference pixel unit 32A into Za (x) in the above equation (2) to obtain the equation Numerical integration of the section (X1 ≦ x ≦ X2) is performed according to (2), and the calculated Zave (= ∫Za2 (x) dx / ∫dx) is calculated as the reference level value Za2_ave of the first reference pixel unit 32A ( S12).

Further, the streak level correction unit 63 applies the reference level value Za2_ave of the first reference pixel unit 32A and the first reference pixel unit 32A to the pixel signal of each effective pixel in the same manner as in the first embodiment, using Expression (3). The muscle level value Za2 (x) is inserted, and the muscle level difference L (x) (= Za2 (x) −Za2_ave) is calculated. Then, the offset correction is performed by subtracting the muscle level difference L (x) from the pixel signal orig_image (x, y) of the effective pixel at the coordinates (x, y) to obtain a corrected pixel signal new_image1 (x, y). The streak level correction is performed by performing the processing on the pixel signals of all the effective pixels (S13).

Subsequently, the muscle level correction unit 63 performs shading correction (S14).

Here, the infrared image I before the signal correction processing is shown on the upper side of FIG. In the infrared image I shown in FIG. 9, the above-described streak noise appears, and image unevenness that gradually darkens as it approaches the upper side is generated. In the infrared image, the pixel signal of the effective pixel is non-uniform due to various causes, and the pixel signal may be uneven for each pixel position as shown in the infrared image I in FIG. The cause of the non-uniformity of the pixel signals of such effective pixels is, for example, incident infrared rays generated on the imaging surface of the two-dimensional infrared detection element, such as a decrease in the amount of infrared rays at the periphery of the image circle due to the optical system. Or non-uniformity of pixel signals of effective pixels caused by non-uniformity of infrared rays generated from the circuit board by energizing the circuit board, non-uniformity of infrared rays from the outside of the lens or camera body, etc. (pixel position) Non-uniformity of pixel signals). The signal correction unit 6 uses the pixel signals of the reference pixel unit 32A and the reference pixel unit 32B to perform shading correction for reducing the unevenness of the pixel signal for each pixel position as described above.

Here, the shading correction amount S (x, y) corresponding to the effective pixel at the coordinates (x, y) is calculated using the muscle level value Za1 (x) and the muscle level value Za2 (x) according to the equation (6). Is obtained by linear interpolation. For the calculation of the shading correction amount S (x, y), any interpolation method for interpolating two values may be applied. Instead of linear interpolation, interpolation using a specific curve such as a spline curve may be applied. May be performed. The muscle level correction unit 63 applies the pixel signal new_image1 (x) of each effective pixel to the effective coordinate of each coordinate (x, y) in the section (X1 ≦ x ≦ X2, Y1 ≦ y ≦ Y2) according to Expression (5). , y), shading correction is performed by offset correction of the shading correction amount S (x, y). Then, an infrared image after shading correction is acquired based on the corrected pixel signal new_image2 (x, y).
new_image2 (x, y) = new_image1 (x, y)-S (x, y) (5)
S (x, y) = {(Za2 (x)-Za1 (x)) / (Y2-Y1)} * (y-Y1)… (6)
In the equations (5) and (6),
Za1 (x): the streak level value of the reference pixel in the x column of the second reference pixel unit 32B (see A1 in FIG. 8) Za2 (x): the x column of the first reference pixel unit 32A (see A2 in FIG. 8) Reference pixel stripe level value new_image1 (x, y): pixel signal after correction of the stripe level at coordinates (x, y) new_image2 (x, y): pixel signal after the shading correction of coordinates (x, y).

According to the second embodiment, the muscle level calculation unit 62 calculates the first muscle level value for the first reference pixel unit 32A and the second muscle level value for the second reference pixel unit 32B for each pixel column. For each effective pixel, using the first muscle level value and the second muscle level value corresponding to the pixel column to which the effective pixel belongs, the shading correction amount for each effective pixel is interpolated and calculated, The shading correction is performed by correcting the pixel signal of the effective pixel using the shading correction amount corresponding to the effective pixel. For this reason, nonuniformity of the pixel signal in the vertical direction can be reduced, and unevenness of the image can be suitably reduced. A corrected image Ip which is an infrared image after signal correction is shown at the bottom of FIG. It can be seen that by performing the line level correction and the shading correction in the column direction with high accuracy, the line noise and the image unevenness are suitably reduced in the corrected image Ip.

In addition, since the reference pixel unit 32 includes the second reference pixel unit 32B that is positioned opposite to the first reference pixel unit 32A with the effective pixel unit 31 interposed therebetween on the other end side in the column direction, the effective pixel is preferably used. The shading correction can be performed by reflecting the image unevenness in the column direction of the unit 31, and a corrected image in which the image unevenness is suitably reduced can be provided. Further, since the rectangular first reference pixel portion 32A is adjacent to the rectangular effective pixel portion 31 in the column direction, and the rectangular second reference pixel portion 32B is adjacent to the rectangular effective pixel portion 31 in the column direction. The above effect can be enhanced by preferably reflecting the image unevenness in the column direction of the effective pixel portion 31. If the effective pixel portion 31 and the first reference pixel portion 32A (or the second reference pixel portion 32B) are provided at a distance sufficiently close to the column direction, the same effect can be obtained even if they are not directly adjacent to each other. .

Further, in the second embodiment, the muscle level correction unit 63 calculates the muscle level difference L (x) (= Za2 (x) −Za2_ave) using the pixel signal of the first reference pixel unit 32A in Expression (4). For example, the muscle level correction unit 63 uses a pixel signal of the second reference pixel unit 32B instead of the first reference pixel unit 32A. The muscle level correction may be performed by substituting L (x) (= Za1 (x) −Za1_ave) into Equation (4).

Further, for example, the muscle level correction unit 63 uses the first muscle level difference L2 (x) (= Za2 (x) −Za2_ave) using the pixel signal of the first reference pixel unit 32A and the pixels of the second reference pixel unit 32B. The average value with the second muscle level difference L1 (x) (= Za1 (x) -Za1_ave) using the signal is the muscle level difference L (x) (= (L1 (x) + L2 ( The muscle level correction may be performed by substituting into equation (4) as x)) / 2). As described above, the reference pixel unit 32 includes the second reference pixel unit 32B positioned opposite to the first reference pixel unit 32A with the effective pixel unit 31 interposed therebetween on the other end side in the column direction. It is advantageous to employ a muscle level correction of. In the above case, the reference level value Za1_ave of the second reference pixel unit 32B is Z1_ave (= ∫Za1 (x) by the above equation (2) using the muscle level value Za1 (x) of the first reference pixel unit 32B. dx / ∫dx).

The reference level calculation process (process corresponding to S12 in FIG. 7) is not limited to the above example as long as it is performed prior to the muscle level correction process (process corresponding to S13 in FIG. 7). The muscle level calculation processing (processing corresponding to S11 in FIG. 7) is performed prior to the muscle level correction processing (processing corresponding to S13 in FIG. 7) and the shading correction processing (processing corresponding to S14 in FIG. 7). If it is. In addition, the muscle level correction process and the shading correction process may be performed in a reversed order, or may be performed in parallel. In addition, the signal correction unit 6 includes a reference level calculation process, a muscle level calculation process, a muscle level correction process, and a shading correction as shown in S11 to S14 for each infrared image capturing. It is preferable to perform signal correction processing according to the embodiment.

In the second and third embodiments, the signal correction unit 6 determines the reference level value Zave and the streak level value Za (x) of each x column when shooting a plurality of infrared images such as moving images. A correction parameter such as a shading correction amount at each position may be stored in the storage unit 8, and the stored correction parameter may be used for a muscle level correction process and a shading correction process for a plurality of infrared images. In this case, it is preferable that the signal correction unit 6 appropriately updates the correction parameters at time intervals. It should be noted that the time interval for updating the correction parameters can be any required time interval, and may be a constant time interval or a different time interval depending on the specifications and requirements of the apparatus. Good. For example, if the amount of infrared rays due to external factors unrelated to the subject such as the substrate temperature (especially the sensor temperature) and the housing temperature of the infrared imaging device fluctuates frequently, set the above time interval short and external If the amount of infrared rays due to the factor does not vary much, it is conceivable to set the time interval longer. As an example, the fixed time interval for updating may be set to 0.1 second to 60 seconds, and may be set to 1 second to 10 seconds.

Hereinafter, the processing of the image sensor 3 and the signal correction unit 6 and the shielding unit 9 according to the third embodiment will be described with reference to FIGS. FIG. 10 is a schematic diagram showing an image sensor 3 according to the third embodiment, FIG. 11 is a diagram for explaining signal correction processing according to the third embodiment, and FIG. 12 is according to the third embodiment. It is a figure which shows the after-correction image Ip which is the infrared image I before a signal correction process, and the infrared image after a signal correction process.

The third embodiment is an application example of the second embodiment, and the shielding unit 9 is configured to provide reference regions BR in both end portions in the row direction in addition to both end portions in the column direction of the image sensor 3. The image sensor 3 includes a first reference pixel unit 32A, a second reference pixel unit 32B, a third reference pixel unit 32C, and a fourth reference pixel unit 32D in the reference region BR, and a signal correction unit. 6 differs from the second embodiment in that signal correction processing is performed using image signals of the first reference pixel unit 32A, the second reference pixel unit 32B, the third reference pixel unit 32C, and the fourth reference pixel unit 32D. Hereinafter, the processing of the shielding unit 9, the image sensor 3, and the signal correction unit 6 will be described with a focus on differences from the second embodiment, and description of other common parts will be omitted.

In the third embodiment, the shielding unit 9 is positioned between the optical system 2 and the image sensor 3 as shown in FIG. A shielding body provided with a reference region BR at the peripheral edge of Here, the shield is a single plate-like member that is supported by a plurality of support parts (not shown) standing from a substrate (not shown) as in the first embodiment, and extends from the support part toward the optical axis. . The shield has an opening 9C positioned so as to include the optical axis, whereby the image sensor 3 has an effective area AR in which infrared light that has passed through the optical system 2 passes through the opening 9C and enters the detection surface, and A reference area BR, which is an area obtained by removing the effective area AR from the detection surface included in the imaging area IR, is provided. As shown in FIG. 10, the reference area BR is a frame-shaped area around the effective area AR.

11, the reference pixel unit 32 includes a first reference pixel unit 32A on one end side in the column direction, and the first reference pixel unit 32A sandwiching the effective pixel unit 31 on the other end side in the column direction. Is provided with a second reference pixel portion 32B located opposite to. The reference pixel unit 32 includes a third reference pixel unit 32C positioned on one end side in the row direction, and a third reference pixel unit 32C sandwiching the effective pixel unit 31 on the other end side in the row direction of the effective pixel unit. A fourth reference pixel portion 32D is provided so as to be opposed to the fourth reference pixel portion 32D.

The signal correction unit 6 of the third embodiment has both a streak noise correction function and a shading correction function. The second embodiment is different from the first reference pixel unit 32A, the second reference pixel unit 32B, and the third reference. The difference is that shading correction is performed using the pixel signals of the pixel portion 32C and the fourth reference pixel portion 32D. The flow of steps S11 to S14 of the processing of the signal correction unit 6 of the third embodiment is the same as that of the second embodiment, but the details of each step are different, so the details of each step will be described below. The processing of the signal correction unit 6 of the third embodiment will be described in detail using FIG.

In step S11, the muscle level calculation unit 62 of the third embodiment performs, for each x column belonging to the section (X1 ≦ x ≦ X2), the first reference pixel unit 32A, as in the second embodiment. The muscle level value Za (x) is calculated using the above-described equation (1), and the calculated muscle level value Za (x) is used as the muscle level value Za2 (x) (first muscle of the first reference pixel unit 32A). Level value). Similarly, the muscle level calculation unit 62 uses the equation (1) to calculate the muscle level value Za (x) for each x column belonging to the section (X1 ≦ x ≦ X2) with respect to the second reference pixel unit 32B. The calculated muscle level value Za (x) is set as the muscle level value Za1 (x) (second muscle level value) of the second reference pixel unit 32B.

Further, the muscle level calculation unit 62 uses the following expression (7) for the third reference pixel unit 32C for each y row belonging to the section (Y1 ≦ y ≦ Y2). The muscle level value Zb (y) is calculated by performing numerical integration in the direction range, and the calculated muscle level value Zb (y) is used as the muscle level value Zb1 (y) (third muscle level) of the third reference pixel unit 32C. Value). Similarly, the muscle level calculation unit 62 uses the following expression (7) for the fourth reference pixel unit 32D for each y row belonging to the section (Y1 ≦ y ≦ Y2). The muscle level value Zb (y) is calculated by performing numerical integration in the range in the x direction, and the calculated muscle level value Zb (y) is used as the muscle level value Zb2 (y) (fourth muscle) of the fourth reference pixel unit 32D. Level value). Thus, the third muscle level value is a value representing the average level of the pixel signal of each reference pixel belonging to the third reference pixel unit 32C (here, the addition average value), and the fourth muscle level value is the fourth reference pixel unit. It is calculated as a value (here, an added average value) representing the average level of pixel signals of each reference pixel belonging to 32D.
Zb (y) = ∫Z (x, y) dx / ∫dx (7)
In equation (7),
Z (x, y): pixel signal of the reference pixel in the y row of the third reference pixel unit 32C (see B1 in FIG. 11) or pixel signal of the reference pixel in the y row of the fourth reference pixel unit 32D (in FIG. 11) B2).

In step S12, the reference level calculation unit 61 according to the third embodiment uses the line level value Za2 (x) of the first reference pixel unit 32A as Za (x) in Expression (2), as in the second embodiment. And the numerical integration of the section (X1 ≦ x ≦ X2) is performed by the equation (2), and the calculated Zave is used as the reference level value Za2_ave (= ∫Za2 (x) dx / ∫) of the first reference pixel unit 32A. dx).

In step S <b> 13, the streak level correction unit 63 of the third embodiment calculates the reference level value of the first reference pixel unit 32 </ b> A according to Equation (3) in the same manner as in the first embodiment for the pixel signal of each effective pixel. By inserting Za2_ave and the muscle level value Za2 (x) of the first reference pixel unit 32A, the muscle level difference L (x) (= Za2 (x) −Za2_ave) is calculated. Then, for the pixel signals of all the effective pixels in the section (X1 ≦ x ≦ X2, Y1 ≦ y ≦ Y2), the muscle level difference L (x) is expressed by coordinates (x, y) using Equation (4). ) Is subtracted from the pixel signal orig_image (x, y) of the effective pixel, and offset correction is performed to obtain a corrected pixel signal new_image1 (x, y) (S13).

Subsequently, in step S14, the muscle level correction unit 63 of the third embodiment performs the following shading correction.

Here, the infrared image I before the signal correction processing is shown on the upper side of FIG. In the infrared image I shown in FIG. 12, the above-described streak noise appears, and image unevenness that gradually becomes darker as it approaches the upper left corner is generated. In the third embodiment, when the streak level correction unit 63 causes such non-uniform pixel signals of the effective pixels in both the column direction and the row direction, the first reference pixel unit 32A and the second reference pixel Using the pixel signals of the third reference pixel unit 32C and the fourth reference pixel unit 32D as well as the unit 32B, shading correction is performed to reduce pixel signal unevenness at each pixel position.

Here, using equations (8) to (10), shading at coordinates (x, y) based on the four muscle level values Za1 (x), Za2 (x), Zb1 (y), Zb2 (y) The correction amount S (x, y) is calculated by linear interpolation, and using the equation (11), the pixel signal new_image1 (x, y) of the effective pixel at the coordinates (x, y) after the muscle level correction processing in S13. Then, the shading correction amount S (x, y) is offset-corrected to calculate a corrected pixel signal new_image3 (x, y). In order to calculate the shading correction amount S (x, y), an arbitrary interpolation method for interpolating four values may be applied. Instead of linear interpolation, interpolation using a specific curved surface such as spline interpolation may be performed. You may go. The streak level correcting unit 63 applies the pixel signals of the effective pixels corresponding to the coordinates (x, y) in the section (X1 ≦ x ≦ X2, Y1 <x <Y2) according to equations (8) to (11). Shading correction processing is performed by correcting the pixel signal new_image1 (x, y) of each effective pixel. Then, an infrared image after shading correction is acquired based on the corrected pixel signal new_image3 (x, y).
rate1 (y) = Zb1 (y) / [{Zb1 (Y1) + Zb1 (Y2) + Zb2 (Y1) + Zb2 (Y2)} / 4]… (8)
rate2 (y) = Zb2 (y) / [{Zb1 (Y1) + Zb1 (Y2) + Zb2 (Y1) + Zb2 (Y2)} / 4]… (9)
S (x, y) = {(Za2 (x) -Za1 (x)) / (Y2-Y1)} * (y-Y1) * {rate1 (y) * (X2-x) + rate2 (y) * (x-X1)} / (X2-X1) (10)
new_image3 (x, y) = new_image1 (x, y)-S (x, y) (11)
In the formulas (8) to (10),
Za1 (x): the streak level value of the reference pixel in the x column of the second reference pixel unit 32B (see A1 in FIG. 11) Za2 (x): the x column of the first reference pixel unit 32A (see A2 in FIG. 11) Streak level value Zb1 (y) of the reference pixel: Streak level value Zb2 (y) of the reference pixel in the y row of the third reference pixel unit 32C (see B1 in FIG. 11): Fourth reference pixel unit 32D (B2 in FIG. 11) (Reference) is the streak level value of the reference pixel in the y-th row.
In the formula (11),
new_image1 (x, y): Pixel signal after correcting the streak level of the effective pixel at the coordinate (x, y) new_image3 (x, y): Pixel signal after the shading correction of the effective pixel at the coordinate (x, y).

According to the third embodiment, the muscle level calculation unit 62 calculates the third muscle level value for the third reference pixel unit 32C and the fourth muscle level value for the fourth reference pixel unit 32D for each pixel row. Further, for each effective pixel, by further using the third muscle level value and the fourth muscle level value corresponding to the pixel column to which the effective pixel belongs, the shading correction amount for each effective pixel is interpolated and calculated, Shading correction is performed by correcting the pixel signal of each effective pixel using a shading correction amount corresponding to the effective pixel. For this reason, nonuniformity of pixel signals not only in the column direction but also in the row direction can be reduced, and unevenness of the image can be preferably reduced. Even when the shading shape in the row direction changes very finely, it can be corrected appropriately. A corrected image Ip which is an infrared image after signal correction is shown at the bottom of FIG. It can be seen that by performing the line level correction and the shading correction in the column direction with high accuracy, the line noise and the image unevenness are suitably reduced in the corrected image Ip.

In addition, since the reference pixel unit 32 includes the fourth reference pixel unit 32D positioned opposite to the third reference pixel unit 32C with the effective pixel unit sandwiched between the other end side in the row direction, the effective pixel unit is preferably used. The shading correction can be performed by reflecting the image unevenness in the row direction 31 and a corrected image in which the image unevenness is suitably reduced can be provided. Further, the rectangular third reference pixel portion 32C is adjacent to the rectangular effective pixel portion 31 in the row direction, and the rectangular fourth reference pixel portion 32D is adjacent to the rectangular effective pixel portion 31 in the row direction. The above effect can be enhanced by preferably reflecting the image unevenness in the row direction of the effective pixel portion 31. If the effective pixel portion 31 and the third reference pixel portion 32C (or the fourth reference pixel portion 32D) are provided at a distance sufficiently close to the row direction, the same effect can be obtained even if they are not directly adjacent to each other. .

The reference level calculation process (the process corresponding to S12) is not limited to the above example, and may be performed prior to the muscle level correction process (the process corresponding to S13), and the muscle level calculation process (S11). The processing corresponding to (2) may be performed prior to the muscle level correction processing (processing corresponding to S13) and the shading correction processing (processing corresponding to S14). In addition, the muscle level correction process and the shading correction process may be performed in a reversed order, or may be performed in parallel.

Hereinafter, as a fourth embodiment, a modified example of the shielding unit 9 of the second embodiment will be described. FIG. 13 is a schematic view (cut end view) showing a cross section of the infrared imaging device 1 of the fourth embodiment. FIG. 14 is a schematic diagram (cut end view) showing a cross section of an infrared imaging device 1 according to a modification of the fourth embodiment. In the fourth embodiment, only the configuration of the shielding unit 9 is different from the shielding unit 9 of the second embodiment, and the configurations and functions of the other units are the same as those of the second embodiment. Hereinafter, only the outline of the internal configuration of the infrared imaging device 1 will be described with reference to FIG. 13, and the description of the other common parts will be omitted. In FIGS. 13 and 14, the light beam from the optical system 2 is a schematic example and does not represent an actual light beam.

As shown in FIG. 13, the infrared imaging apparatus 1 includes an apparatus main body 12 including a main body 11 and a connecting portion 10 that connects and supports a base end portion of a lens barrel (not shown) in which the optical system 2 is accommodated from the periphery. ing. Inside the main body 11, the image sensor 3 is positioned on a substrate (not shown) so that the detection surface 30 of the image sensor 3 is positioned on the imaging surface of the optical system 2. Such an internal configuration is common to that of the first to third embodiments.

The device body 12 is made of stainless steel. For example, a metal material such as an aluminum alloy or stainless steel or a resin material such as plastic can be used as the material of the apparatus main body 12.

In the example of FIG. 13, the shielding unit 9 is located between the optical system 2 and the image sensor 3, extends from the apparatus main body 12 toward the optical axis O, and receives infrared rays from the optical system 2 at the tip portion on the optical axis side. It is comprised from the shielding body which shields. Here, the shield is two plate-like members. The shielding unit 9 partially shields infrared rays that have passed through the optical system 2 and provides reference regions BR at both ends of the image sensor 3 in the column direction.

Here, as described above, of the infrared rays detected by the image sensor 3, the infrared rays not caused by the infrared rays from the optical system 2 are incident infrared rays from the apparatus body 12 and incident from internal elements such as the image sensor 3 itself. It can be considered to be the sum of infrared rays.

However, when the incident infrared rays are shielded by the shielding unit 9 and the reference pixel is provided, the shielding unit 9 not only shields the incident infrared rays from the optical system 2 but also shields the incident infrared rays from the apparatus main body 12. It will be. That is, the effective pixel unit 31 has incident infrared rays from the apparatus main body, whereas the reference pixel unit 32 has no incident infrared rays from the apparatus main body 12. As a result, when there is a temperature difference between the apparatus main body 12 and the shielding unit 9, a difference occurs in the amount of incident infrared rays not caused by the incident infrared rays from the optical system 2 between the effective pixels and the reference pixels. .

In each embodiment of the present invention, the pixel signal detected by the reference pixel is used as a pixel signal that does not originate from the incident infrared ray from the optical system 2 (a pixel signal unrelated to the incident infrared ray from the optical system 2). Processing is in progress. For this reason, in order to improve the accuracy of the signal correction processing as described above, even if there is a temperature difference between the apparatus main body 12 and the shielding unit 9, the optical system 2 is between the effective pixel and the reference pixel. It is preferable that the difference in the amount of incident infrared rays not caused by the incident infrared rays from is small.

In view of this, in the fourth embodiment, the shield 9 is thermally coupled to the apparatus main body 12. For this reason, the temperature of the shielding part 9 can be brought close to the temperature of the apparatus main body 12. As a result, an amount of infrared rays equivalent to the infrared rays from the apparatus main body is generated from the shielding unit 9, and the amount of incident infrared rays not caused by the incident infrared rays from the optical system 2 is between the effective pixels and the reference pixels. The difference can be reduced.

In the example of FIG. 13, the base end portions of the plate-like members constituting the shielding portion 9 are thermally coupled by being in contact with and fixed to the apparatus main body 12. Thereby, when the temperature of the apparatus main body 12 rises due to a change in the environmental temperature outside the apparatus, the temperature of the shielding unit 9 also rises. The shielding unit 9 is preferably configured such that the maximum temperature difference with the apparatus main body 12 is 1 degree or less, and more preferably the maximum temperature difference with the apparatus main body 12 is 0.5 degree or less. It is preferable that it is comprised. For example, it is preferable to employ a material having a high thermal conductivity of 200 (W / (m K)) or higher, such as aluminum (aluminum alloy) or copper, as the material of the shielding part 9.

In addition, you may employ | adopt the arbitrary structures which can couple | bond the shielding part 9 and the apparatus main body 12 thermally. For example, when the shield 9 is not in direct contact with the apparatus main body 12 as in the first embodiment, the shield 9 and the apparatus main body 12 are thermally connected to each other via a heat conduction member having a desired high thermal conductivity. May be combined. For example, the heat conducting member may be a highly flexible member such as a heat conducting sheet or a heat conducting tape, or may be a highly rigid member, adopting an arbitrary shape such as a plate shape or a linear shape. It's okay. As shown in FIG. 13, when the shielding part 9 is configured by a plate-like member extending from the apparatus main body 12 toward the optical axis, the shielding part 9 and the apparatus main body 12 are preferably thermally coupled with a simple configuration. be able to.

Further, the position where the shield 9 and the apparatus main body 12 are thermally coupled may be appropriately changed according to various matters such as design. As shown in the fourth embodiment, when the main body part 11 and the shielding part 9 are thermally coupled, the temperature of the main body part 11 and the shielding part 9 is brought close to the reference pixel and the effective pixel, The difference in the amount of incident infrared rays not caused by the infrared rays that have passed through the optical system 2 can be suitably reduced, and the muscle level correction and the reference level value can be accurately calculated using the reference pixels, so that the signal correction processing can be accurately performed. It can be performed.

13, for example, as shown in FIG. 14, the shielding portion 9 is configured to thermally connect the shielding body 9A, which is a plate-like member having an opening 9C, and the shielding body 9A and the apparatus main body 12. And a support portion 9B that supports the shield 9A. The tip on the optical axis side of the shield 9A extending in the direction approaching the optical axis from the support portion 9B forms the opening 9C, and the tip on the optical axis side of the shield 9A (the peripheral portion of the opening 9C) is the optical system 2. Shields incident infrared rays from. In this case, it is easy to provide the reference region BR having a desired shape by changing the shape of the opening 9C of the shield 9A. Further, as shown in FIG. 14, when the support portion 9B is brought into contact with the front surface of the apparatus main body 12 and the shielding portion 9 and the apparatus main body 12 are thermally coupled, the temperature change on the front surface of the apparatus main body 12 is shielded. This can be suitably reflected in the part 9.

Further, as shown by a broken line in FIG. 14, the shielding portion 9 may be configured by a shielding body 9D that is a plate-like member having an opening 9C and extending from the coupling portion 10 toward the optical axis. The tip of the shield 9D extending in the direction approaching the optical axis from the connecting portion 10 forms the opening 9C, and the tip of the shield 9A on the optical axis side (the peripheral portion of the opening 9C) is the optical system 2. Shields incident infrared rays from. In this case, it is easy to provide the reference region BR having a desired shape by making the shape of the opening 9C different, and the temperature change of the connecting portion 10 located around the optical system 2 is suitably reflected on the shielding portion 9. be able to. In the example of FIG. 14, a cross section including the optical axis O is shown. However, the openings 9C in FIG. 14 are all rectangular, and reference regions BR are provided at both ends in the column direction. First and second reference pixel portions 32A and 32B as shown in the embodiment are provided. Further, the shape of the opening 9C in FIG. 13 is varied to provide a frame-like reference region BR as in the third embodiment, and the first to fourth reference pixel portions 32A as shown in the third embodiment. To 32D may be provided, and signal correction processing similar to that of the third embodiment may be performed.

When the shielding unit 9 and the apparatus main body 12 are thermally coupled as in the fourth embodiment, the difference in the incident amount of infrared rays not caused by the infrared rays that have passed through the optical system 2 with respect to the reference pixels and the effective pixels. Can be suitably reduced, and the muscle level value and the reference level value can be accurately calculated using the reference pixel. As a result, the accuracy of the signal correction processing exemplified in each of the above embodiments can be improved, and a corrected infrared image suitable for observation can be provided. Further, in the infrared imaging device 1 equipped with an infrared image sensor whose output level of the pixel signal is smaller than that of the photoelectric conversion type image sensor, the reference pixel and the effective pixel are caused by infrared rays that have passed through the optical system 2. The effect of reducing the difference in the amount of incident infrared rays is greatly beneficial for improving the image quality of infrared images.

Note that, as illustrated in FIGS. 13 and 14, the shielding unit 9 is preferably thermally coupled to the apparatus main body 12 at a position closer to the optical system 2 in the optical axis direction than the detection surface 30. In this case, the temperature of the apparatus main body 12 on the detection surface side of the image sensor 3 that is greatly affected by the incident infrared rays on the image sensor 3 is reflected in the shielding unit 9, and preferably, for the reference pixel and the effective pixel, The difference in the incident amount of infrared rays not caused by the infrared rays that have passed through the optical system 2 can be suitably reduced.

According to each embodiment of the present invention, each effect described above is preferably applied to noise generated based on far-infrared light (wavelength 8 to 15 μm) in infrared light (wavelength 0.7 μm to 1 mm). Is obtained. In addition, the infrared imaging device 1 according to each embodiment of the present invention can be suitably applied to a security imaging device, a vehicle-mounted imaging device, and the like, and may be configured as a single imaging device that captures infrared images. It may be configured to be incorporated in an imaging system having an infrared image capturing function.

The above embodiments are merely examples, and all the above descriptions should not be used to limit the technical scope of the present invention. The aspect of the present invention is not limited to the above-described individual examples (first to fourth embodiments, other modifications and applications), and any combination of the elements of the individual examples is not limited to this. The invention includes various modifications that can be conceived by those skilled in the art. That is, various additions, modifications, and partial deletions can be made without departing from the concept and spirit of the present invention derived from the contents defined in the claims and their equivalents.

DESCRIPTION OF SYMBOLS 1 Infrared imaging device 2 Optical system 3 Image sensor (infrared image sensor)
4 Analog signal processing circuit 5 Conversion circuit 6 Signal correction unit 7 Output unit 8 Storage unit 9 Shield unit 31 Effective pixel unit 32 Reference pixel unit 32A to 32D First to fourth reference pixel unit 61 Reference level calculation unit 62 Muscle level calculation unit 63 Muscle level correction part IR Imaging area AR Effective area BR Reference area BP Substrate

Claims (14)

Optical system,
An effective pixel portion in which effective pixels as infrared detection elements are arranged in a matrix in an effective region where infrared rays from the optical system are incident, and a reference region where infrared rays from the optical system are shielded by the infrared detection element A reference pixel portion in which one or more rows are arranged for each pixel column of the effective pixel portion, and the effective region and the reference region are provided in one detection region, and the optical system is imaged An infrared image sensor located on the surface;
A shielding part that is located apart from the reference pixel and shields the infrared rays from the optical system from entering the reference region;
A signal correction unit that performs correction processing on the pixel signal detected by the infrared image sensor,
The signal correction unit is
A reference level calculation unit for calculating a reference level value representing a reference level of a pixel signal of the reference pixel;
A streak level calculation unit that calculates a streak level value representing an average level of pixel signals of the reference pixels belonging to the pixel column of the reference pixel for each pixel column of the reference pixel corresponding to each pixel column of the effective pixels; ,
For each effective pixel, a streak level for performing a streak level correction process for correcting the pixel signal of the effective pixel according to the difference between the streak level value corresponding to the pixel column to which the effective pixel belongs and the reference level value. An infrared imaging device comprising: a correction unit. The infrared imaging device according to claim 1, wherein the reference pixel unit includes a plurality of rows of the reference pixels with respect to the pixel columns of the effective pixel unit. The infrared imaging device according to claim 1 or 2, wherein the reference pixel unit includes a first reference pixel unit located on one end side in a column direction of the effective pixel unit. The muscle level correction unit performs muscle level correction by adding or subtracting the difference between the muscle level value corresponding to the pixel column to which the effective pixel belongs and the reference level value to the pixel signal of each effective pixel. The infrared imaging device according to claim 3 to be performed. The infrared imaging device according to claim 3, wherein the reference pixel unit further includes a second reference pixel unit positioned opposite to the first reference pixel unit with the effective pixel unit interposed therebetween on the other end side in the column direction. The muscle level calculation unit has a first muscle level value that is the muscle level value for the first reference pixel unit and a muscle level value that is the second muscle level value for the second reference pixel unit for each pixel row of the effective pixels. Each muscle level value is calculated,
The muscle level correction unit uses, for each effective pixel, the shading correction for the effective pixel using the first muscle level value and the second muscle level value corresponding to the pixel column to which the effective pixel belongs. The infrared imaging apparatus according to claim 5, wherein shading correction is performed by interpolating the amount and correcting the pixel signal of each effective pixel using the shading correction amount corresponding to the effective pixel. A third reference pixel portion located on one end side in the row direction of the effective pixel portion, and opposite to the third reference pixel portion across the effective pixel portion on the other end side in the row direction of the effective pixel portion. A fourth reference pixel portion located;
The muscle level calculation unit, for each pixel row of the effective pixel part, a third muscle level value representing an average level of pixel signals of the reference pixels belonging to the third reference pixel part and the fourth reference pixel part A fourth muscle level value representing the average level of the pixel signal of each of the reference pixels belonging to the
The muscle level correction unit further uses, for each effective pixel, the third muscle level value and the fourth muscle level value corresponding to the pixel row to which the effective pixel belongs, to The infrared imaging device according to claim 6, wherein the shading correction amount is calculated by interpolation, and the shading correction is performed by correcting a pixel signal of each effective pixel using the shading correction amount corresponding to the effective pixel. . It has a device body,
The infrared imaging device according to claim 1, wherein the shielding portion is thermally coupled to the device main body. The infrared imaging device according to claim 8, wherein the shielding portion includes a plate-like member extending from the device main body toward the optical axis. The infrared imaging device according to claim 8, wherein the shielding portion includes a plate-like member having an opening, and a support portion that thermally couples the plate-like member and the apparatus main body to support the plate-like member. 11. The infrared imaging device according to claim 1, wherein the reference level value is a value representing an average level of pixel signals of the reference pixels belonging to each of the pixel columns of the reference pixels. The infrared imaging device according to any one of claims 1 to 11, wherein the reference level value is an average value of the stripe level values of a pixel column of each of the reference pixels. The infrared imaging device according to any one of claims 1 to 12, wherein the stripe level value is an addition average value of pixel signals of the reference pixels belonging to a pixel column of the reference pixels. In an optical system, an effective region where infrared rays from the optical system are incident, an effective pixel portion in which effective pixels that are infrared detection elements are arranged in a matrix, and a reference region in which infrared rays from the optical system are shielded, A reference pixel unit in which one or more rows of reference pixels as infrared detection elements are arranged for each pixel column of the effective pixel unit, and the effective region and the reference region are provided in one detection region, and the optical An infrared image sensor located on the imaging plane of the system, a shielding part that is located apart from the reference pixel and shields the incidence of infrared rays from the optical system to the reference region, and detected by the infrared image sensor A signal correction method using an infrared imaging device including a signal correction unit that performs correction processing on a pixel signal,
The signal correction unit is
A reference level calculation step of calculating a reference level value representing a reference level of a pixel signal of the reference pixel;
A streak level calculation step of calculating a streak level value representing an average level of pixel signals of the reference pixels belonging to the pixel column of the reference pixel for each pixel column of the reference pixel corresponding to each pixel column of the effective pixels; ,
For each effective pixel, a streak level for performing a streak level correction process for correcting the pixel signal of the effective pixel according to the difference between the streak level value corresponding to the pixel column to which the effective pixel belongs and the reference level value. And a correction step. A signal correction method using an infrared imaging device.

Images