WO2018124047A1 - Imaging device and camera - Google Patents

Abstract

An imaging device (100) provided with: a photoelectric conversion element (102), arranged in matrix form, for converting the image of a subject into electric charge; accumulation units (103) each accumulating the electric charge of the photoelectric conversion element (102); an information acquisition unit (104) for acquiring electric charge information for each frame as frame information that indicates the amount of electric charge accumulated in each of the accumulation unit (103) by a rolling shutter operation, and acquiring electric charge information during exposure before the frame information is acquired, nondestructively as interim accumulation information; and a new information creation unit (105) for creating new frame information on the basis of the frame information included in successive first and second frames and the interim accumulation information.

Description

Imaging device and camera

The present invention relates to an imaging apparatus that captures an image of a subject as image information by photoelectric conversion, and a camera including the imaging apparatus.

2. Description of the Related Art Conventionally, an imaging apparatus that captures an image using an image sensor is known (see, for example, Patent Document 1).

JP 2008-042180 A

In an imaging device that acquires images continuously by a rolling shutter operation, for example, a peculiar image in which the lower region of the image becomes whiter than the upper region of the image due to a sudden event such as a flash. A certain band image may occur.

Therefore, an object of the present disclosure is to provide an imaging apparatus capable of creating an image that replaces the band image when there is a possibility that the band image is generated.

An imaging apparatus according to one embodiment of the present disclosure is arranged in a matrix, and includes a photoelectric conversion element that converts an image of a subject into charges, a storage unit that stores charges converted by each of the photoelectric conversion elements, and the storage Charge information indicating the amount of charge accumulated in each of the units is acquired for each frame as frame information by a rolling shutter operation, and charge information during exposure before acquiring the frame information is nondestructively stored as intermediate storage information An information acquisition unit to acquire, and a new information generation unit that generates new frame information based on the frame information and the intermediate storage information included in the first frame and the second frame that are continuous.

According to the imaging apparatus according to the present disclosure, it is possible to provide a moving image that does not include a band image.

FIG. 1 is a block diagram illustrating an overall configuration of a camera including an imaging apparatus according to Embodiment 1. FIG. 2 is a block diagram illustrating a configuration of an image sensor including a plurality of photoelectric conversion elements and an accumulation unit. 3 is a plan view of the photoelectric conversion element according to Embodiment 1. FIG. 4 is a side view of the photoelectric conversion element according to Embodiment 1. FIG. FIG. 5 is a block diagram illustrating a configuration of the pixel circuit according to the first embodiment. FIG. 6 is a diagram illustrating a generation state of a band image. FIG. 7 is a diagram showing a state in which new frame information for the first frame according to Embodiment 1 is created. FIG. 8 is a diagram showing a state in which new frame information for the first frame according to the first embodiment is created. FIG. 9 is a diagram illustrating a state in which new frame information for the first frame according to the second embodiment is created. FIG. 10 is a diagram illustrating a state in which new frame information for the second frame according to the second embodiment is created. FIG. 11 is a perspective view of a digital still camera according to a modification. FIG. 12 is a perspective view of a video camera according to a modification.

Hereinafter, embodiments will be described in detail. Note that each of the embodiments described below shows a preferred specific example of the present disclosure. Numerical values, shapes, materials, components, arrangement positions and connection forms of components, steps, order of steps, and the like shown in the following embodiments are merely examples, and are not intended to limit the present disclosure. The present disclosure is limited only by the claims. Therefore, among the constituent elements in the following embodiments, constituent elements that are not described in the independent claims of the present disclosure are not necessarily required to achieve the problems of the present disclosure, but constitute more preferable embodiments. It is explained as a thing.

[overall structure]
(Embodiment 1)
FIG. 1 is a block diagram illustrating an overall configuration of a camera including an imaging device according to an embodiment.

As shown in the figure, an imaging apparatus 100 is an apparatus that forms an image of a subject on the surface of a photoelectric conversion element 102 by an optical system 101 and creates image information based on the amount of charge accumulated for each pixel by photoelectric conversion. The storage unit 103, the information acquisition unit 104, and the new information creation unit 105 are provided. In the case of the present embodiment, the imaging apparatus 100 further includes a detection unit 106 and a storage device 107.

In the case of the present embodiment, the information acquisition unit 104, the new information creation unit 105, and the detection unit 106 are processing units realized by the control device 200 executing a program. As an example, the control device 200 includes a memory that stores a program and a processor that executes the program stored in the memory.

[Optical system 101]
The optical system 101 is a combination of optical members for forming an image of a subject on the surface of the photoelectric conversion element 102 arranged in a matrix. For example, in addition to a lens and a mirror, an aperture blade, a mechanical shutter, a focus It is configured by combining arbitrary optical members from an adjustment drive mechanism, a camera shake prevention mechanism, and the like.

[Photoelectric conversion element 102]
The photoelectric conversion element 102 is an element that is arranged in a matrix and converts light at each location forming an image of a subject formed on the surface by the optical system 101 into electric charges.

FIG. 2 is a block diagram illustrating a configuration of an image sensor including a plurality of photoelectric conversion elements and a storage unit.

As shown in the figure, the image sensor 110 includes a photoelectric conversion element 102, a pixel circuit array 120, a readout circuit 121, an output circuit 122, a row scanning circuit 123, a timing control circuit 124, and a voltage application circuit 125. And.

FIG. 3 is a plan view of the photoelectric conversion element.

FIG. 4 is a side view of the photoelectric conversion element.

As shown in these drawings, the photoelectric conversion element 102 includes a thin-film photoelectric conversion member 126, a transparent electrode 127 that is in close contact with one surface of the photoelectric conversion member 126, and an N surface that is in close contact with the other surface of the photoelectric conversion member 126. And N × M pixel electrodes 128 arranged in a two-dimensional array of rows and M columns (N and M are integers of 1 or more).

[Photoelectric conversion member 126]
The photoelectric conversion member 126 generates charges due to the internal photoelectric effect by receiving light in a state where a voltage of a first predetermined range not including 0V is applied, and in a state where a voltage of a second predetermined range including 0V is applied. It is a member that does not generate charges due to the internal photoelectric effect even when receiving light.

Here, the photoelectric conversion member 126 will be described as an organic thin film having the above characteristics. That is, in this embodiment, the image pickup device 110 is an example of an organic CMOS image sensor using an organic thin film as a photoelectric conversion member.

In the case of the present embodiment, the photoelectric conversion member 126 is formed of one thin film in one imaging element 110, but a portion overlapping with one pixel electrode 128 functions as one photoelectric conversion element 102. Yes.

[Transparent electrode 127]
The transparent electrode 127 is an electrode to which a voltage that generates a potential difference including 0 is applied to the pixel electrode 128, and transmits light that forms an image of a subject formed on the surface of the photoelectric conversion member 126 by the optical system 101. It has transparency.

[Pixel electrode 128]
The pixel electrode 128 is an electrode disposed in close contact with the surface opposite to the transparent electrode 127 with respect to the photoelectric conversion member 126, and is an electrode to which a voltage causing a positive potential difference is applied to the transparent electrode 127. . The pixel electrodes 128 are a plurality of electrodes arranged in a two-dimensional array of N rows and M columns on the other surface of one photoelectric conversion member 126. Adjacent pixel electrodes 128 are arranged at predetermined intervals. On the surface of the photoelectric conversion member 126, they are insulated from each other.

When charges are generated by the photoelectric conversion member 126 during a period in which a voltage that causes a positive potential difference with respect to the transparent electrode 127 is applied to the pixel electrode 128, the charge is generated in the region of the photoelectric conversion member 126 that overlaps the pixel electrode 128 and Among charges generated in the vicinity, positive charges are collected.

Furthermore, the amount of charge generated in the photoelectric conversion member 126 changes corresponding to the voltage applied to the transparent electrode 127 and the pixel electrode 128, and the amount of charge collected by the transparent electrode 127 also changes. Therefore, the exposure can be adjusted by the applied voltage without using a mechanical member such as a diaphragm blade.

[Pixel circuit array]
As shown in FIG. 2, the pixel circuit array 120 is a semiconductor device in which N × M pixel circuits 129 corresponding to the pixel electrodes 128 are arranged in a two-dimensional array of N rows and M columns. The pixel circuit 129 is arranged on the pixel electrode 128 side of each photoelectric conversion element 102 so as to overlap the photoelectric conversion element 102.

FIG. 5 is a block diagram showing the configuration of the pixel circuit.

As shown in the figure, the pixel circuit 129 includes a reset transistor 191, an amplification transistor 192, a selection transistor 193, and an accumulation unit 103.

[Storage unit 103]
The storage unit 103 is a so-called charge storage node that stores the charges converted by each of the photoelectric conversion elements 102, the pixel electrode 128 corresponding to the pixel circuit 129 to which the storage unit 103 belongs, the source of the reset transistor 191, and the amplification transistor 192. And a capacitor for accumulating positive charges collected by the connected pixel electrode 128.

[Reset transistor 191]
The reset transistor 191 has a gate connected to the reset signal line 194, a drain supplied with a reset voltage VRST, and a source connected to the storage unit 103.

The reset transistor 191 is turned on by a reset signal delivered from a row scanning circuit 123 (described later) via a reset signal line 194, thereby resetting (initializing) the amount of charge accumulated in the accumulation unit 103. .

Here, in the specification and claims, the term “exposure period” is used to mean a period from when the storage unit 103 is reset to when it is reset next. Further, the term “during exposure” is used to mean any timing within the exposure period. In other words, the word “exposure” is not used to mean a state in which any member is exposed to light, but a series of operations until obtaining frame information indicating a desired image. Is used to mean.

In the case of the rolling shutter operation, the exposure period is such that the start time and the end time are sequentially shifted for each row of the photoelectric conversion elements 102, and the time included in the exposure period for one frame and the exposure for the next frame. The time included in the period may match.

[Amplification transistor 192]
In the amplification transistor 192, the storage unit 103 is connected to the gate, the power supply voltage VDD is supplied to the drain, and the drain of the selection transistor 193 is connected to the source.

A voltage corresponding to the charge accumulated in the accumulation unit 103 is applied to the gate of the amplification transistor 192, and the voltage becomes charge information.

Therefore, when the selection transistor 193 is turned on, the amplification transistor 192 flows a current corresponding to the charge accumulated in the accumulation unit 103, and can output the charge information as a current value. Since the output of the charge information does not affect the amount of charge stored in the storage unit 103, the charge amount of the storage unit 103 can be read out nondestructively [Selection transistor 193]
The selection transistor 193 has a gate connected to the selection signal line 195, a drain connected to the source of the amplification transistor 192, and a source connected to the vertical signal line 196.

The selection transistor 193 is turned on by a selection signal delivered from the row scanning circuit 123 (described later) via the selection signal line 195, and thereby outputs a current flowing through the amplification transistor 192 to the vertical signal line 196.

As will be described later, when the amount of current output to the vertical signal line 196 is detected by a column readout circuit 197 (described later), the storage unit of the pixel circuit 129 including the selection transistor 193 turned on by the selection signal. The amount of charge accumulated in 103 is read out.

The pixel circuit 129 can sequentially read out the amount of charge accumulated in the accumulation unit 103 in a non-destructive manner with the above configuration.

[Row scanning circuit 123]
As shown in FIG. 2, the row scanning circuit 123 has both a stored charge amount reset function and a readout pixel circuit selection function.

[Stored charge amount reset function]
In the pixel circuit array 120, the stored charge amount resetting function is sequentially performed row by row from the row farthest to the readout circuit 121 (first row) to the row closest to the readout circuit 121 (Nth row). The reset signal line 194 connected to each pixel circuit 129 belonging to the corresponding row is supplied with a reset signal for resetting the positive charge accumulated in the storage unit 103 in each pixel circuit 129 belonging to the corresponding row at a predetermined time t1 interval. It is a function to deliver via.

Thereby, the resetting of the charges accumulated in the accumulation units 103 of all the pixel circuits 129 included in the pixel circuit array 120 is sequentially executed in units of rows from the first row to the Nth row.

[Reading pixel circuit selection function]
In the pixel circuit array 120, the readout pixel circuit selection function turns on the selection transistor 193 in each of the pixel circuits 129 belonging to the corresponding row at predetermined time intervals t1 in order from the first row to the Nth row. This is a function for delivering a selection signal for selection via a selection signal line 195 connected to each pixel circuit 129 belonging to the corresponding row.

Thereby, the reading of the charge amount accumulated in the accumulation unit 103 of all the pixel circuits 129 included in the pixel circuit array 120 is sequentially executed in units of rows from the first row to the Nth row.

In the case of the rolling shutter operation, when the reading of the nth row (n is a natural number equal to or less than N) is completed, the nth row is reset, and the exposure of the nth row for the next frame starts. This operation is sequentially executed for each row.

[Read circuit 121]
The readout circuit 121 reads out the amount of charge accumulated in each of the pixel circuits 129 constituting the pixel circuit array 120.

The readout circuit 121 includes M column readout circuits 197 corresponding to the M columns of the pixel circuit array 120, respectively.

[Column Read Circuit 197]
The column readout circuit 197 includes a pixel circuit 129 (this pixel circuit 129) including a selection transistor 193 that is turned on by a selection signal via a vertical signal line 196 connected to each pixel circuit 129 belonging to the corresponding column. Is also referred to as a “readout-to-image pixel circuit 129”). As a result, the amount of electric charge accumulated in the accumulating unit 103 of the pixel circuit 129 for reading image can be read out. A current value indicating the amount of the read charge is converted into a K-bit digital signal (K is a positive integer, for example, 8), and is output as the charge information of the pixel circuit 129 for the read image.

[Output Circuit 122]
The output circuit 122 outputs the charge information output from the column readout circuit 197 to the outside. The charge information respectively corresponding to the pixel circuit 129 of N rows and M columns becomes one piece of image information. In addition, since the charge information can be read out non-destructively, it is possible to output information during exposure to the outside as accumulated information.

[Voltage application circuit 125]
The voltage application circuit 125 applies a voltage to the photoelectric conversion element 102. More specifically, the voltage application circuit 125 applies a voltage to the transparent electrode 127 with respect to the pixel electrode 128. The voltage application circuit 125 can adjust the voltage applied to the transparent electrode 127. Therefore, exposure can be adjusted by the voltage application circuit 125, and it also functions as an electronic shutter.

[Timing control circuit 124]
The timing control circuit 124 controls the operation timing of the row scanning circuit 123 and the operation timing of the readout circuit 121. The timing control circuit 124 may control the operation timing of the voltage application circuit 125. That is, the timing control circuit 124 controls the timing for executing the stored charge amount reset function and the timing for executing the readout pixel circuit selection function by the row scanning circuit 123, and is selected by the selection signal by the readout circuit 121. The timing for reading the amount of charge accumulated in the accumulation unit 103 of the pixel circuit 129 is controlled. In addition, the timing control circuit 124 can control the voltage application timing to the photoelectric conversion element 102 by the voltage application circuit 125 and cause the voltage application circuit 125 to function as a shutter. Furthermore, exposure can be adjusted by repeatedly turning on and off the voltage application to the photoelectric conversion element 102 within the exposure period.

[Information acquisition unit 104]
The information acquisition unit 104 obtains N rows and M columns of charge information indicating the amount of charges stored in the storage units 103 provided on a one-to-one basis for each of the photoelectric conversion elements 102 by a rolling shutter operation to frame the frame information. It is a processing unit that obtains the charge information during exposure before obtaining frame information and obtains N rows and M columns as intermediate storage information.

In the case of the present embodiment, the information acquisition unit 104 acquires for each row output from the output circuit 122 and constructs frame information for each frame. Further, the frame information is sequentially connected for each frame to become one moving image information, and is stored in the storage device 107 such as a flash memory, for example. In addition, the information acquisition unit 104 stores the intermediate accumulation information in a ring memory or the like. The information acquisition unit 104 holds a plurality of pieces of intermediate storage information acquired at each stage acquired in one frame until at least the frame information of the next frame is acquired.

The intermediate accumulation information is image information acquired from when the storage unit 103 is refreshed until frame information is acquired, and the light amount (luminance) of each pixel is based on the light amount of the corresponding pick cell in the frame information. Is also small. In addition, when the accumulated information is acquired a plurality of times, the charge amount is integrated. That is, the light quantity of each pixel is integrated for each pixel. Specifically, for example, as shown in FIG. 6, when the intermediate accumulation information is acquired three times so that the intervals of the time centroids of intermediate information (described later) are equal in one frame, Assuming that the light quantity of each pixel is integrated into intermediate information a, the light quantity of subsequently acquired intermediate storage information B is a value obtained by adding intermediate information b to intermediate information a. Further, the light amount of the intermediate storage information C acquired subsequently becomes a value obtained by adding the intermediate information b and the intermediate information c to the intermediate information a. Accordingly, the frame information is D in the figure, and is a value obtained by adding the intermediate information a, the intermediate information b, the intermediate information c, and the intermediate information d.

In the case of the present embodiment, the information acquisition unit 104 gives an instruction to read out the amount of charge stored in the storage unit 103 of each pixel circuit 129 to the timing control circuit 124, and outputs N from the output circuit 122. Charge information for M rows and columns is acquired. Further, the information acquisition unit 104 is performed once or a plurality of times until the instruction for reading the amount of charge is reset.

[Detection unit 106]
The detection unit 106 is a processing unit that detects a band image in which a sudden event that suddenly occurs in the subject is reflected only in the second half of the frame information of the first frame.

Here, the sudden event means an event that suddenly occurs in the subject, and as shown in FIG. 6, the sudden event 202 is performed for N rows after the reset of the first row of one frame. This means an event in which the state of the subject changes almost over the entire subject instantly until the reset is completed. Specifically, an instantaneous event that suddenly starts and immediately returns to the original state, such as when the subject hits a flash, is also a sudden event 202, and a high-speed moving object (such as a car) is in front of the subject. The sudden event 202 includes a case where it suddenly starts and continues for a certain period of time, or when the color of the subject itself suddenly changes and the state continues for a plurality of frames. . In addition, the specification and the claims are described as including the case where the direction of the optical system 101 is suddenly moved as an unexpected event. In the present embodiment, the sudden event 202 will be described as a flash. In this case, the band image may be referred to as a flash band.

In the case of the present embodiment, the detection unit 106 acquires frame information from the information acquisition unit 104, and first information 211 (FIG. 6) that is a part of information acquired in the second half by the rolling shutter operation from the frame information. 2) indicates a so-called whiteout state (a state in which all pixels have a light amount equal to or greater than the threshold value), the second information 212 that is a part of the information acquired in the first half is further acquired, When it is determined that the information 212 is not in a whiteout state, information indicating that a band image has been generated is transmitted to the new information creation unit 105.

In addition, the detection method of the band image of the detection part 106 is not limited above, Arbitrary methods can be employ | adopted. For example, the first information 211 and the second information 212 may be acquired for each frame, and it may be determined that a band image has occurred if the difference between these is equal to or greater than a threshold value. Further, it may be determined that a band image has occurred by comparing the latter half of the halfway information d and the latter half of the halfway information c.

Furthermore, the detection unit 106 detects the second band image reflected in only the first half of the frame information of the second frame following the first frame, and the sudden event that suddenly occurs in the subject is an instantaneous event. It does not matter if it is detected.

In this case, for example, as in the case of the first frame, the detection unit 106 acquires the frame information of the second frame from the information acquisition unit 104, and the information acquired in the first half by the rolling shutter operation from the frame information. If the third information 213 (see FIG. 6), which is a part, indicates a so-called whiteout state, the fourth information 214 that is a part of the information acquired in the second half is further acquired, and the fourth information 214 is acquired. If it is determined that is not in a state of being out of focus, information indicating that a band image has also occurred in the second frame may be transmitted to the new information creation unit 105.

[New information creation unit 105]
The new information creation unit 105 is a processing unit that creates new frame information based on, for example, frame information and intermediate storage information included in the continuous first frame F1 and second frame F2.

In the case of the present embodiment, when the new information creation unit 105 acquires information indicating that a band image has been generated from the detection unit 106, the new information creation unit 105 creates new frame information that is not a band image from the frame information in which the band image has occurred. Replace with frame information.

Specifically, as shown in FIG. 6, when a band image is generated in the first frame F1 due to the instantaneous sudden event 202, the new information creation unit 105, as shown in FIG. The intermediate storage information A of the second frame F2 (same as the intermediate information a in the case of the present embodiment) is added to the frame information D of F1. As a result, the amount of light corresponding to the first frame F1 is larger than the amount of light actually subjected to photoelectric conversion. Therefore, new frame information is created by adjusting the gain so that the total amount of light becomes 4/5, and the first frame F1. As frame information, old frame information is replaced with new frame information.

Further, since a band image is also generated in the second frame F2, the new information creation unit 105, in the middle of the frame information D of the first frame F1 and the frame information D of the second frame F2, as shown in FIG. Intermediate information d, which is a difference from accumulated information C, is added. Also in this case, since the amount of light of the second frame F2 increases by the amount of the intermediate information d, new frame information D is created with 4/5 gain, and is replaced with the frame information of the second frame F2.

[Effects]
As described above, according to the imaging apparatus 100, since the frame information of the frame in which the band image is generated is replaced with new frame information based on the accumulated information, moving image information that does not feel strange even if a sudden event occurs in the subject is created. It becomes possible to do.

In addition, since new frame information is created from the accumulated information before the frame information of the next frame is acquired, it is possible to shoot video without delay even if a sudden event occurs multiple times in a short time.

Also, since the gain is adjusted in the direction of decreasing the light amount by combining the frame information and the halfway accumulated information, it is possible to create frame information with a good S / N ratio without enhancing noise included in the frame information.

(Embodiment 2)
Next, another embodiment of the imaging device 100 will be described. In addition, the same code | symbol may be attached | subjected to what has the effect | action and function similar to the said Embodiment 1, and the same shape, mechanism, and structure (part) may be abbreviate | omitted.

In the case of the present embodiment, only the new information creation unit 105 is different from the first embodiment, so the new information creation unit 105 will be described.

[New information creation unit 105]
For example, as shown in FIG. 7, when new frame information is created by combining the frame information of the first frame F1 with the frame information and the intermediate information of the second frame F2, the image center of gravity position of the new frame information is the old It will deviate from the image gravity center position of the frame information.

Therefore, when the new information creation unit 105 according to the present embodiment acquires information indicating that a band image has been generated from the detection unit 106, the new information creation unit 105 synthesizes the frame information in which the band image has been generated and the intermediate information of the previous and subsequent frames. New frame information that is not a band image is created by adjusting the gain, and the old frame information is replaced.

When a band image is generated in the first frame F1 due to the instantaneous sudden event 202, the new information creation unit 105 adds the second frame F2 immediately after the frame information D of the first frame F1, as shown in FIG. The intermediate storage information A (same as intermediate information a in the case of the present embodiment) and the immediately preceding third frame F3 are added. As a result, frame information in which the entire image is blown out is created without shifting the time center of gravity position of the frame information to be newly created. Since the amount of light corresponding to the first frame F1 is larger than the amount of light actually photoelectrically converted, new frame information is created by adjusting the gain so that the total amount of light is 4/6, and the first frame F1 As frame information, old frame information is replaced with new frame information.

When a band image is also generated in the second frame F2, the new information creation unit 105, as shown in FIG. 10, adds the frame information D of the first frame F1 to the frame information D of the second frame F2. And the intermediate information d which is the difference between the intermediate storage information C and the intermediate storage information A (same as the intermediate information a in the case of the present embodiment) of the immediately following fourth frame F4. Also in this case, since the light amount of the second frame F2 increases by the amount of the intermediate information d and the intermediate information a, new frame information D is created with 4/6 gain and replaced with the frame information of the second frame F2.

[Effects]
As described above, according to the imaging apparatus 100 of the present embodiment, in addition to the effects of the imaging apparatus 100 shown in the first embodiment, a new frame based on the intermediate accumulation information of the frames before and after the frame in which the band image is generated. In order to create information, there is no deviation from the time centroid position of the new frame information that replaces the time centroid position of the old frame information, and even if a sudden event occurs in the subject, moving image information that does not feel strange in the movement of the subject Is possible.

In the case of the present embodiment, the case of overexposure has been described. However, the remarkable effect of the present embodiment can be obtained when the contour of the subject is captured as frame information, such as when the subject suddenly moves. appear.

[Summary]
An imaging apparatus 100 according to an aspect of the present disclosure is arranged in a matrix and includes a photoelectric conversion element 102 that converts an image of a subject into charges, and a storage unit 103 that stores charges converted by the photoelectric conversion elements 102, respectively. The charge information indicating the amount of charge accumulated in each of the storage units 103 is acquired as frame information for each frame by the rolling shutter operation, and the charge information during exposure before acquiring the frame information is not stored as intermediate storage information. An information acquisition unit 104 that acquires by destruction, and a new information generation unit 105 that generates new frame information based on frame information and intermediate storage information included in the continuous first frame and the second frame.

The imaging apparatus 100 creates new frame information in which the sudden event is reflected in the entire image by adjusting the frame information that becomes the band image based on the intermediate accumulation information of the frame before or after the frame. Can do.

Also, for example, the new information creation unit 105 may create new frame information by adding halfway accumulated information of the second frame to the frame information of the first frame and adjusting the gain.

According to this, new frame information can be created without deteriorating the S / N ratio.

Also, for example, the new information creation unit 105 may create new frame information by adding the difference between the frame information of the first frame and the intermediate storage information to the frame information of the second frame and adjusting the gain.

According to this, new frame information can be created for subsequent frames without degrading the S / N ratio.

Further, for example, the new information creation unit 105 further adds the difference between the frame information of the third frame immediately before the first frame and the intermediate storage information to the frame information of the first frame, and adjusts the gain to obtain new frame information. You may create it.

According to this, it is possible to create new frame information in which the position of the center of gravity of the time does not shift further.

In addition, for example, the new information creation unit 105 may add new frame information by adjusting the gain by further adding the intermediate storage information of the fourth frame, which is the frame immediately after the second frame, to the frame information of the second frame. Good.

According to this, it is possible to create new frame information in which the position of the center of gravity of the time does not further shift for subsequent frames.

Further, for example, a detection unit that detects a band image in which a sudden event that suddenly occurs in the subject is reflected in the second half of the frame information of the first frame is further included, and the new information creation unit 105 includes a band in the detection unit. When an image is detected, new frame information of at least a first frame in which a band image has occurred may be created.

This makes it possible to change only the frame information of the frame in which the band image is generated.

For example, the photoelectric conversion element 102 may include an organic thin film as a photoelectric conversion member.

Thereby, it is possible to effectively acquire the accumulated information on the way, and to effectively process the frame information of the frame in which the band image is generated.

(Supplement)
As described above, two embodiments have been described as examples of the technology disclosed in the present application. However, the technology in the present disclosure is not limited to these, and can be applied to embodiments in which changes, replacements, additions, omissions, and the like are appropriately performed.

In the embodiment, in the imaging device 100, the photoelectric conversion member 126 generates charges due to the internal photoelectric effect by receiving light in a state where a voltage is applied. Even if light is received in a state where no voltage is applied, the photoelectric conversion member 126 is affected by the internal photoelectric effect. It has been described that the organic thin film has a function of not generating charges.

However, the photoelectric conversion member 126 is not necessarily limited to the organic thin film as long as it can control the presence or absence of charge generation due to the internal photoelectric effect by the applied voltage. As an example, the imaging apparatus 100 may be an example in which the photoelectric conversion member 126 is a diode having a PN junction surface.

Needless to say, the present disclosure includes an electronic device in which the imaging device 100 according to the embodiment is incorporated.

As shown in FIG. 1 and FIG. 11, a camera 300 that is an example of an electronic device in which the imaging device 100 is incorporated is also included.

Furthermore, the imaging apparatus 100 may be provided in a video camera 400 as shown in FIG.

Further, although the case where the intermediate accumulation information is acquired three times in one frame has been described, the present invention is not limited to this, and the intermediate accumulation information may be acquired once or a plurality of times. Further, not only when the intermediate storage information is acquired evenly in time, but also before and after the reset, the intermediate storage information may be acquired at an arbitrary timing such as acquiring the intermediate storage information with a shorter time interval.

Further, although the case where the new frame information is created based on the detection of the band image by the detection unit 106 has been described, the present invention is not limited to this. For example, when the imaging apparatus 100 is changed to the band image continuous removal mode based on instruction information from a photographer or the like, the new information creation unit 105 for each frame during the mode changes the new frame information. Can be created and replaced with the old frame information to create video information.

In addition, each component (functional block) in the imaging apparatus 100 may be individually made into one chip by a semiconductor device such as an IC (Integrated Circuit) or an LSI (Large Scale Integration), or may include a part or all of it. Alternatively, one chip may be used. Further, the method of circuit integration is not limited to LSI's, and implementation using dedicated circuitry or general purpose processors is also possible. An FPGA (Field Programmable Gate Array) that can be programmed after manufacturing the LSI or a reconfigurable processor that can reconfigure the connection and setting of circuit cells inside the LSI may be used. Furthermore, if integrated circuit technology that replaces LSI appears as a result of progress in semiconductor technology or other derived technology, functional blocks may be integrated using this technology. Biotechnology can be applied as a possibility.

In addition, all or part of the various processes described above may be realized by hardware such as an electronic circuit or may be realized by using software. Note that the software processing is realized by a processor included in the imaging apparatus 100 executing a program stored in the memory. Further, the program may be recorded on a recording medium and distributed or distributed. For example, by installing the distributed program in a device having another processor and causing the processor to execute the program, it is possible to cause the device to perform each of the above processes.

Also, a form realized by arbitrarily combining the constituent elements and functions shown in the above-described embodiments is also included in the scope of the present disclosure.

The present disclosure can be widely used for imaging devices that capture images.

DESCRIPTION OF SYMBOLS 100 Image pick-up apparatus 101 Optical system 102 Photoelectric conversion element 103 Storage part 104 Information acquisition part 105 New information creation part 106 Detection part 107 Memory | storage device 110 Image pick-up element 120 Pixel circuit array 121 Reading circuit 122 Output circuit 123 Row scanning circuit 124 Timing control circuit 125 Voltage application circuit 126 Photoelectric conversion member 127 Transparent electrode 128 Pixel electrode 129 Pixel circuit 191 Reset transistor 192 Amplification transistor 193 Selection transistor 194 Reset signal line 195 Selection signal line 196 Vertical signal line 197 Column readout circuit 200 Controller 202 Sudden event 211 First Information 212 Second information 213 Third information 214 Fourth information 300 Camera 400 Video camera

Claims (8)

A photoelectric conversion element that is arranged in a matrix and converts an image of a subject into an electric charge;
An accumulator for accumulating the charges converted by the photoelectric conversion elements;
Charge information indicating the amount of charge stored in each of the storage units is acquired for each frame as frame information by a rolling shutter operation, and charge information during exposure before acquiring the frame information is not stored as intermediate storage information. An information acquisition unit acquired by destruction,
An imaging apparatus comprising: a continuous first frame; and a new information creation unit that creates new frame information based on the frame information and the intermediate storage information included in the second frame. The new information creation unit
The imaging apparatus according to claim 1, wherein the frame information of the first frame is added to the halfway accumulated information of the second frame to adjust gain to create new frame information. The new information creation unit
The imaging apparatus according to claim 1, wherein a new frame information is created by adding a difference between the frame information of the first frame and the intermediate storage information to the frame information of the second frame and adjusting a gain. The new information creation unit
The frame information of the first frame is further added with a difference between the frame information of the third frame immediately before the first frame and the intermediate storage information to adjust the gain, thereby creating new frame information. The imaging device described. The new information creation unit
The imaging apparatus according to claim 3, wherein the frame information of the second frame is further added with the intermediate storage information of the fourth frame, which is a frame immediately after the second frame, to adjust the gain and create new frame information. A detection unit that detects a band image in which a sudden event that suddenly occurs in a subject is reflected in the second half of the frame information of the first frame;
The new information creation unit
The imaging device according to any one of claims 1 to 5, wherein when the detection unit detects a band image, new frame information of at least the first frame in which the band image is generated is created. The photoelectric conversion element is
The imaging device according to any one of claims 1 to 6, comprising an organic thin film as the photoelectric conversion member. The imaging device according to any one of claims 1 to 7,
An optical system that forms an image of a subject on the photoelectric conversion element.

Images