[go: up one dir, main page]

WO2023129566A1 - Capteurs d'image comportant une compression de données can sur puce pour sorties de nombres d'électrons multibit - Google Patents

Capteurs d'image comportant une compression de données can sur puce pour sorties de nombres d'électrons multibit Download PDF

Info

Publication number
WO2023129566A1
WO2023129566A1 PCT/US2022/054105 US2022054105W WO2023129566A1 WO 2023129566 A1 WO2023129566 A1 WO 2023129566A1 US 2022054105 W US2022054105 W US 2022054105W WO 2023129566 A1 WO2023129566 A1 WO 2023129566A1
Authority
WO
WIPO (PCT)
Prior art keywords
value
digital signals
digital
image sensor
threshold
Prior art date
Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
Ceased
Application number
PCT/US2022/054105
Other languages
English (en)
Inventor
Jiaju MA
Current Assignee (The listed assignees may be inaccurate. Google has not performed a legal analysis and makes no representation or warranty as to the accuracy of the list.)
Gigajot Technology Inc
Original Assignee
Gigajot Technology Inc
Priority date (The priority date is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the date listed.)
Filing date
Publication date
Priority claimed from US18/145,838 external-priority patent/US12225316B2/en
Application filed by Gigajot Technology Inc filed Critical Gigajot Technology Inc
Priority to CN202280082733.7A priority Critical patent/CN118383039A/zh
Priority to KR1020247019986A priority patent/KR20240112303A/ko
Priority to JP2024535779A priority patent/JP2024546919A/ja
Publication of WO2023129566A1 publication Critical patent/WO2023129566A1/fr
Anticipated expiration legal-status Critical
Ceased legal-status Critical Current

Links

Classifications

    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/76—Addressed sensors, e.g. MOS or CMOS sensors
    • H04N25/78—Readout circuits for addressed sensors, e.g. output amplifiers or A/D converters
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N23/00—Cameras or camera modules comprising electronic image sensors; Control thereof
    • H04N23/80—Camera processing pipelines; Components thereof
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/60—Noise processing, e.g. detecting, correcting, reducing or removing noise
    • H04N25/63—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to dark current
    • H04N25/633—Noise processing, e.g. detecting, correcting, reducing or removing noise applied to dark current by using optical black pixels
    • H—ELECTRICITY
    • H04—ELECTRIC COMMUNICATION TECHNIQUE
    • H04N—PICTORIAL COMMUNICATION, e.g. TELEVISION
    • H04N25/00—Circuitry of solid-state image sensors [SSIS]; Control thereof
    • H04N25/70—SSIS architectures; Circuits associated therewith
    • H04N25/71—Charge-coupled device [CCD] sensors; Charge-transfer registers specially adapted for CCD sensors
    • H04N25/75—Circuitry for providing, modifying or processing image signals from the pixel array

Definitions

  • This disclosure relates generally to an image sensor and more specifically to design of on-chip image signal processing circuits of an image sensor.
  • Image capturing devices such as cameras, are widely used in various electronic devices, such as mobile devices (e g., smart phones, tablets, laptops, etc ), robotic equipment, or security monitoring devices, among others.
  • An image capturing device may include an image sensor having a plurality of light-gathering pixels.
  • a pixel may include a photodiode.
  • the image capturing device may capture light from an environment and pass the light to the image sensor When exposed to light, the photodiodes of the pixels may accumulate photoelectrons. At readout, the photoelectrons may transfer out of the photodiodes and generate analog image signals.
  • the image sensor may also include one or more image signal processing circuits which may process the analog image signals to digital image signals and transfer them to an image signal processor.
  • the image signal processor may further process the digital image signals to produce images.
  • FIG. 1 is a schematic diagram showing an example image signal processing system of an image sensor, according to some embodiments.
  • FIG. 2 is a flowchart showing an example method for converting one set of digital image signals to another set of digital image signals, according to some embodiments.
  • FIG. 3 shows an example probability distribution of analog image signals of an image sensor, according to some embodiments.
  • FIG. 4 shows an example probability distribution of digital image signals of an image sensor, according to some embodiments.
  • FIG. 5 shows an example probability distribution of digital image signals of an image sensor with additional quantization, according to some embodiments.
  • FIG. 6 is a schematic diagram of an example image sensor including at least one disclosed quantization circuit, according to some embodiments.
  • FIG. 7 is a flowchart showing an example method for processing image signals of an image sensor, according to some embodiments.
  • FIG. 8 illustrates a schematic representation of an example device that may include an image capturing device (e.g., a camera) having an image sensor that includes at least one disclosed quantization circuit, according to some embodiments.
  • an image capturing device e.g., a camera
  • an image sensor that includes at least one disclosed quantization circuit
  • FIG. 9 illustrates a schematic block diagram of an example computing device that may include or host embodiments of an image capturing device (e.g., a camera) having an image sensor that includes at least one disclosed quantization circuit, according to some embodiments.
  • an image capturing device e.g., a camera
  • an image sensor that includes at least one disclosed quantization circuit
  • a unit/circuit/component is “configured to” perform one or more tasks is expressly intended not to invoke 35 U.S.C. ⁇ 112(f) for that unit/circuit/component.
  • “configured to” can include generic structure (e.g., generic circuitry) that is manipulated by software and/or firmware (e.g., an FPGA or a general-purpose processor executing software) to operate in manner that is capable of performing the task(s) at issue.
  • “Configure to” may also include adapting a manufacturing process (e.g., a semiconductor fabrication facility) to fabricate devices (e g., integrated circuits) that are adapted to implement or perform one or more tasks.
  • this term is used to describe one or more factors that affect a determination. This term does not foreclose additional factors that may affect a determination. That is, a determination may be solely based on those factors or based, at least in part, on those factors.
  • a determination may be solely based on those factors or based, at least in part, on those factors.
  • first, second, etc. may be used herein to describe various elements, these elements should not be limited by these terms. These terms are only used to distinguish one element from another.
  • a first contact could be termed a second contact, and, similarly, a second contact could be termed a first contact, without departing from the intended scope.
  • the first contact and the second contact are both contacts, but they are not the same contact.
  • the term “if may be construed to mean “when” or “upon” or “in response to determining” or “in response to detecting,” depending on the context.
  • the phrase “if it is determined” or “if [a stated condition or event] is detected” may be construed to mean “upon determining” or “in response to determining” or “upon detecting [the stated condition or event]” or “in response to detecting [the stated condition or event],” depending on the context.
  • the image sensor may include a plurality of light-gathering pixels. At least some pixels may each include at least one photodiode and one or more readout circuits.
  • the signal readout circuits may be formed using one or more transistors.
  • the photodiodes may generate and accumulate photoelectrons when exposed to light. At readout, the photoelectrons may transfer out of the photodiodes, e.g., using the readout circuits, to generate analog image signals.
  • the image sensor may include one or more image signal processing circuits. The image signal processing circuits may process the analog image signals from the pixels to digital image signals.
  • the digital image signals may be transferred from the image sensor to one or more devices external to the image sensor, e.g., an image signal processor (ISP).
  • the ISP may further process the digital image signals to generate one or more images.
  • the image sensor may be implemented using one or more integrated circuits on a semiconductor die.
  • the image sensor may be part of an image capturing device, such as a camera, and the image capturing device may be part of an electronic device, such as a mobile device (e.g., a smart phone, tablet, laptop, etc.), robotic equipment, or a security monitoring device, among others.
  • the image signal processing circuits may include one or more analog-to-digital conversion (ADC) circuits which may convert the analog image signals of the pixels to digital image signals.
  • the image signal processing circuits may further include one or more darkness correction circuits, such as an auto black level correction (ABLC) circuit and an optical black (OB) pixel subtraction circuit.
  • ABLC auto black level correction
  • OB optical black
  • the ABLC circuit may correct darkness associated with the digital image signals of pixels on multiple rows or columns (e.g., pixels on all rows or columns of an image sensor), whereas the OB pixel subtraction circuit may correct blackness associated with the digital image signals of pixels on single row or column.
  • the ABLC and/or OB pixel subtraction circuits may be described with regards to rows or columns, because the pixels are generally organized into a pixel array and their image signals are read out and processed in groups row-by-row or column-by- column.
  • the ABLC circuit may determine a value representing a black level based on the values of digital image signals from pixels within a dark (or black) region of the image sensor.
  • these pixels may be optically and/or electrically shielded from the incident photons, and thus produce image signals only at very low values, e.g., below a threshold.
  • these pixels are also referred to as black pixels in the disclosure.
  • the black pixels may reside in a region covering multiple rows or columns of the pixel array of an image sensor.
  • the image signals from these black pixels may provide a representation of noises (e.g., thermal noises and/or other system noises), and their values may determine the darkness level of an image.
  • the ABLC circuit may subtract the determined value that represents the black level from the values of digital image signals of active (or non-black) pixels to thus cancel out the dark level from these active pixels’ output.
  • the OB subtraction circuit may operate similarly but with respect to pixels on a single row or column. For examples, given one row, the OB pixel subtraction circuit may determine a value representing a black level of this row based on the digital image signals of black pixels on this row, and then subtract the value from values of the digital image signals from the active (or non-black) pixels on the row.
  • the ABLC circuit may be considered to provide black correction on a “global” level of an image sensor
  • the OB pixel subtraction circuit may be considered to provide black correction on a “local” level on each row or column.
  • the digital image signals may be divided into a first set of digital image signals and a second set of digital image signals.
  • the first set of digital image signals may each have a value corresponding to an integer number of photoelectrons
  • the second set of image digital signals may each have a value between the values of the first subset of digital signals so as to correspond to a non-integer number of photoelectrons.
  • the digital image signals may include only the first set of digital image signals, without the second set of digital image signals.
  • the second set of digital image signals may be caused by noises, such as thermal noises (e.g., caused by temperature variation) and/or other system noises.
  • the second set of digital image signals may be useful for the above described darkness correction, because they provide the signal precision for the small values of image signals of black pixels. However, once these image signal processing is complete, the second set of digital image signals may not necessarily provide additional useful information as to the final image generation and displaying.
  • the image signal processing circuits of the image sensor may include a quantization circuit which may further quantize the digital image signals.
  • the quantization circuit may convert the second set of digital image signals to a third set of digital image signals, and the third set of digital image signals may each have a value corresponding to an integer number of photoelectrons.
  • the quantization circuit may compare the value of each one of the second set of digital image signals with one or more thresholds. Based on the comparison, the quantization circuit may identify a (new) value to replace the (original) value of the digital image signal to thus generate a (new) digital signal to form the third set of digital image signals.
  • the further quantized digital signal may have less bit depth so that the data throughput is compressed.
  • the values of the second set of digital image signals may be replaced with values of the third set of digital image signals (corresponding to integer numbers of discrete photoelectrons) or may be eliminated (e.g., when they are smaller than a specific threshold).
  • This may thus reduce the size of digital image signals to be transferred from an image sensor to the external devices, e.g., an image signal processor.
  • the data compression may increase data transfer rate and accelerate downstream image generation and processing.
  • the second set of digital image signals may be caused by noises.
  • the conversion of their values may be considered correction of non-ideal noisy image signals. This may improve the image signals quality and increase clarity and/or sharpness of the ultimately produced images.
  • FIG. 1 is a schematic diagram showing an example image signal processing system of an image sensor, according to some embodiments.
  • image sensor 100 may include a plurality of light-gathering pixels 102.
  • pixels 102 may each include at least one photodiode and one or more readout circuits.
  • the readout circuits may be formed using one or more transistors.
  • the photodiodes may generate and accumulate photoelectrons when exposed to light.
  • the photoelectrons may transfer out of the photodiodes, e.g., using the readout circuits, to generate analog image signals (e.g., analog voltages).
  • the at least one photodiode of the pixel may become coupled with a floating diffusion region (hereinafter “FD”),
  • the photoelectrons may transfer from the photodiode to the FD.
  • the FD may have capacitance, and transfer of the photoelectrons may cause a current to flow through the capacitance and thus generate an analog voltage.
  • the FD may be further coupled with a pixel signal output line (hereinafter “output”), through which the voltage of the FD may be accessed and sampled.
  • image sensor 100 may also include one or more amplifiers 104, and one or more analog-to-digital conversion (ADC) circuits and memory (e g., volatile (e.g., SRAM) or non-volatile memory) 106.
  • ADC analog-to-digital conversion
  • memory e g., volatile (e.g., SRAM) or non-volatile memory
  • amplifiers 104 may amplify the analog image signals of pixels 102
  • ADC circuits and memory 106 may convert the image signals from analog signals to digital signals and store them in the memory.
  • pixels 102 of image sensor 100 may be organized into a pixel array having multiple rows and columns.
  • the image signals of pixels 102 may be read out in groups row-by-row or column-by-column.
  • pixels 102 on the same row may be read out at or around the same time, whereas pixels 102 on the same column but different rows may be read out sequentially one row after another.
  • the column-by-column readout may be similar except that readout with respect to the rows and columns are exchanged.
  • image sensor 100 may also include one or more image signal processing circuits 108.
  • image signal processing circuits 108 may include at least one CDS circuit to perform correlated double sampling (CDS) calculations. With CDS, the analog image signal of each pixel may be sampled twice at the output to generate two digital values, one before and one after the transfer of photoelectrons. Then, the first sample may be subtracted from the second sample to cancel out effects of a reset voltage (of the FD). The final differential value may be considered the final digital signal value of the pixel.
  • image signal processing circuits 108 may also include one or more darkness correction circuits. As shown in FIG.
  • image signal processing circuits 108 may include at least one ABLC circuit 112.
  • ABLC circuit 112 may correct darkness associated with the digital image signals of pixels 102 on multiple rows or columns (e.g., pixels on all rows or columns of image sensor 100).
  • ABLC circuit 112 may identify a plurality of black pixels of pixels 102 that reside within one or more dark (or black) regions of image sensor 100. These black pixels may be identified because the values of their digital image signals may be less than a threshold.
  • ABLC circuit 112 may calculate an average value of these digital image signals, which may be considered representing the average black level. ABLC circuit 112 may then subtract the average value from the values of digital image signals of other active (or non-black) pixels to thus cancel out the average black level.
  • image signal processing circuits 108 may also include at least one OB pixel subtraction circuit 114.
  • OB pixel subtraction circuit 114 may operate similarly to above described ABLC circuit 112, but with respect to pixels on a single row or column. For example, given one row, OB pixel subtraction circuit 114 may identify black pixels on the row and then calculate an average from their digital image signals to represent black color of this row. OB pixel subtraction circuit 114 may then subtract this value from the values of digital image signals of other active (or non-black) pixels on the row. The same operations may be repeated on each row (or column) to thus compensate for the darkness row-by-row (or column-by-column).
  • ABLC circuit 112 and OB pixel subtraction circuit 114 may be used in combination within image sensor 100 to provide darkness correction, as shown in FIG. 1.
  • ABLC circuit 112 may subtract the average value representing the overall darkness level from also the values of digital image signals of the black pixels, such that only the remaining values (after the ABLC correction) of the black pixels (for each row or column) may be used by OB pixel subtraction 114 to correct darkness. This may avoid over-correction at the subsequent row or column level.
  • FIG. 1 is provided only as an example for purposes of illustration.
  • image signal processing circuits 108 may include less or more circuits.
  • image signal processing circuits 108 may include the defect pixel correction (DPC) circuit, digital gain (DG) circuit, etc.
  • DPC defect pixel correction
  • DG digital gain
  • the digital image signals of image sensor 100 may include a first set of digital image signals and a second set of digital image signals.
  • the first set of digital image signals may each have a value corresponding to an integer number of photoelectrons
  • the second set of image digital signals may each have a value between the values of the first set of digital signals and thus correspond to a non-integer number of photoelectrons.
  • the second set of digital image signals may be useful for darkness correction, because they provide the signal precision for the small values of image signals of black pixels.
  • the second set of digital image signals may not have values necessarily corresponding to “real” photoelectrons.
  • the second set of digital image signals may not provide additional useful information as to the final image generation and displaying
  • image signal processing circuit 108 may include quantization circuit 116 to further quantize the digital image signal.
  • quantization circuit 116 may convert the second set of digital signals to a third set of digital image signals, and the third set of digital image signals may each have a value corresponding to an integer number of discrete photoelectrons.
  • quantization circuit 116 may compare the value of the digital signal with a threshold. Based on the comparison, quantization circuit 116 may identify a (new) value to replace the (original) value of the digital signal to thus generate a (new) digital signal to form the third set of digital image signals.
  • image sensor 100 may also include data interface 118, through which the digital image signals including the first and third sets of digital image signals both only having values corresponding to integer numbers of discrete photoelectrons may be transferred from image sensor 100 to one or more devices external to image sensor 100, e.g., an image signal processor (ISP).
  • ISP image signal processor
  • the ISP may further process these digital images to finally reconstruct and generate one or more images.
  • FIG. 2 is a flowchart showing an example method for converting one set of digital image signals to another set of digital image signals, according to some embodiments.
  • a digital image signal e.g., one of the above described second set of digital image signals
  • a value corresponding to a non-integer number of discrete photoelectrons may be obtained (e g., by quantization circuit 116) as shown in block 202.
  • the value of the digital image signal may be compared with one or more thresholds, as shown in block 204. In some embodiments, based on the comparison, a new value may be identified, as shown in block 206.
  • the threshold may be value between two values (e.g., a and b) each corresponding to an integer number of discrete photoelectrons.
  • a and b may be the two closest values (each corresponding to an integer numbers of photoelectrons) to the original value of the digital image signal (corresponding to a non-integer number of photoelectrons).
  • the threshold may be value between two values (e.g., a and b) each corresponding to an integer number of discrete photoelectrons.
  • a and b may be the two closest values (each corresponding to an integer numbers of photoelectrons) to the original value of the digital image signal (corresponding to a non-integer number of photoelectrons).
  • the larger one of a and b may be identified as the new value.
  • the smaller one of a and b may be identified as the new value.
  • the original value of the digital image signal may be replaced with the identified new value, as shown in block 208.
  • the original value of the digital image signal may be replaced with one of the two values (e.g., a and b) based on the comparison. In other words, the original value may be rounded up or down to one of the closest values corresponding to an integer number of photoelectrons.
  • other implementations may be utilized to convert the non-integer digital image signals, such as division, multiplication, addition, subtraction, bit shifting, etc. For example, a non-integer value of 0.12 may be quantized to 1 via 3-bit shift (e.g., multiplied by 8) in combination with further addition and/or other arithmetic operations.
  • FIG. 3 shows an example probability distribution of analog image signals of an image sensor, according to some embodiments.
  • the diagram of FIG. 3 may represent the probability distribution of values of the analog image signals of FIG. 1 (before or after the image signal processing of 110, 112, and/or 114).
  • the horizonal axis represents values of the analog image signals
  • the vertical axis represents probability densities of the values.
  • the diagram may include several peaks, and each peak may correspond to an analog image signal corresponding to an integer number of photoelectrons. Since photoelectrons are discrete particles, these values may appear as peaks in the diagram.
  • the 1 st peak from the left may represent analog image signals having values corresponding to an integer (m) number of discrete photoelectrons
  • the 2 nd peak from the left may represent analog image signals having values corresponding to an integer (n) number of discrete photoelectrons
  • the diagram may also include some non-peak values between the peak values. These values may represent analog image signals having values corresponding to non-integer numbers of photoelectrons.
  • these signals may be caused by noises and thus tend to have values less than the peak. Thus, they show up as values between the peak values and surround the peak values following a Poisson-Gaussian distribution, for example.
  • the more the noise effects, the more these non-peak values, the wider the Poisson-Gaussian distribution curves, and eventually the Poisson-Gaussian curves of the peaks may become overlapped with each other.
  • FIG. 4 shows an example probability distribution of digital image signals of an image sensor, e.g., the analog image signals of FIG. 3 after amplification (e.g., by amplifiers 104) and digitalization (e.g., by ADC circuits 106), according to some embodiments.
  • FIG. 4 in the diagram of FIG. 4 is similar to the diagram of FIG. 3 except that values of FIG. 4 are quantized from analog signals to discrete digital values.
  • a zoomed view of the diagram between the values a and b is shown at the bottom of FIG. 4.
  • the digital image signals may include a first set of digital image signals, e.g., the two peaks at a and b each having a value corresponding to an integer number of discrete photoelectrons.
  • the 1 st peak at value a may correspond to an integer (m) number of discrete photoelectrons
  • the 2 nd peak at value b may correspond to an integer ( «) number of discrete photoelectrons.
  • the digital image signals in the zoomed view may also include a second set of digital image signals, e.g., the non-peak values between the two peaks of a and b. Thus, these values may be considered corresponding to non-integer numbers of discrete photoelectrons.
  • an image sensor may use a quantization circuit (e.g., quantization circuit 116) to convert these non-integer values each corresponding to a non-integer number of discrete photoelectrons) to values corresponding to integer numbers of discrete photoelectrons.
  • a quantization circuit e.g., quantization circuit 116
  • the quantization circuit may compare the non- integer values with a threshold having a value of (a + Z>)/2. When a non- integer value is larger than the threshold, the quantization circuit may round up the non- integer value to b.
  • the quantization circuit may round down the non- integer value to a.
  • the quantization circuit may use different thresholds to convert non- integer values residing within different ranges. For example, if the closest peaks change from (a, b) to (c, d), a different threshold may be used to perform the conversion of the non-integer values.
  • FIG. 5 shows an example probability distribution of digital image signals of an image sensor, e.g., the digital image signals after further quantization of FIG. 1 (by quantization circuit 116), according to some embodiments.
  • the non-peak values (corresponding to non-integer numbers of discrete photoelectrons) of FIG 4 are converted to integer values and thus disappear in FIG. 5.
  • the conversion of the non- integer values to integer values may cause changes of the probability densities of peak values (corresponding to integer numbers of discrete photoelectrons) compared to FIG. 4.
  • the above described quantization circuit 116 of image sensor 100 may be deactivated so that quantization circuit 116 may retain the values of the second set of digital image signals, without converting them to the second set of digital image signals to the third set of digital image signals having values corresponding to integer numbers of discrete photoelectrons.
  • image sensor 100 may have a bypass circuit, which when activated may enable to the digital image signals (including the second set of digital image signals) to bypass quantization circuit 116, so that the second set of digital image signals may not be converted to the third set of digital image signals.
  • image sensor 100 may separate the second set of digital image signals from the first set of digital image signals and transfer the second set of digital image signals from image sensor 100 to one or more external devices.
  • the second set of digital image signals may be transferred and thus available from image sensor 100, based on which the value(s) of the threshold(s) for the conversion may be determined.
  • the second set of digital image signals may be used for one or other more purposes, e.g., noise analysis, calibration, testing, etc.
  • FIG. 6 is a schematic diagram of an example image sensor including the above described quantization circuit, according to some embodiments.
  • image sensor 600 may include a plurality of light-gathering pixels 602 (e.g., similar to pixels 102) organized as a pixel array 620.
  • image sensor 600 may include one or more amplifiers 604 (e.g., similar to amplifiers 104 of FIG. 1), one or more ADC circuits and memory 608 (e.g., similar to ADC circuits and memory 104), and one or more image signal processing and data interface circuits 622 (e.g., including image signal processing circuits 108 and data interface circuit 118).
  • amplifiers 604 e.g., similar to amplifiers 104 of FIG. 1
  • ADC circuits and memory 608 e.g., similar to ADC circuits and memory 104
  • image signal processing and data interface circuits 622 e.g., including image signal processing circuits 108 and data interface circuit 118.
  • image signals of pixels 602 of pixel array 602 may be read out row-by-row or column-by-column.
  • image sensor 600 may also include row logic circuit 624 to provide control signals to perform row- by-row readout of pixels 602.
  • row logic circuit 624 to provide control signals to perform row- by-row readout of pixels 602.
  • the above readout of image signals of pixels 602 may be implemented using one or more readout circuits, e.g., implemented using transistors.
  • pixels 602 may each include at least one photodiode 632 and transistors 642, 644, 646, and 648.
  • Photodiode 632 may generate and accumulate photoelectrons when exposed to light.
  • RS row selection transistor
  • SF source follower transistor
  • RG reset gate transistor
  • reset voltage VDD 638 may be coupled with floating diffusion region (“FD”) 634, and FD 634 may be coupled with pixel signal output line (“output”) 636.
  • the voltage of FD 634 (e.g., having a value of reset voltage VDD) may be amplified by amplifiers 604, sampled by ADC circuits 608, and stored by memory 608. As described above, this may be the first sample of the CDS (correlated double sampling). Then, under control signals from row logic circuit 624, RG 644 may be turned off, and transfer gate transistor (“TG”) 642 may be turned on. Thus, the photoelectrons of photodiode 632 may transfer out of photodiode 632 to capacitance C of FD 634.
  • the transfer of photoelectrons may cause a current flowing through the capacitance C and thus result in an analog voltage, which may be further amplified by amplifiers 604, sampled by ADC circuits 608, and stored by memory 608. As described above, this may be the second sample of the CDS.
  • a CDS circuit e.g., similar to CDS circuit 110
  • the above described operations may be repeated on different pixels, e.g., according to the rows or columns.
  • image signal processing and data interface circuits 622 may include one or more darkness correction circuits (e.g., ABLC circuit 112 and/or OB pixel subtraction circuit 114 of FIG. 1), and at least one quantization circuit (e.g., quantization circuit 116).
  • the quantization circuit may further quantize the digital image signals of pixels 602, and convert values of the digital image signals corresponding to non-integer number of discrete photoelectrons to values corresponding to integer numbers of discrete photoelectrons. Also, as described above, in some embodiments, the conversion may include rounding up or down the non-integer values to closest integer values.
  • image sensor 600 may be implemented using one or more integrated circuits on a semiconductor die.
  • FIG. 7 is a flowchart showing an example method for processing image signals of an image sensor, according to some embodiments.
  • analog image signals may be generated using a plurality of pixels (e.g., the light-gathering pixels described above) based on photoelectrons accumulated by the pixels when exposed to light, as shown by block 702. For example, as described above, when exposed to light, the photodiodes of the pixels may generate and accumulate photoelectrons.
  • these photoelectrons may transfer out of the photodiodes to generate analog voltage signals at FD regions of the pixels, which may be further accessed at the output of the pixels.
  • the analog image signals e.g., the analog voltage signals
  • the analog image signals may be converted, using one or more ADC circuits (e.g., the ADC circuits described above), to digital signals, wherein the digital signals include (a) a first set of digital signals individually having a value corresponding to an integer number of discrete photoelectrons and (b) a second set of digital signals individually having a value between values of the first set of digital signals so as to correspond to a non-integer number of discrete photoelectron, as shown by block 704.
  • ADC circuits e.g., the ADC circuits described above
  • the analog image signals and the subsequently converted digital image signals may include a first set of peak values each corresponding to an integer number of discrete photoelectrons (accumulated by and transferred out of photodiodes) and a second set of non-peak value each corresponding to a non-integer number of discrete photoelectrons.
  • the second set of digital signals may be converted, e.g., using a quantization circuit (e.g., the quantization circuits described above), to a third set of digital signals, wherein the third set of digital signals individually has a value corresponding to an integer number of discrete photoelectrons, as shown by block 706.
  • the second set of non-peak values may be converted to a third set of values each corresponding to an integer number of discrete photoelectrons.
  • FIG. 8 illustrates a schematic representation of an example device 800 that may include an image capturing device (e.g., a camera) having an image sensor that includes at least one abovedescribed quantization circuit, according to some embodiments.
  • the device 800 may be a mobile device and/or a multifunction device.
  • the device 800 may be any of various types of devices, including, but not limited to, a personal computer system, desktop computer, laptop, notebook, tablet, slate, pad, or netbook computer, mainframe computer system, handheld computer, workstation, network computer, a camera, a set top box, a mobile device, an augmented reality (AR) and/or virtual reality (VR) headset, a consumer device, video game console, handheld video game device, application server, storage device, a television, a video recording device, a peripheral device such as a switch, modem, router, or in general any type of computing or electronic device.
  • a personal computer system desktop computer, laptop, notebook, tablet, slate, pad, or netbook computer
  • mainframe computer system handheld computer
  • workstation network computer
  • a camera a set top box
  • a mobile device an augmented reality (AR) and/or virtual reality (VR) headset
  • AR augmented reality
  • VR virtual reality
  • consumer device video game console
  • handheld video game device application server
  • storage device a television
  • the device 800 may include a display system 802 (e.g., comprising a display and/or a touch- sensitive surface) and/or one or more cameras 804.
  • the display system 802 and/or one or more front-facing cameras 804a may be provided at a front side of the device 800, e.g., as indicated in FIG. 8.
  • one or more rear-facing cameras 804b may be provided at a rear side of the device 800.
  • some or all of the cameras may be the same as, or similar to, each other. Additionally, or alternatively, some or all of the cameras may be different from each other.
  • the location(s) and/or arrangement(s) of the camera(s) 804 may be different than those indicated in FIG. 8.
  • the device 800 may include memory 806 (e g., comprising an operating system 808 and/or application(s)/program instructions 810), one or more processors and/or controllers 812 (e.g., comprising CPU(s), memory controller(s), display controller(s), and/or camera controller(s), etc.), and/or one or more sensors 816 (e.g., orientation sensor(s), proximity sensor(s), and/or position sensor(s), etc.).
  • the device 800 may communicate with one or more other devices and/or services, such as computing device(s) 818, cloud service(s) 820, etc., via one or more networks 822.
  • the device 800 may include a network interface (e.g., network interface 810) that enables the device 800 to transmit data to, and receive data from, the network(s) 822. Additionally, or alternatively, the device 800 may be capable of communicating with other devices via wireless communication using any of a variety of communications standards, protocols, and/or technologies.
  • a network interface e.g., network interface 810
  • the device 800 may be capable of communicating with other devices via wireless communication using any of a variety of communications standards, protocols, and/or technologies.
  • FIG. 9 illustrates a schematic block diagram of an example computing device, referred to as computer system 900, that may include or host embodiments of an image capturing device (e g., a camera) having an image sensor that includes at least one above described quantization circuit, according to some embodiments.
  • computer system 900 may implement methods for controlling operations of the camera and/or for performing image processing images captured with the camera.
  • the device 900 (described herein with reference to FIG. 9) may additionally, or alternatively, include some or all of the functional components of the computer system 900 described herein.
  • the computer system 900 may be configured to execute any or all of the embodiments described above.
  • computer system 900 may be any of various types of devices, including, but not limited to, a personal computer system, desktop computer, laptop, notebook, tablet, slate, pad, or netbook computer, mainframe computer system, handheld computer, workstation, network computer, a camera, a set top box, a mobile device, an augmented reality (AR) and/or virtual reality (VR) headset, a consumer device, video game console, handheld video game device, application server, storage device, a television, a video recording device, a peripheral device such as a switch, modem, router, or in general any type of computing or electronic device.
  • AR augmented reality
  • VR virtual reality
  • computer system 900 includes one or more processors 902 coupled to a system memory 904 via an input/output (I/O) interface 906.
  • Computer system 900 further includes one or more cameras 908 coupled to the I/O interface 906.
  • Computer system 900 further includes a network interface 910 coupled to I/O interface 906, and one or more input/output devices 912, such as cursor control device 914, keyboard 916, and display(s) 918.
  • embodiments may be implemented using a single instance of computer system 900, while in other embodiments multiple such systems, or multiple nodes making up computer system 900, may be configured to host different portions or instances of embodiments.
  • some elements may be implemented via one or more nodes of computer system 900 that are distinct from those nodes implementing other elements.
  • computer system 900 may be a uniprocessor system including one processor 902, or a multiprocessor system including several processors 902 (e.g., two, four, eight, or another suitable number).
  • processors 902 may be any suitable processor capable of executing instructions.
  • processors 902 may be general- purpose or embedded processors implementing any of a variety of instruction set architectures (ISAs), such as the x86, PowerPC, SPARC, or MIPS ISAs, or any other suitable ISA.
  • ISAs instruction set architectures
  • processors 902 may include additional types of processors, such as graphics processing units (GPUs), application specific integrated circuits (ASICs), etc.
  • processors 902 may commonly, but not necessarily, implement the same ISA.
  • computer system 900 may be implemented as a system on a chip (SoC).
  • SoC system on a chip
  • processors 902, memory 904, I/O interface 906 (e g. a fabric), etc. may be implemented in a single SoC comprising multiple components integrated into a single chip.
  • an SoC may include multiple CPU cores, a multi-core GPU, a multi-core neural engine, cache, one or more memories, etc. integrated into a single chip.
  • an SoC embodiment may implement a reduced instruction set computing (RISC) architecture, or any other suitable architecture.
  • RISC reduced instruction set computing
  • System memory 904 may be configured to store program instructions 920 accessible by processor 902.
  • system memory 904 may be implemented using any suitable memory technology, such as static random access memory (SRAM), synchronous dynamic RAM (SDRAM), nonvolatile/Flash-type memory, or any other type of memory.
  • existing camera control data 922 of memory 904 may include any of the information or data structures to implement the techniques described above.
  • program instructions 920 and/or data 922 may be received, sent or stored upon different types of computer- accessible media or on similar media separate from system memory 904 or computer system 900.
  • some or all of the functionality described herein may be implemented via such a computer system 900.
  • I/O interface 906 may be configured to coordinate I/O traffic between processor 902, system memory 904, and any peripheral devices in the device, including network interface 910 or other peripheral interfaces, such as input/output devices 912.
  • I/O interface 906 may perform any necessary protocol, timing or other data transformations to convert data signals from one component (e.g., system memory 904) into a format suitable for use by another component (e.g., processor 902).
  • I/O interface 906 may include support for devices attached through various types of peripheral buses, such as a variant of the Peripheral Component Interconnect (PCI) bus standard or the Universal Serial Bus (USB) standard, for example.
  • PCI Peripheral Component Interconnect
  • USB Universal Serial Bus
  • I/O interface 906 may be split into two or more separate components, such as a north bridge and a south bridge, for example. Also, in some embodiments some or all of the functionality of I/O interface 906, such as an interface to system memory 904, may be incorporated directly into processor 902.
  • Network interface 910 may be configured to allow data to be exchanged between computer system 900 and other devices attached to a network 924 (e.g., carrier or agent devices) or between nodes of computer system 900.
  • Network 924 may in various embodiments include one or more networks including but not limited to Local Area Networks (LANs) (e.g., an Ethernet or corporate network), Wide Area Networks (WANs) (e.g., the Internet), wireless data networks, some other electronic data network, or some combination thereof.
  • LANs Local Area Networks
  • WANs Wide Area Networks
  • wireless data networks some other electronic data network, or some combination thereof.
  • network interface 910 may support communication via wired or wireless general data networks, such as any suitable type of Ethernet network, for example; via telecommunications/telephony networks such as analog voice networks or digital fiber communications networks; via storage area networks such as Fibre Channel SANs, or via any other suitable type of network and/or protocol.
  • general data networks such as any suitable type of Ethernet network, for example; via telecommunications/telephony networks such as analog voice networks or digital fiber communications networks; via storage area networks such as Fibre Channel SANs, or via any other suitable type of network and/or protocol.
  • Input/output devices 912 may, in some embodiments, include one or more display terminals, keyboards, keypads, touchpads, scanning devices, voice or optical recognition devices, or any other devices suitable for entering or accessing data by one or more computer systems 900. Multiple input/output devices 912 may be present in computer system 900 or may be distributed on various nodes of computer system 900. In some embodiments, similar input/output devices may be separate from computer system 900 and may interact with one or more nodes of computer system 900 through a wired or wireless connection, such as over network interface 910.
  • computer system 900 is merely illustrative and is not intended to limit the scope of embodiments.
  • the computer system and devices may include any combination of hardware or software that can perform the indicated functions, including computers, network devices, Internet appliances, PDAs, wireless phones, pagers, etc.
  • Computer system 900 may also be connected to other devices that are not illustrated, or instead may operate as a stand-alone system.
  • the functionality provided by the illustrated components may in some embodiments be combined in fewer components or distributed in additional components.
  • the functionality of some of the illustrated components may not be provided and/or other additional functionality may be available.
  • instructions stored on a computer-accessible medium separate from computer system 900 may be transmitted to computer system 900 via transmission media or signals such as electrical, electromagnetic, or digital signals, conveyed via a communication medium such as a network and/or a wireless link.
  • Various embodiments may further include receiving, sending or storing instructions and/or data implemented in accordance with the foregoing description upon a computer-accessible medium.
  • a computer-accessible medium may include a non-transitory, computer-readable storage medium or memory medium such as magnetic or optical media, e.g., disk or DVD/CD-ROM, volatile or non-volatile media such as RAM (e.g. SDRAM, DDR, RDRAM, SRAM, etc.), ROM, etc.
  • a computer-accessible medium may include transmission media or signals such as electrical, electromagnetic, or digital signals, conveyed via a communication medium such as network and/or a wireless link.

Landscapes

  • Engineering & Computer Science (AREA)
  • Multimedia (AREA)
  • Signal Processing (AREA)
  • Transforming Light Signals Into Electric Signals (AREA)

Abstract

Un capteur d'image peut comprendre une pluralité de pixels, un ou plusieurs circuits de conversion analogique-numérique (CAN) et au moins un circuit de quantification. Les pixels peuvent générer des signaux analogiques sur la base de photoélectrons accumulés par les pixels lorsque ceux-ci sont exposés à la lumière. Les circuits CAN peuvent convertir les signaux analogiques en signaux numériques, les signaux numériques pouvant comprendre (a) un premier ensemble de signaux numériques comportant individuellement une valeur correspondant à un nombre entier de photoélectrons discrets, et (b) un deuxième ensemble de signaux numériques comportant individuellement une valeur choisie entre des valeurs du premier ensemble de signaux numériques de façon à correspondre à un nombre non entier de photoélectrons discrets. Le circuit de quantification peut convertir le deuxième ensemble de signaux numériques en un troisième ensemble de signaux numériques, le troisième ensemble de signaux numériques comportant individuellement une valeur correspondant à un nombre entier de photoélectrons discrets.
PCT/US2022/054105 2021-12-28 2022-12-27 Capteurs d'image comportant une compression de données can sur puce pour sorties de nombres d'électrons multibit Ceased WO2023129566A1 (fr)

Priority Applications (3)

Application Number Priority Date Filing Date Title
CN202280082733.7A CN118383039A (zh) 2021-12-28 2022-12-27 具有用于多位电子数输出的片上adc数据压缩的图像传感器
KR1020247019986A KR20240112303A (ko) 2021-12-28 2022-12-27 다중 비트 전자수 출력들을 위한 온 칩 adc 데이터 압축을 갖는 이미지 센서들
JP2024535779A JP2024546919A (ja) 2021-12-28 2022-12-27 マルチビット電子数出力のためのオンチップａｄｃデータ圧縮を有する画像センサ

Applications Claiming Priority (4)

Application Number Priority Date Filing Date Title
US202163294263P 2021-12-28 2021-12-28
US63/294,263 2021-12-28
US18/145,838 2022-12-22
US18/145,838 US12225316B2 (en) 2021-12-28 2022-12-22 Image sensors with on-chip ADC data compression for multi-bit electron-number outputs

Publications (1)

Publication Number Publication Date
WO2023129566A1 true WO2023129566A1 (fr) 2023-07-06

Family

ID=85199598

Family Applications (1)

Application Number Title Priority Date Filing Date
PCT/US2022/054105 Ceased WO2023129566A1 (fr) 2021-12-28 2022-12-27 Capteurs d'image comportant une compression de données can sur puce pour sorties de nombres d'électrons multibit

Country Status (3)

Country Link
JP (1) JP2024546919A (fr)
KR (1) KR20240112303A (fr)
WO (1) WO2023129566A1 (fr)

Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180091748A1 (en) * 2016-09-28 2018-03-29 Semiconductor Components Industries, Llc Image sensors having dark pixels
EP3716612A1 (fr) * 2017-11-24 2020-09-30 Hamamatsu Photonics K.K. Dispositif de comptage de photons et procédé de comptage de photons

Family Cites Families (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
JP2006229443A (ja) * 2005-02-16 2006-08-31 Casio Comput Co Ltd 画像処理装置、デジタルカメラおよびオプティカルブラック補正方法。
CN110572586A (zh) * 2012-05-02 2019-12-13 株式会社尼康 拍摄元件及电子设备

Patent Citations (2)

* Cited by examiner, † Cited by third party
Publication number Priority date Publication date Assignee Title
US20180091748A1 (en) * 2016-09-28 2018-03-29 Semiconductor Components Industries, Llc Image sensors having dark pixels
EP3716612A1 (fr) * 2017-11-24 2020-09-30 Hamamatsu Photonics K.K. Dispositif de comptage de photons et procédé de comptage de photons

Non-Patent Citations (2)

* Cited by examiner, † Cited by third party
Title
ABHIRAM GNANASAMBANDAM ET AL: "Megapixel Photon-Counting Color Imaging using Quanta Image Sensor", ARXIV.ORG, CORNELL UNIVERSITY LIBRARY, 201 OLIN LIBRARY CORNELL UNIVERSITY ITHACA, NY 14853, 21 March 2019 (2019-03-21), pages 1 - 13, XP081377438, DOI: 10.1364/OE.27.017298 *
FOSSUM ERIC R: "Photon Counting Error Rates in Single-Bit and Multi-Bit Quanta Image Sensors", IEEE JOURNAL OF THE ELECTRON DEVICES SOCIETY, IEEE, USA, vol. 4, no. 3, 1 May 2016 (2016-05-01), pages 136 - 143, XP011607397, DOI: 10.1109/JEDS.2016.2536722 *

Also Published As

Publication number Publication date
KR20240112303A (ko) 2024-07-18
JP2024546919A (ja) 2024-12-26

Similar Documents

Publication Publication Date Title
US9344649B2 (en) Floating point image sensors with different integration times
US10277843B2 (en) Oversampled image sensor with conditional pixel readout
US11812169B2 (en) Dynamic vision sensor with in-pixel digital change detection
US11956560B2 (en) Digital pixel sensor having reduced quantization operation
US9307172B2 (en) Floating point image sensors with tile-based memory
US20210051284A1 (en) Imaging systems and methods for performing analog domain regional pixel level feature extraction
GB2575137A (en) An image sensor and an image dynamic information processing method
JP2024513276A (ja) 画像センサ、その画像出力方法及び使用
CN117544865B (zh) 面向多列像素复用读出的低面积数字相关双采样电路
US12407948B2 (en) Imaging circuitry with high frame rate edge detection
US12225316B2 (en) Image sensors with on-chip ADC data compression for multi-bit electron-number outputs
CN116055905B (zh) 图像传感器、摄像模组及电子设备
WO2023129566A1 (fr) Capteurs d'image comportant une compression de données can sur puce pour sorties de nombres d'électrons multibit
US12231793B2 (en) Low row noise ramp generator
TWI695623B (zh) 局部曝光感測器及其操作方法
CN118383039A (zh) 具有用于多位电子数输出的片上adc数据压缩的图像传感器
US12452562B1 (en) Fast DCG pixel readout scheme for HDR CMOS image sensors
US20250365518A1 (en) Self-Adaptive Multi-Conversion Gain Pixel
US12231792B1 (en) Pixel output settling assist with dynamic bias current
US20220014696A1 (en) Image sensor with dark reference pixel oversampling
García-Lesta et al. HOPBAS10K: A 98× 98 Pixels CMOS Vision Sensor for Background Subtraction
US11962923B2 (en) Image processing system and method of operating the same
US12126928B2 (en) Column readout amplifier for image sensors
US20220141405A1 (en) Digital pixel sensor with adaptive noise reduction
CN118402252A (zh) 低行噪声斜坡发生器

Legal Events

Date Code Title Description
121 Ep: the epo has been informed by wipo that ep was designated in this application

Ref document number: 22854624

Country of ref document: EP

Kind code of ref document: A1

WWE Wipo information: entry into national phase

Ref document number: 202280082733.7

Country of ref document: CN

ENP Entry into the national phase

Ref document number: 20247019986

Country of ref document: KR

Kind code of ref document: A

WWE Wipo information: entry into national phase

Ref document number: 2024535779

Country of ref document: JP

Ref document number: 1020247019986

Country of ref document: KR

NENP Non-entry into the national phase

Ref country code: DE

122 Ep: pct application non-entry in european phase

Ref document number: 22854624

Country of ref document: EP

Kind code of ref document: A1