WO2024110959A1 - Multiple gated pixel per light pulse and/or gated event counter for single photon avalanche diode (spad) - Google Patents
Multiple gated pixel per light pulse and/or gated event counter for single photon avalanche diode (spad) Download PDFInfo
- Publication number
- WO2024110959A1 WO2024110959A1 PCT/IL2023/051194 IL2023051194W WO2024110959A1 WO 2024110959 A1 WO2024110959 A1 WO 2024110959A1 IL 2023051194 W IL2023051194 W IL 2023051194W WO 2024110959 A1 WO2024110959 A1 WO 2024110959A1
- Authority
- WO
- WIPO (PCT)
- Prior art keywords
- given pixel
- environment
- pixels
- respective given
- dof
- Prior art date
- Legal status (The legal status is an assumption and is not a legal conclusion. Google has not performed a legal analysis and makes no representation as to the accuracy of the status listed.)
- Ceased
Links
Classifications
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/02—Systems using the reflection of electromagnetic waves other than radio waves
- G01S17/06—Systems determining position data of a target
- G01S17/08—Systems determining position data of a target for measuring distance only
- G01S17/10—Systems determining position data of a target for measuring distance only using transmission of interrupted, pulse-modulated waves
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S17/00—Systems using the reflection or reradiation of electromagnetic waves other than radio waves, e.g. lidar systems
- G01S17/88—Lidar systems specially adapted for specific applications
- G01S17/89—Lidar systems specially adapted for specific applications for mapping or imaging
- G01S17/894—3D imaging with simultaneous measurement of time-of-flight at a 2D array of receiver pixels, e.g. time-of-flight cameras or flash lidar
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/483—Details of pulse systems
- G01S7/486—Receivers
- G01S7/4861—Circuits for detection, sampling, integration or read-out
- G01S7/4863—Detector arrays, e.g. charge-transfer gates
-
- G—PHYSICS
- G01—MEASURING; TESTING
- G01S—RADIO DIRECTION-FINDING; RADIO NAVIGATION; DETERMINING DISTANCE OR VELOCITY BY USE OF RADIO WAVES; LOCATING OR PRESENCE-DETECTING BY USE OF THE REFLECTION OR RERADIATION OF RADIO WAVES; ANALOGOUS ARRANGEMENTS USING OTHER WAVES
- G01S7/00—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00
- G01S7/48—Details of systems according to groups G01S13/00, G01S15/00, G01S17/00 of systems according to group G01S17/00
- G01S7/483—Details of pulse systems
- G01S7/486—Receivers
- G01S7/4865—Time delay measurement, e.g. time-of-flight measurement, time of arrival measurement or determining the exact position of a peak
Definitions
- the present invention relates to the field of active imaging by a multiple gated pixel per light pulse method and/or a gated event counter for Single Photon Avalanche Diode (SPAD) method.
- a multiple gated pixel per light pulse method and/or a gated event counter for Single Photon Avalanche Diode (SPAD) method is a multiple gated pixel per light pulse method and/or a gated event counter for Single Photon Avalanche Diode (SPAD) method.
- SPAD Single Photon Avalanche Diode
- a current solution for active imaging of two or more Depths of Field (DoF’ s) in an environment in a single frame uses a gated image sensor that includes pixels having a single memory node. Since the pixels have only a single memory node, only a single digital image is generated for a frame. As a result thereof, the range of an object that is present in one of the two or more DoF’s that are actively imaged cannot be determined.
- An object of the present disclosure is to provide a solution for this problem.
- an active-gated imaging system for imaging a Depth of Field (DoF) in an environment at a predefined distance
- the system comprising: a light source configured to emit light pulses towards the environment; and a gated sensor comprising: (i) an array of pixels, wherein given pixels of the pixels include: a photodetector, a first transfer gate switch, a second transfer gate switch, a first memory node associated with the first transfer gate switch, and a second memory node associated with the second transfer gate switch; and (ii) one or more gating controllers associated with the given pixels, each of the gating controllers being configured, for one or more of the given pixels, to: (A) activate the first transfer gate switch of the respective given pixel, in a first synchronization scheme with an emission of the light pulses towards the environment, to accumulate, at the first memory node of the respective given pixel, a converted light signal that is based on returns of the light pulses from the
- the second converted light signal represents the ambient light of the environment without said returns
- the system further comprises: at least one second controller configured, for each of the given pixels, to: associate: (i) the converted light signal of the respective given pixel with a first grey level for the respective given pixel and (ii) the second converted light signal of the respective given pixel with a second grey level for the respective given pixel; and perform image subtraction for the respective given pixel, based on the first grey level and the second grey level, to provide a compensation grey level for the respective given pixel, the compensation grey level being the first grey level as compensated to reduce an effect of the ambient light on the first grey level; wherein an image of the DoF at the predefined distance is formed from the compensation grey levels of the given pixels.
- an imaging system for imaging a Depth of Field (DoF) in an environment at a predefined distance, the system comprising: a light source configured to emit light pulses towards the environment; and a digital camera comprising: (i) a Single Photon Avalanche Diode (SPAD) sensor having an array of pixels, wherein given pixels of the pixels include: (a) an event counter for counting avalanche events at the respective given pixel and (b) a transfer gate switch for gating the event counter; (ii) one or more gating controllers associated with the given pixels, each of the gating controllers being configured, for one or more of the given pixels, to control the transfer gate switch of the respective given pixel, in synchronization with an emission of the light pulses towards the environment, to: (a) pause the event counter of the respective given pixel prior and subsequent to the respective given pixel capturing a reflection signal indicative of a reflection of the light pulses returning from the DoF
- DoF Depth of Field
- an active-gated imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance, the method comprising: emitting light pulses, by a light source, towards the environment; and for given pixels in a gated sensor comprising an array of pixels, the given pixels including a photodetector, a first transfer gate switch, a second transfer gate switch, a first memory node associated with the first transfer gate switch, and a second memory node associated with the second transfer gate switch: (A) activating the first transfer gate switch of the respective given pixel, in a first synchronization scheme with an emission of the light pulses towards the environment, to accumulate, at the first memory node of the respective given pixel, a converted light signal that is based on returns of the light pulses from the DoF and an ambient light of the environment; and (B) activating the second transfer gate switch of the respective given pixel, in a second synchronization scheme with the emission of the light pulse
- the second converted light signal represents the ambient light of the environment without said returns
- the method further comprises: for each of the given pixels: associating: (i) the converted light signal of the respective given pixel with a first grey level for the respective given pixel and (ii) the second converted light signal of the respective given pixel with a second grey level for the respective given pixel; and performing image subtraction for the respective given pixel, based on the first grey level and the second grey level, to provide a compensation grey level for the respective given pixel, the compensation grey level being the first grey level as compensated to reduce an effect of the ambient light on the first grey level; wherein an image of the DoF at the predefined distance is formed from the compensation grey levels of the given pixels.
- an imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance comprising: emitting light pulses, by a light source, towards the environment; for a digital camera comprising a Single Photon Avalanche Diode (SPAD) sensor having an array of pixels, wherein given pixels of the pixels include: (a) an event counter for counting avalanche events at the respective given pixel and (b) a transfer gate switch for gating the event counter: for the given pixels: controlling the transfer gate switch of the respective given pixel, in synchronization with an emission of the light pulses towards the environment, to: (a) pause the event counter of the respective given pixel prior and subsequent to the respective given pixel capturing a reflection signal indicative of a reflection of the light pulses returning from the DoF and (b) activate the event counter of the respective given pixel to count the avalanche events at the respective given pixel resulting from the capturing of the
- a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, executable by at least one controller of a computer to perform an active-gated imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance, the method comprising: emitting light pulses, by a light source, towards the environment; and for given pixels in a gated sensor comprising an array of pixels, the given pixels including a photodetector, a first transfer gate switch, a second transfer gate switch, a first memory node associated with the first transfer gate switch, and a second memory node associated with the second transfer gate switch: (A) activating the first transfer gate switch of the respective given pixel, in a first synchronization scheme with an emission of the light pulses towards the environment, to accumulate, at the first memory node of the respective given pixel, a converted light signal that is based on returns of the light pulses from
- a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, executable by at least one controller of a computer to perform an imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance, the method comprising: emitting light pulses, by a light source, towards the environment; for a digital camera comprising a Single Photon Avalanche Diode (SPAD) sensor having an array of pixels, wherein given pixels of the pixels include: (a) an event counter for counting avalanche events at the respective given pixel and (b) a transfer gate switch for gating the event counter: for the given pixels: controlling the transfer gate switch of the respective given pixel, in synchronization with an emission of the light pulses towards the environment, to: (a) pause the event counter of the respective given pixel prior and subsequent to the respective given pixel capturing a reflection signal indicative of a reflection of the
- Fig. 1 is a block diagram schematically illustrating one example of a pixel in a gated image sensor, in accordance with the presently disclosed subject matter
- Fig. 2 is a flowchart illustrating one example of a sequence of operations for active imaging of two or more Depths of Field (DoF’s) in an environment using a gated image sensor having pixels that include two or more memory nodes, in accordance with the presently disclosed subject matter;
- DoF Depths of Field
- Fig. 3 is one example of a timing diagram for a respective given pixel in the gated image sensor that includes two memory nodes for active imaging of two DoF’s in an environment, in accordance with the presently disclosed subject matter;
- Fig. 4 is a flowchart illustrating one example of a sequence of operations for active imaging of a given DoF in an environment while reducing an effect of ambient light and parasitic light on the imaging using a gated image sensor having pixels that include at least two memory nodes, in accordance with the presently disclosed subject matter;
- Fig. 5 is one example of a timing diagram for a respective given pixel in the gated image sensor that includes two memory nodes for reducing an effect of ambient light and parasitic light on an image of a given DoF in an environment, in accordance with the presently disclosed subject matter;
- Fig. 6 is a flowchart illustrating one example of a sequence of operations for active imaging of a given DoF in an environment while reducing an effect of parasitic light on the imaging using a gated image sensor having pixels that include at least two memory nodes, in accordance with the presently disclosed subject matter;
- Fig. 7 is one example of a timing diagram for a respective given pixel in the gated image sensor that includes two memory nodes for reducing an effect of parasitic light on an image of a given DoF in an environment, in accordance with the presently disclosed subject matter;
- Fig. 8 is a block diagram schematically illustrating one example of a pixel in a Single Photon Avalanche Diode (SPAD) sensor, in accordance with the presently disclosed subject matter; and
- Fig. 9 is a flowchart illustrating one example of a sequence of operations for active gated imaging of a given DoF in an environment using a SPAD sensor, in accordance with the presently disclosed subject matter.
- ⁇ should be expansively construed to cover any kind of electronic device with data processing capabilities, including, by way of non-limiting example, a personal desktop/laptop computer, a server, a computing system, a communication device, a smartphone, a tablet computer, a smart television, a processor (e.g. digital signal processor (DSP), a microcontroller, a field-programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), a group of multiple physical machines sharing performance of various tasks, virtual servers co-residing on a single physical machine, any other electronic computing device, and/or any combination thereof.
- DSP digital signal processor
- FPGA field-programmable gate array
- ASIC application specific integrated circuit
- the phrase “for example,” “such as”, “for instance” and variants thereof describe non-limiting embodiments of the presently disclosed subject matter.
- Reference in the specification to “one case”, “some cases”, “other cases” or variants thereof means that a particular feature, structure or characteristic described in connection with the embodiment(s) is included in at least one embodiment of the presently disclosed subject matter.
- the appearance of the phrase “one case”, “some cases”, “other cases” or variants thereof does not necessarily refer to the same embodiment(s).
- fewer, more and/or different stages than those shown in Figs. 2, 4, 6 and 9 may be executed.
- one or more stages illustrated in Figs. 2, 4, 6 and 9 may be executed in a different order and/or one or more groups of stages may be executed simultaneously.
- Any reference in the specification to a method should be applied mutatis mutandis to a system capable of executing the method and should be applied mutatis mutandis to a non- transitory computer readable medium that stores instructions that once executed by a computer result in the execution of the method.
- Any reference in the specification to a system should be applied mutatis mutandis to a method that may be executed by the system and should be applied mutatis mutandis to a non- transitory computer readable medium that stores instructions that may be executed by the system.
- Any reference in the specification to a non-transitory computer readable medium should be applied mutatis mutandis to a system capable of executing the instructions stored in the non-transitory computer readable medium and should be applied mutatis mutandis to method that may be executed by a computer that reads the instructions stored in the non- transitory computer readable medium.
- FIG. 1 a block diagram schematically illustrating one example of a pixel 100 in a gated image sensor, in accordance with the presently disclosed subject matter.
- each pixel 100 in a gated image sensor includes a photodetector (PD) 110.
- the photodetector 110 is a photodiode.
- the photodiode 110 is a pinned photodiode.
- the photodetector 110 converts return light pulses that arrive at the photodetector 110 from an environment into photo-electrons.
- TX1 transfer gate switch
- TX2 second transfer gate switch
- MN1 first memory node
- MN2 second memory node
- PD RST photodetector reset switch
- a pixel may also include three or more transfer gate switches and a corresponding three or more memory nodes.
- each of the given pixels can have a standard electric signal chain after the “gate-able” PD 110, TX1 112, TX2 114 and PD RST 122 configuration.
- This standard electric signal chain can include at least one Reset transistor (RST) 124 with the role of resetting the memory nodes (e.g., MN1 116 & MN2 118) using the pixel voltage (VDD). It is to be noted that, in some cases, one Reset transistor (RST) 124 may be used to reset the memory nodes (e.g., MN1 116 & MN2 118), as illustrated in Fig. 1.
- RST Reset transistor
- this standard electrical signal chain can also include a number of Source Follower (SF) transistors (e.g., 126 and 128), wherein each of the SF transistors converts an accumulated signal (i.e., electrons) at a respective memory node (e.g., MN1 or MN2) to a voltage.
- SF Source Follower
- the standard electrical signal chain can also include a number of Select transistors (e.g., SEL1 132 & SEL2 134) connected to a column and/or row of the pixel array of the gated image sensor, each of the Select transistors (e.g., SEL1 132 & SEL2 134) being associated with a different memory node of the memory nodes (e.g., MN1 & MN2), and enabling the contents of the respective memory node associated with the respective Select transistor to be read-out after each frame on a given output channel (e.g., Readoutl or Readout2).
- a given output channel e.g., Readoutl or Readout2
- SEL1 132 is activated to enable the contents of MN 1 116 to be read out after each frame on a first output channel (e.g., Readoutl), and SEL2 134 is activated to enable the contents of MN2 118 to be read out after each frame on a second output channel (e.g., Readout2) that is distinct from the first output channel (e.g., Readoutl).
- first output channel e.g., Readoutl
- SEL2 134 is activated to enable the contents of MN2 118 to be read out after each frame on a second output channel (e.g., Readout2) that is distinct from the first output channel (e.g., Readoutl).
- FIG. 2 a flowchart illustrating one example of a sequence of operations 200 for active imaging of two or more Depths of Field (DoF’s) in an environment using a gated image sensor having pixels 100 that include two or more memory nodes (e.g., MN1 116 and MN2 118), in accordance with the presently disclosed subject matter.
- DoF Depths of Field
- the controllers can include one or more gating controllers associated with given pixels 100 of the pixels (e.g., all of the pixels) of the gated image sensor that include multiple transfer gate switches and corresponding multiple memory nodes.
- the active gated imaging system can be configured to control the light source to emit light pulses of a frame towards the environment.
- Fig. 3 one (non-limiting) example of a timing diagram 250 for a respective given pixel 100 in the gated image sensor that includes two memory nodes (e.g., MN1 116 and MN2 118) for active imaging of two DoF’s in an environment, in accordance with the presently disclosed subject matter.
- Fig. 3 illustrates two successive light pulses (252, 254) of the light pulses of the frame that are emitted towards the environment.
- At least a subset (e.g., all) of the given pixels 100 in the gated image sensor that include two or more memory nodes, including, inter alia, the respective given pixel 100, are synchronized to the light pulses (e.g., 252, 254) that are emitted towards the environment in accordance with the same timing diagram (e.g., 250) to achieve active imaging of two or more DoFs in the environment.
- the light pulses e.g., 252, 254
- a gating controller (being the same or different than the controller that controls the emission of light pulses) that is associated with a respective given pixel 100 can be configured to gate the respective given pixel 100 in a first synchronization scheme with the emission of the given light pulse (e.g., light pulse 252).
- the respective given pixel 100 can be gated, as illustrated in Fig.
- the gating controller can also be configured to complete the gating of the respective given pixel 100 in the first synchronization scheme by deactivating the first transfer gate switch (TX1 112) of the respective given pixel 100 at a later time (e.g., ti + At) following the emission of the given light pulse (e.g., by switching TX1 112 from 1 to 0), as illustrated in Fig. 3.
- TX1 112 the first transfer gate switch
- the gating controller In gating the respective given pixel 100 in the first synchronization scheme, the gating controller enables accumulating, at the first memory node (MN1 116) of the respective given pixel 100, a converted light reading that is based on returns of the given light pulse (e.g., 252) that arrive at the respective given pixel 100 during the time period between the deactivation of PD RST 122 at time t2 following the emission of the given light pulse and the deactivation of TX1 112 at time ti + At following the emission of the given light pulse.
- MN1 116 the first memory node
- a converted light reading that is based on returns of the given light pulse (e.g., 252) that arrive at the respective given pixel 100 during the time period between the deactivation of PD RST 122 at time t2 following the emission of the given light pulse and the deactivation of TX1 112 at time ti + At following the emission of the given light pulse.
- the gating controller can be configured to activate PD RST 122 (e.g., by switching PD RST from 0 to 1) following the deactivation of TX1 112.
- the gating controller that is associated with the respective given pixel 100 can be configured to gate the respective given pixel 100 in a second synchronization scheme with the emission of the given light pulse (e.g., light pulse 252).
- the respective given pixel 100 can be gated in the second synchronization scheme, as illustrated in Fig.
- the gating controller can be further configured to complete the gating of the respective given pixel 100 in the second synchronization scheme by deactivating the second transfer gate switch (TX2 114) of the respective given pixel 100 at a later time (e.g., t3 + Ati) following the emission of the given light pulse (e.g., by switching TX2 114 from 1 to 0), as illustrated in Fig. 3.
- TX2 114 the second transfer gate switch
- the gating controller enables accumulating, at the second memory node (MN2 118) of the respective given pixel 100, a second converted light reading that is based on returns of the given light pulse (e.g., 252) that arrive at the respective given pixel 100 during the time period between the deactivation of PD RST 122 at time b + U following the emission of the given light pulse and the deactivation of TX2 114 at time t3 + Ati following the emission of the given light pulse.
- These returns of the given light pulse e.g., 252 are from a second Depth of Field (DoF) in the environment, distinct from the first DoF (in Fig.
- the second DoF is at a greater distance from the active gated imaging system than the first DoF), and not from other DoF’s in the environment, other than the second DoF (block 212).
- the gating controller can be configured to activate PD RST 122 following the deactivation of TX2 114.
- the period of time during which TX1 112 is active can be the same or different than the period of time during which TX2 114 is active (e.g., Ati).
- the respective given pixel 100 can include three or more memory nodes, each of which accumulate a converted light reading that is based on returns of the given light pulse (e.g., 252) from a different DoF in the environment.
- the gating controller can be configured to repeat the aforementioned operations discussed above with respect to the given light pulse 252 (including, inter alia, blocks 208 and 212) for each emitted light pulse in the frame (e.g., light pulse 254, etc.), as illustrated in Fig. 3 (block 216).
- a controller (the same or different than the gating controller) can be configured, for each of the given pixels 100 in the subset of the given pixels (e.g., for each of the given pixels in the gated image sensor), to read-out on a first output channel (e.g., Readoutl) of the respective given pixel 100, into a digital domain, a converted light signal from MN1 116 of the respective given pixel 100, the converted light signal being generated based on the converted light readings accumulated at MN1 116 of the respective given pixel 100 over the course of the frame, giving rise to a first digital image of the first DoF (block 220).
- a first output channel e.g., Readoutl
- this controller can be configured, for each of the given pixels 100 in the subset of the given pixels, to read-out on a second output channel (e.g., Readout2) of the respective given pixel, into a digital domain, a second converted light signal from MN2 118 of the respective given pixel 100, the second converted light signal being generated based on the second converted light readings accumulated at MN2 118 of the respective given pixel 100 over the course of the frame, giving rise to a second digital image of the second DoF (block 224).
- a second output channel e.g., Readout2
- N digital images of N DoFs can be generated, provided that the respective given pixel 100 includes N memory nodes.
- the range of any illuminated object in any one of the images can be determined (if the object is illuminated in the first digital image, its range corresponds to the first DoF; if it is illuminated in the second digital image, its range corresponds to the second DoF).
- FIG. 4 a flowchart illustrating one example of a sequence of operations 300 for active imaging of a given DoF in an environment while reducing an effect of ambient light and parasitic light on the imaging using a gated image sensor having pixels 100 that include at least two memory nodes (e.g., MN1 116 & MN2 118), in accordance with the presently disclosed subject matter.
- MN1 116 & MN2 118 memory nodes
- the controllers can include one or more gating controllers associated with given pixels 100 of the pixels (e.g., all of the pixels) of the gated image sensor that include multiple transfer gate switches and corresponding multiple memory nodes.
- the active gated imaging system can be configured to control the light source to emit light pulses of a frame towards the environment.
- Fig. 5 one (non-limiting) example of a timing diagram 350 for a respective given pixel 100 in the gated image sensor that include two memory nodes (e.g., MN1 116 and MN2 118) for reducing an effect of ambient light and parasitic light on an image of a given DoF in the environment, in accordance with the presently disclosed subject matter.
- Fig. 5 illustrates two successive light pulses (252, 254) of the light pulses of the frame that are emitted towards the environment.
- At least a subset of the given pixels 100 (e.g., all of the given pixels 100) in the gated image sensor that include two or more memory nodes, including, inter alia, the respective given pixel 100, are synchronized to the light pulses (e.g., 252, 254) that are emitted towards the environment in accordance with the same timing diagram (e.g., 350) to reduce an effect of ambient light and parasitic light on an image of a given DoF in the environment.
- the light pulses e.g., 252, 254
- a gating controller (being the same or different than the controller that controls the emission of light pulses) that is associated with a respective given pixel 100 can be configured to gate the respective given pixel 100 in a first synchronization scheme with the emission of the given light pulse (e.g., light pulse 252).
- the respective given pixel 100 can be gated, as illustrated in Fig.
- the gating controller can be further configured to complete the gating of the respective given pixel 100 in the first synchronization scheme by deactivating the first transfer gate switch (TX1 112) of the respective given pixel 100 at a later time (e.g., tl + At) following the emission of the given light pulse (e.g., by switching TX1 112 from 1 to 0), as illustrated in Fig. 5.
- TX1 112 the first transfer gate switch
- the gating controller enables accumulating, at the first memory node (MN1 116) of the respective given pixel 100, a converted light reading that is based on: (i) returns of the given light pulse (e.g., 252) from a given Depth of Field (DoF) in the environment and (ii) an ambient light in the environment (e.g., light from non-pulsed Continuous Wave (CW) ambient sources) that arrives at the respective given pixel 100 during the time period between the deactivation of PD RST 122 at time t2 following the emission of the given light pulse and the deactivation of TX1 112 at time ti + At following the emission of the given light pulse.
- CW Continuous Wave
- the returns of the given light pulse are not from other DoFs in the environment, other than the given DoF (block 308).
- the gating controller can be configured to activate PD RST 122 (e.g., by switching PD RST 122 from 0 to 1) following the deactivation of TX1 112.
- the gating controller that is associated with the respective given pixel 100 can be configured to gate the respective given pixel 100 in a second synchronization scheme with the emission of the given light pulse (e.g., 252).
- the respective given pixel 100 can be gated in the second synchronization scheme, as illustrated in Fig.
- the gating controller can be further configured to complete the gating of the respective given pixel 100 in the second synchronization scheme by deactivating the second transfer gate switch (TX2 114) of the respective given pixel 100 at a later time (e.g., t6 + At) following the emission of the given light pulse (e.g., by switching TX2 114 from 1 to 0), as illustrated in Fig. 5.
- TX2 114 the second transfer gate switch
- the gating controller enables accumulating, at the second memory node (MN2 118) of the respective given pixel 100, a second converted light reading that is based on the ambient light in the environment during the time period between the deactivation of PD RST 122 at time t2 + ts following the emission of the given light pulse (e.g., 252) and the deactivation of TX2 114 at time t6 + At following the emission of the given light pulse.
- the second converted light reading does not include or substantially does not include returns of the given light pulse (e.g., 252) (no or negligible reflections from the given light pulse are collected) (block 312).
- the second converted light reading is collected over the same time period as the converted light reading (the converted light reading being based on returns of the given light pulse (e.g., 252) from a given Depth of Field (DoF) in the environment and an ambient light in the environment (e.g., light from non-pulsed (CW) ambient sources), as discussed above).
- the converted light reading being based on returns of the given light pulse (e.g., 252) from a given Depth of Field (DoF) in the environment and an ambient light in the environment (e.g., light from non-pulsed (CW) ambient sources), as discussed above).
- TX1 112 and TX2 114 are opened for the same duration (At), as illustrated in Fig. 5.
- TX2 114 is activated after all relevant returns from the given light pulse have already passed the camera (“long delay”).
- TX2 114 is activated to ensure that no or a negligible amount of the second converted light reading is based on returns of the given light pulse (e.g., 252).
- the gating controller can be configured to activate PD RST 122 (e.g., by switching PD RST 122 from 0 to 1) following the deactivation of TX2 114.
- the gating controller can be configured to repeat the aforementioned operations discussed above with respect to the given light pulse 252 (including, inter alia, blocks 308 and 312) for each emitted light pulse in the frame (e.g., light pulse 254, etc.), as illustrated in Fig. 5 (block 316).
- a controller (the same or different than the gating controller) can be configured, for each of the given pixels 100 in the subset of the given pixels, to read-out, into a digital domain, a converted light signal from MN1 116 of the respective given pixel 100, the converted light signal being generated based on the converted light readings accumulated at MN1 116 of the respective given pixel 100 over the course of the frame, giving rise to a first digital image of the given DoF based on the returns of the light pulses (e.g., 252, 254) of the frame from the given DoF and the ambient light (block 320).
- the light pulses e.g., 252, 254
- this controller can be configured, for each of the given pixels 100 in the subset of the given pixels, to read-out, into a digital domain, a second converted light signal from MN2 118 of the respective given pixel 100, the second converted light signal being generated based on the second converted light readings accumulated at MN2 118 of the respective given pixel 100 over the course of the frame, giving rise to a second digital image that is based on the ambient light in the environment (the same or approximately the same quantity of ambient light as the first digital image), and that is not based (or is negligibly based) on returns of the lights pulses of the frame arriving at the gated image sensor (block 324).
- the controller can be configured to subtract the second digital image from the first digital image, giving rise to a compensated digital image of the given DoF that is less affected by the ambient light than the first digital image (or even unaffected by the ambient light). This is advantageous for several reasons, including, inter alia, that the ambient light inserts light into the given pixels un-correlated to the given DoF, it might reduce image contrast, etc.
- the memory nodes MN1 116 and MN2 118
- MN1 116 and MN2 118 each collect some light directly (not thorough the PD) during the whole frame time (from reset till readout) both when the gate is open or closed, resulting in the parasitic light being accumulated in MN1 116 and MN2 118. Accordingly, by subtracting the second digital image from the first digital image to generate the compensated digital image of the given DoF, the parasitic light present in the first digital image can be reduced or eliminated in the compensated digital image.
- a variant of the ambient subtraction method described above is to perform the subtraction in an analog domain without the need to readout MN1 116 and MN2 118 into a digital domain.
- the analog signal (photo-electrons) stored in each of MN1 116 and MN2 118 is transferred to a corresponding capacitor on the sensor column line: capl or cap 2, respectively.
- the column line includes a circuit for subtracting the charge or voltage on capl by the charge or voltage on cap 2 to achieve a “total_analog_value”. This “total analog value” is read-out to the digital domain, such that with a single digitization per pixel, a compensated digital image (as defined above) is obtained.
- FIG. 6 a flowchart illustrating one example of a sequence of operations 400 for active imaging of a given DoF in an environment while reducing an effect of parasitic light on the imaging using a gated image sensor having pixels 100 that include at least two memory nodes (e.g., MN1 116 & MN2 118), in accordance with the presently disclosed subject matter.
- MN1 116 & MN2 118 memory nodes
- the controllers can include one or more gating controllers associated with given pixels 100 of the pixels (e.g., all of the pixels) of the gated image sensor that include multiple transfer gate switches and corresponding multiple memory nodes.
- the active gated imaging system can be configured to control the light source to emit light pulses of a frame towards the environment.
- Fig. 7 one (non-limiting) example of a timing diagram 450 for a respective given pixel 100 in the gated image sensor that includes two memory nodes (e.g., MN1 116 and MN2 118) for reducing an effect of parasitic light on an image of a given DoF in the environment, in accordance with the presently disclosed subject matter.
- Fig. 7 illustrates two successive light pulses (252, 254) of the light pulses of the frame that are emitted towards the environment.
- At least a subset of the given pixels 100 in the gated image sensor that include two or more memory nodes, including, inter alia, the respective given pixel 100, are synchronized to the light pulses (e.g., 252, 254) that are emitted towards the environment in accordance with the same timing diagram (e.g., 450) to reduce an effect of parasitic light on an image of a given DoF in the environment.
- the light pulses e.g., 252, 254
- a gating controller (being the same or different than the controller that controls the emission of light pulses) that is associated with a respective given pixel 100 can be configured to gate the respective given pixel 100 in a first synchronization scheme with the emission of the given light pulse (e.g., light pulse 252).
- the respective given pixel 100 can be gated, as illustrated in Fig.
- the gating controller can be further configured to complete the gating of the respective given pixel 100 in the first synchronization scheme by deactivating the first transfer gate switch (e.g., TX1 112) of the respective given pixel 100 at a later time (e.g., ti + At) following the emission of the given light pulse (e.g., by switching TX1 112 from 1 to 0), as illustrated in Fig. 7.
- the first transfer gate switch e.g., TX1 112
- the gating controller In gating the respective given pixel 100 in the first synchronization scheme, the gating controller enables accumulating, at the first memory node (e.g., MN1 116) of the respective given pixel 100, a converted light reading that is based on returns of the given light pulse (e.g., 252) that arrive at the respective given pixel 100 during the time period between the deactivation of PD RST 122 at time t2 following the emission of the given light pulse and the deactivation of TX1 112 at time ti + At following the emission of the given light pulse.
- These returns of the given light pulse e.g., 252 are from a given Depth of Field (DoF) in the environment, and not from other DoFs in the environment (block 408).
- the gating controller can be configured to activate PD RST 122 (e.g., by switching PD RST 122 from 0 to 1) following the deactivation of TXl 112.
- Parasitic light can accumulate at MN1 116 since a pixel 100 can be responsive to light (i.e.: it can collect photo-electrons) even at times when the gate of the pixel 100 is nominally closed. Put differently, MN1 116 collects some light directly (not through the PD) during the whole frame time (from reset till readout) both when the gate is open or closed, resulting in the parasitic light being accumulated in MN 1 116.
- the controller can be configured to subtract the second digital image from the first digital image, giving rise to a compensated digital image of the given DoF that is less affected by the parasitic light accumulated at the first memory nodes of the given pixels than the first digital image (unaffected by the parasitic light if both MN1 116 and MN2 118 of each of the given pixels have the same parasitic light value) (block 424).
- a variant of this ambient subtraction method is to perform the subtraction in an analog domain without the need to readout MN1 116 and MN2 118 into a digital domain.
- the analog signal (photo-electrons) stored in each of MN1 116 and MN2 118 is transferred to a corresponding capacitor on the sensor column line: capl or cap 2, respectively.
- the column line includes a circuit for subtracting the charge or voltage on capl by the charge or voltage on cap 2 to achieve a “total_analog_value”. This “total_analog_value” is read-out to the digital domain, such that with a single digitization per pixel, a compensated digital image (as defined above) is obtained.
- the parasitic light accumulated in the first memory node (MN1 116) is reduced or eliminated but the ambient light accumulated in the first memory node (MN1 116) is retained. This allows for retaining in the compensated digital image important information regarding ambient sources in the environment, such as car tail-lamps, etc.
- FIG. 8 a block diagram schematically illustrating one example of a pixel 500 in a Single Photon Avalanche Diode (SPAD) sensor, in accordance with the presently disclosed subject matter.
- SBA Single Photon Avalanche Diode
- an imaging system for imaging at least one Depth of Field (DoF) is an environment.
- the imaging system includes: (i) a light source (e.g., a laser) (not shown), (ii) a digital camera comprising a SPAD sensor having an array of pixels and (iii) at least one controller 520.
- Each of the pixels 500 in the array of pixels includes one or more event counters (e.g., 532, 542) for counting avalanche events at the respective given pixel 500, based on light absorbed in a SPAD 510 of the respective given pixel 500, and a corresponding one or more transfer gate switches (e.g., 534, 544), each of the transfer gate switches gating a different event counter.
- the SPAD 510 is actively or passively quenched, this being determinative of the dead time of the SPAD.
- FIG. 9 a flowchart illustrating one example of a sequence of operations 600 for active gated imaging of a given DoF in an environment using a SPAD sensor, in accordance with the presently disclosed subject matter.
- controller 520 can be configured to control the light source of the imaging system, the imaging system including the SPAD sensor, to emit a given light pulse of a frame towards the environment (block 604).
- controller 520 can be configured to control a transfer gate switch (e.g., 534 or 544) of the respective given pixel 500, in synchronization with an emission of the given light pulse towards the environment, to: (a) activate an event counter (e.g., 532, 542) of the respective given pixel 500 associated with the transfer gate switch (e.g., 534, 544) to count the avalanche events at the respective given pixel 500 based on returns of the given light pulse from a given DoF within the environment and (b) pause the event counter (e.g., 532, 542) of the respective given pixel 500 to not count avalanche events at the respective given pixel 500 based on returns of the light pulse from other DoF’s within the environment, other than the given DoF (block 608).
- an event counter e.g., 532, 542
- the transfer gate switch e.g., 534 or 544
- Controller 520 can be configured to repeat block 608 for each light pulse in the frame (block 612).
- Controller 520 (the same or distinct from the gating controllers and/or the light emission controlling controller) can be further configured, for each of the given pixels 500, to associate the avalanche events counted by the event counter (e.g., 532, 542) of the respective given pixel 500 based on the returns of the light pulses in the frame from the given DoF with a grey level, giving rise to a digital image of the given DoF.
- the grey level is proportional to the number of avalanche events in the frame time slot.
- each pixel in the SPAD sensor can include multiple transfer gate controllers (e.g., 534, 544) and a corresponding number of event counters (e.g., 532, 542), as illustrated in Fig. 8.
- transfer gate controllers e.g., 534, 544
- event counters e.g., 532, 542
- Fig. 8 By activating the different transfer gate controllers (e.g., 534, 544) in different synchronization schemes with an emission of each light pulse towards the environment, different digital images can be provided for different DoFs in the environment, as detailed earlier herein, inter alia with reference to Figs. 2 and 3; or a digital image of a given DoF in the environment that is less affected (or unaffected) by ambient light can be generated, as detailed earlier herein, inter alia with reference to Figs. 4 and 5.
- system can be implemented, at least partly, as a suitably programmed computer.
- the presently disclosed subject matter contemplates a computer program being readable by a computer for executing the disclosed method.
- the presently disclosed subject matter further contemplates a machine-readable memory tangibly embodying a program of instructions executable by the machine for executing the disclosed method.
Landscapes
- Engineering & Computer Science (AREA)
- Physics & Mathematics (AREA)
- Computer Networks & Wireless Communication (AREA)
- General Physics & Mathematics (AREA)
- Radar, Positioning & Navigation (AREA)
- Remote Sensing (AREA)
- Electromagnetism (AREA)
- Transforming Light Signals Into Electric Signals (AREA)
Abstract
A system comprises a light source for emitting light pulses (LPs) towards an environment and a gated sensor comprising an array of pixels. For given pixels in the sensor, (A) a first transfer gate switch (TGS) of the respective given pixel (RGP) is activated, in a first synchronization scheme with an emission of the LPs, to accumulate, at a first memory node of the RGP, a converted light signal (CLS) that is based on returns of the LPs from a Depth of Field (DoF) in the environment and an ambient light of the environment; and (B) a second TGS of the RGP is activated, in a second synchronization scheme with the emission of the LPs, to accumulate, at a second memory node of the RGP, a second CLS representing at least one of: (a) returns of the LPs from a second DoF in the environment and (b) the ambient light.
Description
MULTIPLE GATED PIXEL PER LIGHT PULSE AND/OR GATED EVENT COUNTER FOR SINGLE PHOTON AVALANCHE DIODE (SPAD)
TECHNICAL FIELD
The present invention relates to the field of active imaging by a multiple gated pixel per light pulse method and/or a gated event counter for Single Photon Avalanche Diode (SPAD) method.
BACKGROUND
A current solution for active imaging of two or more Depths of Field (DoF’ s) in an environment in a single frame uses a gated image sensor that includes pixels having a single memory node. Since the pixels have only a single memory node, only a single digital image is generated for a frame. As a result thereof, the range of an object that is present in one of the two or more DoF’s that are actively imaged cannot be determined. An object of the present disclosure is to provide a solution for this problem.
GENERAL DESCRIPTION
In accordance with a first aspect of the presently disclosed subject matter, there is provided an active-gated imaging system for imaging a Depth of Field (DoF) in an environment at a predefined distance, the system comprising: a light source configured to emit light pulses towards the environment; and a gated sensor comprising: (i) an array of pixels, wherein given pixels of the pixels include: a photodetector, a first transfer gate switch, a second transfer gate switch, a first memory node associated with the first transfer gate switch, and a second memory node associated with the second transfer gate switch; and (ii) one or more gating controllers associated with the given pixels, each of the gating controllers being configured, for one or more of the given pixels, to: (A) activate the first transfer gate switch of the respective given pixel, in a first synchronization scheme with an emission of the light pulses towards the environment, to accumulate, at the first memory node of the respective given pixel, a converted light signal that is based on returns of the light pulses from the DoF and an ambient light of the environment; and (B) activate the second transfer gate switch of the respective given pixel, in a second synchronization scheme with the emission of the light pulses towards the environment, to accumulate, at the second memory
node of the respective given pixel, a second converted light signal, wherein the second synchronization scheme is different from the first synchronization scheme; wherein the second converted light signal represents at least one of: (a) said pulses returning from a second DoF of the environment at a second predefined distance and (b) the ambient light of the environment without said returns.
In some cases, the second converted light signal represents the ambient light of the environment without said returns, and the system further comprises: at least one second controller configured, for each of the given pixels, to: associate: (i) the converted light signal of the respective given pixel with a first grey level for the respective given pixel and (ii) the second converted light signal of the respective given pixel with a second grey level for the respective given pixel; and perform image subtraction for the respective given pixel, based on the first grey level and the second grey level, to provide a compensation grey level for the respective given pixel, the compensation grey level being the first grey level as compensated to reduce an effect of the ambient light on the first grey level; wherein an image of the DoF at the predefined distance is formed from the compensation grey levels of the given pixels.
In accordance with a second aspect of the presently disclosed subject matter, there is provided an imaging system for imaging a Depth of Field (DoF) in an environment at a predefined distance, the system comprising: a light source configured to emit light pulses towards the environment; and a digital camera comprising: (i) a Single Photon Avalanche Diode (SPAD) sensor having an array of pixels, wherein given pixels of the pixels include: (a) an event counter for counting avalanche events at the respective given pixel and (b) a transfer gate switch for gating the event counter; (ii) one or more gating controllers associated with the given pixels, each of the gating controllers being configured, for one or more of the given pixels, to control the transfer gate switch of the respective given pixel, in synchronization with an emission of the light pulses towards the environment, to: (a) pause the event counter of the respective given pixel prior and subsequent to the respective given pixel capturing a reflection signal indicative of a reflection of the light pulses returning from the DoF and (b) activate the event counter of the respective given pixel to count the avalanche events at the respective given pixel resulting from the capturing of the reflection signal; and (ii) at least one second controller configured, for each of the given pixels, to associate the avalanche events counted by the event counter of the respective given pixel with a grey level for the respective given pixel; wherein an image of the DoF is formed from the grey levels of the given pixels.
In accordance with a third aspect of the presently disclosed subject matter, there is provided an active-gated imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance, the method comprising: emitting light pulses, by a light source, towards the environment; and for given pixels in a gated sensor comprising an array of pixels, the given pixels including a photodetector, a first transfer gate switch, a second transfer gate switch, a first memory node associated with the first transfer gate switch, and a second memory node associated with the second transfer gate switch: (A) activating the first transfer gate switch of the respective given pixel, in a first synchronization scheme with an emission of the light pulses towards the environment, to accumulate, at the first memory node of the respective given pixel, a converted light signal that is based on returns of the light pulses from the DoF and an ambient light of the environment; and (B) activating the second transfer gate switch of the respective given pixel, in a second synchronization scheme with the emission of the light pulses towards the environment, to accumulate, at the second memory node of the respective given pixel, a second converted light signal, wherein the second synchronization scheme is different from the first synchronization scheme; wherein the second converted light signal represents at least one of: (a) said pulses returning from a second DoF of the environment at a second predefined distance and (b) the ambient light of the environment without said returns.
In some cases, the second converted light signal represents the ambient light of the environment without said returns, and the method further comprises: for each of the given pixels: associating: (i) the converted light signal of the respective given pixel with a first grey level for the respective given pixel and (ii) the second converted light signal of the respective given pixel with a second grey level for the respective given pixel; and performing image subtraction for the respective given pixel, based on the first grey level and the second grey level, to provide a compensation grey level for the respective given pixel, the compensation grey level being the first grey level as compensated to reduce an effect of the ambient light on the first grey level; wherein an image of the DoF at the predefined distance is formed from the compensation grey levels of the given pixels.
In accordance with a fourth aspect of the presently disclosed subject matter, there is provided an imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance, the method comprising: emitting light pulses, by a light source, towards the environment; for a digital camera comprising a Single Photon Avalanche Diode (SPAD) sensor having an array of pixels, wherein given pixels of the pixels include: (a) an event
counter for counting avalanche events at the respective given pixel and (b) a transfer gate switch for gating the event counter: for the given pixels: controlling the transfer gate switch of the respective given pixel, in synchronization with an emission of the light pulses towards the environment, to: (a) pause the event counter of the respective given pixel prior and subsequent to the respective given pixel capturing a reflection signal indicative of a reflection of the light pulses returning from the DoF and (b) activate the event counter of the respective given pixel to count the avalanche events at the respective given pixel resulting from the capturing of the reflection signal; and associating the avalanche events counted by the event counter of the respective given pixel with a grey level for the respective given pixel; wherein an image of the DoF is formed from the grey levels of the given pixels.
In accordance with a fifth aspect of the presently disclosed subject matter, there is provided a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, executable by at least one controller of a computer to perform an active-gated imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance, the method comprising: emitting light pulses, by a light source, towards the environment; and for given pixels in a gated sensor comprising an array of pixels, the given pixels including a photodetector, a first transfer gate switch, a second transfer gate switch, a first memory node associated with the first transfer gate switch, and a second memory node associated with the second transfer gate switch: (A) activating the first transfer gate switch of the respective given pixel, in a first synchronization scheme with an emission of the light pulses towards the environment, to accumulate, at the first memory node of the respective given pixel, a converted light signal that is based on returns of the light pulses from the DoF and an ambient light of the environment; and (B) activating the second transfer gate switch of the respective given pixel, in a second synchronization scheme with the emission of the light pulses towards the environment, to accumulate, at the second memory node of the respective given pixel, a second converted light signal, wherein the second synchronization scheme is different from the first synchronization scheme; wherein the second converted light signal represents at least one of: (a) said pulses returning from a second DoF of the environment at a second predefined distance and (b) the ambient light of the environment without said returns.
In accordance with a sixth aspect of the presently disclosed subject matter, there is provided a non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, executable by at
least one controller of a computer to perform an imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance, the method comprising: emitting light pulses, by a light source, towards the environment; for a digital camera comprising a Single Photon Avalanche Diode (SPAD) sensor having an array of pixels, wherein given pixels of the pixels include: (a) an event counter for counting avalanche events at the respective given pixel and (b) a transfer gate switch for gating the event counter: for the given pixels: controlling the transfer gate switch of the respective given pixel, in synchronization with an emission of the light pulses towards the environment, to: (a) pause the event counter of the respective given pixel prior and subsequent to the respective given pixel capturing a reflection signal indicative of a reflection of the light pulses returning from the DoF and (b) activate the event counter of the respective given pixel to count the avalanche events at the respective given pixel resulting from the capturing of the reflection signal; and associating the avalanche events counted by the event counter of the respective given pixel with a grey level for the respective given pixel; wherein an image of the DoF is formed from the grey levels of the given pixels.
BRIEF DESCRIPTION OF THE DRAWINGS
In order to understand the presently disclosed subject matter and to see how it may be carried out in practice, the subject matter will now be described, by way of non-limiting examples only, with reference to the accompanying drawings, in which:
Fig. 1 is a block diagram schematically illustrating one example of a pixel in a gated image sensor, in accordance with the presently disclosed subject matter;
Fig. 2 is a flowchart illustrating one example of a sequence of operations for active imaging of two or more Depths of Field (DoF’s) in an environment using a gated image sensor having pixels that include two or more memory nodes, in accordance with the presently disclosed subject matter;
Fig. 3 is one example of a timing diagram for a respective given pixel in the gated image sensor that includes two memory nodes for active imaging of two DoF’s in an environment, in accordance with the presently disclosed subject matter;
Fig. 4 is a flowchart illustrating one example of a sequence of operations for active imaging of a given DoF in an environment while reducing an effect of ambient light and
parasitic light on the imaging using a gated image sensor having pixels that include at least two memory nodes, in accordance with the presently disclosed subject matter;
Fig. 5 is one example of a timing diagram for a respective given pixel in the gated image sensor that includes two memory nodes for reducing an effect of ambient light and parasitic light on an image of a given DoF in an environment, in accordance with the presently disclosed subject matter;
Fig. 6 is a flowchart illustrating one example of a sequence of operations for active imaging of a given DoF in an environment while reducing an effect of parasitic light on the imaging using a gated image sensor having pixels that include at least two memory nodes, in accordance with the presently disclosed subject matter;
Fig. 7 is one example of a timing diagram for a respective given pixel in the gated image sensor that includes two memory nodes for reducing an effect of parasitic light on an image of a given DoF in an environment, in accordance with the presently disclosed subject matter;
Fig. 8 is a block diagram schematically illustrating one example of a pixel in a Single Photon Avalanche Diode (SPAD) sensor, in accordance with the presently disclosed subject matter; and
Fig. 9 is a flowchart illustrating one example of a sequence of operations for active gated imaging of a given DoF in an environment using a SPAD sensor, in accordance with the presently disclosed subject matter.
DETAILED DESCRIPTION
In the following detailed description, numerous specific details are set forth in order to provide a thorough understanding of the presently disclosed subject matter. However, it will be understood by those skilled in the art that the presently disclosed subject matter may be practiced without these specific details. In other instances, well-known methods, procedures, and components have not been described in detail so as not to obscure the presently disclosed subject matter.
In the drawings and descriptions set forth, identical reference numerals indicate those components that are common to different embodiments or configurations.
Unless specifically stated otherwise, as apparent from the following discussions, it is appreciated that throughout the specification discussions utilizing terms such as “activating”,
“associating”, “performing”, “subtracting”, “controlling”, “pausing”, “forming” or the like, include actions and/or processes, including, inter alia, actions and/or processes of a computer, that manipulate and/or transform data into other data, said data represented as physical quantities, e.g. such as electronic quantities, and/or said data representing the physical objects. The terms “computer”, “processor” and “controller” should be expansively construed to cover any kind of electronic device with data processing capabilities, including, by way of non-limiting example, a personal desktop/laptop computer, a server, a computing system, a communication device, a smartphone, a tablet computer, a smart television, a processor (e.g. digital signal processor (DSP), a microcontroller, a field-programmable gate array (FPGA), an application specific integrated circuit (ASIC), etc.), a group of multiple physical machines sharing performance of various tasks, virtual servers co-residing on a single physical machine, any other electronic computing device, and/or any combination thereof.
As used herein, the phrase "for example," "such as", "for instance" and variants thereof describe non-limiting embodiments of the presently disclosed subject matter. Reference in the specification to "one case", "some cases", "other cases" or variants thereof means that a particular feature, structure or characteristic described in connection with the embodiment(s) is included in at least one embodiment of the presently disclosed subject matter. Thus the appearance of the phrase "one case", "some cases", "other cases" or variants thereof does not necessarily refer to the same embodiment(s).
It is appreciated that, unless specifically stated otherwise, certain features of the presently disclosed subject matter, which are, for clarity, described in the context of separate embodiments, may also be provided in combination in a single embodiment. Conversely, various features of the presently disclosed subject matter, which are, for brevity, described in the context of a single embodiment, may also be provided separately or in any suitable subcombination.
In embodiments of the presently disclosed subject matter, fewer, more and/or different stages than those shown in Figs. 2, 4, 6 and 9 may be executed. In embodiments of the presently disclosed subject matter one or more stages illustrated in Figs. 2, 4, 6 and 9 may be executed in a different order and/or one or more groups of stages may be executed simultaneously.
Any reference in the specification to a method should be applied mutatis mutandis to a system capable of executing the method and should be applied mutatis mutandis to a non-
transitory computer readable medium that stores instructions that once executed by a computer result in the execution of the method.
Any reference in the specification to a system should be applied mutatis mutandis to a method that may be executed by the system and should be applied mutatis mutandis to a non- transitory computer readable medium that stores instructions that may be executed by the system.
Any reference in the specification to a non-transitory computer readable medium should be applied mutatis mutandis to a system capable of executing the instructions stored in the non-transitory computer readable medium and should be applied mutatis mutandis to method that may be executed by a computer that reads the instructions stored in the non- transitory computer readable medium.
Attention is now drawn to Fig. 1, a block diagram schematically illustrating one example of a pixel 100 in a gated image sensor, in accordance with the presently disclosed subject matter.
In accordance with the presently disclosed subject matter, each pixel 100 in a gated image sensor (for example, a gated Complementary Metal Oxide Semiconductor (CMOS) image sensor) includes a photodetector (PD) 110. In some cases, the photodetector 110 is a photodiode. In some cases, the photodiode 110 is a pinned photodiode. The photodetector 110 converts return light pulses that arrive at the photodetector 110 from an environment into photo-electrons.
At least given pixels of the pixels (e.g., all of the pixels) in a gated image sensor, including, inter alia, pixel 100, further include: a first transfer gate switch (TX1) 112, a second transfer gate switch (TX2) 114, a first memory node (MN1) 116 associated with the first transfer gate switch 112, a second memory node (MN2) 118 associated with the second transfer gate switch 114, and a photodetector reset switch (PD RST) 122. It is to be noted that a pixel may also include three or more transfer gate switches and a corresponding three or more memory nodes.
In some cases, each of the given pixels can have a standard electric signal chain after the “gate-able” PD 110, TX1 112, TX2 114 and PD RST 122 configuration. This standard electric signal chain can include at least one Reset transistor (RST) 124 with the role of resetting the memory nodes (e.g., MN1 116 & MN2 118) using the pixel voltage (VDD). It is to be noted that, in some cases, one Reset transistor (RST) 124 may be used to reset the memory nodes (e.g., MN1 116 & MN2 118), as illustrated in Fig. 1. Alternatively, in some
cases, more than one Reset transistor (RST) may be used to reset the memory nodes (e.g., a first RST may be used to reset a first memory node (e.g., MN1 116) and a second RST may be used to reset a second memory node (e.g., MN2 118)). In some cases, this standard electrical signal chain can also include a number of Source Follower (SF) transistors (e.g., 126 and 128), wherein each of the SF transistors converts an accumulated signal (i.e., electrons) at a respective memory node (e.g., MN1 or MN2) to a voltage. The standard electrical signal chain can also include a number of Select transistors (e.g., SEL1 132 & SEL2 134) connected to a column and/or row of the pixel array of the gated image sensor, each of the Select transistors (e.g., SEL1 132 & SEL2 134) being associated with a different memory node of the memory nodes (e.g., MN1 & MN2), and enabling the contents of the respective memory node associated with the respective Select transistor to be read-out after each frame on a given output channel (e.g., Readoutl or Readout2). In the example of Fig. 1, SEL1 132 is activated to enable the contents of MN 1 116 to be read out after each frame on a first output channel (e.g., Readoutl), and SEL2 134 is activated to enable the contents of MN2 118 to be read out after each frame on a second output channel (e.g., Readout2) that is distinct from the first output channel (e.g., Readoutl).
Attention is now drawn to Fig. 2, a flowchart illustrating one example of a sequence of operations 200 for active imaging of two or more Depths of Field (DoF’s) in an environment using a gated image sensor having pixels 100 that include two or more memory nodes (e.g., MN1 116 and MN2 118), in accordance with the presently disclosed subject matter.
In accordance with the presently disclosed subject matter, an active-gated imaging system (not shown) for active imaging of two or more Depths of Field (DoF’s) in an environment comprises: (a) a light source (e.g., a laser) (not shown) configured to emit light pulses (e.g., laser pulses) towards the environment, (b) a gated image sensor, as detailed earlier herein, inter alia with reference to Fig. 1, and (c) one or more controllers (not shown) for controlling the light source and the pixels 100 in the gated sensor. The controllers can include one or more gating controllers associated with given pixels 100 of the pixels (e.g., all of the pixels) of the gated image sensor that include multiple transfer gate switches and corresponding multiple memory nodes.
The active gated imaging system can be configured to control the light source to emit light pulses of a frame towards the environment. To illustrate this, attention is drawn to Fig. 3, one (non-limiting) example of a timing diagram 250 for a respective given pixel 100 in the
gated image sensor that includes two memory nodes (e.g., MN1 116 and MN2 118) for active imaging of two DoF’s in an environment, in accordance with the presently disclosed subject matter. Fig. 3 illustrates two successive light pulses (252, 254) of the light pulses of the frame that are emitted towards the environment. At least a subset (e.g., all) of the given pixels 100 in the gated image sensor that include two or more memory nodes, including, inter alia, the respective given pixel 100, are synchronized to the light pulses (e.g., 252, 254) that are emitted towards the environment in accordance with the same timing diagram (e.g., 250) to achieve active imaging of two or more DoFs in the environment.
Following the emission of a given light pulse of the frame (e.g., light pulse 252) towards the environment (block 204), the following can be performed for each pixel 100 of the at least a subset of the given pixels 100 in the gated image sensor. A gating controller (being the same or different than the controller that controls the emission of light pulses) that is associated with a respective given pixel 100 can be configured to gate the respective given pixel 100 in a first synchronization scheme with the emission of the given light pulse (e.g., light pulse 252). The respective given pixel 100 can be gated, as illustrated in Fig. 3, by deactivating the photodetector reset switch (PD RST) 122 of the respective given pixel 100 at a time t2 following the emission of the given light pulse (e.g., 252) (e.g., by switching PD RST 122 from 1 to 0), and by shortly thereafter activating the first transfer gate switch (TX1 112) of the respective given pixel 100 at a time ti following the emission of the given light pulse (e.g., 252) (e.g., by switching TX1 112 from 0 to 1). The gating controller can also be configured to complete the gating of the respective given pixel 100 in the first synchronization scheme by deactivating the first transfer gate switch (TX1 112) of the respective given pixel 100 at a later time (e.g., ti + At) following the emission of the given light pulse (e.g., by switching TX1 112 from 1 to 0), as illustrated in Fig. 3. In gating the respective given pixel 100 in the first synchronization scheme, the gating controller enables accumulating, at the first memory node (MN1 116) of the respective given pixel 100, a converted light reading that is based on returns of the given light pulse (e.g., 252) that arrive at the respective given pixel 100 during the time period between the deactivation of PD RST 122 at time t2 following the emission of the given light pulse and the deactivation of TX1 112 at time ti + At following the emission of the given light pulse. These returns of the given light pulse (e.g., 252) are from a first Depth of Field (DoF) in the environment, and not from other DoF’s in the environment, other than the first DoF (block 208). The gating controller can be
configured to activate PD RST 122 (e.g., by switching PD RST from 0 to 1) following the deactivation of TX1 112.
Moreover, the gating controller that is associated with the respective given pixel 100 can be configured to gate the respective given pixel 100 in a second synchronization scheme with the emission of the given light pulse (e.g., light pulse 252). The respective given pixel 100 can be gated in the second synchronization scheme, as illustrated in Fig. 3, by deactivating the photodetector reset switch (PD RST) 122 of the respective given pixel 100 at a time t2 + following the emission of the given light pulse (e.g., 252) (e.g., by switching PD RST 122 from 1 to 0), and by shortly thereafter activating the second transfer gate switch (TX2 114) of the respective given pixel 100 at a time t3 following the emission of the given light pulse (e.g., 252) (e.g., by switching TX2 114 from 0 to 1). The gating controller can be further configured to complete the gating of the respective given pixel 100 in the second synchronization scheme by deactivating the second transfer gate switch (TX2 114) of the respective given pixel 100 at a later time (e.g., t3 + Ati) following the emission of the given light pulse (e.g., by switching TX2 114 from 1 to 0), as illustrated in Fig. 3. In gating the respective given pixel 100 in the second synchronization scheme, the gating controller enables accumulating, at the second memory node (MN2 118) of the respective given pixel 100, a second converted light reading that is based on returns of the given light pulse (e.g., 252) that arrive at the respective given pixel 100 during the time period between the deactivation of PD RST 122 at time b + U following the emission of the given light pulse and the deactivation of TX2 114 at time t3 + Ati following the emission of the given light pulse. These returns of the given light pulse (e.g., 252) are from a second Depth of Field (DoF) in the environment, distinct from the first DoF (in Fig. 3, the second DoF is at a greater distance from the active gated imaging system than the first DoF), and not from other DoF’s in the environment, other than the second DoF (block 212). The gating controller can be configured to activate PD RST 122 following the deactivation of TX2 114.
It is to be noted that the period of time during which TX1 112 is active (e.g., At) can be the same or different than the period of time during which TX2 114 is active (e.g., Ati). Moreover, it is possible for the respective given pixel 100 to include three or more memory nodes, each of which accumulate a converted light reading that is based on returns of the given light pulse (e.g., 252) from a different DoF in the environment.
The gating controller can be configured to repeat the aforementioned operations discussed above with respect to the given light pulse 252 (including, inter alia, blocks 208
and 212) for each emitted light pulse in the frame (e.g., light pulse 254, etc.), as illustrated in Fig. 3 (block 216).
Following the end of the frame, a controller (the same or different than the gating controller) can be configured, for each of the given pixels 100 in the subset of the given pixels (e.g., for each of the given pixels in the gated image sensor), to read-out on a first output channel (e.g., Readoutl) of the respective given pixel 100, into a digital domain, a converted light signal from MN1 116 of the respective given pixel 100, the converted light signal being generated based on the converted light readings accumulated at MN1 116 of the respective given pixel 100 over the course of the frame, giving rise to a first digital image of the first DoF (block 220).
Moreover, this controller can be configured, for each of the given pixels 100 in the subset of the given pixels, to read-out on a second output channel (e.g., Readout2) of the respective given pixel, into a digital domain, a second converted light signal from MN2 118 of the respective given pixel 100, the second converted light signal being generated based on the second converted light readings accumulated at MN2 118 of the respective given pixel 100 over the course of the frame, giving rise to a second digital image of the second DoF (block 224). It is to be noted that N (>2) digital images of N DoFs can be generated, provided that the respective given pixel 100 includes N memory nodes. Since the first digital image shows illuminated pixels arising from the returns of the emitted light pulses from the first DoF only and the second digital image shows illuminated pixels arising from the returns of the emitted light pulses from the second DoF only, the range of any illuminated object in any one of the images can be determined (if the object is illuminated in the first digital image, its range corresponds to the first DoF; if it is illuminated in the second digital image, its range corresponds to the second DoF).
Attention is now drawn to Fig. 4, a flowchart illustrating one example of a sequence of operations 300 for active imaging of a given DoF in an environment while reducing an effect of ambient light and parasitic light on the imaging using a gated image sensor having pixels 100 that include at least two memory nodes (e.g., MN1 116 & MN2 118), in accordance with the presently disclosed subject matter.
In accordance with the presently disclosed subject matter, an active-gated imaging system (not shown) for active imaging of a given DoF in an environment comprises: (a) a light source (e.g., a laser) (not shown) configured to emit light pulses (e.g., laser pulses) towards an environment, (b) a gated image sensor, as detailed earlier herein, inter alia with
reference to Fig. 1, and (c) one or more controllers (not shown) for controlling the light source and the pixels 100 in the gated sensor. The controllers can include one or more gating controllers associated with given pixels 100 of the pixels (e.g., all of the pixels) of the gated image sensor that include multiple transfer gate switches and corresponding multiple memory nodes.
The active gated imaging system can be configured to control the light source to emit light pulses of a frame towards the environment. To illustrate this, attention is drawn to Fig. 5, one (non-limiting) example of a timing diagram 350 for a respective given pixel 100 in the gated image sensor that include two memory nodes (e.g., MN1 116 and MN2 118) for reducing an effect of ambient light and parasitic light on an image of a given DoF in the environment, in accordance with the presently disclosed subject matter. Fig. 5 illustrates two successive light pulses (252, 254) of the light pulses of the frame that are emitted towards the environment. At least a subset of the given pixels 100 (e.g., all of the given pixels 100) in the gated image sensor that include two or more memory nodes, including, inter alia, the respective given pixel 100, are synchronized to the light pulses (e.g., 252, 254) that are emitted towards the environment in accordance with the same timing diagram (e.g., 350) to reduce an effect of ambient light and parasitic light on an image of a given DoF in the environment.
Following the emission of a given light pulse of the frame (e.g., light pulse 252) towards the environment (block 304), the following can be performed for each pixel 100 of the at least a subset of the given pixels in the gated image sensor. A gating controller (being the same or different than the controller that controls the emission of light pulses) that is associated with a respective given pixel 100 can be configured to gate the respective given pixel 100 in a first synchronization scheme with the emission of the given light pulse (e.g., light pulse 252). The respective given pixel 100 can be gated, as illustrated in Fig. 5, by deactivating the photodetector reset switch (PD RST) 122 of the respective given pixel 100 at a time t2 following the emission of the given light pulse (e.g., 252) (e.g., by switching PD RST 122 from 1 to 0), and by shortly thereafter activating the first transfer gate switch (TX1 112) of the respective given pixel 100 at a time ti following the emission of the given light pulse (e.g., 252) (e.g., by switching TX1 112 from 0 to 1). The gating controller can be further configured to complete the gating of the respective given pixel 100 in the first synchronization scheme by deactivating the first transfer gate switch (TX1 112) of the respective given pixel 100 at a later time (e.g., tl + At) following the emission of the given
light pulse (e.g., by switching TX1 112 from 1 to 0), as illustrated in Fig. 5. In gating the respective given pixel 100 in the first synchronization scheme, the gating controller enables accumulating, at the first memory node (MN1 116) of the respective given pixel 100, a converted light reading that is based on: (i) returns of the given light pulse (e.g., 252) from a given Depth of Field (DoF) in the environment and (ii) an ambient light in the environment (e.g., light from non-pulsed Continuous Wave (CW) ambient sources) that arrives at the respective given pixel 100 during the time period between the deactivation of PD RST 122 at time t2 following the emission of the given light pulse and the deactivation of TX1 112 at time ti + At following the emission of the given light pulse. The returns of the given light pulse (e.g., 252) are not from other DoFs in the environment, other than the given DoF (block 308). The gating controller can be configured to activate PD RST 122 (e.g., by switching PD RST 122 from 0 to 1) following the deactivation of TX1 112.
Moreover, the gating controller that is associated with the respective given pixel 100 can be configured to gate the respective given pixel 100 in a second synchronization scheme with the emission of the given light pulse (e.g., 252). The respective given pixel 100 can be gated in the second synchronization scheme, as illustrated in Fig. 5, by deactivating the photodetector reset switch (PD RST) 122 of the respective given pixel 100 at a time t2 + ts following the emission of the given light pulse (e.g., by switching PD RST 122 from 1 to 0), and by shortly thereafter activating the second transfer gate switch (TX2 114) of the respective given pixel 100 at a time te) following the emission of the given light pulse (e.g., 252) (e.g., by switching TX2 114 from 0 to 1). The gating controller can be further configured to complete the gating of the respective given pixel 100 in the second synchronization scheme by deactivating the second transfer gate switch (TX2 114) of the respective given pixel 100 at a later time (e.g., t6 + At) following the emission of the given light pulse (e.g., by switching TX2 114 from 1 to 0), as illustrated in Fig. 5. In gating the respective given pixel 100 in the second synchronization scheme, the gating controller enables accumulating, at the second memory node (MN2 118) of the respective given pixel 100, a second converted light reading that is based on the ambient light in the environment during the time period between the deactivation of PD RST 122 at time t2 + ts following the emission of the given light pulse (e.g., 252) and the deactivation of TX2 114 at time t6 + At following the emission of the given light pulse. The second converted light reading does not include or substantially does not include returns of the given light pulse (e.g., 252) (no or negligible reflections from the given light pulse are collected) (block 312). It is to be noted
that in order to generate a digital image of the given DoF with no (or minimal) ambient light, the second converted light reading is collected over the same time period as the converted light reading (the converted light reading being based on returns of the given light pulse (e.g., 252) from a given Depth of Field (DoF) in the environment and an ambient light in the environment (e.g., light from non-pulsed (CW) ambient sources), as discussed above). To achieve this, TX1 112 and TX2 114 are opened for the same duration (At), as illustrated in Fig. 5. As illustrated in Fig. 5, TX2 114 is activated after all relevant returns from the given light pulse have already passed the camera (“long delay”). As such, in all cases, TX2 114 is activated to ensure that no or a negligible amount of the second converted light reading is based on returns of the given light pulse (e.g., 252). The gating controller can be configured to activate PD RST 122 (e.g., by switching PD RST 122 from 0 to 1) following the deactivation of TX2 114.
The gating controller can be configured to repeat the aforementioned operations discussed above with respect to the given light pulse 252 (including, inter alia, blocks 308 and 312) for each emitted light pulse in the frame (e.g., light pulse 254, etc.), as illustrated in Fig. 5 (block 316).
In some cases, following the end of the frame, a controller (the same or different than the gating controller) can be configured, for each of the given pixels 100 in the subset of the given pixels, to read-out, into a digital domain, a converted light signal from MN1 116 of the respective given pixel 100, the converted light signal being generated based on the converted light readings accumulated at MN1 116 of the respective given pixel 100 over the course of the frame, giving rise to a first digital image of the given DoF based on the returns of the light pulses (e.g., 252, 254) of the frame from the given DoF and the ambient light (block 320).
Moreover, this controller can be configured, for each of the given pixels 100 in the subset of the given pixels, to read-out, into a digital domain, a second converted light signal from MN2 118 of the respective given pixel 100, the second converted light signal being generated based on the second converted light readings accumulated at MN2 118 of the respective given pixel 100 over the course of the frame, giving rise to a second digital image that is based on the ambient light in the environment (the same or approximately the same quantity of ambient light as the first digital image), and that is not based (or is negligibly based) on returns of the lights pulses of the frame arriving at the gated image sensor (block 324).
The controller can be configured to subtract the second digital image from the first digital image, giving rise to a compensated digital image of the given DoF that is less affected by the ambient light than the first digital image (or even unaffected by the ambient light). This is advantageous for several reasons, including, inter alia, that the ambient light inserts light into the given pixels un-correlated to the given DoF, it might reduce image contrast, etc. It is to be noted that the memory nodes (MN1 116 and MN2 118) can accumulate parasitic light during the time period of the frame, even at times when the transfer gate switches (TX1 112, TX2 114) associated with the respective memory node are nominally closed. Put differently, MN1 116 and MN2 118 each collect some light directly (not thorough the PD) during the whole frame time (from reset till readout) both when the gate is open or closed, resulting in the parasitic light being accumulated in MN1 116 and MN2 118. Accordingly, by subtracting the second digital image from the first digital image to generate the compensated digital image of the given DoF, the parasitic light present in the first digital image can be reduced or eliminated in the compensated digital image.
A variant of the ambient subtraction method described above is to perform the subtraction in an analog domain without the need to readout MN1 116 and MN2 118 into a digital domain. During the readout of each respective pixel 100 of at least the subset of given pixels, the analog signal (photo-electrons) stored in each of MN1 116 and MN2 118 is transferred to a corresponding capacitor on the sensor column line: capl or cap 2, respectively. The column line includes a circuit for subtracting the charge or voltage on capl by the charge or voltage on cap 2 to achieve a “total_analog_value”. This “total analog value” is read-out to the digital domain, such that with a single digitization per pixel, a compensated digital image (as defined above) is obtained.
Attention is now drawn to Fig. 6, a flowchart illustrating one example of a sequence of operations 400 for active imaging of a given DoF in an environment while reducing an effect of parasitic light on the imaging using a gated image sensor having pixels 100 that include at least two memory nodes (e.g., MN1 116 & MN2 118), in accordance with the presently disclosed subject matter.
In accordance with the presently disclosed subject matter, an active-gated imaging system (not shown) for active imaging of a given DoF in an environment comprises: (a) a light source (e.g., a laser) (not shown) configured to emit light pulses (e.g., laser pulses) towards the environment, (b) a gated image sensor, as detailed earlier herein, inter alia with reference to Fig. 1, and (c) one or more controllers (not shown) for controlling the light
source and the pixels 100 in the gated sensor. The controllers can include one or more gating controllers associated with given pixels 100 of the pixels (e.g., all of the pixels) of the gated image sensor that include multiple transfer gate switches and corresponding multiple memory nodes.
The active gated imaging system can be configured to control the light source to emit light pulses of a frame towards the environment. To illustrate this, attention is drawn to Fig. 7, one (non-limiting) example of a timing diagram 450 for a respective given pixel 100 in the gated image sensor that includes two memory nodes (e.g., MN1 116 and MN2 118) for reducing an effect of parasitic light on an image of a given DoF in the environment, in accordance with the presently disclosed subject matter. Fig. 7 illustrates two successive light pulses (252, 254) of the light pulses of the frame that are emitted towards the environment. At least a subset of the given pixels 100 in the gated image sensor that include two or more memory nodes, including, inter alia, the respective given pixel 100, are synchronized to the light pulses (e.g., 252, 254) that are emitted towards the environment in accordance with the same timing diagram (e.g., 450) to reduce an effect of parasitic light on an image of a given DoF in the environment.
Following the emission of a given light pulse of the frame (e.g., light pulse 252) towards the environment (block 404), the following can be performed for each pixel 100 of the at least a subset of the given pixels 100 in the gated image sensor. A gating controller (being the same or different than the controller that controls the emission of light pulses) that is associated with a respective given pixel 100 can be configured to gate the respective given pixel 100 in a first synchronization scheme with the emission of the given light pulse (e.g., light pulse 252). The respective given pixel 100 can be gated, as illustrated in Fig. 7, by deactivating the photodetector reset switch (PD RST) 122 of the respective given pixel 100 at a time t2 following the emission of the given light pulse (e.g., 252) (e.g., by switching PD RST 122 from 1 to 0), and by shortly thereafter activating the first transfer gate switch (e.g., TX1 112) of the respective given pixel 100 at a time ti subsequent to the emission of the given light pulse (e.g., 252) (e.g., by switching TX1 112 from 0 to 1). The gating controller can be further configured to complete the gating of the respective given pixel 100 in the first synchronization scheme by deactivating the first transfer gate switch (e.g., TX1 112) of the respective given pixel 100 at a later time (e.g., ti + At) following the emission of the given light pulse (e.g., by switching TX1 112 from 1 to 0), as illustrated in Fig. 7. In gating the respective given pixel 100 in the first synchronization scheme, the gating controller enables
accumulating, at the first memory node (e.g., MN1 116) of the respective given pixel 100, a converted light reading that is based on returns of the given light pulse (e.g., 252) that arrive at the respective given pixel 100 during the time period between the deactivation of PD RST 122 at time t2 following the emission of the given light pulse and the deactivation of TX1 112 at time ti + At following the emission of the given light pulse. These returns of the given light pulse (e.g., 252) are from a given Depth of Field (DoF) in the environment, and not from other DoFs in the environment (block 408). The gating controller can be configured to activate PD RST 122 (e.g., by switching PD RST 122 from 0 to 1) following the deactivation of TXl 112.
The gating controller can be configured to repeat the aforementioned operations discussed above with respect to the given light pulse (including, inter alia, block 408) for each emitted light pulse in the frame (e.g., light pulse 254, etc.), as illustrated in Fig. 7 (block 412).
In some cases, following the end of the frame, a controller (the same or different than the gating controller) can be configured, for each of the given pixels 100 in the subset of the given pixels, to read-out, into a digital domain, a converted light signal from MN1 116 of the respective given pixel 100, the converted light signal being generated based on the converted light readings over the course of the frame and the parasitic light accumulated at MN1 116 of the respective given pixel 100 over the course of the frame, giving rise to a first digital image of the given DoF (block 416). Parasitic light can accumulate at MN1 116 since a pixel 100 can be responsive to light (i.e.: it can collect photo-electrons) even at times when the gate of the pixel 100 is nominally closed. Put differently, MN1 116 collects some light directly (not through the PD) during the whole frame time (from reset till readout) both when the gate is open or closed, resulting in the parasitic light being accumulated in MN 1 116.
Moreover, the controller can be configured, for each of the given pixels 100 in the subset of the given pixels, to read-out, into a digital domain, a second converted light signal from MN2 118 of the respective given pixel 100, the second converted light signal being generated based on the parasitic light accumulated at MN2 118 of the respective given pixel 100 over the course of the frame (MN2 118 also collects some light directly (not through the PD) during the whole frame time (from reset till readout) both when the gate is open or closed), giving rise to a second digital image that includes only the parasitic light (block 420).
The controller can be configured to subtract the second digital image from the first digital image, giving rise to a compensated digital image of the given DoF that is less affected by the parasitic light accumulated at the first memory nodes of the given pixels than the first digital image (unaffected by the parasitic light if both MN1 116 and MN2 118 of each of the given pixels have the same parasitic light value) (block 424).
A variant of this ambient subtraction method is to perform the subtraction in an analog domain without the need to readout MN1 116 and MN2 118 into a digital domain. During the readout of each respective pixel 100 of at least the subset of given pixels, the analog signal (photo-electrons) stored in each of MN1 116 and MN2 118 is transferred to a corresponding capacitor on the sensor column line: capl or cap 2, respectively. The column line includes a circuit for subtracting the charge or voltage on capl by the charge or voltage on cap 2 to achieve a “total_analog_value”. This “total_analog_value” is read-out to the digital domain, such that with a single digitization per pixel, a compensated digital image (as defined above) is obtained.
It is to be noted that in the present embodiment, unlike the embodiment described in Figs. 4 and 5, the parasitic light accumulated in the first memory node (MN1 116) is reduced or eliminated but the ambient light accumulated in the first memory node (MN1 116) is retained. This allows for retaining in the compensated digital image important information regarding ambient sources in the environment, such as car tail-lamps, etc.
Attention is now drawn to Fig. 8, a block diagram schematically illustrating one example of a pixel 500 in a Single Photon Avalanche Diode (SPAD) sensor, in accordance with the presently disclosed subject matter.
In accordance with the presently disclosed subject matter, an imaging system for imaging at least one Depth of Field (DoF) is an environment is disclosed. The imaging system includes: (i) a light source (e.g., a laser) (not shown), (ii) a digital camera comprising a SPAD sensor having an array of pixels and (iii) at least one controller 520. Each of the pixels 500 in the array of pixels includes one or more event counters (e.g., 532, 542) for counting avalanche events at the respective given pixel 500, based on light absorbed in a SPAD 510 of the respective given pixel 500, and a corresponding one or more transfer gate switches (e.g., 534, 544), each of the transfer gate switches gating a different event counter. The SPAD 510 is actively or passively quenched, this being determinative of the dead time of the SPAD. Controller 520 can be configured to control (activate and deactivate) the transfer gate switches (e.g., 534, 544), reset the event counters (e.g., 532, 542) following the read-out
of each frame, and output one or more digital images based on avalanche events counted by the event counters (e.g., 532, 542), as detailed further herein, inter alia with reference to Fig. 9.
Attention is now drawn to Fig. 9, a flowchart illustrating one example of a sequence of operations 600 for active gated imaging of a given DoF in an environment using a SPAD sensor, in accordance with the presently disclosed subject matter.
In accordance with the presently disclosed subject matter, controller 520 can be configured to control the light source of the imaging system, the imaging system including the SPAD sensor, to emit a given light pulse of a frame towards the environment (block 604).
For given pixels (e.g., all of the pixels) in the SPAD sensor, controller 520 (the same controller that controls the emission of the light pulses or one or more different gating controllers) can be configured to control a transfer gate switch (e.g., 534 or 544) of the respective given pixel 500, in synchronization with an emission of the given light pulse towards the environment, to: (a) activate an event counter (e.g., 532, 542) of the respective given pixel 500 associated with the transfer gate switch (e.g., 534, 544) to count the avalanche events at the respective given pixel 500 based on returns of the given light pulse from a given DoF within the environment and (b) pause the event counter (e.g., 532, 542) of the respective given pixel 500 to not count avalanche events at the respective given pixel 500 based on returns of the light pulse from other DoF’s within the environment, other than the given DoF (block 608).
Controller 520 can be configured to repeat block 608 for each light pulse in the frame (block 612).
Controller 520 (the same or distinct from the gating controllers and/or the light emission controlling controller) can be further configured, for each of the given pixels 500, to associate the avalanche events counted by the event counter (e.g., 532, 542) of the respective given pixel 500 based on the returns of the light pulses in the frame from the given DoF with a grey level, giving rise to a digital image of the given DoF. It is to be noted that the grey level is proportional to the number of avalanche events in the frame time slot.
It is to be noted that each pixel in the SPAD sensor can include multiple transfer gate controllers (e.g., 534, 544) and a corresponding number of event counters (e.g., 532, 542), as illustrated in Fig. 8. By activating the different transfer gate controllers (e.g., 534, 544) in different synchronization schemes with an emission of each light pulse towards the environment, different digital images can be provided for different DoFs in the environment,
as detailed earlier herein, inter alia with reference to Figs. 2 and 3; or a digital image of a given DoF in the environment that is less affected (or unaffected) by ambient light can be generated, as detailed earlier herein, inter alia with reference to Figs. 4 and 5.
It is to be noted that, with reference to Figs. 2, 4, 6 and 9, some of the blocks can be integrated into a consolidated block or can be broken down to a few blocks and/or other blocks may be added. Furthermore, in some cases, the blocks can be performed in a different order than described herein. It should be also noted that whilst the flow diagram is described also with reference to the system elements that realizes them, this is by no means binding, and the blocks can be performed by elements other than those described herein.
It is to be understood that the presently disclosed subject matter is not limited in its application to the details set forth in the description contained herein or illustrated in the drawings. The presently disclosed subject matter is capable of other embodiments and of being practiced and carried out in various ways. Hence, it is to be understood that the phraseology and terminology employed herein are for the purpose of description and should not be regarded as limiting. As such, those skilled in the art will appreciate that the conception upon which this disclosure is based may readily be utilized as a basis for designing other structures, methods, and systems for carrying out the several purposes of the present presently disclosed subject matter.
It will also be understood that the system according to the presently disclosed subject matter can be implemented, at least partly, as a suitably programmed computer. Likewise, the presently disclosed subject matter contemplates a computer program being readable by a computer for executing the disclosed method. The presently disclosed subject matter further contemplates a machine-readable memory tangibly embodying a program of instructions executable by the machine for executing the disclosed method.
Claims
1. An active-gated imaging system for imaging a Depth of Field (DoF) in an environment at a predefined distance, the system comprising: a light source configured to emit light pulses towards the environment; a gated sensor comprising: an array of pixels, wherein given pixels of the pixels include: a photodetector, a first transfer gate switch, a second transfer gate switch, a first memory node associated with the first transfer gate switch, and a second memory node associated with the second transfer gate switch; and one or more gating controllers associated with the given pixels, each of the gating controllers being configured, for one or more of the given pixels, to:
(A) activate the first transfer gate switch of the respective given pixel, in a first synchronization scheme with an emission of the light pulses towards the environment, to accumulate, at the first memory node of the respective given pixel, a converted light signal that is based on returns of the light pulses from the DoF and an ambient light of the environment; and
(B) activate the second transfer gate switch of the respective given pixel, in a second synchronization scheme with the emission of the light pulses towards the environment, to accumulate, at the second memory node of the respective given pixel, a second converted light signal, wherein the second synchronization scheme is different from the first synchronization scheme; wherein the second converted light signal represents at least one of: (a) said pulses returning from a second DoF of the environment at a second predefined distance and (b) the ambient light of the environment without said returns.
2. The imaging system of claim 1, wherein the second converted light signal represents the ambient light of the environment without said returns, and wherein the system further comprises: at least one second controller configured, for each of the given pixels, to: associate: (i) the converted light signal of the respective given pixel with a first grey level for the respective given pixel and (ii) the second converted light signal
of the respective given pixel with a second grey level for the respective given pixel; and perform image subtraction for the respective given pixel, based on the first grey level and the second grey level, to provide a compensation grey level for the respective given pixel, the compensation grey level being the first grey level as compensated to reduce an effect of the ambient light on the first grey level; wherein an image of the DoF at the predefined distance is formed from the compensation grey levels of the given pixels.
3. An imaging system for imaging a Depth of Field (DoF) in an environment at a predefined distance, the system comprising: a light source configured to emit light pulses towards the environment; and a digital camera comprising: a Single Photon Avalanche Diode (SPAD) sensor having an array of pixels, wherein given pixels of the pixels include: (a) an event counter for counting avalanche events at the respective given pixel and (b) a transfer gate switch for gating the event counter; one or more gating controllers associated with the given pixels, each of the gating controllers being configured, for one or more of the given pixels, to control the transfer gate switch of the respective given pixel, in synchronization with an emission of the light pulses towards the environment, to: (a) pause the event counter of the respective given pixel prior and subsequent to the respective given pixel capturing a reflection signal indicative of a reflection of the light pulses returning from the DoF and (b) activate the event counter of the respective given pixel to count the avalanche events at the respective given pixel resulting from the capturing of the reflection signal; and at least one second controller configured, for each of the given pixels, to associate the avalanche events counted by the event counter of the respective given pixel with a grey level for the respective given pixel; wherein an image of the DoF is formed from the grey levels of the given pixels.
4. An active-gated imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance, the method comprising:
emitting light pulses, by a light source, towards the environment; and for given pixels in a gated sensor comprising an array of pixels, the given pixels including a photodetector, a first transfer gate switch, a second transfer gate switch, a first memory node associated with the first transfer gate switch, and a second memory node associated with the second transfer gate switch:
(A) activating the first transfer gate switch of the respective given pixel, in a first synchronization scheme with an emission of the light pulses towards the environment, to accumulate, at the first memory node of the respective given pixel, a converted light signal that is based on returns of the light pulses from the DoF and an ambient light of the environment; and
(B) activating the second transfer gate switch of the respective given pixel, in a second synchronization scheme with the emission of the light pulses towards the environment, to accumulate, at the second memory node of the respective given pixel, a second converted light signal, wherein the second synchronization scheme is different from the first synchronization scheme; wherein the second converted light signal represents at least one of: (a) said pulses returning from a second DoF of the environment at a second predefined distance and (b) the ambient light of the environment without said returns.
5. The imaging method of claim 4, wherein the second converted light signal represents the ambient light of the environment without said returns, and wherein the method further comprises: for each of the given pixels: associating: (i) the converted light signal of the respective given pixel with a first grey level for the respective given pixel and (ii) the second converted light signal of the respective given pixel with a second grey level for the respective given pixel; and performing image subtraction for the respective given pixel, based on the first grey level and the second grey level, to provide a compensation grey level for the respective given pixel, the compensation grey level being the first grey level as compensated to reduce an effect of the ambient light on the first grey level; wherein an image of the DoF at the predefined distance is formed from the compensation grey levels of the given pixels.
6. An imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance, the method comprising: emitting light pulses, by a light source, towards the environment; for a digital camera comprising a Single Photon Avalanche Diode (SPAD) sensor having an array of pixels, wherein given pixels of the pixels include: (a) an event counter for counting avalanche events at the respective given pixel and (b) a transfer gate switch for gating the event counter: for the given pixels: controlling the transfer gate switch of the respective given pixel, in synchronization with an emission of the light pulses towards the environment, to: (a) pause the event counter of the respective given pixel prior and subsequent to the respective given pixel capturing a reflection signal indicative of a reflection of the light pulses returning from the DoF and (b) activate the event counter of the respective given pixel to count the avalanche events at the respective given pixel resulting from the capturing of the reflection signal; and associating the avalanche events counted by the event counter of the respective given pixel with a grey level for the respective given pixel; wherein an image of the DoF is formed from the grey levels of the given pixels.
7. A non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, executable by at least one controller of a computer to perform an active-gated imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance, the method comprising: emitting light pulses, by a light source, towards the environment; and for given pixels in a gated sensor comprising an array of pixels, the given pixels including a photodetector, a first transfer gate switch, a second transfer gate switch, a first memory node associated with the first transfer gate switch, and a second memory node associated with the second transfer gate switch:
(A) activating the first transfer gate switch of the respective given pixel, in a first synchronization scheme with an emission of the light pulses towards the environment, to accumulate, at the first memory node of the respective given pixel, a
converted light signal that is based on returns of the light pulses from the DoF and an ambient light of the environment; and
(B) activating the second transfer gate switch of the respective given pixel, in a second synchronization scheme with the emission of the light pulses towards the environment, to accumulate, at the second memory node of the respective given pixel, a second converted light signal, wherein the second synchronization scheme is different from the first synchronization scheme; wherein the second converted light signal represents at least one of: (a) said pulses returning from a second DoF of the environment at a second predefined distance and (b) the ambient light of the environment without said returns.
8. A non-transitory computer readable storage medium having computer readable program code embodied therewith, the computer readable program code, executable by at least one controller of a computer to perform an imaging method for imaging a Depth of Field (DoF) in an environment at a predefined distance, the method comprising: emitting light pulses, by a light source, towards the environment; for a digital camera comprising a Single Photon Avalanche Diode (SPAD) sensor having an array of pixels, wherein given pixels of the pixels include: (a) an event counter for counting avalanche events at the respective given pixel and (b) a transfer gate switch for gating the event counter: for the given pixels: controlling the transfer gate switch of the respective given pixel, in synchronization with an emission of the light pulses towards the environment, to: (a) pause the event counter of the respective given pixel prior and subsequent to the respective given pixel capturing a reflection signal indicative of a reflection of the light pulses returning from the DoF and (b) activate the event counter of the respective given pixel to count the avalanche events at the respective given pixel resulting from the capturing of the reflection signal; and associating the avalanche events counted by the event counter of the respective given pixel with a grey level for the respective given pixel; wherein an image of the DoF is formed from the grey levels of the given pixels.
Applications Claiming Priority (2)
| Application Number | Priority Date | Filing Date | Title |
|---|---|---|---|
| US202263384602P | 2022-11-22 | 2022-11-22 | |
| US63/384,602 | 2022-11-22 |
Publications (1)
| Publication Number | Publication Date |
|---|---|
| WO2024110959A1 true WO2024110959A1 (en) | 2024-05-30 |
Family
ID=91195307
Family Applications (1)
| Application Number | Title | Priority Date | Filing Date |
|---|---|---|---|
| PCT/IL2023/051194 Ceased WO2024110959A1 (en) | 2022-11-22 | 2023-11-20 | Multiple gated pixel per light pulse and/or gated event counter for single photon avalanche diode (spad) |
Country Status (1)
| Country | Link |
|---|---|
| WO (1) | WO2024110959A1 (en) |
Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050269481A1 (en) * | 2002-08-05 | 2005-12-08 | Elbit Systems Ltd. | Vehicle mounted night vision imaging system and method |
| US9810785B2 (en) * | 2012-05-29 | 2017-11-07 | Brightway Vision Ltd. | Gated imaging using an adaptive depth of field |
| US20190056498A1 (en) * | 2016-03-01 | 2019-02-21 | Brightway Vision Ltd. | Gated imaging apparatus, system and method |
| US10564267B2 (en) * | 2014-10-27 | 2020-02-18 | Brightway Vision Ltd. | High dynamic range imaging of environment with a high intensity reflecting/transmitting source |
-
2023
- 2023-11-20 WO PCT/IL2023/051194 patent/WO2024110959A1/en not_active Ceased
Patent Citations (4)
| Publication number | Priority date | Publication date | Assignee | Title |
|---|---|---|---|---|
| US20050269481A1 (en) * | 2002-08-05 | 2005-12-08 | Elbit Systems Ltd. | Vehicle mounted night vision imaging system and method |
| US9810785B2 (en) * | 2012-05-29 | 2017-11-07 | Brightway Vision Ltd. | Gated imaging using an adaptive depth of field |
| US10564267B2 (en) * | 2014-10-27 | 2020-02-18 | Brightway Vision Ltd. | High dynamic range imaging of environment with a high intensity reflecting/transmitting source |
| US20190056498A1 (en) * | 2016-03-01 | 2019-02-21 | Brightway Vision Ltd. | Gated imaging apparatus, system and method |
Similar Documents
| Publication | Publication Date | Title |
|---|---|---|
| US9461088B2 (en) | Image sensor pixel with multiple storage nodes | |
| Gupta et al. | Photon-flooded single-photon 3d cameras | |
| KR102332287B1 (en) | A high dynamic range pixel and a method for operating it | |
| US10827135B2 (en) | BDI based pixel for synchronous frame-based and asynchronous event-driven readouts | |
| US8426797B2 (en) | CMOS readout architecture and method for photon-counting arrays | |
| US20210223398A1 (en) | Imaging systems with single-photon avalanche diodes and ambient light level detection | |
| US9006630B2 (en) | Quality of optically black reference pixels in CMOS iSoCs | |
| US20200326414A1 (en) | Neuromorphic Single Photon Avalanche Detector (SPAD) Array Microchip | |
| US9888200B2 (en) | Image sensor and operating method thereof | |
| US9848148B1 (en) | Methods and apparatus for a multiple storage pixel imaging system | |
| US11442153B2 (en) | Systolic processor system for a light ranging system | |
| WO2013127450A1 (en) | Self-reset asynchronous pulse frequency modulated droic with extended counting and having reduced quantization noise | |
| CN109884663B (en) | Time resolution sensor, three-dimensional imaging system and time resolution method | |
| US20250142226A1 (en) | Imaging device, optoelectronic device and method for operating an imaging device | |
| US11169270B2 (en) | Solid-state imaging device | |
| WO2024110959A1 (en) | Multiple gated pixel per light pulse and/or gated event counter for single photon avalanche diode (spad) | |
| CN106454156A (en) | Image sensor and operating method thereof | |
| Jaklin et al. | Global shutter CMOS vision sensors and event cameras for on‐chip dynamic information | |
| CN114829970A (en) | Time-of-flight imaging circuit, time-of-flight imaging system, and time-of-flight imaging method | |
| CN114071040A (en) | Image sensor and method for reading out signals | |
| US20250044423A1 (en) | Electronic device, method and computer program | |
| US20210006742A1 (en) | Image sensor and timing controller thereof | |
| US20250016474A1 (en) | Multi-storage gated imaging system | |
| Xiang et al. | A Token-Passing-Based Trigger-Prediction Methodology for Event-Driven ToF Sensors | |
| CN117857940A (en) | Signal processing circuit and electronic device |
Legal Events
| Date | Code | Title | Description |
|---|---|---|---|
| 121 | Ep: the epo has been informed by wipo that ep was designated in this application |
Ref document number: 23894109 Country of ref document: EP Kind code of ref document: A1 |
|
| NENP | Non-entry into the national phase |
Ref country code: DE |
|
| 122 | Ep: pct application non-entry in european phase |
Ref document number: 23894109 Country of ref document: EP Kind code of ref document: A1 |