TW201918060A - Digital pixel with extended dynamic range - Google Patents

Abstract

An example device is disclosed. In some examples, a device can include a photodiode, a first charge storage unit, and a second charge storage unit, wherein the first charge storage unit is configured to store the charge generated by the photodiode and has a first capacitance. The second charge storage unit is configured to store the charge generated by the photodiode and has a second capacitance greater than the first capacitance. The apparatus may further include an analog-to-digital converter (ADC) circuit for measuring a first amount of charge stored in the first charge storage unit and a second amount of charge stored in the second charge storage unit, and based on representing the first amount of charge A first metering or a second metering representative of the second amount of charge produces a digital output representative of the intensity of light incident on the photodiode.

Description

Digital pixels with extended dynamic range

The present invention relates to an image sensor, and more particularly to a pixel cell structure including an interface circuit to determine the intensity of light produced by an image.

A typical image sensor includes a photodiode for sensing incident light by converting the photon into electrical charges, such as electrons or holes. The image sensor further includes a floating node configuration as a capacitor to collect the charge generated by the photodiode during the exposure period. The collected charge can evolve into a voltage among the capacitors. This voltage can be buffered and passed to an analog digital converter (ADC), which converts this voltage into a digital value representative of the intensity of the incident light.

The present invention relates to an image sensor. In particular, but not limited to, a pixel unit is formed using a stacked structure. The invention also relates to controlling a pixel unit circuit to measure incident light intensity in two different measurement modes.

In one example, a device is proposed. The device can include a photodiode, a first charge storage unit, and a second charge storage unit. The first charge storage unit and the second charge storage unit are both used to store the charge generated by the photodiode. The first charge storage unit has a first capacitance and the second charge storage unit has a second capacitance greater than the first capacitance. The apparatus can further include an analog-to-digital converter (ADC) circuit for generating a first ramp voltage using the first meter in the first measurement mode; and comparing the first voltage with the first ramp voltage to generate a first decision output, The first voltage represents a first amount of charge stored in the first charge storage unit, and the first decision output sets a first meter in the first meter. The analog-to-digital converter circuit is also configured to generate a second ramp voltage using the second meter in the second measurement mode; and compare the second voltage with the second ramp voltage to generate a second decision output, the second voltage representative being stored in the second A second amount of charge in the charge storage unit, the second decision output setting a second meter in the second meter. The analog digital converter circuit is also operative to generate a digital output based on the first meter or the second meter, the digital output representing the intensity of light incident on the photodiode.

In some aspects, the device further includes a switching gate coupled between the first charge storage unit and the second charge storage unit. In the second measurement mode, the analog-to-digital converter circuit is configured to control the switching gate to prevent the first amount of charge from moving to the second charge storage unit through the switching gate; and to form the second charge storage unit by using the first comparator. The second voltage and the second ramp voltage are used to generate a second meter.

In some aspects, in the first measurement mode, the analog digital converter circuit is configured to control the switching gate to move the first amount of charge through the switching gate to the second charge storage unit. The analog-to-digital converter circuit is also operative to compare the first voltage formed in the second charge storage unit with the first ramp voltage using a second comparator to generate a first meter.

In some aspects, the analog digital converter circuit is configured to reset the second charge storage unit between the second measurement mode and the first measurement mode.

In some aspects, the analog-to-digital converter circuit includes a third capacitor coupled to the second charge storage unit and at least one of the first comparator or the second comparator. The third capacitor is configured to store a charge during resetting the second charge storage unit to compensate for at least one of: reset noise introduced to the second charge storage unit or the first comparator and the second comparator An offset voltage of at least one of them.

In some aspects, the second capacitance of the second charge storage unit is configurable. An analog to digital converter circuit is operative to reduce a second capacitance in the first measurement mode and increase a second capacitance in the second measurement mode.

In some aspects, the analog digital converter circuit is configured to perform the first mode measurement mode after the second measurement mode. The analog-to-digital converter circuit is further configured to: determine to store the first meter or the second meter in the memory based on the first decision output and the second decision output; and provide the first meter or the second meter stored in the memory as A digital output representing the intensity of light.

In some aspects, at least one of the first meter or the second meter is configured to generate at least one of the first ramp voltage or the second ramp voltage, respectively, to have a non-uniform ramping slope with respect to time.

In some aspects, the analog digital converter circuit is further used in the third measurement mode: comparing the second voltage with a fixed threshold to generate a third decision output, and the third decision output indicating whether the second voltage exceeds the fixed threshold; And generating a digital output based on the time of the third decision output, the digital output representing the intensity of light incident on the photodiode. In some aspects, the analog-to-digital converter circuit is further used in the third measurement mode: after driving the photodiode to transfer charge to the second charge storage unit, the third meter is enabled, and the third decision output is The third meter is set in the three gauges. The digital output representing the light intensity can be generated based on the third meter.

In some aspects, the analog to digital converter circuit is configured to perform a third measurement mode prior to the second measurement mode and the first measurement mode. The analog-to-digital converter circuit is further configured to: store the third meter in the memory; determine, based on the third decision output, that the third meter is not overwritten by the first meter or the second meter in the memory; and provide the memory in the memory The third measurement of the volume acts as a digital output representing the intensity of the light.

In some aspects, the first meter can include a second meter. The first comparator can also include a second comparator.

In another example, a method is proposed. The method includes exposing a photodiode to incident light to cause a photodiode to generate a charge, wherein the photodiode is coupled to the first charge storage unit and the second charge storage unit. The first charge storage unit has a first capacitance and the second charge storage unit has a second capacitance greater than the first capacitance. This method further includes performing a first measurement mode. This first measurement mode includes generating a first ramp voltage using the first meter and comparing the first voltage to the first ramp voltage to produce a first decision output. The first voltage represents a first amount of charge stored in the first charge storage unit, and the first decision output sets a first meter in the first meter. This method further includes performing a second measurement mode. The second measurement mode includes generating a second ramp voltage using the second metering; and comparing the second voltage to the second ramp voltage to generate a second decision output. The second voltage represents a second amount of charge stored in the second charge storage unit, and the second decision output is set in the second meter to set a second meter. The method further includes generating a digital output representative of the intensity of the incident light based on the first meter or the second meter.

In some aspects, the first charge storage unit is coupled to the second charge storage unit through the switching gate, and the performing the second measurement mode further comprises: controlling the switching gate to prevent the first charge amount from moving to the second charge storage unit through the switching gate; And comparing the second voltage formed by the second charge storage unit with the second ramp voltage to generate a second meter.

In some aspects, performing the first measurement mode further includes: controlling the switching gate to move the first amount of charge through the switching gate to the second charge storage unit; and comparing the first voltage formed in the second charge storage unit with the first The ramp voltage is used to generate the first meter.

In some aspects, at least one of the first ramp voltage or the second ramp voltage is generated to have a non-uniform ramp slope with respect to time.

In some aspects, this method may further include performing a third measurement mode. Performing the third measurement mode may include comparing the second voltage with a fixed threshold to generate a third decision output, the third decision output indicating whether the second voltage exceeds the fixed threshold; and generating the time based on the third decision output A digital output representing the intensity of light incident on the photodiode. The third measurement mode may be performed prior to performing the first measurement mode and the second measurement mode.

In the following description, for the purposes of illustration However, it will be apparent that various embodiments may be practiced without these specific details. The drawings and the description are not limitative.

A typical image sensor includes a photodiode for sensing incident light by converting a photon into a charge, such as an electron or a hole. The image sensor further includes a floating node configuration as a capacitor to collect the charge generated by the photodiode during the exposure period. The collected charge can evolve into a voltage in the capacitor. This voltage can be buffered and passed to an analog digital converter (ADC), which converts this voltage into a digital value representative of the intensity of the incident light.

The [0001] generated by the analog-to-digital converter can be associated with the intensity of the incident light, which reflects the amount of charge stored at this floating node for a specific time. However, the degree of association is affected by different factors. First, the amount of charge stored at the floating node is directly related to the intensity of the incident light until the floating node reaches the saturation limit. Beyond this saturation limit, the floating node may not be able to stably receive the extra charge generated by the photodiode, and the extra charge may leak out and cannot be stored. This will cause the amount of charge stored at the floating node to be lower than the amount of charge actually produced by the photodiode. This saturation limit determines the upper limit of the intensity of the photometric light of the image sensor.

Various factors can also set the lower limit of the measurable light intensity of the image sensor. For example, the charge collected at the floating node can include a noise charge that is not associated with the intensity of the incident light. One source of the noise charge may be a dark current, which may be due to a pn junction of the photodiode or a pn junction of other semiconductor devices connected to the capacitor due to crystallographic defects. Leakage current. Dark current can flow into the capacitor and increase the charge that is not associated with the intensity of the incident light. In general, the dark current generated in the photodiode is smaller than the dark current generated in other semiconductor devices. Another source of noise charge can be capacitively coupled to other circuits. For example, when the analog digital conversion circuit performs a read operation to determine the amount of charge stored in the floating node, the analog digital conversion circuit can introduce the noise charge to the floating node through capacitive coupling.

In addition to the noise charge, the analog digital conversion can also introduce measurement errors when determining the amount of charge. This measurement error reduces the correlation of the digital output to the intensity of the incident light. One source of measurement error can be the quantization error. In the quantization process, a set of discrete levels of levels can be used to represent a set of consecutive amounts of charge, with each quantity level representing a predetermined amount of charge. Analog-to-digital conversion compares the amount of input charge to this level of magnitude, determines the number level that is closest to the input, and outputs the determined level of magnitude (eg, in the form of a digitally encoded code representing the number of levels). A quantization error may occur when the amount of charge represented by the quantity level does not match the amount of input charge mapped to this number level. This quantization error can be reduced by a smaller quantization step (e.g., by reducing the difference in charge levels between two adjacent levels). Other sources of measurement error may include, for example, device noise that increases the uncertainty of the charge amount measurement (e.g., device noise of an analog digital conversion circuit) and the offset of the comparator. The noise charge and analog to digital converter measurement error can define the lower limit of the photometric intensity of the image sensor. The ratio of the upper and lower limits defines a dynamic range that allows the image sensor to set a range of operable light intensities.

Images may be generated based on intensity data provided by a plurality of image sensor arrays, wherein each image sensor forms a pixel unit corresponding to the image pixels. The array of pixel cells can be configured as a plurality of rows and columns, wherein each pixel produces a voltage that represents the intensity of a pixel at a particular location in the associated image. Multiple pixels in the array can determine the resolution of the resulting image. The voltage can be converted to digital intensity data by an analog digital converter, and the image can be reconstructed based on the digital intensity data for each pixel. With current technology, some of the plurality of digital units may need to access the analog digital converter in turn to generate digital strength data. For example, a set of analog digital converters can be provided to simultaneously process the voltage generated by each pixel cell in a column. However, adjacent columns of pixel cells may require sequential access to the set of analog digital converters. In one example, to produce an image, the pixel array can operate in a rolling shutter mode in which each pixel column is exposed to incident light to sequentially generate intensity data. For example, a pixel column of an image sensor can be exposed to incident light during an exposure period. Each of the pixel units in this column can generate a voltage based on the charge generated by the photodiode during the exposure period, and transfer this voltage to an analog digital converter. Such a ratio digital converter can generate a set of digital data representing the intensity of incident light received by a column of pixels. After generating the set of digit data for a pixel column, the next pixel column can be exposed to incident light during subsequent exposure periods to produce another set of digital intensity data until all of the pixel columns have been exposed to incident light and output intensity data. In yet another example, the exposure times of different columns of pixels may overlap slightly, but the pixels of each column still need to convert the voltage generated from the photodiode charge into digital data in turn. An image can be generated based on the digital intensity data for each pixel column.

Image sensors are available for a variety of different applications. As an example, an image sensor included in a digital image device (eg, a digital camera, smart phone, etc.) can provide a digital image. In another example, an image sensor is configured to function as an input device to control or affect the operation of the device, such as controlling or affecting a wearable virtual reality (VR) system and/or augmented reality (AR) and/or Display content of a near-eye display in a hybrid reality (MR) system. For example, an image sensor can be used to generate a physical image of a physical environment in which the user is located. The entity image can be provided to the location tracking system for tracking by a synchronous positioning and map creation (SLAM) algorithm, such as the location of the user in the physical environment, the direction of the user, and/or the path of movement of the user. The image sensor can also be used to generate solid image data that includes stereo depth information for measuring the distance between the user and the object in a physical environment. The image sensor can also be used as a near infrared (NIR) sensor. An illuminant can project light of a near-infrared type to the eye of the user. The internal structure of the eyeball, such as the pupil, produces an optical pattern from the NIR light. The image sensor captures the image of the optical pattern and provides the image to the system to track the movement of the user's eye to determine the user's gaze point. Based on these physical image data, the virtual reality/augmented reality/hybrid reality system can generate and update virtual image data for display to the user via the near-eye display, thereby providing the interactive experience to the user. For example, the virtual reality/augmented reality/hybrid reality system may update this virtual image material based on the user's gaze direction (which may indicate the user's interest in the object), the user's location, and the like.

Wearable VR/Augmented Reality/Mixed Reality systems operate in environments with a wide range of light intensities. For example, a wearable virtual reality/augmented reality/hybrid reality system can operate in an indoor environment or in an outdoor environment, and/or operate at different times of the day, and wearable virtual reality/amplification The light intensity of the operating environment of a real/hybrid reality system may vary substantially. Furthermore, the wearable VR/Augmented Reality/Mixed Reality system may also include the aforementioned NIR eye tracking system, which may need to project very low intensity light to the user's eye to avoid injury to the eye. Therefore, an image sensor of a wearable VR/Augmented Reality/Mixed Reality system may need to have a wide dynamic range to be able to function properly over a wide range of light intensities in different operating environments (eg, generate Output of incident light intensity). Image sensors that wear virtual reality/augmented reality/hybrid reality systems may also need to generate images at a sufficiently high speed to allow tracking of the user's position, orientation, gaze point, and the like. Image sensors with limited dynamic range and producing images at relatively slow speeds may not be suitable for such wearable virtual reality/amplified reality/hybrid reality systems.

The present invention relates to a pixel unit that provides enhanced dynamic range and improved processing speed. The pixel unit may include a photodiode, a first charge storage unit, a second charge storage unit, a switching gate between the first charge storage unit and the second charge storage unit, and an analog digital converter (ADC) circuit. Both the first charge storage unit and the second charge storage unit are capable of storing the charge generated by the photodiode, and the transfer gate controls the flow of charge from the first charge storage unit to the second charge storage unit. For example, the switching gate can be controlled by a bias voltage such that when the photodiode generates a charge due to contact with the incident light, the charge can be first accumulated in the first charge storage unit as a residual charge until formed in the first charge storage unit. The voltage exceeds the threshold set by this bias voltage. When the voltage formed in the first charge storage unit exceeds the threshold, additional charge (generated by the photodiode) can be transferred to the second charge storage unit through the transfer gate as an overflow charge (overcharge). The first charge storage unit may be a device capacitor of the photodiode. The second charge storage unit may be a device capacitor of a switching gate, a metal capacitor, a metal oxide semiconductor (MOS) capacitor, or a combination thereof. In general, the second charge storage unit has a much higher capacitance than the first charge storage unit has.

The pixel unit can operate under a plurality of measurement modes to perform light intensity measurements, the different measurement modes being for different light intensity ranges. In a first measurement mode for a range of low light intensities for which the first charge storage unit is not expected to reach full load, the analog digital converter can be operated to measure the amount of residual charge stored in the first charge storage unit to determine the light intensity. In a second measurement mode for the first charge storage unit to be expected to reach a range of light intensities in the full load, the analog digital converter can be operated to measure the amount of overflow charge stored in the second charge storage unit to determine the light intensity. And in a third measurement mode in which the first charge storage unit and the second charge storage unit are both expected to reach a full light intensity range, the analog digital converter is operative to measure an accumulation rate of the overflow charge in the second charge storage unit. And then determine the light intensity.

The disclosed techniques can be extended for the dynamic range of pixel units for low light intensity measurements and high light intensity measurements, and produce a digital output representative of the measurements. For example, the accumulation rate of the overflow charge (for the high light intensity range) provides a fairly accurate representation of the light intensity, and even when the second charge storage unit reaches the capacitance limit, the light intensity can be measured. Therefore, the upper limit of the photometric intensity of the image sensor can be increased, and the dynamic range can be expanded.

Furthermore, for the low light intensity range, the determination of the light intensity based on the measurement of the residual charge stored in the device capacitor of the photodiode can also improve the accuracy of the light intensity determination. As described above, in general, the photodiode generates less dark current than other semiconductor devices. Therefore, by judging the light intensity based on the measurement of the residual charge stored in the device capacitor of the photodiode, the influence of the dark current on the accuracy of the residual charge measurement (and the judgment of the light intensity) can be reduced. Therefore, less noise charge is introduced (eg, due to dark current), which in turn can lower the lower limit of the photometric intensity of the image sensor and further extend the dynamic range. Other techniques have been disclosed, such as variable charge-to-voltage conversion ratios, non-uniform quantization, etc., to further improve the accuracy of light intensity determination, particularly for low intensity light ranges.

Further, by providing an analog digital converter in the pixel unit, each pixel unit of the pixel array contacts the incident light and simultaneously produces a digital representation of the incident light intensity received by the pixel unit to provide a global shutter. Operation. For high-speed mobile capture, the advantage of the global shutter is that it can avoid the problem of moving distortion caused by the pixel unit being listed at different times to capture different parts of the image. This movement distortion and rolling shutter operation Related. Further, the processing time for generating an image using this pixel unit can be reduced compared to the current method in which the pixel unit column takes turns to expose and generate intensity data. Therefore, the disclosed technology can not only expand the dynamic range, but also improve the running speed of the pixel unit, and can also improve the application relying on the digital output of the pixel unit (for example, virtual reality/augmented reality/hybrid reality system). Performance, and the user experience.

Embodiments of the invention may be embodied or implemented in conjunction with an artificial reality system. The artificial reality system is a real-world type, which is adjusted in some way before being presented to the user, which may include, for example, virtual reality (VR), augmented reality (AR), Mixed reality (MR), hybrid reality, or any combination of the above and/or derivatives thereof. Artificial reality content may contain fully generated content or generated content combined with captured (real-world) content. The artificial reality content may include video, audio, haptic feedback, or any combination of the above, and any of the above may be presented via a single channel or multiple channels (eg, stereoscopic audio and video that produces a 3D effect to the viewer). Moreover, in some embodiments, the artificial reality may also be associated with an application, product, accessory, service, or any combination of the above, for example, to create content in an artificial reality and/or otherwise in a human reality. Use (eg perform activities). Artificial reality systems that provide artificial reality content can be implemented on a variety of platforms, including head-mounted displays (HMDs) connected to the host computer system, stand-alone HMDs, mobile devices, or any other capable of providing artificial reality content to one or more The hardware platform of the viewers.

FIG. 1A is a schematic diagram of a near-eye display 100 of an embodiment. The near-eye display 100 presents the media to the user. The medium presented by the near-eye display 100 includes, for example, one or more images, videos, and/or audio. In some embodiments, the audio system is presented by an external device (eg, a speaker and/or a headset) that receives audio information from the near-eye display 100, the console, or both, and based on the audio information. Present audio materials. In general, near-eye display 100 is used to operate as a virtual reality (VR) display. In some embodiments, the near-eye display 100 can be modified as an augmented reality (AR) display and/or a hybrid reality (MR) display.

The near-eye display 100 includes a frame 105 and a display 110. The frame 105 is coupled to one or more optical components. The display 110 is used for the user to view the content presented by the near-eye display 100. In some embodiments, display 110 includes a waveguide display assembly for directing light from one or more images to the user's eyes.

The near-eye display 100 further includes image sensors 120a, 120b, 120c, and 120d. Each of the image sensors 120a, 120b, 120c, and 120d can include an array of pixels for generating image data representing different fields of view in different directions. For example, sensors 120a and 120b can be used to provide image data representative of two fields of view in direction A along the Z-axis, while sensor 120c can be used to provide image data representative of A field of view in the direction B of the X-axis, the sensor 120d is used to provide image data representative of a field of view in a direction C along the X-axis.

In some embodiments, sensors 120a-120d may be used as input devices to control or influence the display content of near-eye display 100 to provide an interactive virtual reality/amplified reality/mixed reality experience to wear with near-eye display 100 User. For example, the sensors 120a-120d can generate physical image data of the physical environment in which the user is located. The physical image data is provided to a location tracking system to track the location and/or movement path of the user in the physical environment. The system can update the image data provided to display 100 based on, for example, the location and orientation of the user to provide an interactive experience. In some embodiments, the location tracking system can run a synchronous location and map creation (SLAM) algorithm to track a set of objects within the physical environment and the user's field of view as the user moves through the physical environment. The location tracking system can construct and update a map of the physical environment based on the set of objects and track the location of the user in the map. By providing image data corresponding to multiple fields of view, the sensors 120a-120d can provide a more comprehensive view of the physical environment to the location tracking system, which allows more objects to be included in the construction and update of the map. With such a configuration, the accuracy and reliability of tracking the location of the user in a physical environment can be improved.

In some embodiments, the near-eye display 100 can further include one or more active illuminators 130 that can project light into a physical environment. The projected light can be correlated to different spectra (eg, visible light, infrared light, ultraviolet light) and can be used for a variety of purposes. For example, the illuminator 130 can project light (or an environment with low intensity infrared light, ultraviolet light, etc.) in a dark environment to assist the sensors 120a-120d in capturing images of different objects to be able to track the position of the user. . The illuminator 130 can project a particular marker to an item in the environment to assist the location tracking system in identifying the item for map construction/update.

In some embodiments, the illuminator 130 can also achieve stereoscopic imaging. For example, sensor 120a or 120b can include a first array of pixels for sensing visible light and a second array of pixels for sensing infrared light (IR). The first pixel array may be covered by a color filter (eg, a Bayer filter), each pixel of the first pixel array being used to measure light intensity associated with a particular color (eg, red, green, blue) One). This second pixel array (for sensing infrared light) can also be covered by a filter that allows only infrared light to pass through, and each pixel of the second pixel array is used to measure the intensity of the infrared light. The pixel array can produce an RGB image of the object and an IR image, each pixel of the IR image being mapped to each pixel of the RGB image. The illuminator 130 can project a set of IR markers onto the object, and the set of IR-tagged images can be captured by the IR pixel array. Based on the distribution of the IR markers of the object in the image, the system can estimate the distance of different portions of the object from the IR pixel array and generate a stereoscopic image of the object based on the distances. Based on the stereoscopic image of the object, the system can determine, for example, the relative position of the object to the user, and can update the image data provided to display 100 based on the relative position information to provide an interactive experience.

As discussed above, the near-eye display 100 can operate in an environment having an extremely wide range of light intensities. For example, the near-eye display 100 can operate in an indoor or outdoor environment, and/or at different times of the day. The near-eye display 100 can also operate with the illuminator 130 turned on or off. Thus, image sensors 120a-120d may need to have a wide dynamic range to function properly over an extremely wide range of light intensities in different operating environments of near-eye display 100 (eg, to produce an output related to incident light intensity).

FIG. 1B is a schematic diagram of a near-eye display 100 of another embodiment. FIG. 1B illustrates the side of the near-eye display 100 facing the user's eye 135 wearing the near-eye display 100. As shown in FIG. 1B, the near-eye display 100 may further include a plurality of illuminants 140a, 140b, 140c, 140d, 140e, and 140f. The near-eye display 100 further includes a plurality of image sensors 150a and 150b. Illuminants 140a, 140b, and 140c can emit light (eg, near infrared light (NIR)) of a particular frequency range toward direction D (which is relative to direction A of FIG. 1A). The emitted light can be associated with a particular pattern and can be reflected by the user's left eye. The sensor 150 can include an array of pixels to receive the reflected light and produce an image of the reflected pattern. Similarly, illuminants 140d, 140e, and 140f can emit NIR rays with this pattern. This NIR light can be reflected by the user's right eye and can be received by the sensor 150b. The sensor 150b can also include an array of pixels to produce an image of the reflected pattern. Based on the images of the reflection patterns from the sensors 150a and 150b, the system can determine the gaze point of the user and update the image data provided to the display 100 based on the determined gaze point to provide an interactive experience to the user.

As discussed above, in order to avoid damage to the user's eyeballs, the illuminants 140a, 140b, 140c, 140d, 140e, and 140f are generally used to output light of very low intensity. In the case where the image sensors 150a and 150b and the image sensors 120a-120d of FIG. 1A include the same sensor device, when the incident light intensity is extremely low, the image sensors 120a-120d may need to be able to Producing an output related to the intensity of the incident light may further increase the dynamic range requirement of the image sensor.

Additionally, image sensors 120a-120d may need to be able to produce an output at high speed to track the movement of the eyeball. For example, the user's eye can perform a very fast motion (eg, a saccade) that quickly jumps from one eye position to another. In order to track the rapid movement of the user's eyeballs, the image sensors 120a-120d need to produce an image of the eyeball at high speeds. For example, the rate at which the image sensor produces an image frame (frame rate) needs to match at least the rate of eye movement. A high frame rate requires that the total exposure time of all pixel cells involved in the generation of the image frame must be short-lived and that the sensor output needs to be quickly converted to a digital value to produce an image. Moreover, as discussed above, image sensors also need to be able to operate in environments with low light intensities.

2 is a cross-sectional view 200 of the near-eye display 100 of FIG. 1 as an embodiment. Display 110 includes at least one waveguide display assembly 210. The exit pupil 230 is a position in which the user's single eyeball 220 is aligned in the eyebox region when the user wears the near-eye display 100. For purposes of illustration, FIG. 2 depicts a cross-section 200 associated with the eyeball 22 and the single waveguide display assembly 210, but a second waveguide display assembly can be used for the second eye of the user.

The waveguide display assembly 210 is used to direct image light to a viewable area of the exit pupil 230 and the eyeball 220. The waveguide display assembly 210 can be comprised of one or more materials (eg, plastic, glass, etc.) with one or more refractive indices. In some embodiments, the near-eye display 100 includes one or more optical elements between the waveguide display assembly 210 and the eyeball 220.

In some embodiments, waveguide display assembly 210 includes a stack of one or more waveguide displays including, but not limited to, a stacked waveguide display, a varifocal waveguide display, and the like. The stacked waveguide display is a multi-color display (eg, a red-green-blue (RGB) display) constructed by stacking a plurality of waveguide displays having respective monochromatic light sources of different colors. The stacked waveguide display can also be a multi-color display (eg, a multi-plane color display) that is projected on multiple planes. In some versions, the stacked waveguide display is a multi-color display (eg, a multi-planar monochrome display) that is projected on multiple planes. The zoom waveguide display is a display that adjusts the focus position of the image light emitted from the waveguide display. In an alternative embodiment, the waveguide display assembly 210 can include the stacked waveguide display and the zoomed waveguide display.

FIG. 3 illustrates an isometric view of a waveguide display 300 in accordance with an embodiment. In some embodiments, waveguide display 300 is an element of near-eye display 100 (such as waveguide display assembly 210). In some embodiments, waveguide display 300 is part of some other near-eye display or other system that directs image light to a particular location.

[0002] The waveguide display 300 includes a source component 310, an output waveguide 320, and a controller 330. For purposes of illustration, FIG. 3 shows a waveguide display 300 associated with a single eye 220, but in some embodiments, another waveguide display that is separate or partially separate from the waveguide display 300 provides image light to the user's other Only eyes.

Source component 310 produces image light 355. Source component 310 generates and outputs image light 355 to a coupling element 350 located on first side 370-1 of output waveguide 320. Output waveguide 320 is an optical waveguide that outputs expanded image light 340 to a user's eye 220. Output waveguide 320 receives image light 355 at one or more coupling elements 350 on first side 370-1 and directs the received input image light 355 to a steering element 360. In some embodiments, coupling element 350 couples image light 355 from source component 310 to output waveguide 320. The coupling element 350 can be, for example, a diffraction grating, a holographic grating, one or more cascaded reflectors, one or more tantalum surface elements, and/or a holographic reflection. Array of holographic reflectors.

The guiding element 360 redirects the received output image light 355 to a decoupling element 365 such that the received input image light 355 is decoupled outside the output waveguide 320 via the decoupling element 365. The guiding element 360 is a portion of the first side 370-1 of the output waveguide 320 or is fixed to the first side 370-1 of the output waveguide 320. The decoupling element 365 is part of the second side 370-2 of the output waveguide 320 or is fixed to the second side 370-2 of the output waveguide 320 such that the guiding element 360 is opposite the decoupling element 365. The guiding element 360 and/or the decoupling element 365 can be, for example, a diffraction grating, a holographic grating, one or more tandem reflectors, one or more tantalum surface elements, and/or a holographic reflector array.

The second side 370-2 represents a plane along the x dimension and the y dimension. Output waveguide 320 may be comprised of one or more materials that promote total internal fling of image light 355. The output waveguide 320 can be composed of, for example, tantalum, plastic, glass, and/or a polymer. Output waveguide 320 has a relatively small form factor. For example, the output waveguide 320 can be about 50 millimeters (mm) wide along the x dimension, 30 millimeters long along the y dimension, and 0.5 to 1 millimeter thick along the z dimension.

The controller 330 controls the scanning operation of the source component 310. Controller 330 determines scan instructions for source component 310. In some embodiments, output waveguide 320 outputs extended image light 340 having a field of view (FOV) to the user's eye 220. For example, the extended image light 340 is provided to the user's eye 220 with a diagonal field of view (in the x and y dimensions) having 60 degrees and/or greater and/or 150 degrees and/or less. The output waveguide 320 is for providing a viewable area having a length of 20 mm or more and/or 50 mm or less; and/or a width of 10 mm or more and/or less than or equal to 50 mm.

Further, based on the image data provided by image sensor 370, controller 330 also controls image light 355 produced by source component 310. Image sensor 370 can be located on first side 370-1 and can include image sensors 120a-120d, such as of FIG. 1A, to generate image material of a physical environment in front of the user (eg, for confirmation) position). Image sensor 370 may also be located on second side 370-2 and may include image sensors 150a and 150b of FIG. 1B to generate image data of the user's eye 220 (eg, for confirming a gaze point) ). Image sensor 370 can connect a remote console that is not located within waveguide display 300. The image sensor 370 can provide image data to a remote console that can confirm, for example, the location of the user, the gaze point of the user, etc., and determine the content of the image displayed to the user. The remote console can transmit instructions related to the determined content to the controller 330. Based on the instructions, controller 330 can control the generation and output of image light 355 through source component 310.

4 depicts a cross section 400 of a waveguide display 300 in accordance with an embodiment. This profile 400 includes a source component 310, an output waveguide 320, and an image sensor 370. In the example of FIG. 4, image sensor 370 can include a set of pixel units 402 located on first side 370-1 to produce an image of the physical environment in front of the user. In some embodiments, a mechanical shutter 404 can be inserted between the set of pixel units 402 and the physical environment to control exposure of the set of pixel units 402. In some embodiments, the mechanical shutter 404 can be replaced by an electronic shutter that will be discussed below. Each pixel unit 402 can correspond to one pixel of an image. Although not shown in FIG. 4, it is understood that each pixel unit 402 can also be covered with a filter to control the frequency range of light sensed by the pixel unit.

Upon receipt of an instruction from the remote console, the mechanical shutter 404 can open and expose the set of pixel units 402 during an exposure period. During the exposure period, image sensor 370 can obtain samples of light incident on the set of pixel units 402 and generate image data based on an intensity distribution of incident light samples detected by the set of pixel units 402. Image sensor 370 can then provide the image material to a remote console that determines the display content and provide display content information to controller 330. The controller 330 can then determine the image light 355 based on the displayed content information.

Source component 310 produces image light 355 in accordance with instructions from controller 330. Source component 310 includes a source 410 and an optical system 415. Source 410 is a light source that produces coherent or partial dimming. The source 410 is, for example, a laser diode, a vertical cavity surface emitting laser, and/or a light emitting diode.

Optical system 415 includes one or more optical elements that modulate light from source 410. Adjusting the light from source 410 includes, for example, expanding, collimating, and/or adjusting the orientation in accordance with instructions from controller 330. The one or more optical elements can include one or more lenses, fluid lenses, mirrors, apertures, and/or gratings. In some embodiments, optical system 415 includes a fluid lens having a plurality of electrodes that allow scanning of a beam of light having a scan angle threshold to move the beam to an area other than the fluid lens. Light emitted from optical system 415 (and source component 310) is treated as image light 355.

Output waveguide 320 receives image light 355. Coupling element 350 couples image light 355 from source assembly 310 to output waveguide 320. In embodiments where the coupling element 350 is a diffraction grating, the pitch of the diffraction grating is selected such that total internal reflection occurs in the output waveguide 320 and the image light 355 is directed toward the decoupling element 365 at the output waveguide. 320 propagates internally (eg, by total internal reflection).

The guiding element 360 redirects the image light 355 toward the decoupling element 365 to be decoupled from the output waveguide 320. In embodiments where the guiding element 360 is a diffraction grating, the pitch of the selected diffraction grating causes the incident image light 355 to exit the output waveguide 320 at an oblique angle relative to the surface of the decoupling element 365.

In some embodiments, the guiding element 360 and/or the decoupling element 365 are structurally similar. The extended image light 340 exiting the output waveguide 320 is expanded along one or more dimensions (eg, may be elongated along the x dimension). In some embodiments, waveguide display 300 includes a plurality of source components 310 and a plurality of output waveguides 320. Each source component 310 emits monochromatic image light having a particular wavelength band corresponding to a primary color (eg, red, green, blue). Each of the output waveguides 320 may be stacked together at a spaced apart distance to output multi-color extended image light 340.

FIG. 5 is a block diagram of a system 500 including a near-eye display 100 in accordance with an embodiment. System 500 includes a near-eye display 100, an image device 535, an output/output interface 540, and image sensors 120a-120d and 150a-150d that are each coupled to a control circuitry 510. System 500 can be used as a head mounted device, a wearable device, or the like.

The near-eye display 100 is a display that presents media to a user. The media presented by the near-eye display 100 can include, for example, one or more images, video, and/or audio. In some embodiments, the audio is presented by an external device (eg, a speaker and/or a headset) that receives audio information from near-eye display 100 and/or control circuitry 510 and presents the user based on the audio information Audio data. In some embodiments, the near-eye display 100 can also function as an AR glasses. In some embodiments, the near-eye display augments the physical real-world view with computer generated elements such as images, video, sound, and the like.

The near-eye display 100 includes a waveguide display assembly 210, one or more position sensors 525, and/or an inertial measurement unit (IMU) 530. The waveguide display assembly 210 includes a source component 310, an output waveguide 320, and a controller 330.

The IMU 530 is an electronic device that generates fast correction data based on measurement signals received from one or more position sensors 525 that indicate an estimated position of the near-eye display 100 relative to the initial position of the near-eye display 100.

Image device 535 can generate image material for a variety of applications. For example, image device 535 can generate image information based on correction parameters received from control circuitry 510 to provide slow correction data. Image device 535 may include image sensors 120a-120d, such as FIG. 1A, to generate image data of the physical environment in which the user is located, thereby performing location tracking by the user. Image device 535 can further include image sensors 150a-150b, such as FIG. 1B, to generate image data that confirms the user's gaze point, thereby identifying objects of interest to the user.

Input/output interface 540 is a device that allows a user to send an action request to control circuitry 510. An action request is a request to perform a specific action. For example, an action request can be to start or end an application or perform a specific action within an application.

Control circuitry 510 provides media to near-eye display 100 for presentation to the user based on information received from one or more of image device 535, near-eye display 100, and input/output interface 540. In some examples, control circuitry 510 can be wrapped within system 500 as a head mounted device. In some examples, control circuitry 510 can be a separate component of the console device communication coupling system 500. In the example shown in FIG. 5, the control circuitry 510 includes an application store 545, a tracking module 550, and an engine 555.

The application store 545 stores one or more applications for execution by the control circuitry 510. An application is a set of instructions that, when executed by a processor, can produce content for presentation to a user. The application can include, for example, a gaming application, a conferencing application, a video playback application, or other suitable application.

The tracking module 550 uses one or more calibration parameter correction systems 500 and can adjust one or more calibration parameters to reduce errors in confirming the position of the near-eye display 100.

The tracking module 550 tracks the movement of the near-eye display 100 using the slow correction information from the image device 535. The tracking module 550 also uses the position information from the quick correction information to confirm the position of the reference point of the near-eye display 100.

The engine 555 executes the application within the system 500 and receives location information, acceleration information, speed information, and/or predicted future locations of the near-eye display 100 from the tracking module 550. In some embodiments, the information received by engine 555 can be used to generate a signal (eg, display instructions) to waveguide display component 210 to determine the type of content presented to the user. For example, to provide an interactive experience, engine 555 is based on the user's location (eg, provided by tracking module 550), the user's gaze point (eg, based on image data provided from image device 535), objects, and The distance between the users (e.g., based on image data provided from image device 535) may determine the content to be presented to the user.

FIG. 6 illustrates an example of a pixel unit 600. Pixel unit 600 can be part of a pixel array and can generate digital intensity data for a pixel corresponding to an image. For example, pixel unit 600 can be part of a plurality of pixel units 402 of FIG. As shown in FIG. 6, the pixel unit 600 may include a photodiode 602, a residual charge capacitor 603, a shutter switch 604, a switching gate 606, a reset switch 607, a measurement capacitor 608, a buffer 609, and a pixel analog-to-digital converter 610.

In some embodiments, the photodiode 602 can comprise a P-N diode or a P-I-N diode. Each of the shutter switch 604, the switching gate 606, and the reset switch 607 may include a transistor. This transistor includes, for example, a metal oxide semiconductor field effect transistor (MOSFET), a bipolar junction diode (BJT), or the like. Shutter switch 604 acts as an electronic shutter gate (which may be replaced or incorporated into mechanical shutter 404 of Figure 4) to control the exposure period of pixel unit 600. During the exposure period, the shutter switch 604 can be disabled (turned off) by the exposure enable signal 611. The exposure enable signal 611 can allow the charge generated by the photodiode 602 to move to the residual charge capacitor 603 and/or the measurement capacitor 608. . At the end of the exposure period, the shutter switch 604 can be caused to divert the charge generated by the photodiode 602 to the photodiode current slot 617. In addition, reset switch 607 can also be disabled (turned off) by reset signal 618, which can allow measurement capacitor 608 to accumulate charge and form a voltage that reflects the accumulated charge. After completing a measurement mode, reset switch 607 can be enabled to empty the charge stored in measurement capacitor 608 to charge slot 620, allowing measurement capacitor 608 to make the next measurement.

The residual charge capacitor 603 can be a device capacitor of the photodiode 602 and can store the charge generated by the photodiode 602. Residual charge capacitor 603 can include, for example, a junction capacitance at the P-N diode connection interface or other device capacitors that connect photodiode 602. Since the residual charge capacitor 603 is close to the photodiode 602, the charge generated by the photodiode 602 can be accumulated in the capacitor 603. Measurement capacitor 608 can be a device capacitor located at switching gate 606, metal capacitor, MOS capacitor, or floating termination of any combination thereof. Measurement capacitor 608 can be used to store an amount of charge that can be measured by pixel analog to digital converter 610 to provide a digital output representative of the intensity of the incident light. The charge stored in the measurement capacitor 608 may be an overflow charge (from the photodiode 602) that is not added to the residual charge capacitor 603, or a residual charge that is emptied from the residual charge capacitor 603.

Referring to FIG. 7, [0003] illustrates the operation of charge accumulation at residual charge capacitor 603 and measurement capacitor 608 for different target light intensity ranges. FIG. 7 illustrates the total amount of charge accumulated (or expected to accumulate) among the residual charge capacitor 603 and the measurement capacitor 608 for different light intensity ranges versus time. The total amount of charge accumulated may reflect the total charge generated by the photodiode 602 during exposure, which also reflects the intensity of light incident on the photodiode 602 during exposure. When the exposure period ends, the amount can be measured. Threshold 702 and threshold 704 may be defined as a threshold charge amount that defines a low light intensity range 706, a medium light intensity range 708, and a high light intensity range 710 of incident light intensity. For example, if the total accumulated charge is below a threshold 702 (eg, Q1), the incident light intensity is within the low light intensity range 706. If the total accumulated charge is between threshold 704 and threshold 702 (e.g., Q2), the incident light intensity is in the mid-light intensity range 708. If the total accumulated charge is above threshold 704, the incident light intensity is in the high light intensity range 710.

Thresholds 702 and 704 can be set to control the accumulation of charge among residual charge capacitor 603 and measurement capacitor 608 to ensure that when the incident light intensity falls within low light intensity range 706 or medium light intensity range 708, the accumulated charge at the capacitor will Associated with the intensity of the incident light. For example, thresholds 702 and 704 can be set lower than the capacitance of residual charge capacitor 603 and measurement capacitor 608. As discussed above, once the residual charge capacitor 603 and the measurement capacitor 608 reach full capacity, the capacitor may begin to overflow the charge, and the voltage formed at the capacitor may not accurately represent or reflect the exposure of the photodiode 602. The total amount of charge produced during the period. By setting the thresholds 702 and 704 below the capacitance of the residual charge capacitor 603 and the measurement capacitor 608, measurement errors due to charge overflow can be avoided. In some examples, threshold 702 can be set at 2000e- (2000 charges) and threshold 704 can be set at 63000e- (63000 charges).

The accumulation of charge within residual charge capacitor 603 and measurement capacitor 608 can be controlled by thresholds 702 and 704. For example, the incident light intensity falling within the low light intensity range 706 can be based on the total charge accumulated in the residual charge capacitor 603. Assuming that the residual charge capacitor 603 has not reached full load at the end of the exposure period, the total charge accumulated in the residual charge capacitor 603 can reflect the total charge generated by the photodiode 602 during exposure, and can be used to determine the incident light intensity. . When the total amount of charge accumulated in the residual charge capacitor 603 exceeds the threshold 702, the additional charge generated by the photodiode 602 can be transferred to the measurement capacitor 608 as an overflow charge. Assuming that the measurement capacitor 608 has not reached full load at the end of the exposure period, the total overflow charge accumulated in the measurement capacitor 608 can also reflect the total charge generated by the photodiode 602 during the exposure period, and can be used to determine the incident light. Intensity (in the medium light intensity range 708).

On the other hand, where the incident light intensity is in the high light intensity range 710, the total overflow charge accumulated at the measurement capacitor 608 may exceed the threshold 704 before the end of the exposure period. When additional charge is accumulated, the measurement capacitor 608 may reach full load before the end of the exposure period, and charge overflow may occur. To avoid measuring errors caused by the measurement capacitor 608 reaching full load, a time-to-saturation measurement can be performed to measure the time it takes for the total accumulated charge accumulated by the measurement capacitor 608 to reach the threshold 704. The charge accumulation rate at the measurement capacitor 608 is determined based on the ratio of the threshold 704 to the saturation time, and at the end of the exposure period, the estimated charge amount (Q3) accumulated in the measurement capacitor 608 can be determined based on the estimation of the charge accumulation rate. (assuming the capacitor has an infinite capacitance). This estimated charge amount (Q3) provides a reasonable and accurate representation of the incident light intensity in the high light intensity range 710.

Returning to FIG. [0004] 6, the switching gate 606 can be controlled by the measurement control signal 612 to control the charge accumulation at the residual charge capacitor 603 and the measurement capacitor 608 for different light intensity ranges as described above. For example, for low light intensity range 706, switching gate 606 can be controlled to operate in a partially turned-on state. During the exposure period, when the total accumulated charge at the residual charge capacitor 603 reaches the threshold 702, the gate voltage of the switching gate 606 can be set according to the voltage formed in the residual charge capacitor 603. With such an arrangement, the charge generated by the photodiode 602 will first be stored in the residual charge capacitor 603 until the accumulated charge reaches a threshold 702. Before the end of the exposure period, the switching gate 606 can be controlled to operate in a fully turned-on state to move the charge stored in the residual charge capacitor 603 to the measurement capacitor 608. After the end of charge transfer, the switching gate 606 can be controlled to operate in a fully turned-off state to retain the charge stored in the measuring capacitor 608. At this time, the charge stored in the measurement capacitor 608 can be stored in the residual charge capacitor 603 and can be used to determine the incident light intensity. On the other hand, for the medium light intensity range 708 and the high light intensity range 710, when the switching gate 606 is still in the partially open state and the charge stored in the residual charge capacitor 603 has not been transferred to the measuring capacitor 608, it is accumulated in the measuring capacitor 608. The overflow charge can also be measured before the end of the exposure period.

The charge accumulated at the measurement capacitor 608 can be sensed by the buffer 609 to produce a replica of the analog voltage (but with a greater driving force) at the analog output node 614. The analog voltage at analog output node 614 can be converted to a set of digital data (e.g., containing logic 1 and 0) by pixel analog to digital converter 610. The analog voltage formed at measurement capacitor 608 can be sampled and the digital output can be generated before exposure (eg, for medium light intensity range 708 and high light intensity range 710), or after exposure (for low light intensity range 706) . This digital data can be transmitted to control circuitry 510, such as FIG. 5, through a set of pixel output busses 616 to represent the light intensity during the exposure period.

In some examples, the capacitance of the measurement capacitor 608 can be configured to improve the accuracy of the light intensity determination for low light intensity ranges. For example, when the measurement capacitor 608 is used to measure the residual charge stored in the residual charge capacitor 603, the capacitance of the measurement capacitor 608 can be reduced. The reduction in the capacitance of the measurement capacitor 608 can increase the charge-to-voltage conversion ratio at the measurement capacitor 608, which allows a certain amount of stored charge to form a higher voltage. The higher charge voltage conversion rate reduces the effect of measurement errors (eg, quantization errors, comparator offsets, etc.) produced by the pixel analog digital conversion 610 on the accuracy of low light intensity determination. The measurement error can set a limit on the minimum voltage difference that can be detected and/or discerned by the pixel analog to digital conversion 610. By increasing the charge voltage conversion ratio, the amount of charge corresponding to this minimum voltage difference can be reduced, and the lower limit of the measurable light intensity of the pixel unit 600 can be reduced and the dynamic range can be expanded. On the other hand, for medium light intensity, the capacitance of the measurement capacitor 608 can be boosted to ensure that the measurement capacitor 608 has sufficient capacitance to store the amount of charge as defined, for example, by the threshold 704.

FIG. 8 illustrates an example of internal components of a pixel analog to digital converter 610. As shown in FIG. 8, pixel analog to digital conversion 610 includes a threshold generator 802, a comparator 804, and a digital output generator 806. The digital output generator 806 can further include a counter 808 and a memory 810. The meter 808 can generate a set of count values based on the free-running clock signal 812, and the memory 810 can store the measured values generated by the meter 808. At least some measurement values (such as the latest measurement value). In some embodiments, memory 810 can be part of meter 808. Memory 810 can be, for example, a latch circuit to store metering values based on local pixel values as described below. Threshold generator 802 includes a digital analog converter (DAC) 813 that accepts a set of digital values and outputs a reference voltage (VREF) 815 that represents the set of digital values. As will be discussed in greater detail below, threshold generator 802 can accept static digit values to generate a fixed threshold or accept output 814 of meter 808 to generate a ramp threshold.

Although FIG. 8 illustrates digital analog converter 813 (and threshold generator 802) as part of pixel analog digital conversion 610, it will be appreciated that digital analog converter 813 (and threshold generator 802) may be from different pixel units. Coupled to multiple digital output generators. In addition, digital output generator 806 can also be shared by multiple multi-pixel units to produce digital values.

Comparator 804 can compare the analog voltage formed at analog output node 614 with a threshold provided by threshold generator 802 and generate a decision 816 based on the comparison. For example, if the analog voltage at analog output node 614 equals or exceeds a threshold generated by threshold generator 802, comparator 804 can generate a logical one as decision 816. Comparator 804 may also generate a logical zero as decision 816 if the analog voltage is below a threshold. Decision 816 can control the metering operation of meter 808 and/or the metered value stored in memory 810 to perform a time-of-saturation measurement of the ramp analog voltage of the aforementioned bit at analog output node 614, and execute at The quantization process of the analog voltage of the analog output node 614 is used to make a determination of the incident light intensity.

FIG. 9A illustrates an example of saturation time measurement of pixel analog to digital conversion 610. To perform saturation time measurements, threshold generator 802 can control digital analog converter 813 to generate a fixed reference voltage 815. The fixed reference voltage 815 can be set to a voltage corresponding to the charge amount threshold, which is between the mid-light intensity range and the high light intensity range (eg, threshold 704 of FIG. 7). Meter 808 can begin metering during exposure (eg, after shutter switch 604 is disabled). When the analog voltage at analog output node 614 drops (or rises based on operation), clock signal 812 remains toggling to update the metered value at meter 808. The analog voltage can reach this fixed threshold at a particular point in time, causing comparator 804 to flip decision 816. The flipping of this decision 816 may stop the metering of the meter 808, and this metering value of the meter 808 may represent a time-to-saturation. As will be discussed in more detail below, the charge accumulation rate at the measurement capacitor 608 can also be determined based on this duration and the incident light intensity can be determined based on the charge accumulation rate.

FIG. 9B illustrates an example of quantizing analog voltages by pixel analog to digital conversion 610. After the start of the measurement, the digital analog converter 813 can be programmed by the metered output 714 to generate a ramp reference voltage 815 that can ramp up (as in the example of Figure 9B) or ramp down depending on the implementation. In the example of FIG. 9B, the quantization process can be performed using a uniform quantization step, with the reference voltage 815 increasing (or decreasing) by the same amount for each clock cycle of the clock signal 812. The amount of increase (or decrease) of the reference voltage 815 corresponds to the quantization step. When the reference voltage 815 reaches the analog voltage of the analog output node 614 within a quantization step, the decision 816 of the comparator 804 flips. The flipping of decision 816 can stop the metering of meter 808, and the metering value can correspond to the total number of quantization steps, the total number of which is accumulated in a quantization step to match the analog voltage. This measurement can be a digital representation of the charge stored in measurement capacitor 608, as well as a digital representation of the intensity of the incident light. As noted above, quantification of the analog voltage can occur during the exposure period (e.g., for medium light intensity range 708) and after the exposure period (e.g., for low light intensity range 706).

As discussed above, the amount of charge represented by the quantum level of the analog-to-digital conversion 610 and the actual amount of input charge mapped to this magnitude by the analog-to-digital conversion 610 (eg, expressed as the total number of quantization steps) When there is no match between, the analog digital conversion 610 causes a quantization error. The quantization error can be reduced by using a smaller quantization step. In the example of FIG. 9B, the quantization error can be reduced by the amount of increase (or decrease) of the reference voltage 815 for each clock cycle.

Although a smaller quantization step can be used to reduce quantization error, the region and performance speed may limit how much the quantization step can be reduced. With a smaller quantization step, the total number of quantization steps required to represent a particular range of charge amounts (and light intensities) may increase. A larger number of data bits may be required to represent the increased number of quantization steps (eg, 8 bits for 255 steps, 7 bits for 127 steps, etc.). The larger number of data bits may require additional bus bars to be added to the pixel output bus 616. However, if the pixel unit 600 is used for a head mounted device or other wearable device with limited space, it may not be feasible. Sex. Furthermore, using a larger amount of quantization step, the analog digital conversion 610 may need to cycle through a larger number of quantization steps before finding a level of compliance (with a quantization step), which results in an increase in processing power consumption time. And reduce the rate at which image data is produced. For applications that require high frame rates, such as applications that track eye movements, this reduced rate is unacceptable.

One way to reduce the quantization error is to use a non-uniform quantization scheme whose quantization steps are non-uniform in the input range. FIG. 10A illustrates the correspondence between the analog-to-digital conversion coding (output of the quantization process) and the input charge magnitude for the non-uniform quantization process and the uniform quantization process. The broken line indicates the correspondence of the non-uniform quantization process, and the solid line indicates the correspondence of the uniform quantization process. For the uniform quantization process, the quantization step size (expressed as Δ1) is the same throughout the input charge amount range. Conversely, in the non-uniform quantization process, the quantization step size differs depending on the amount of input charge. For example, the quantization step (represented by ΔS) for a low input charge amount is less than the quantization step size (expressed by ΔL) for a high input charge amount. Furthermore, for the same low input charge amount, the quantization step size (ΔS) of the non-uniform quantization process may be smaller than the quantization step size (Δ1) of the uniform quantization process.

One advantage of employing a non-uniform quantization scheme is that the quantization step for quantizing the low input charge amount can be reduced, the quantization error for quantizing the low input charge amount can also be reduced, and the analogy can be reduced by the analog bit converter 610. The minimum amount of input charge. Therefore, the reduced quantization error can lower the lower limit of the measurable light intensity of the image sensor and can increase the dynamic range. Furthermore, although the quantization error increases for a high input charge amount, the quantization error may remain small compared to the high input charge amount. Therefore, the overall quantization error of the introduced charge measurement can be reduced. On the other hand, the total number of quantization steps covering the entire body range of the input charge amount may remain the same (or even decrease), and is associated with the aforementioned potential problems of increasing the number of quantization steps (eg, increasing the area, reducing the processing speed, etc.) ) can be avoided.

FIG. 10B illustrates an example in which the pixel analog to digital converter 610 quantizes the analog voltage using a non-uniform quantization process. In contrast to Figure 9B (using a uniform quantization process), the reference voltage 815 increases in a non-linear manner with each clock cycle, initially having a more suppressed slope and a steeper slope thereafter. The difference in these slopes is due to the uneven quantization step. For lower meter values (corresponding to lower input range), the quantization step becomes smaller, so the reference voltage 815 increases at a slower rate. For higher gauge values (corresponding to a higher input range), the quantization step becomes larger, so the reference voltage 815 increases at a higher rate. The uneven quantization step of the reference voltage 815 can be introduced using different schemes. For example, as discussed above, digital analog converter 813 is used to output voltages to different meter values (from meter 808). The digital analog converter 813 can be configured such that the output voltage difference between two adjacent meter count values is different for different meter count values. As another example, the meter 808 can also be configured to generate jumps in the meter count value instead of increasing or decreasing through the same count step to generate a non-uniform quantization step. In some examples, the non-uniform quantization process of FIG. 10B can be used for light intensity determinations for low light intensity range 706 and medium light intensity range 708.

Please refer to FIG. 11 . FIG. 11 illustrates an example of a pixel unit 1100. The pixel unit 1100 can be an embodiment of the pixel unit 600 of FIG. 6 . In the example of FIG. 11, the PD may correspond to the photodiode 602, the transistor M0 may correspond to the shutter switch 604, the transistor M1 may correspond to the switching gate 606, and the transistor M2 may correspond to the reset switch 607. Additionally, PDCAP may correspond to residual charge capacitor 603, and a combination of COF and CEXT capacitances may correspond to measurement capacitor 608. The capacitance of the measurement capacitor 608 is configured by the signal LG. When LG is enabled, measurement capacitor 608 provides a combined capacitance of the COF and CEXT capacitors. When LG is disabled, the CEXT capacitor can be detached from this parallel combination, and measurement capacitor 608 contains only COF capacitors (plus other parasitic capacitances). As discussed above, the capacitance of the measurement capacitor 608 can be lowered to increase the charge voltage conversion rate for low light intensity determination, and the capacitance of the measurement capacitor 608 can be increased to provide the necessary capacitance for the medium light intensity determination.

Pixel unit 1100 further includes an example of buffer 609 and an example of pixel analog digital conversion 610. For example, transistors M3 and M4 form a source follower that can be tied to buffer 609 of FIG. 6 for buffering analog voltages formed at node OF, such voltages being stored in capacitor COF (or in capacitors) The amount of charge in COF and CEXT). Further, the capacitor CC, the comparator 1110, the transistor M5, and the NOR gate 1112 together with the memory 810 can be part of the pixel analog-to-digital converter 610 to generate a representative voltage representative of the analog voltage at the OF node. Digital output. As described above, the quantization may be based on a comparison result (voltage VOUT) between the analog voltage formed at the node OF and the reference voltage (VREF) generated by the comparator 1110. Here, capacitor CC is used to generate a voltage VIN (located at an input of comparator 1110) that tracks the output of buffer 609 and provides voltage VIN to comparator 1110 for comparison with a reference voltage (VREF). The reference voltage (VREF) can be used as a constant voltage for saturation time measurements (for high light intensity ranges) or ramp voltages for quantification of analog voltages (for low and medium light intensity ranges). An analog code conversion input can be generated by a free-running counter (eg, meter 808), and the comparison result produced by comparator 1110 can determine that the analog digital conversion code input is to be stored in The memory 810 and the output are represented as digits of the intensity of the incident light. In some examples, the generation of a reference voltage for low and medium light intensity determinations may be based on a non-uniform quantization scheme, as discussed in Figures 10A and 10B.

In addition to the techniques disclosed above, pixel unit 1100 includes techniques that more accurately improve the determination of incident light intensity. For example, a combination of capacitor CC and transistor M5 can be used to compensate for measurement errors caused by comparator 1110 (eg, offset of the comparator), as well as other error signals caused by comparator 1110, such that comparator 1110 The accuracy can be improved. The noise may include, for example, a reset noise charge caused by the reset switch 607, an output noise of the buffer 609 due to a source follower threshold mismatch, and the like. When both transistors M2 and M5 are enabled, the amount of charge reflecting the comparator offset and the error signal can be stored in capacitor CC during the reset phase. Due to the stored voltage, a voltage difference can also be formed in the capacitor CC during the reset phase. During the measurement phase, the voltage difference of capacitor CC remains unchanged, and capacitor CC can generate the voltage VIN by subtracting (or adding) this voltage difference to track the output voltage of buffer 609. Therefore, the voltage VIN can be compensated for measurement errors and error signals, which improves the accuracy of the comparison between the voltage VIN and the reference voltage VREF and subsequent quantization.

In some embodiments, pixel unit 1100 can be operated during a three-phase measurement process. Each of these three stages may correspond to one of the three light intensity ranges of FIG. 7 (eg, low light intensity range 706, medium light intensity range 708, and high light intensity range 710). In each stage, the pixel unit 1100 can be operated in a measurement mode that targets a corresponding range of light intensities, and based on the output of the comparator 1110, determines whether the incident light intensity falls within a corresponding range of light intensities. If the incident light intensity falls within the corresponding light intensity range, the pixel unit 1100 can latch the analog digital converter code input (from the meter 808) to the memory 810 and latch the memory 810 (using the flag FLAG_1) And the combination of the flag FLAG_2) to avoid overwriting the memory 810 in the subsequent measurement phase. At the end of the three-phase measurement process, the analog digital converter coded input stored in memory 810 can be provided as a digital output representative of the intensity of the incident light.

Please refer to FIGS. 12A-12D, which illustrate the change of the control signal of the pixel unit 1100 with respect to time during the three-stage measurement. Referring to FIG. 12A, the period between the time point T0 and the time point T1 corresponds to the first reset phase, and the period between the time point T1 and the time point T4 corresponds to the exposure period. During the exposure period, the period between the time point T1 and the time point T2 corresponds to the first stage of measuring the high light intensity range (for example, the high light intensity range 710), and the period between the time point T2 and the time point T3 corresponds to the measured light intensity range. The second phase (eg, medium light intensity range 708), and the time period between time point T3 and time point T4 corresponds to the second reset phase. Further, the period between the time point T4 and the time point T5 corresponds to the third stage of measuring the low light intensity range (eg, the low light intensity range 706). The pixel unit 1100 can provide a digital output representative of the intensity of the incident light at time T5 and then begin the next three-phase measurement process.

As shown in FIG. 12A, between the time points T0 and T1, the signals RST1 and RST2, and the signal LG, the signal TX and the shutter signal SHUTTER are asserted. Therefore, they are stored in the capacitors PDCAP, CEXT and COF. The charge is removed. Further, since the charge generated by the photodiode PD is transferred by the transistor M0, no charge is applied to the capacitors. Further, the comparator 1110 is also in the reset phase, and the capacitor CC can store the charge reflecting the reset noise introduced by the transistor M2, the comparator offset, the buffer 609 threshold mismatch, and the like. At the end of this phase, this TX gate is shifted to a threshold level to acquire a predetermined amount of charge (e.g., threshold 702) at capacitor PDCAP. This threshold level can be set based on the voltage corresponding to this predetermined number of charges.

During the period between time points T1 and T2, the shutter signal is disabled and the signal LG remains enabled, so that the charge generated by the photodiode PD flows into the capacitor PDCAP, and if the voltage formed in the capacitor PDCAP exceeds The threshold level set by the TX gate allows the charge to flow into the capacitors COF and CEXT. FIG. 12B illustrates the measurement operations performed by the analog digital conversion 610 during this time period. As shown in FIG. 12B, the analog digital conversion 610 can perform saturation time measurements, and when the meter 808 is free to run, the buffered and error compensated version of the analog voltage at the node OF (VIN) can be compared to the charge representative of the threshold 704. A threshold voltage is compared. If the total amount of charge stored in capacitor COF and capacitor CEXT exceeds threshold 704 (based on node OF voltage), comparator 1110 will trip, and the count value produced by meter 808 can be stored in memory when tripped. 810. The trip of comparator 1110 also causes the register storing FLAG_1 to store a value of one. Regardless of the other input system of the NOR gate, this non-zero FLAG_1 value can keep the output of the inverse OR gate 1112 low and can latch this memory and avoid the subsequent measurement phase overwriting the meter 808. On the other hand, if the comparator 1110 never trips during the period between the time points T1 and T2, which represents that the incident light intensity is below the high light intensity range, FLAG_1 remains at zero. Regardless of whether comparator 1110 is tripped, FLAG_2, which is updated by the subsequent measurement phase, remains at zero.

In the period corresponding to the second measurement phase between the time points T2 and T3, the analog voltage at the node OF can be quantized by the analog digital conversion 610. FIG. 12C illustrates the measurement operations performed by analog digital conversion 610 during this time period. As shown in Figure 12C, a ramp reference voltage VREF is provided to comparator 1110 for comparison with a buffered and error compensated version of the analog voltage at node OF (VIN). Although FIG. 12C illustrates a ramp reference voltage VREF corresponding to a uniform quantization process, it will be appreciated that as described with respect to FIG. 10B, the ramp reference voltage VREF may also include a non-uniform slope corresponding to a non-uniform quantization process. The second phase measurement ends at time point T3, where the analog digital conversion input code representing the overall mid-incident light range has been cycled. If the ramp reference voltage VREF matches VIN (within a quantization step), the comparator 1110 will trip, and if the memory 810 is not latched by the first measurement phase (as indicated by the value 0 of FLAG_1), then the measurement This count value generated by the 808 when tripping can be stored to the memory 810. If the memory is latched, this count value will not be stored in the memory 810. On the other hand, if the memory is not latched, the count value generated by the meter 808 when it trips can be stored in the memory 810, and can be latched by writing a value of 1 to a register stored with FLAG_2. Memory.

At the beginning of the time period between time points T3 and T4, signals RST1 and RST2 can be enabled again for the second reset phase. The purpose of the second reset phase is to reset the capacitors CEXT and COF and prepare to store the capacitor COF for the charge transferred by the capacitor PDCAP during the third measurement phase (for the low light intensity range). This signal LG can also be disabled to disconnect the capacitor CEXT from the capacitor COF and reduce the capacitance of the measurement capacitor 608. As described above, the decrease in capacitance is to increase the charge voltage conversion rate to improve the low light intensity judgment. Comparator 1110 also enters a reset state in which capacitor CC can be used to store the noise charge generated by the reset of capacitors CEXT and COF. At time T4, after the reset is completed, signals RST1 and RST2 are disabled, and the offset signal TX can be increased to fully conduct the crystal M1. The charge stored in capacitor PDCAP can then be moved through transistor M1 to capacitor COF.

At a time interval between time points T4 and T5, a third measurement phase is performed for the low light intensity range. During this time period, the shutter signal is enabled to end the exposure period, and the signal TX is disabled to disconnect the capacitor PDCA from the photodiode PD to ensure that the capacitor COF is stored only during the exposure period during measurement. The charge of the capacitor PDCAP. FIG. 12D illustrates the measurement operations performed by analog digital conversion 610 during this time period. As shown in Figure 12D, a ramp reference voltage VREF can be provided to comparator 1110 for comparison with a buffered and error compensated version of the analog voltage at node OF (VIN). Although FIG. 12D illustrates a ramp voltage VREF corresponding to a uniform quantization process, it can be understood that the ramp voltage VREF can also include a non-uniform slope corresponding to a non-uniform quantization process, as described in relation to FIG. 10B. The third measurement phase ends at time T5, at which point the analog digital conversion input code representing the overall low incident light range has been cycled. As shown in FIG. 12D, if the ramp reference voltage VREF matches VIN (within a quantization step), the comparator 1110 will trip and the count value generated by the meter 808 when tripped can be stored to the memory 810. If the memory 810 is not latched by the first and second measurement stages (as indicated by the value 0 of FLAG_1 and FLAG_2). If the memory 810 is latched, this count value is not stored in the memory 810. This count value stored in memory 810 can be provided as a digital output representative of the intensity of the incident light.

Figure 13 is a diagram showing an example of signal to noise ratio over a range of incident light intensities that can be achieved by embodiments of the present invention. The signal-to-noise ratio (SNR) can be determined based on the ratio of the signal power representing the incident light intensity to the noise power. The noise power can include, for example, noise charges generated by dark current, switching noise (such as read noise), device noise (such as scattering noise), and errors introduced by measurement of signal power. For example, quantization error, comparator offset, device mismatch, and the like. As shown in Figure 13, this SNR pattern can be smooth and continuous over the entire range of light intensities and boundaries between adjacent intensity ranges. The smoothness of this SNR pattern is desirable to ensure predictability of pixel cell performance. In this example, device noise (eg, scattered noise and dark current) is substantially larger than other noises (eg, noise caused by measurement errors, such as quantization error, reset) in the entire set of incident light intensity ranges. Noise, etc.), so the SNR graph can be smooth. These noises can be reduced by using exposed techniques such as increasing the charge voltage conversion rate for low light intensity measurements, non-uniform quantization, and compensation for comparator offset, device mismatch, reset noise, etc. Program.

14 is a flow diagram of a process 1400 for determining incident light intensity at a pixel unit (e.g., pixel unit 600, pixel unit 1100, etc.), in accordance with an embodiment. Process 1400 can be performed by a controller and pixel unit 600 along with various components of pixel unit 1100. Process 1400 begins at step 1402, where a pixel unit is executed in an exposure mode in which a photodiode can transfer charge to a residual charge capacitor and/or a measurement capacitance. In step 1404, the pixel unit is operable to compare a voltage formed in the measurement capacitor with a fixed threshold voltage to generate a first decision and a first meter at the meter. Step 1404 can be for a first measurement phase of the high light intensity range, and this first measurement can represent a saturation time measurement. If the first decision system is positive (at step 1406), the pixel unit can proceed to step 1408 and store the first meter in a memory and then pass a first flag (eg, FLAG_1). This memory is latched and then proceeds to step 1410 to perform a second measurement phase. If the first decision is not positive, the pixel unit can also directly enter the step to perform the second measurement phase.

In step 1408, the pixel unit is operated to compare the voltage formed in the measuring capacitor with the first ramp voltage to generate a second decision and a second metering at the meter. Step 1408 can be a second measurement phase for the mid-light intensity range. Then, in step 1412, the pixel unit can determine whether the second decision is positive and determine whether the memory is latched (eg, based on the first flag, FLAG_1 remains disabled). If the second decision system is positive and the memory is not latched, the pixel unit may perform step 1414 and store the second meter in the memory and enable the second flag (eg, FLAG_2) to lock. This memory is stored and then proceeds to step 1416 to perform the second measurement phase. If the first decision is not positive, the pixel unit may also proceed directly to step 1416 to perform the third measurement phase.

In step 1416, as part of the third measurement phase, the pixel unit can reset the measurement capacitor to empty the stored charge. In step 1418, the pixel unit can also reduce the capacitance of the measuring capacitor to increase the charge voltage conversion rate. In step 1420, the pixel unit can transfer the residual charge stored in the residual charge capacitor of the photodiode to the measurement container. However, the pixel unit proceeds to step 1422 to compare the voltage formed within the measurement capacitor with a second ramp voltage to produce a third decision and a third meter at the meter. Then, in step 1424, the pixel unit performs a determination as to whether the third decision is positive and whether the memory is not latched (eg, based on whether any of the first flag FLAG_1 and the second flag FLAG_2 is caused. can). If the third decision is positive and the memory is not latched, then in step 1426, the pixel unit stores the third meter in the memory and then proceeds to 1428 for output to be stored here. The measured value in the memory. On the other hand, if the third decision is not positive or the memory has been latched, the pixel unit will proceed directly to step 1428 to output the measured value stored in the memory (which may be One of the first measurement and the second measurement).

The foregoing description of the embodiments of the present invention has been presented for purposes of illustration Those skilled in the relevant art will appreciate that many modifications and variations are possible in light of the above disclosure.

Some portions of this embodiment describe embodiments of the invention based on aspects of algorithms and symbolic representations of information operations. Descriptions and representations of these algorithms are typically used by those skilled in the data processing arts to effectively convey the essence of their work to other skilled in the art. These operations are described in terms of functional, computational or logical aspects, but should be understood to be implemented by computer programs or equivalent circuits, microcode or the like. In addition, it has sometimes been proven that the arrangement of these operations is referred to as a modular system without loss of generality. The described operations and their associated modules may be embodied on software, firmware and/or hardware.

The described steps, operations or processes may be performed or implemented by one or more hardware or software modules, either alone or in combination with other devices. In some embodiments, the software module is implemented by a computer program product comprising a computer readable medium containing computer code, the computer code being executable by a computer processor to implement any or all of the described Step, operation or process.

Embodiments of the invention may also relate to apparatus for performing the operations described. The device may be specially constructed for the required purposes and/or include a general purpose computing device that is selectively enabled or reconfigured by a computer program stored in the computer. Such computer programs can be stored in a non-transitory state, a specific computer readable storage medium, or any medium suitable for storing electronic commands that can be coupled to a computer system bus. In addition, any computer system mentioned in the specification may include a single processor or an architecture that employs a multi-processor design to increase computing power.

Embodiments of the invention may also relate to products produced by the computing processes described herein. Such products may include information generated by a computing process in which the information is stored in a non-transitory state, a particular computer readable storage medium, and may include any embodiment of a computer program product or other combination of materials described herein.

The language used in the specification is primarily for readability and instructional purposes and is not chosen to describe or limit the subject matter of the invention. The scope of the present invention is therefore not limited by the scope of the invention, but is limited by the scope of the appended claims. Therefore, the disclosure of the embodiments is intended to be illustrative, and not to limit the scope of the claims

100‧‧‧ near-eye display

105‧‧‧Architecture

135, 220‧ ‧ eyeballs

110‧‧‧ display

120a~120d, 150a, 150b‧‧‧ sensors

130, 140a~140f‧‧‧ illuminants

A, B, C‧‧‧ directions

200, 400‧‧ ‧ section

210‧‧‧Waveguide Display Components

230‧‧‧Output

300‧‧‧Waveguide display

310‧‧‧ source components

320‧‧‧Output waveguide

330‧‧‧ Controller

340, 355‧‧‧ image light

350‧‧‧Coupling components

360‧‧‧guide elements

365‧‧‧Decoupling components

370‧‧‧Image Sensor

370-1‧‧‧ first side

370-2‧‧‧ second side

402‧‧‧Pixel unit

404‧‧ ‧Shutter

410‧‧‧Source

415‧‧‧Optical system

500‧‧‧ system

510‧‧‧Control circuitry

525‧‧‧ position sensor

530‧‧‧Inertial Sensing Unit

535‧‧‧Image device

540‧‧‧Input/Output Interface

545‧‧‧App Store

550‧‧‧Tracking module

555‧‧‧ engine

600, 1100‧‧ ‧ pixel unit

602‧‧‧Photoelectric diode

603‧‧‧Residual charge capacitor

604‧‧‧Shutter switch

606‧‧‧Transition gate

607‧‧‧Reset switch

608‧‧‧Measurement capacitor

609‧‧‧buffer

610‧‧‧Pixel Analog Digital Converter

611‧‧‧Exposure enable signal

612‧‧‧Measurement control signal

614‧‧‧ analog output node

616‧‧‧Pixel Output Bus

617‧‧‧Photodiode current slot

702, 704‧‧‧ threshold

706‧‧‧Low light intensity range

708‧‧‧ Medium light intensity range

710‧‧‧High light intensity range

Q1~Q3‧‧‧Charge

802‧‧‧Threshold Generator

804‧‧‧ comparator

806‧‧‧Digital Output Generator

808‧‧‧meter

810‧‧‧ memory

812‧‧‧clock signal

813‧‧‧Digital Analog Converter

814‧‧‧meter output

815‧‧‧reference voltage

816‧‧ decision

1110‧‧‧ Comparator

1112‧‧‧Anti-gate

Δ1, ΔS, ΔL‧‧ ‧ quantization step

PD‧‧‧Photoelectric diode

PDCAP, CEXT, COF, CC‧‧‧ capacitors

M0~M6‧‧‧O crystal

RST1, RST2, LG, TX‧‧‧ signals

FLAG_1, FLAG2‧‧‧ flag

VIN, VOUT‧‧‧ voltage

VREF‧‧‧reference voltage

SHUTTER‧‧·Shutter Signal

OF‧‧‧ node

T0~T5‧‧‧

Exemplary embodiments are described with reference to the following figures.

1A and 1B are schematic diagrams of a near-eye display according to an embodiment.

2 is a cross-sectional view of a near-eye display, in accordance with an embodiment.

3 is an isometric view of a waveguide display having a single source component, in accordance with an embodiment.

4 is a cross-sectional view of a waveguide display 300 in accordance with an embodiment.

5 is a block diagram of a system including a near-eye display, in accordance with an embodiment.

FIG. 6 is a block diagram of a pixel unit according to an embodiment.

Figure 7 is a diagram showing the operation of determining different ranges of light intensities according to the embodiment of Figure 6.

FIG. 8 is a diagram showing an example of internal components of the pixel unit of FIG. 6.

9A and 9B are diagrams showing an exemplary method of determining light intensity.

10A and 10B illustrate techniques for performing quantization.

FIG. 11 is a block diagram of a pixel unit according to an embodiment.

12A, 12B, 12C, and 12D illustrate an exemplary method of determining light intensity.

Figure 13 is a diagram showing an example of signal to noise ratio over a range of incident light intensities that can be achieved by embodiments of the present invention.

14 is a flow chart of a process for determining light intensity, in accordance with an embodiment.

The drawings depict embodiments of the invention for purposes of illustration only. Those skilled in the art will appreciate from the following description that alternative embodiments of the illustrated structures and methods may be employed without departing from the principles of the invention.

In the accompanying drawings, similar components and/or features may have the same reference numerals. Further, various components of the same type may be distinguished by reference numerals below the dashed line and second symbols that distinguish similar components. If only the first reference number is used in the specification, this description applies to any one of the similar components having the same first reference number, regardless of the second reference number.

Claims (20)

A device comprising: a photodiode; a first charge storage unit for storing the charge generated by the photodiode, the first charge storage unit having a first capacitance; and a second charge storage unit For storing the charge generated by the photodiode, the second charge storage unit has a second capacitance greater than the first capacitance; and an analog-to-digital converter circuit for: a first measurement In the mode: generating a first ramp voltage by using a first meter; and comparing a first voltage with the first ramp voltage to generate a first decision output, where the first voltage is stored in the first charge storage unit a first amount of charge, the first decision output sets a first meter in the first meter; and in a second measurement mode: generating a second ramp voltage using a second meter; and comparing a first a second voltage and the second ramp voltage to generate a second decision output, the second voltage representing a second amount of charge stored in the second charge storage unit, the second decision output setting the second In a second metering gauge; and generating a digital output based on the first measurement or the second measurement, the digital output representative of the intensity of light incident on one of the photodiode. The device of claim 1, further comprising a switching gate coupled between the first charge storage unit and the second charge storage unit; wherein, in the first measurement mode, the analog-to-digital converter circuit And configured to: control the switching gate to prevent the first amount of charge from moving to the second charge storage unit through the switching gate; and compare the second voltage formed in the second charge storage unit using a first comparator And the second ramp voltage to generate the second meter. The device of claim 2, wherein in the first measurement mode, the analog-to-digital converter circuit is configured to: control the switching gate to drive the first amount of charge to move to the second charge storage unit through the switching gate And using a second comparator to compare the first voltage formed in the second charge storage unit with the first ramp voltage to generate the first meter. The device of claim 3, wherein the analog to digital converter circuit is configured to reset the second charge storage unit between the first measurement mode and the second measurement mode. The device of claim 4, wherein the analog-to-digital converter circuit includes a third capacitor coupled between the second charge storage device and at least one of the first comparator and the second comparator; The third capacitor is configured to store a charge to compensate for at least one of: reset noise introduced to the second charge storage unit or the first comparator and the resetting the second charge storage unit An offset voltage of at least one of the second comparators. The device of claim 1, wherein the second capacitance of the second charge storage unit is configurable; and wherein a digital analog circuit is configured to reduce the second capacitance in the first measurement mode and The second capacitance in the second measurement mode is increased. The apparatus of claim 1, wherein the analog-to-digital converter circuit is configured to perform the first measurement mode after the second measurement mode; wherein the analog-to-digital converter circuit is further configured to: based on the first decision output Determining, by the second decision output, storing the first meter and the second meter in a memory; and providing the first meter or the second meter stored in the memory as the representative of the photodiode This digital output of light intensity. The device of claim 1, wherein at least one of the first meter and the second meter is used to generate at least one of the first ramp voltage and the second ramp voltage, respectively, to have a non-time Uniform ramping slope. The device of claim 1, wherein in the third measurement mode, the analog-to-digital converter circuit is further configured to: compare the second voltage with a fixed threshold to generate a third decision output, the third decision The output indicates whether the second voltage exceeds the fixed threshold; and the digital output is generated based on the timing of the third decision output, the digital output representing a light intensity incident on the photodiode. The device of claim 9, wherein in the third measurement mode, the analog-to-digital converter circuit is further configured to: after driving the photodiode to transfer charge to the second charge storage unit, enabling a third meter; and setting a third meter in the third meter with the third decision output; and wherein the digital output representative of the light intensity is generated based on the third meter. The apparatus of claim 10, wherein the analog-to-digital converter circuit is configured to perform the third measurement mode before the second measurement mode and the first measurement mode; wherein the analog-to-digital converter circuit is further configured to: The third meter is stored in a memory; determining, based on the third decision output, that the third meter is not overwritten by the first meter or the second meter in the memory; and providing the third meter stored in the memory The third meter is output as the digit representing the intensity of the light. The device of claim 1 wherein the first meter comprises the second meter. The device of claim 3, wherein the first comparator comprises the second comparator. The device of claim 10, wherein the first meter comprises the third meter. A method includes: exposing a photodiode to an incident light to cause the photodiode to generate a charge, wherein the photodiode is coupled to a first charge storage unit and a second charge storage unit, wherein the photodiode is coupled to a first charge storage unit and a second charge storage unit a charge storage unit has a first capacitance and the second charge storage unit has a second capacitance greater than the first capacitance; performing a first measurement mode, comprising: generating a first using a first meter a ramp voltage; and comparing a first voltage with the first ramp voltage to generate a first decision output, the first voltage representing a first amount of charge stored in the first charge storage unit, the first decision output setting a first metering in the first meter; performing a second measuring mode, comprising: generating a second ramp voltage using a second meter; comparing a second voltage with the second ramp voltage to generate a second decision Outputting, the second voltage represents a second amount of charge stored in the second charge storage unit, the second decision output setting a second meter in the second meter; and based on the first Metering the amount or generates a second digital output represents one of the intensity of the incident light. The method of claim 15, wherein the first charge storage unit is coupled to the second charge storage unit via a switching gate; wherein performing the second measurement mode comprises: controlling the switching gate to prevent the first charge amount from passing The switching gate moves to the second charge storage unit; and compares the second voltage formed in the second charge storage unit with the second ramp voltage to generate the second meter. The method of claim 16, wherein the performing the first measurement mode further comprises: controlling the switching gate to drive the first amount of charge to move to the second charge storage unit through the switching gate: and comparing to form the second The first voltage of the charge storage unit and the first ramp voltage to generate the first meter. The method of claim 15 wherein at least one of the first ramp voltage and the second ramp voltage is generated to have a non-uniform ramping slope associated with time. The method of claim 15, further comprising: performing a third measurement mode comprising: comparing the second voltage with a fixed threshold to generate a third decision output, the third decision output indicating whether the second voltage exceeds The fixed threshold value; and generating the digital output representative of a light intensity incident on the photodiode based on the time of the third decision output, wherein the performing before the performing the first measurement mode and the second measurement mode is performed This third measurement mode. An apparatus comprising: a plurality of pixels, each of the pixels comprising: a photodiode; a first charge storage unit for storing a charge generated by the photodiode, the first charge storage unit having a first a second charge storage unit for storing the charge generated by the photodiode, the second charge storage unit having a second capacitance greater than the first capacitance; and an analog-to-digital converter a circuit for: in a first measurement mode: generating a first ramp voltage using a first meter; and comparing a first voltage with the first ramp voltage to generate a first decision output, the first voltage Representing a first amount of charge stored in the first charge storage unit, the first decision output setting a first meter in the first meter; and in a second measurement mode: generating using a second meter a second ramp voltage; and comparing a second voltage to the second ramp voltage to generate a second decision output, the second voltage representing a second amount of charge stored in the second charge storage unit, A second decision output of the second set in a second measurement gauge; and generating a digital output based on the first measurement or the second measurement, the digital output represents one of the intensity of light incident on the photoelectric diode.