KR20020077793A - Method for storing and retrieving digital image data from an imaging array - Google Patents

Abstract

CLAIMS 1. A method for storing digital information from an image sensor, comprising: providing an image sensor for generating three color output data at each of a plurality of pixel locations; Providing a digital storage device coupled to the image sensor; Sensing three-color digital output data from an image sensor; And storing the tricolor output data as digital data in a digital storage device without performing any interpolation on the tricolor output data. The data can be compressed before storage and decompressed after retrieval from the storage device. In a preferred embodiment, the image sensor comprises a triple junction active pixel sensor array.

Description

METHOD FOR STORING AND RETRIEVING DIGITAL IMAGE DATA FROM AN IMAGING ARRAY

The process of capturing, processing, storing and retrieving digital data is common in the field of digital imaging.

In general, digital images are provided from a source such as a camera. Many types of cameras are widely used in the field of digital imaging, including digital, video, and television cameras. Whatever the type of camera used, it is often desirable for images to be captured and stored in digital format, so that the images can later be edited or otherwise processed. In the prior art, it is common to interpolate and compress digital image data prior to storage. Processing the data prior to storage has certain disadvantages inherent in the processes used so far in the prior art.

First, the interpolation process can introduce an irreversible change in the digital image data. Interpolation is the process of correcting data for errors that occur according to the type of sensor or camera used in the camera. Thus, the type of interpolation used or the need for interpolation processing is entirely determined by the nature of the imaging process used. For example, it is common in the art to use digital sensors that include charge-coupled device (CCD) or metal oxide semiconductor (MOS) transistors. Within a sensor, the smallest resolution full-color image component ("pixel") generally comprises four individual sensors; It consists of two greens, one blue and one red. These sensors are used to generate three color digital outputs. However, interpolation is essential to correct for distortion caused by small but limited distances that separate the four individual sensors that make up each pixel. The result of such interpolation often increases the size of the original digital image significantly. Often this increase in data size is about three times. With the increase in size, interpolation compromises the integrity of the original data if executed before storage.

Secondly, after the interpolation step, the digital image data is often compressed before storage. Compression is often necessary not only because of the size increase after the interpolation just described, but also to facilitate transmission over a limited bandwidth system such as a television system. However, in the compression method generally used, once the digital image is compressed, it cannot be restored to its original state. This is a significant disadvantage if access to the original uncompressed digital image data is required.

The problem with interpolation and compression of digital image data prior to storage is evident in poor quality output when viewed on screen or in print. In practice, interpolation or compression techniques often generate moiré patterns on fine pitch fabrics or result in distortion and / or loss of detail along or between the fine lines of the object.

In view of the above background, it is apparent that there is a need for a digital imaging storage and retrieval method that eliminates the problems associated with interpolation and compression of digital image data.

Further, in view of the above background, it is advantageous for the digital image storage and retrieval system to be combined with an active pixel sensor array.

Summary of the Invention

According to the present invention, a method for storing digital information from an image sensor array comprises the steps of providing an image sensor array for generating three color output data at each of a plurality of pixel locations; Providing a digital storage device coupled to the image sensor array; Sensing tricolor output data from the image sensor array; And storing the tricolor digital output data as digital data in the digital storage device without performing any interpolation on the tricolor output data. The storing step can be performed using a semiconductor memory device such as a RAM or the like.

Another method according to the invention uses the method for an image obtained from an image sensor array further comprising a triple junction structure in which each pixel in the array measures respective primary colors at the same location.

Furthermore, the method according to the invention optionally comprises the step of performing a lossless compression operation on the three color digital output data prior to the step of storing the three color digital output data in the digital storage device.

The present invention relates to the capture, storage, and retrieval of digital images. More specifically, the present invention relates to a new method for storing and retrieving pixel data from a full-color RGB imaging array embedded in a device such as a digital camera.

The invention also relates to an active pixel sensor and an active pixel sensor array. More specifically, the present invention relates to an array of active pixel sensors in which each of the active pixel sensors has a triple junction structure to ensure that each pixel sensor in the array measures each of the three primary colors (R-G-B) at the same location.

The invention also relates to a device such as a digital camera employing a triple junction active pixel sensor array and new methods of capturing, storing and retrieving data provided by the array.

1 illustrates a well known Bayer color filter array (CFA) pattern.

FIG. 2 shows the Nyquist domains for red, green and blue derived from the Bayer CFA of FIG. 1.

3 is a partial cross-sectional view showing a conventional twin well CMOS structure.

4 is a partial cross-sectional view showing a conventional triple junction CMOS structure.

5 is a block diagram of an imager suitable for use in an embodiment of an active pixel sensor in accordance with the present invention.

6 is a schematic diagram of an existing active pixel sensor circuit having a single storage node as an N-channel MOS.

FIG. 7 is a timing diagram illustrating the operation of the active pixel sensor shown in FIG. 6.

8 is a graph showing the light absorption length versus wavelength in silicon.

9 is a partial cross-sectional view showing a three color pixel sensor using a triple junction structure in accordance with the inventive concept.

10 is a graph illustrating a set of predicted sensitivity curves for the triple junction photodiode structure of FIG. 8 in accordance with the present invention.

11, 12, 13, 14 and 15 are schematic diagrams of active pixel sensors with multiple storage nodes according to the first to fifth embodiments of the present invention.

16A and 16B are optional timing diagrams for the operation of the active pixel sensor shown in FIG. 15 in accordance with the present invention.

FIG. 17 is a block diagram of a prior art image capture and display system showing interpolation steps, lossy compression steps, and data storage, data retrieval, and decompression steps performed on three color digital output data from an imaging array.

18A and 18B are block diagrams of alternative embodiments of an uncompressed image capture and display system and method in accordance with the present invention.

19A and 19B are block diagrams of alternative embodiments of image capture and display systems and methods using compression in accordance with the present invention.

It will be understood by those skilled in the art that the following description of the invention is illustrative only and is not intended to be limiting. Other embodiments of the invention will be readily apparent to those skilled in the art having the benefit of this disclosure.

Full RGB imagers suitable for use in the present invention are described in co-pending application number 09 / 290,361, filed December 4, 1999. The application describes an active pixel imaging array comprising a triple junction structure. The advantage of the triple junction structure is that each pixel in the array measures each primary at the same location, minimizing or eliminating the need for interpolation.

Another advantage of a full RGB imager is that all of the red, green, and blue image information captured for a single pixel is contained within a smaller space than the pixel cluster of prior art imaging systems, enabling finer resolution. will be. In a typical system according to the present invention, a full RGB imager consists of an array of, for example, a 640 x 480 three-layer RGB full color pixel sensor, providing a total M = 921,600 individual bytes of pixel data in the image dataset. An illustrative, non-limiting example of a denser imager that can be used in accordance with this aspect of the present invention is a 3000 pixel sensor x 2000 pixel sensor (red, green, blue) for the total M = 18,000,000 bytes of pixel data in the image dataset. Is an imager that can be configured as an array of x3). An alternative embodiment of a full RGB imager is an assembly of three simple sensor arrays in a tricolor color separation prism with three arrays optically aligned with each other as is known in the video camera art.

The full RGB imager used in the present invention employs a triple junction pixel cell structure to exploit the difference in absorption length in silicon of light with different wavelengths to measure different colors at the same location with sensitive areas as large as spacing. It relates to color separation of an active pixel MOS imaging array.

In the present invention, a color photosensor structure for separating blue, green, and red light is formed in the P-type silicon body. The color photosensor structure includes a vertical PNPN device that implements a triple stack photodiode and includes a first N-doped region formed in a P-type silicon body, a P-doped region formed in a first N-doped region, and a P-doped region. A second N-doped region formed therein. According to the present invention a triple well process is used to fabricate the color light sensor structure. A typical N well of a triple well CMOS process is not used in the color light sensor structure of the present invention, although it would be useful to use it on the same chip outside of an array of imager cells.

In a color photosensor structure, the pn junction formed between the P-type silicon body and the first N-doped region defines a red-sensitive photodiode at a predetermined depth in silicon that is approximately equal to the absorption length of red light in the silicon, and the first The pn junction formed between the N-doped region and the P-doped region defines a green-sensitive photodiode at a predetermined depth in silicon that is approximately equal to the absorption length of the green light in the silicon, and the P-doped region and the second N-doped region The pn junction formed between defines a blue-sensitive photodiode at a predetermined depth in silicon that is approximately equal to the absorption length of blue light. The sense circuitry is coupled to the red, green, and blue photodiodes to integrate and store respective photodiode currents.

The full RGB imager used in the present invention reduces color aliasing artifacts by ensuring that all pixels within the imaging error can measure the red, green, and blue color response at the same location in the pixel structure. Color filtration occurs using the difference in absorption length in silicon of red, green and blue light.

The full color RGB imager used in the present invention provides other advantages besides the reduction of color aliasing. For example, it eliminates the complex polymer color filter array process steps that are common in the prior art. Instead, triple bonding processes are commonly used in the semiconductor industry. In addition, the overall use efficiency of the available photons is increased. In the traditional approach, photons that do not pass through the filter material are absorbed and consumed by the filter. In the approach of the present invention, photons are separated by absorption depth, but all are collected and used. Thus, the overall improvement of the quantum is made efficiently by a factor of about 3.

The full RGB imager used in the present invention provides an excellent example of an imager that is difficult to implement with conventional CCD technology. In addition, the present invention has the advantage of the availability of a scaled CMOS process in that there are many support transistors in each of the three color pixels.

Semiconductor devices for measuring the color of light are known in the non-imaging art. These devices are manufactured with a variety of techniques that rely on changes in photon absorption depth with wavelength. Embodiments are disclosed in US Pat. No. 4,011,016 entitled "Semiconductor Radiation Wavelength Detector" and US Pat. No. 4,309,604, entitled "Apparatus for Sensing the Wavelength and Intensity of Light." However, the structure for a tricolor integrated circuit color sensor or imaging array is not disclosed in any patent.

In the field of imaging, CCD devices having a plurality of buried channels for accumulating and moving photocharges are known. These devices are difficult and expensive to manufacture and are not practical for tricolor applications. US Patent No. 4,613,895 entitled "Color Responsive Imaging Device Employing Wavelength Dependent Semiconductor Optical Absorption" discloses one example of such a device. This category also includes devices that use layers of thin film photosensitive material applied on top of the imager integrated circuit. Examples of such techniques are disclosed in US Pat. No. 4,677,289 entitled "Color Sensor" and US Pat. No. 4,651,001 entitled "Visible / Infrared Imaging Device with Stacked Cell Structure." Such structures are also difficult to manufacture, expensive and impractical.

Color imaging integrated circuits using color filter mosaics to select different wavelength bands at different light sensor locations are known in the imaging art. US Patent No. 3,971,065 entitled "Color Imaging Array" discloses this technique. As reviewed by Parulski et al. "Enabling Technologies for Family of Digital Cameras", 1996, 156 / SPIE Vol. 2654, one pixel mosaic pattern commonly used in digital cameras is a Bayer color filter array (CFA) pattern.

As shown in FIG. 1, the Bayer CFA has 50% green pixels arranged on a checkerboard. Alternate lines of red and blue pixels are used to fill the rest of the pattern. As shown in FIG. 2, the Bayer CFA pattern generates a Nyquist region in a diamond shape for green and a Nyquist region in a rectangular shape for red and blue. The human eye is more sensitive to high spatial frequencies of luminance rather than chrominance, and the luminance consists mainly of green light. Thus, because Bayer CFA provides the same Nyquist frequency for horizontal and vertical spatial frequencies as a monochrome imager, the perceived sharpness of the digital image is improved.

This mosaic approach and the serious color aliasing problem caused by the fact that the sensors are small when compared to their spacing locally sample the image signal, and that the sensors for different colors are in different locations, cause the samples to not be aligned between colors. It is known in the art that it is related. Image frequency components outside the Nyquist region are aliased to the sampled image with small attenuation and small correlations between colors.

As pointed out in the discussion of the CCD color imaging array, semiconductor processes used to fabricate the array are difficult and expensive to implement. However, it is known that CMOS technology is less expensive and easier to implement.

Referring to FIG. 3, many modern CMOS integrated circuit fabrication processes have shown that N well region 12 and P well region 10 having a doping density of approximately 10 ¹⁷ atoms / cm 3 are P-channel and N-channel transistors, respectively. It uses a "twin-well" or "twin-tub" structure, which is used as the region for making the cavities. Substrate material 14 is generally lower doped P-type silicon (10 ¹⁵ atoms / cm 3) and P well 10 is not separated from substrate 14. N- channel FET formed on the P well 10, 16 having an N- 10 ¹⁸ atoms / ㎤ than the concentration of a typical N + source / drain diffusion 18, and approximately 10 ¹⁸ atoms / ㎤ degree having a greater impurity concentration A shallow shallow doped diffusion (20) region of the type. The P-channel FET 22 formed in the N well region 20 is similarly constructed using a typical P + source / drain region 24 and a shallow LDD region 26 having similar impurity concentrations.

In FIG. 4, in an improved process known as "triple well", an additional deep N isolation well 28 is used to bond the P well 10 away from the P-type substrate 14. The impurity concentration of the N separation well 28 (10 ¹⁶ atoms / cm 3) is determined between the impurity concentration of the P-type substrate 14 and the P wells (10 ¹⁵ atoms / cm 3 and 10 ¹⁷ atoms / cm 3, respectively). Examples of triple well techniques are described in US Pat. No. 5,397,734, "Method of Fabricating a Semiconductor Device Having a Triple Well Structure."

Triple well processing is popular and economical for manufacturing MOS memory (DRAM) devices because it provides effective separation of dynamic charge storage nodes from stray minority carriers that can diffuse through the substrate.

Storage pixel sensors are known in the art. In storage pixels, data indicative of the intensity of light received by a phototransducer is stored in a storage element that is readable and can be erased with an appropriate control circuit.

5 is a block diagram of an active pixel imager 30 suitable for use in accordance with the present invention. In the imager 30, the active pixel sensors are arranged in a matrix form in the pixel sensor array 32. In order to extract analog pixel information from the pixel sensor array 32 and process it with an analog-to-digital converter (ADC) 34, a row decoder 36, a column sampling circuit 38, and a counter 40 are provided. The row decoder 34 selects a row in the pixel sensor array 32 in response to the row enable signal 42 and the signal from the counter 40. The column sampling circuit 38 further comprises a multiplexer, driven by the counter 40, for coupling the sampled columns to the ADC, preferably in response to a signal from the counter 40.

In a typical implementation, the higher-order bits from the counter 40 are used to drive the row decoder circuit 36 and lower-order before the next row is selected by the row decoder circuit 36. Bit is used to drive the column sampling circuit 38 to extract all pixel information from the rows of the pixel sensor array 32. Row decoders, column sampling circuits with built-in multiplexers, and counters, which are mainly used in imager 30, are well known to those of ordinary skill in the art and are complicated to explain, thus making the present invention unclear. No detailed explanation is given to avoid.

In FIG. 6, a schematic diagram of a known active pixel sensor 50 with a single embedded storage element is disclosed. The active pixel sensor 50 is realized with an N channel MOS transistor. In addition, those skilled in the art will recognize that the active pixel sensor 50 can be realized as a P channel MOS transistor or a combination of P and N channel MOS transistors. In the active pixel sensor 50, the anode of the photodiode 52 is connected to ground and the cathode is connected to the source of the N-channel MOS reset transistor 54. The drain of the N-channel MOS reset transistor 54 is connected to Vref, and the gate of the N-channel MOS reset transistor 54 is connected to the global RESET line indicated by reference numeral 44 in FIG. The reset line RESET is preferably driven at a threshold voltage of at least Vref or higher to set the cathode of the photodiode 52 to Vref.

The cathode of photodiode 52 is also connected to the first source / drain of N-channel MOS transfer transistor 56. The second source / drain of the N-channel MOS transfer transistor 56 is connected to the first terminal of the storage element 58 and to the gate of the N-channel MOS read transistor 60. The second terminal of the storage element 58 is connected to a reference potential shown as ground. The gate of the N-channel MOS transfer transistor 56 is connected to the global XFR line, indicated at 46 in FIG. The second source / drain of the N-channel MOS transfer transistor 56 is connected to the first terminal of the storage element 58 and the gate of the N-channel MOS transistor 60 to form the storage node 62. The drain of the N-channel MOS read transistor 60 is connected to Vcc, and the source of the N-channel MOS read transistor 60 is connected to the drain of the N-channel MOS row select transistor 64. The gate of the N-channel MOS row select transistor 64 is connected to the ROW SELECT line, one of which is indicated by reference numeral 48 in FIG. 5, and the source of the N-channel MOS row select transistor 64 is a column output line. Is connected to.

With regard to storage node 62, N-channel MOS transfer transistor 56 isolates storage node 62 so that photocharge no longer collects by the cathode of photodiode 52 at the end of the subsequent integration period, It should be appreciated that the N channel MOS read transistor 60 detects the charge that accumulates in the storage node 62 and the storage element 58 stores the charge. Also in US Patent Application No. 09 / 099,116, "ACTIVE PIXEL SENSOR WITH BOOTSTRAP AMPLIFICATION," filed June 17, 1998, filed on June 17, 1998, assigned to the assignee of the present invention by RB Merrill and Richard F. Lyon. As disclosed and also explicitly incorporated herein by reference, a storage element 58 may be omitted and a charge stored in the gate of the N-channel MOS read transistor 60 or in other charge charging means for storing the charge may be used.

In order to understand the operation of the active pixel sensor 50 in more detail, in the timing diagram of FIG. 7, the timing of the RESET signal, the XFR signal, and the ROW SELECT signal shown in FIG. The active pixel 50 is reset by turning on the N-channel MOS reset transistor 54 and the N-channel MOS transfer transistor 56 as indicated by the high levels of the RESET signal and the XFR signal at 66 and 68. Thereafter, the N-channel MOS reset transistor 54 is turned off at the falling edge 70 of the RESET 66, so that the photocurrent from the photodiode 52 begins to integrate. The integration period of the photocurrent is indicated by reference numeral 72.

While the N-channel MOS transfer transistor 56 is turned on, the capacitance of the storage element 58 is added to the capacitance of the photodiode 52 during the integration period, so that the charge capacitance and the range of the active pixel sensor 50 are Increases. This also reduces the variation in pixel output due to capacitance instability because the gate oxide capacitance on which the storage element 58 is formed is better controlled than the junction capacitance of the photodiode 52.

Once the integration is complete (as determined by external exposure control), the N-channel MOS transfer transistor 56 is turned off at the falling edge 74 of the XFR to isolate the voltage level corresponding to the integrated photocharge in the storage element 58. . Immediately thereafter, the photodiode 52 itself is preferably reset to the reference voltage by turning on the N-channel MOS reset transistor 54 again, as indicated by the rising edge 76 of RESET. This operation prevents the photodiode 52 from continuing to integrate during the readout process and excess charge overflow into the body, affecting the integration of the signal to the storage element 58.

After the N-channel MOS transfer transistor 56 is turned off, the readout process can begin. Each active pixel sensor in one row is read when the ROW SELECT signal pulse shown in FIG. 7 is applied to the gate of the N-channel MOS row select transistor 64 of the active pixel sensor 60. In operation of the active pixel sensor 50, the voltage associated with the voltage appearing at the storage node 62 is sensed by the N-channel MOS read transistor 50 and the column output when the N-channel row select transistor 64 is turned on. It takes a line. The XFR signal remains low until all rows have been read or another cycle has begun.

8 shows the light absorption length in silicon for light in the visible spectrum. It is well known that the longer the wavelength of light incident on the silicon body, the more the light penetrates into the silicon body before being absorbed. As shown, blue light with a wavelength in the range of about 400-490 nm will preferentially be absorbed by the silicon body to a depth of about 0.2-0.5 micron, and green light with a wavelength in the range of about 490-575 nm is about 0.5-1.5. It will be absorbed to a depth of micron and red light with a wavelength in the range of about 575-700 nm will be absorbed to a depth of about 1.5-3.0 microns.

In FIG. 9, a triple junction color photosensor structure 78 is shown formed in a silicon body 80 of P-type conductivity (about 10 ¹⁵ atoms / cm 2) in accordance with the present invention. The color photosensor structure 78 has a first N-type doped well region 82 (about 10 ¹⁶ atoms / cm 3) formed in the P-type silicon body 80 and a P-type formed in the first N doped region 82. N-type conductivity (about 10 ¹⁸ atoms / cm 3) formed as a doped well region 84 of conductivity (about 10 ¹⁷ atoms / cm 3) and a fairly shallow NLDD (N-type low doped drain) layer in P doped region 84. Second doped region 86).

There are three pn junctions in the color light sensor structure 78. The first pn junction is between the P-type silicon body 80 and the first N doped region 82 at a depth of about 1.5 to 3.0 microns. The first pn junction is preferably formed at an approximate absorption depth for red light of about 2 microns. The second pn junction is between the P doped region 84 and the first N doped region 82 at a depth of about 0.5 to 1.5 microns. The second pn junction is preferably formed at an approximate absorption depth for green light of about 0.6 microns. The third pn junction is between the P doped region 84 and the second N doped region 86 at a depth of about 0.2 to 0.5 microns. The third pn junction is preferably formed at an approximate absorption depth for blue light of 0.2 micron. Thus, in the color photosensor structure 78, the first pn junction forms a red-photosensitive photodiode, the second pn junction forms a green-photosensitive photodiode, and the third pn junction forms a blue-photosensitive photodiode Form.

Those skilled in the art will appreciate that the photosensitive depletion layer of the diodes described above is somewhat exceeding or lacking at such junction depths. Also, those skilled in the art will appreciate that the triple junction structure described above may be used in the opposite conductive region shown in the example of FIG. 9, namely in the first P doped region, the first P region in the N-type silicon substrate. It will be appreciated that by using the N doped region and the second P doped region in the N region. However, such a structure is generally not used in industry, and is also preferable because the structure of FIG. 9 uses standard triple junction MOS memory technology. In addition, those skilled in the art can form additional pn junctions at selected depths of the color photosensor structure 78 by forming additional doped regions to provide absorption of photons at additionally selected wavelengths. You will know well.

9 also shows that the color light sensor structure of the present invention includes a sensing mechanism 88 connected to the red, green, and blue photodiodes to respectively measure red, green, and blue photocurrents through the photodiodes. . The photocurrent sensor 88 is shown in a conceptual arrangement comprising a first ammeter 90 connected across the red photosensitive photodiode and measuring the red photocurrent ir. A second ammeter 92 is connected across the green photosensitive photodiode to measure the green photocurrent ig. A third ammeter 94 is connected across the blue photosensitive photodiode to measure the blue photocurrent ib. Assuming that most of the current in the photodiode is concentrated in the depletion layer, those of ordinary skill in the art will appreciate that current ib is primarily the photocurrent from the incident photons at the blue end of the visible spectrum, and current ig is mainly green. It will be appreciated that the current is from the photon and the current ir is mainly from the red photon.

10 shows a set of predicted photosensitivity curves of a triple stacked photodiode array in accordance with the present invention as a function of wavelength in the visible spectrum. The curve is only roughly adjusted based on the color filter, not as detailed as in other color separation schemes. However, as is known in the field of color imaging, it is possible to properly matrix the three measurements from such a set of curves into a more closely colored set of red, green, and blue intensity values. . Methods for properly predicting matrix transformations are known and are described, for example, in US Pat. No. 5,668,596, "Digital Imaging Device Optimized for Color Performance."

According to the present invention, imager 30 as shown in FIG. 5 has a number of storage nodes associated with each pixel of pixel array 32. To capture color images in imager 30, each pixel employs a triple photodiode color sensor structure 78 described with respect to FIG. In each embodiment of the invention, in the storage pixel sensors 100-1 to 100-5 shown in Figs. 11 to 15, each of the three diodes in the triple diode color photosensor structure 78 is at least one separated. It has a terminal connected to the storage and reading circuit. Embodiments of the storage pixel sensors 100-1 to 100-5 shown in Figs. 11 to 15 are realized with N-channel MOS transistors. In addition, those skilled in the art will appreciate that the storage pixel sensor described below can be realized as a P channel MOS transistor or a combination of N and P channel MOS transistors. Corresponding elements shown in Figs. 11 to 15 are denoted by the same reference numerals.

In the operation of the active pixel sensors 100-1 to 100-4 of FIG. 11, the active pixel sensor is reset and charge is accumulated in a manner similar to that described above with respect to the pixel sensor of FIG. 6. For the operation of the active pixel sensor 100-5, another timing diagram is shown in FIGS. 16A and 16B.

In each embodiment of the active pixel sensors 100-1 to 100-5, the first N doped region 82 is connected to the source of the N channel MOS reset transistor 102-1, and the P doped region 84 is It is connected to the drain of the N-channel MOS reset transistor 102-2, and the second N doped region 86 is connected to the source of the N-channel MOS reset transistor 102-3. The drains of the N-channel MOS reset transistors 102-1 and 102-3 are connected to the reference voltage Vn, and the source of the N-channel MOS reset transistor 102-2 is connected to the reference voltage Vp <Vn. Gates of the N-channel MOS reset transistors 102-1 and 102-3 are connected to the RESET-N control line 104, and gates of the N-channel MOS reset transistors 102-2 are connected to the RESET-P control line 106. Is connected to.

The potential Vn connected to the drains of the N-channel MOS reset transistors 102-1 and 102-3 is generally a positive potential with respect to the P-type silicon substrate, and is a potential connected to the drain of the N-channel MOS reset transistor 102-2. Since Vp is a positive potential lower than the potential Vn, all three photodiodes start operating in a reverse biased state when the RESET-N and RESET-P signals are applied. The photodiodes in the triple diode color photosensor structure 78 are lower biased by exposure to light, and may be somewhat forward biased before they "overflow." The three detected voltages will correspond to different linear combinations of photocharges, depending on the value and stray capacitance of the various photodiodes in the circuit. These linear combinations affect the resulting photosensitivity curve with respect to the voltage output, which is modified during matrix transformation resulting in a color-detectable final output.

In addition, each of the active pixel sensors 100-1 through 100-5 includes a plurality of storage nodes 108-1, 108-2, 108-3. For example, storage node 108-1 may include a common connection terminal of a first terminal of storage element 110-1, a first source / drain of N-channel MOS transfer transistor 112-1, and an N-channel MOS read. It consists of the gate of the transistor 114-1. Storage node 108-2 is a common connection terminal of a first terminal of storage element 110-2, a first source / drain of N-channel MOS transfer transistor 112-2, and an N-channel MOS read transistor 114-. It consists of a gate of 2). The storage node 108-3 includes a common connection terminal of the first terminal of the storage element 110-3, a first source / drain of the N-channel MOS transfer transistor 112-3, and an N-channel MOS read transistor 114-. It consists of a gate of 3). Gates of the N-channel MOS transfer transistors 112-1, 112-2, and 112-3 are connected to the global transmission signal of the XFR line 116. The storage elements 110-1, 110-2, 110-3 each have a second terminal connected to a fixed potential which is shown as ground.

In accordance with an embodiment 100-1 of an active pixel sensor in accordance with the present invention as shown in FIG. 11, each voltage appearing in storage nodes 108-1 through 108-3 is a separate column output line 118-1. 118-3), it is read by a single row select signal on the ROW SELECT line 120. Accordingly, the drains of the respective N-channel MOS read transistors 114-1 to 114-3 are connected to Vcc, and the source of each of the N-channel MOS read transistors 114-1 to 114-3 is an N-channel MOS row select transistor. It is connected to the drain of one of (122-1 to 122-3), respectively. The gates of the N-channel MOS row select transistors 122-1 through 122-3 are connected to the ROW SELECT lines 120, respectively, and the source of the N-channel MOS row select transistors 122-1 through 122-3 are column output lines. And 118-1 to 118-3, respectively.

In the operation of the active pixel sensor 100-1, the column circuit connected to the column output lines 118-1 to 118-3 while reading an image on the column output lines 118-1 to 118-3. Each (not shown) will be used to select pixels representing stored images provided to storage nodes 108-1 through 108-3. In addition, the column circuitry can be used to perform certain functions, such as performing a linear combination of sensed color signals on the stored pixels.

12, in the embodiment of the active pixel sensor 100-2 according to the present invention, the voltages on the storage nodes 108-1 through 108-3 assert each of the ROW SELECT3 signals in the ROW SELECT1 signal, respectively. Thereby being read separately into the same column output line 118. Thus, the drain of each N-channel MOS read transistor 114-1 to 114-3 is connected to V _CC, and the source of each N-channel MOS read transistor 114-1 to 114-3 is N-. Are connected to the drains of the channel MOS row select transistors 122-1 through 122-3, respectively. Gates of the N-channel MOS row select transistors 122-1 to 122-3 are respectively connected to ROW SELECT1 line to ROW SELECT3 line 120-1 to 120-3, respectively, and the N-channel MOS row select transistors ( Sources 122-1 through 122-3 are connected to a single column output line 118.

In operation of the active pixel sensor 100-2 of FIG. 12, the image stored in the storage node 108-1 will be read in response to the HIGH ROW SELECT1 signal, and the image stored in the storage node 108-2 is HIGH. The image stored in storage node 108-3 will be read in response to the ROW SELECT2 signal and will be read in response to the HIGH ROW SELECT3 signal. The imager 30 shown in FIG. 5 further includes additional decoding circuitry for providing signals to the ROW SELECT1 line to the ROW SELECT3 line.

Referring to FIG. 13, in the embodiment of the active pixel sensor 100-3, the voltage on the storage nodes 108-1 through 108-3 is the N-channel MOS image selection transistor 124-1 through 124-3. Are individually read into a single column output line 118 in response to signals on IMAGE SELECT1 to IMAGE SELECT3 lines 126-1 to 126-3 that are applied to each of the gates of &lt; RTI ID = 0.0 &gt; and &lt; / RTI &gt; . Thus, the drains of the N-channel MOS read transistors 114-1 through 114-3 are each connected to V _CC , and the source of the N-channel MOS read transistors 114-1 through 114-3 is an N-channel MOS image. Are connected to the drains of the select transistors 124-1 to 124-3, respectively. The gates of the N-channel MOS image select transistors 124-1 through 124-3 are connected to IMAGE SELECT1 lines through IMAGE SELECT3 lines 126-1 through 126-3, respectively. The sources of the N-channel MOS image select transistors 124-1 through 124-3 are all connected to the drain of the N-channel MOS row select transistor 128. The gate of the N-channel MOS row select transistor 128 is connected to the ROW SELECT line 120 and the source of the N-channel MOS row select transistor 128 is connected to the column output line 118.

In operation of the active pixel sensor 100-3, the image stored in the storage node 108-1 is applied to the high signal asserted to the ROW SELECT line 120 and to the high signal asserted to the IMAGE SELECT1 line 126-1. Will be read in response. The image stored at storage node 108-2 will be read in response to the high signal asserted in ROW SELECT line 120 and the high signal asserted in IMAGE SELECT 2 line 126-2. The image stored at storage node 108-3 will be read in response to the high signal asserted to ROW SELECT line 120 and the high signal asserted to IMAGE SELECT 3 line 126-3. The imager 30 shown in FIG. 5 may further include global IMAGE SELECT1 lines to IMAGE SELECT3 lines. The use of global IMAGE SELECT1 through IMAGE SELECT3 lines in combination with the ROW SELECT signal eliminates the need for additional row decoding required in the embodiment of FIG.

Referring to FIG. 14, in the embodiment of the active pixel sensor 100-4, the voltage on the storage nodes 108-1 to 108-3 is the IMAGE SELECT1 line to the IMAGE SELECT3 line 126-1 to 126-3. A single column output line 118 is read in current mode in response to each signal asserted at and the signal asserted at ROW SELECT line 120. Thus, the drains of the N-channel MOS read transistors 114-1 through 114-3 are connected together and connected to the source of the N-channel MOS row select transistor 128. The sources of the N-channel MOS read transistors 114-1 through 114-3 are connected to IMAGE SELECT1 lines through IMAGE SELECT3 lines 126-1 through 126-3, respectively. The gate of the N-channel MOS row select transistor 128 is connected to the ROW SELECT line 120, and the drain of the N-channel MOS row select transistor 128 is connected to the column output line 118.

In operation of the active pixel sensor 100-4, the column output line 118 is connected to the drain of the N-channel MOS row select transistor 128. In order to locate the current representing the image stored in column output line 118, the image stored in storage node 108-1 will be selected by the low signal asserted in IMAGE SELECT1 line 126-1, and the storage node ( The image stored in 108-2 will be selected by the low signal asserted in IMAGE SELECT1 line 126-2, and the image stored in storage node 108-3 is asserted in IMAGE SELECT1 line 126-3. Will be selected by the signal. Thus, the current-mode output on column output line 118 is controlled by signals on IMAGE SELECT3 (126-1 to 126-3) in IMAGE SELECT1. The column output line 118 must be biased to a voltage high enough that the unselected N-channel MOS read transistors 114-1 through 114-3 do not start conducting reversely. Also, the voltage driver of IMAGE SELECT3 (126-1 to 126-3) in IMAGE SELECT1 can lower all column currents from the selected row.

Referring to FIG. 15, the embodiment of the active pixel sensor 100-5 is similar to the embodiment of FIG. 11, with the addition that multiple storage nodes appear to be matrixed using ROW SELECT2 at ROW SELECT1 and COLUMN OUTPUT3 at COLUMN OUTPUT3. It includes a storage node. Most importantly, the embodiment of FIG. 15 functions in the same manner as the embodiment of FIG.

In the active pixel sensor 100-5 shown in FIG. 15, the voltage on the storage nodes 108-1 through 108-3 is controlled by the ROW SELECT1 line 120-1 through the column output lines 118-1 through 118-3. ), And the voltages on storage nodes 108-4 to 108-6 are read from column output lines 118-1 to 118-3 by ROW SELECT2 line 120-2, respectively. Thus, the drain of the N-channel MOS read transistors 114-1 through 114-6 is connected to V _CC and the source of the N-channel MOS read transistors 114-1 through 114-6 selects an N-channel MOS low. Connected to transistors 122-1 to 122-6, respectively. Gates of the N-channel MOS row select transistors 122-1 through 122-3 are connected to ROW SELECT1 line 120-1, respectively, and gates of the N-channel MOS row select transistors 122-4 through 122-6 Are respectively connected to ROW SELECT2 line 120-2. The source of the N-channel MOS row select transistors 122-1 to 122-4 is connected to the first column output line 118-1 and the source of the N-channel MOS row select transistors 122-2 to 122-5. The source is connected to the second column output line 118-2, and the source of the N-channel MOS row select transistors 122-3 to 122-6 is connected to the third column output line 118-3.

In operation of the active pixel sensor 100-5, the charge stored in any of the storage nodes 108-1 through 108-6 is detected by sensing column output lines 118-1 through 118-3 connected to the storage node. To the gate of an N-channel MOS row select transistor 122-1 to 122-3 or 122-4 or 122-6 connected to storage node 108-1 or 108-3 to 108-4 respectively. Read in response to the assertion of one of the applied ROW SELECT1 line and ROW SELECT2 line.

For example, to select pixel information stored at storage node 108-1, the signal on ROW SELECT1 line 120-1 will be asserted and first column output line 118-1 will be sensed. In an embodiment where multiple storage nodes are used, using ROW SELECT1 lines and ROW SELECT2 lines 120-1 and 120-2 and the first, second, and third column output lines 118-1 to 118-3 Matrixing storage nodes 108-1 through 108-6 reduces the number of additional row and column lines required. Instead of the single global XFR line shown in FIG. 1, first and second global transmission lines (XFR1 and XFR2; shown by reference numerals 116-1 and 116-2) will be used, which is motion sensing, multiple exposure time. And so on.

16A and 16B are timing diagrams showing the signals RESET-N, RESET-P, XFR1 and XFR2, and show the operation of the active pixel sensor 100-5. In FIG. 16A, the XFR1 signal high asserted on line 116-1 and the RESET-N and RESET-P signals (shown as a single RESET signal for brevity) are stored at the falling nodes 130 at storage nodes 108-1, 108. FIG. -2 and 108-3) are switched to start the accumulation of charge. The XFR1 signal is then switched at falling edge 132 to stop the accumulation of charge on storage nodes 108-1, 180-2 and 108-3. The RESET signal is switched at the rising edge 134 to reset the voltage of the photodiode in the three diode color photosensor structure 78. And, the XFR2 signal on line 116-2 is switched at the rising edge 136. When the RESET signal transitions on the falling edge 138, the accumulation of charge on storage nodes 108-4, 118-5, and 108-6 begins. The XFR2 signal on line 116-2 switches at falling edge 140 to stop the accumulation of charge on storage nodes 108-4, 118-5, and 108-6.

In FIG. 16B, both XFR1 and XFR2 signals are asserted high, and the RESET signal accumulates charges on storage nodes 108-1, 108-2, 108-3, 108-4, 108-5 and 108-6. Transition at falling edge 150 to initiate The XFR1 signal is then switched at falling edge 152 to stop accumulation of charge on storage nodes 108-1, 180-2 and 108-3. Accumulation of charge on storage nodes 108-4, 118-5, and 108-6 continues. The XFR2 signal is then diverted at the falling edge 154 to stop the accumulation of charge on the storage nodes 108-4, 118-5 and 108-6.

Now that the advantages of a full RGB imager have been fully described, the reader is shown in FIG. 17 and the storage and retrieval method is described below.

17, a block diagram of a general prior art image capture and display system is shown. The image is first captured by a filter-mosaic imager 210 having an M pixel sensor. Prior art color image sensors typically detect only one of the three primary colors at each pixel location through a mosaic of color filters integrated in the image sensor chip, which is a full-color sensor of the present invention that detects each of the three colors at each pixel location. Is distinguished from. In a typical system, the prior art filter-mosaic imager may consist of, for example, an array of 640 pixel sensors x 480 pixel sensors that carry a dataset having a total M = 307,200 bytes of pixel data. A denser imager may consist of an array of 3,000 pixel sensors × 2,000 pixel sensors for the total M = 6,000,000 bytes of pixel data in the dataset.

The output dataset from the pixel sensor of imager 210 is processed by interpolator 212 to convert to a full RGB dataset as known in the art. Interpolation increases the size of the dataset to 3M. And, as known in the art, color conversion and correction are performed on the dataset by color corrector 214.

After interpolation and color correction are made to the output pixel dataset from imager 210, data compression, such as JPEG compression, is performed on the dataset in data compressor 216. JPEG compression is an industry standard and, as a general example, results in an adjustable compression of 0.25x, which reduces the dataset size to 0.75M as shown in FIG.

After the dataset is compressed, it is stored in the storage element 218. Storage element 218 includes many forms, such as magnetic storage (eg, floppy disk), or digital semiconductor memory storage, such as flash or random access memory, in the prior art.

When displaying or printing the stored digital image, the stored compressed data representing the color-corrected image is first retrieved from the reservoir 218 by the storage retrieval element 220. The characteristics of the storage retrieval element 220 depend on the characteristics of the storage element 218, the function of which may be understood by those skilled in the art.

After the stored dataset representing the image is retrieved from the storage element 218 by the storage retrieval element 220, it is decompressed by the decompression element 222 as in the prior art, and according to the user's request, a display or printer. 224 is provided.

The image data storage and retrieval method performed by the system of FIG. 17 can be easily estimated by the block diagram of FIG. The steps of the image data storage and retrieval method performed by the prior art image capture and display system are indicated by using the same reference numerals to identify the elements performing these steps. Thus, in step 210, the image is captured by the imager. Thereafter, at step 212, the dataset representing the image is interpolated to produce a full RBG dataset. The RBG dataset is preferably processed in step 214 to perform color conversion and correction on the dataset. In step 16 the dataset is compressed and in step 218 the dataset is stored.

When attempting to display or print the stored digital image, the stored compressed dataset representing the color-corrected RGB image is first retrieved from the stored one at step 220. The retrieved compressed dataset representing the image is then decompressed at step 222, as in the prior art. Finally, in step 224, the image data is provided to the display or printer 224, as required by a user using conventional techniques.

The interpolation step 212 and the compression step 216 performed by the prior art technique shown in FIG. 17 provide a comprehension for the resolution of the original image data in the dataset obtained from the imager 210. It is a "lossy" step. These steps are not possible to be reversed because the original dataset obtained from imager 210 is not recoverable. More importantly, the original dataset from imager 210 may not fully represent the image belonging to the sensor array. It is well known in the art that in order to achieve a complete representation of an image, the image must be sampled at least twice in size for each cycle of the highest spatial frequency present in the optical image. The highest spatial frequency is typically set by the lens's modulation transmission function, which is equivalent to one cycle per 10 micrometers. The size of a typical optical sensor in a dense imaging array is about 5 micrometers, which meets the sampling standard. However, with filter mosaics, repeating units used to sample an image consisting of four sensors are generally between 10 and 20 micrometers each in size. Such large sampling intervals confuse higher and lower spatial frequencies, which inevitably leads to loss of irreplaceable information, a problem known as aliasing. Alias artifacts caused by this process are shown in the digital image as moiré patterns, or colored highlights along edges and thin lines, on fine pitch fabrics. Alias artifacts are usually maintained, often accentuated by lossy compression techniques and by attempts to sharpen the image.

18A and 18B are block diagrams of alternative embodiments of an image capture and display system without compression in accordance with the present invention. Referring first to FIG. 18A, one embodiment of an image capture and display system 230 in accordance with the present invention is presented.

Image capture and display system 230 preferably includes a full RGB imager 232, i.e., an imager that senses all three primary colors at each pixel location to produce a full RGB image dataset. As will be apparent to those skilled in the art, the full RGB imager 232 is optimally located in an imaging device such as a digital camera.

The full RGB output dataset from the pixel sensor in imager 232 is processed by color corrector 234 to perform color conversion and correction. The color corrector 234 is configured as in the example of the prior art shown in Fig. 17, and its structure and operation are similar to those in the general prior art. Examples of color conversion and correction performed by color corrector 234 are dark signal cancellation, matrixing, bad pixel replacement, linearization, and gamma encoding. Color correction is optional and need not be performed in accordance with the present invention if not needed.

After color correction is performed on the RGB dataset from imager 232 of the present invention, the color corrected dataset is stored directly in storage element 236. Storage element 236 takes many forms, such as magnetic storage (eg, floppy disk), or digital semiconductor memory storage such as flash or random access memory. It is apparent to those skilled in the art that other storage techniques, such as optical storage, used in the systems and methods of the present invention, are not limited to the storage techniques specifically listed herein.

When it is desired to display or print a stored digital image in accordance with the systems and methods of the present invention, a dataset representing the stored color-corrected image is first retrieved from the storage element 236 by the storage retrieval element 238. Those skilled in the art will appreciate that the characteristics of the storage retrieval element 220 depends on the characteristics of the storage element 218 functioning with it. As a non-limiting example, if a semiconductor memory is employed in the present invention, conventional memory addressing and reading circuitry will perform the function of the storage retrieval element 238.

The dataset representing the stored color-corrected image may be interpolated by interpolation element 240 after being retrieved from storage element 236 by storage retrieval element 238. In accordance with the present invention, interpolation element 240 performs a process of interpolating from sensor resolution to high output resolution, for example, to prevent pixel artifacts on the print for data in the dataset prior to display or print. Can be done. Interpolation element 240 may include, for example, a microprocessor that operates interpolation software, as will be appreciated by those skilled in the art. Those skilled in the art will appreciate that no interpolation step is necessary to practice the invention.

Finally, the dataset interpolated from the interpolation element 240 may be provided to the display or printer 242 required by the user, such as when the photographer sends the image to the customer, or may be used for further use or further processing. The resolution may be stored or transmitted. Hardware and software techniques for providing image data to printers or displays are well known to those of ordinary skill in the art.

The image data storage and retrieval method of the present invention performed by the system of FIG. 18A is easily inferred from the block diagram in the figure. The steps of the image data storage and retrieval method performed by the image capture and display system of Fig. 18A are indicated by using the same reference numerals to identify the elements performing these steps. Thus, first, in step 232, the image is captured by the imager, and an image dataset is formed. Thereafter, in step 234, the image dataset is then processed to perform color conversion and / or correction, if desired. Thereafter, the dataset is stored at step 236.

If it is desired to display or print the stored digital image, the stored dataset representing the color-corrected image is retrieved from storage in step 238. If desired, the retrieved dataset representing the stored color-corrected image may then be interpolated at step 240. Finally, at step 242, the image dataset is then provided to a display or printer as required by the user and known to those skilled in the art.

As can be observed from the test of FIG. 18A. The amount of data M in the dataset remains constant throughout the storage and retrieval process until the interpolation step 240, where the size of the dataset is increased by the interpolation process. In the example given in FIG. 18A, an optional interpolation step increases the amount of data in the image dataset from M to 4M. Those skilled in the art will appreciate that the example shown in FIG. 18A is a non-limiting example, and that other interpolation procedures performed in accordance with the principles of the present invention may increase the amount of data by coefficients other than four. Will recognize.

Referring now to FIG. 18B, a variation on the image capture and display system and method of the present invention of FIG. 18A is shown. Since the elements and processing steps of the embodiment of FIG. 18B are presented in the embodiment of FIG. 18A, reference numbers such as those used in FIG. 18A may be employed to identify corresponding elements and steps of the embodiment of FIG. 18B. will be.

In a variation of the image capture and display system and method of the present invention shown in FIG. 18A, the full RGB dataset from imager 232 is stored in storage element 36 without any color conversion or correction being performed. . As can be seen from the test of FIG. 18B, color conversion and / or correction is performed after retrieval from storage in step 238 and prior to interpolation and display or printing. Otherwise, the image capture and display system shown in FIG. 18B may be the same as that shown in FIG. 18A.

The image capture and display method performed by the embodiment of the present invention shown in FIG. 18B begins with step 232, such as the method of FIG. 18A, in which image data is captured by an imager and formed into an image dataset. Thereafter, in step 236 the raw image dataset is stored.

In the case where it is desired to display or print a stored digital image, a dataset representing the stored image is retrieved from storage in step 238. Color correction and / or conversion is then performed on the data retrieved in step 234. If desired, then the dataset representing the color-corrected image may be interpolated at step 240. Finally, at step 242, the dataset is then provided to a display or printer as required by the user and known to those skilled in the art.

As can be observed from the test of FIG. 18B, the amount of data M in the image dataset remains constant throughout the storage and retrieval process until the interpolation step 240, where the amount of data in the dataset is subject to interpolation. Is increased by. In the example given in FIG. 2B, the optional interpolation step increases the amount of data in the image dataset by a factor of 4 from M to 4M. Those skilled in the art will appreciate that the example shown in FIG. 2B is a non-limiting example, and that the amount of data in the image dataset by other coefficients is 4 different interpolation procedures performed in accordance with the principles of the present invention. You will recognize that it will increase.

19A and 19B are block diagrams of alternative embodiments of an image capture and display system and method using compression in accordance with the present invention. According to the embodiments of FIGS. 19A and 19B, the image dataset can be compressed to reduce system storage requirements. Since certain elements and processing steps of the embodiment of FIGS. 19A and 3B are presented in the embodiments of FIGS. 19A and 19B, the same reference numerals as used in FIGS. 19A and 19B refer to the implementations of FIGS. 19A and 19B. Examples will be employed to identify corresponding elements and steps.

Referring now to FIG. 19A, one embodiment of a second image capture and display system 260 using compression in accordance with the present invention is shown.

Image capture and display system 260 includes a full RGB imager 232 as described with reference to the previously described embodiments. Thereafter, the full RGB output dataset from the pixel sensors in imager 232 is processed by color corrector 234 to perform conjugate conversion and correction on the image dataset. The color corrector 234 may be configured as the example of the prior art shown in FIG. 17 and the embodiments of the present invention shown in FIGS. 18A and 18B. Color correction according to this embodiment of the present invention is optional and need not be performed if deemed unnecessary.

After optional color correction is performed on the image dataset from imager 232 of the present invention, the color corrected image dataset is then subjected to a data-compression step at data compressor 262. The data compression step carried out in accordance with the invention in the data compressor 232 is lossless compression, ie the stored data can be decompressed later to produce the same as the data before compression, or a "little loss" compression step. to be. As will be appreciated by those of ordinary skill in the art, various means, such as a microprocessor that operates compression integrated circuits or compression software, may be used to perform this function.

Compared with conventional methods, the present invention disclosed herein provides a better combination of image quality and data storage requirements in a system where quality is a major concern. Conventional methods of sensing, interpolating, and compressing colors through filter mosaics achieve a combination of comparable image resolution and storage requirements, but there is the potential for aliasing in the sensor; Aliasing is a well known artifact that is detected through filter mosaics and cannot be completely corrected by subsequent processing.

Furthermore, by not interpolating before storage, the present invention allows image processing steps such as color correction (matrixing, bad pixel replacement and such steps) to be performed after retrieval of the image data and, therefore, is improved and Allow modified process steps to be used at retrieval time. Thus, image quality is not irreparably compromised in image capture and storage by processing and correction algorithms. In addition, since the full RGB image sensor delivers all three color measurements at each pixel location, the data can be stored in a standard RGB scanned image format file without data interpolation or other parallax operations; This feature of the present invention allows data to be stored and retrieved in a standard way so that subsequent processing can be done with standard color image processing tools.

In embodiments of the present invention employing compression, the same advantages can be retained while further reducing the size of the stored dataset, for example about half. As an example of using standard color image file formats, the tagged image file format (TIFF) standard allows for lossless Lempel-ziv-Welch (LZW) compression that is compatible with standard TIFF file retrieval tools. Since the decompressed dataset exactly matches the dataset before compression, there is no loss of quality needed to achieve this storage advantage.

The storage of data, such as image datasets, generally involves some sort of data precision compromise, such as the number of bits per color per pixel; The tradeoffs are usually observed as expression issues rather than compression issues. For example, an image sensor typically measures light intensity and displays the result using 10 to 14 bits in a linear representation of intensity; But before passing that data as an image, they often switch mostly to nonlinear or gamma-compressed representations, rounding to 8 bits per color per pixel. In this level of precision and in this non-linear representation, the resulting loss of quality is usually below a noticeable level. In the present invention, if the dataset is converted to a conventional 8 bit representation per color per pixel, and stored without compression or with lossless compression, the advantages of storing color corrected data from raw data or RGB imager are maintained. Can be.

In addition, the same advantage can be obtained by storing the image dataset using a "little lossless" compression technique, especially in cases where the dataset is not first converted to a representation with a few bits per pixel per color. For example, if the imager or color corrector delivers an image using 14 bits per pixel per color, as long as the retrieved and decompressed dataset is close enough to the original dataset to maintain sub-level errors Almost no lossless compression algorithm can be used directly on the dataset.

For the purposes of the present invention, if the error between the original image and the decompressed image is not greater than three times the error of the normal representation process of switching 8-bit gamma-compressed data, the image compression / decompression technique is " nearly " Lossless "; The errors indicate that for typical images, the quantization statistic gives an rms error of 1/3 of the 8-bit LSB process for normal quantization, allowing for almost lossless compression / decompression with 8-bit gamma-encoded output. Measured with root-mean-square sense, which allows the error to be equal to 1 LSB step.

Note that most lossy image compression techniques, including JPEGs with a "best" quality setting, cause large errors and therefore do not fall into the "almost lossless" class as defined herein. Threshold determination is roughly defined by the amount of noise added by small curves or some typical image processing steps, such as level adjustment, in a program like Adobe Phtoshop, because they are generally not considered very lossy operations. .

One particular type of lossless or almost lossless data compression used with practical embodiments manufactured in accordance with the principles of the present invention is a big problem of design choice.

After data compression, the compressed image dataset is stored in storage element 236. The storage element 236 may take many of the same forms as magnetic storage (eg, floppy disks) or digital semiconductor memory storage such as flash or random access memory, as in the already described embodiments of the present invention. have.

When it is desired to display or print a stored digital image in accordance with the systems and methods of the present invention, the stored data representing the color-corrected image is first retrieved from the storage element 236 by the storage retrieval element 238. In accordance with an already described embodiment of the present invention, those skilled in the art are well aware that the characteristics of the storage retrieval element 220 depend on the characteristics of the storage element 218 functioning with it. You will know.

Referring again to FIG. 19A, after the stored data is retrieved from the storage element 236 by the storage retrieval element 238, it is expanded or decompressed in the data-expansion element 264. Since the data-expansion element 264 and the data compression element 262 perform opposite or almost the same function, the characteristics of the data-expansion element 264 will depend on the characteristics of the data compression element 262. Data extension techniques are well known in the art.

After the retrieved image dataset is extended, it is interpolated by interpolation element 240. Interpolation element 240 may be the same as the embodiments previously described herein. In accordance with the embodiments already described herein, those skilled in the art will recognize that interpolation steps are not necessary to practice the invention.

Finally, the interpolated image dataset from interpolation element 240 employs well-known hardware and software techniques for providing image data to printers and displays, after which the display or printer ( 242.

The image data storage and retrieval method of the present invention performed by the system of FIG. 19A is easily deduced from the block diagram in the figure. The steps of the image data storage and retrieval method performed by the image capture and display system of Fig. 19A are indicated by using the same reference numerals to identify the elements performing these steps. Thus, first, in step 232, the image is captured by the imager and an image dataset is formed. Thereafter, in step 234, if it is desired to generate a color-corrected image dataset, the image dataset is processed to perform color changes and / or corrections. Thereafter, in step 262, lossless or “almost lossless” data compression is performed on the color-corrected image dataset prior to storage. Thereafter, in step 236, the compressed image dataset is stored.

If it is desired to display or print the stored digital image, the stored data representing the color-corrected image is retrieved from storage in step 238. If desired, the retrieved data representing the color-corrected image may then be interpolated at step 240. Finally, at step 242, the image dataset is provided to a display or printer as required by the user and known to those of ordinary skill in the art.

As can be observed from the test of FIG. 19A, the amount of stored data is smaller than that stored in the examples of FIGS. 18A and 18B. In the embodiments shown in Figures 18A and 18B, the amount of data M in the image dataset remains constant throughout the storage and retrieval process up to the interpolation step 240, where the amount of data can be increased by the interpolation process. have. In the example given in FIG. 3A, the data compression step reduces the amount of data in the image dataset to M / 2, although those of ordinary skill in the art will appreciate that other data compression steps that produce different compression ratios are present invention. It will be appreciated that it may be performed according to.

As in the embodiments of FIGS. 18A and 18B, an optional interpolation step performed after data expansion in the embodiment of FIG. 19A increases the amount of data in the image dataset from M to 4M. Those skilled in the art will appreciate that the example shown in FIG. 18A is a non-limiting example, and that other interpolation procedures performed in accordance with the principles of the present invention may be performed by means of coefficients other than four. You will recognize that you will increase the volume.

Referring now to FIG. 19B, changes to the image capture and display system and method of the present invention of FIG. 19A are presented. Since the elements and processing steps of the embodiment of FIG. 19B are presented in the embodiment of FIG. 19A, reference numerals such as those used in FIG. 19A may be employed to identify corresponding elements and steps of the embodiment of FIG. 19B. will be.

In a variation of the image capture and display system and method of the present invention shown in FIG. 19B, a compressed RGB dataset from imager 232 is stored in storage element 236 without any color conversion or correction being performed. . As can be seen from the test of FIG. 18B, color conversion and / or correction is performed after retrieval of the image dataset from storage in step 238 and data expansion in step 264, and prior to interpolation and display or printing. . Otherwise, the image capture and display system shown in FIG. 19B can be the same as that shown in FIG. 19A.

The image capture and display method performed by the embodiment of the present invention shown in FIG. 19B starts with step 232, such as the method of FIG. 19A, where image data is captured by an imager and an image dataset is formed. do. Thereafter, in step 262, data compression is performed on the image dataset from imager 232. Thereafter, the compressed image data is stored at step 236.

If it is desired to display or print a stored digital image, the stored dataset representing the image is retrieved from storage in step 238. Thereafter, the dataset is decompressed at step 264. Thereafter, color correction and / or conversion is performed on the retrieved image dataset in step 234. If desired, the color corrected image dataset may then be interpolated at step 240. Finally, at step 242, image data is provided to the display or printer as required by the user and as is known by those of ordinary skill in the art.

While embodiments and applications of the present invention have been shown and described, it will be apparent to those skilled in the art that many more modifications are possible than those described above without departing from the inventive concepts herein. Accordingly, the invention is not to be restricted except in the spirit of the appended claims.

Claims (28)

A method for storing a full RGB dataset, Providing an image sensor for generating a full RGB dataset as tricolor output data; Providing a digital storage device coupled to the image sensor; Sensing three-color digital output data representing the full RGB dataset from the image sensor; And Storing the three-color output data as the digital data in the digital storage device without performing any interpolation on the three-color output data. Full RGB dataset storage method comprising the. 2. The method of claim 1, wherein providing said digital storage device comprises providing a semiconductor memory device. 2. The method of claim 1, wherein providing the digital storage device comprises providing a magnetic storage device. 2. The method of claim 1, wherein providing said digital storage device comprises providing an optical storage device. 2. The method of claim 1, further comprising performing a lossless compression operation on the three color output data prior to the step of storing the three color digital output data as digital data in the digital storage device. How to save a full RGB dataset. 2. The method of claim 1, further comprising performing a lossless compression operation on the three color output data prior to the step of storing the three color digital output data as digital data in the digital storage device. How to save a full RGB dataset. In the method for processing digital information from an image sensor, Providing an image sensor generating a full RGB dataset as three color output data; Providing a digital storage device coupled to the image sensor; Sensing tricolor output data representing the full RGB dataset from the image sensor; Storing the three color output data as digital data in the digital storage device without performing any interpolation on the three color digital output data; And Retrieving the three color digital output data as digital data from the digital storage device; Digital information processing method comprising a. 8. The method of claim 7, wherein the step of providing the digital storage device comprises providing a semiconductor memory device. The method of claim 7, wherein Prior to the step of storing the three color output data as digital data in the digital storage device, performing a lossless compression operation on the three color output data; And Performing a lossless decompression operation on the three color output data after the step of retrieving the three color digital output data from the digital storage device as digital data. Digital information processing method further comprising. The method of claim 7, wherein Prior to the step of storing the tricolor output data as digital data in the digital storage device, performing a lossless compression operation on the tricolor output data; And After the step of retrieving the three color digital output data from the digital storage device as digital data, performing a nearly lossless decompression operation on the three color output data. Digital information processing method further comprising. A method for storing digital information from a single chip image sensor, Providing a single chip image sensor generating tricolor output data at each of the plurality of pixel positions; Providing a digital storage device coupled to the single chip image sensor; Sensing three-color digital output data from the single chip image sensor; And Storing the three-color output data as digital data in the digital storage device without performing any interpolation on the three-color output data. Digital information storage method comprising a. 12. The method of claim 11, wherein providing the digital storage device comprises providing a semiconductor memory device. 12. The method of claim 11, wherein providing the digital storage device comprises storing a magnetic storage device. 12. The method of claim 11, wherein providing the digital storage device comprises providing an optical storage device. 10. The method of claim 9, further comprising performing a lossless compression operation on the three color digital output data prior to the step of storing the three color digital output data as digital data in the digital storage device. Digital information storage method. 10. The method of claim 9, further comprising performing a lossless compression operation on the three color output data prior to the step of storing the three color digital output data as digital data in the digital storage device. Digital information storage method. In the method of processing digital data from a single chip image sensor, Providing a single chip image sensor generating tricolor output data at each of the plurality of pixel positions; Providing a digital storage device coupled to the single chip image sensor; Sensing three-color output data from the single chip image sensor; Storing the three color output data as digital data in the digital storage device without performing any interpolation on the three color digital output data; And Retrieving the three-color output data as digital data from the digital storage device Digital data processing method comprising a. 18. The method of claim 17, wherein providing the digital storage device comprises providing a semiconductor memory device. The method of claim 17, Prior to the step of storing the three color output data as digital data in the digital storage device, performing a lossless compression operation on the three color output data; And Performing a lossless decompression operation on the three color output data after the step of retrieving the three color digital output data from the digital storage device as digital data. Digital data processing method further comprising. The method of claim 17, Prior to the step of storing the tricolor output data as digital data in the digital storage device, performing a lossless compression operation on the tricolor output data; And After the step of retrieving the three color digital output data from the digital storage device as digital data, performing a nearly lossless decompression operation on the three color digital output data. Digital data processing method further comprising. A method of processing digital information from a triple-junction active pixel array, the method comprising: Providing a triple junction active pixel array that produces a full RGB image dataset; Providing a digital storage device coupled to the triple junction active pixel array; Sensing the full RGB dataset from the triple junction active pixel array; Storing the full RGB dataset as digital data in the digital storage device without performing any interpolation on the full RGB dataset; And Retrieving the full RGB dataset as digital data from the digital storage device Digital information processing method comprising a. 22. The method of claim 21, wherein providing the digital storage device comprises providing a semiconductor memory device. The method of claim 21, Prior to the step of storing the full RGB dataset as digital data in the digital storage device, performing a lossless compression operation on the full RGB dataset; And After the step of retrieving the full RGB dataset from the digital storage device as digital data, performing a lossless decompression operation on the full RGB dataset. Digital information processing method further comprising. The method of claim 21, Prior to the step of storing the full RGB dataset as digital data in the digital storage device, performing a lossless compression operation on the full RGB dataset; And After the step of retrieving the full RGB dataset from the digital storage device as digital data, performing a nearly lossless decompression operation on the full RGB dataset. Digital information processing method further comprising. A method of processing digital information from a digital camera image capture and display system, Providing a digital camera having a triple junction active pixel array; Providing a digital storage device coupled to the triple junction active pixel array; Generating a full RGB dataset from the triple junction active pixel array; Sensing the full RGB dataset from the triple junction active pixel array into the digital storage device; Storing the full RGB dataset as digital data in the digital storage device without performing any interpolation on the full RGB dataset; And Retrieving the full RGB dataset as digital data from the digital storage device Digital information processing method comprising a. 26. The method of claim 25, wherein providing the digital storage device comprises providing a semiconductor memory device. The method of claim 25, Prior to the step of storing the full RGB dataset as digital data in the digital storage device, performing a lossless compression operation on the full RGB dataset; And After the step of retrieving the full RGB dataset from the digital storage device as digital data, performing a lossless decompression operation on the full RGB dataset. Digital information processing method further comprising. The method of claim 25, Prior to the step of storing the full RGB dataset as digital data in the digital storage device, performing a lossless compression operation on the full RGB dataset; And After the step of retrieving the full RGB dataset from the digital storage device as digital data, performing a nearly lossless decompression operation on the full RGB dataset. Digital information processing method further comprising.