WO2007055494A1

WO2007055494A1 - Apparatus for detecting hot pixel and apparatus for detecting and correcting bad pixel regarding hotpixel

Info

Publication number: WO2007055494A1
Application number: PCT/KR2006/004542
Authority: WO
Inventors: Yo-Hwan Noh
Original assignee: MtekVision Co Ltd
Current assignee: MtekVision Co Ltd
Priority date: 2005-11-14
Filing date: 2006-11-02
Publication date: 2007-05-18
Anticipated expiration: 2008-05-14
Also published as: KR20070051052A; KR100794232B1

Abstract

An apparatus for detecting a hot pixel and an apparatus for detecting and correcting a bad pixel regarding a hot pixel. The apparatus for detecting a hot pixel has a pre-application unit for performing a gain operation or gamma correction on a pixel of an inputted image, a hot pixel detection unit for determining whether a pixel inputted from the pre-application unit is a hot pixel by comparing the pixel with a neighboring pixel, and a flag generation unit for generating a hot pixel flag in case the hot pixel detection unit detects a hot pixel. With the present invention, the spread of noise can be prevented in advance, significantly reducing the distortion of an image and improving the quality of a processed image.

Description

[DESCRIPTION]

[Invention Title]

APPARATUS FOR DETECTING HOT PIXEL AND APPARATUS FOR

DETECTING AND CORRECTING BAD PIXEL REGARDING HOTPIXEL

[Technical Field]

The present invention is directed to an apparatus for detecting and correcting a

bad pixel, more specifically to an apparatus for detecting a hot pixel and an apparatus

for detecting and correcting a bad pixel regarding a hot pixel to maintain a good picture

quality by reducing image distortion and image deterioration in an image sensor.

[Background Art]

An image sensor has a two-dimensional arrangement of pixels, each of which

converts the luminosity of light to a corresponding electrical signal. By measuring this

electrical signal, the amount of light entered into each pixel can be inferred, and using

this, an image made up of pixels can be constructed.

An image sensor comprises pixel arrays, which consists of several hundred

thousand to several million pixels, a device to convert analog data sensed from a pixel

to digital data, and several hundred to several thousand storage devices. Due to the large number of devices, the image sensor is always exposed to a possibility of having a

process error, and such an error can cause bad pixels, which become an important factor

for determining the level and price of the image sensor. The pixel data resulted from a

bad pixel is characterized by being exceedingly larger or smaller than the pixel data of

neighboring pixels.

In the conventional method of processing a bad pixel, a pixel is usually defined

as a bad pixel, and hence is corrected, when the difference compared with a base pixel

value exceeds a certain range. This method, however, uses too much hardware resource

and often treats a normal pixel as a bad pixel and corrects this normal pixel if the image

is crowded with tiny objects or has a high contrast, in which case neighboring pixels

have a big difference. The restored image, therefore, suffers with too much distortion,

compared to the real image.

To address the above issue, the Korean Patent Application Number

2005-0034091 (Apparatus for processing dead pixel) is disclosed.

The above application determines whether to detect or bypass a bad pixel by

comparing a relative difference with an absolute window, while the relaled method uses

the relative difference.

[Disclosure]

[Technical Problem] Therefore, the present invention provides an apparatus for detecting a hot pixel

and an apparatus for detecting and correcting a bad pixel regarding a hot pixel that can

correctly detect the hot pixel and correct the hot pixel with the bad pixel.

Other objects of the present invention will become more apparent through the

embodiments described below.

[Technical Solution]

In order to achieve the above objects, an embodiment of the present invention

features an apparatus for detecting a bad pixel regarding a hot pixel. The apparatus has a

bad pixel detection unit for determining whether a subject pixel is a bad pixel by

comparing the subject pixel with a neighboring pixel, a first flag generation unit for

generating a bad pixel flag in case the bad pixel detection unit detects a bad pixel, a

pre-application unit for performing at least one of gain operation and gamma correction

on a pixel of an inputted image, a hot pixel detection unit for determining whether a

pixel inputted from the pre-application unit is a hot pixel by comparing the pixel with a

neighboring pixel, and a second flag generation unit for generating a hot pixel flag in

case the hot pixel detection unit detects a hot pixel.

The gain operation performed by the pre-application unit is an RGB operation,

and the gamma correction performed by the pre-application unit is performed by

applying a gamma curve. The apparatus also has a correction unit for correcting the subject pixel by

using a neighboring pixel value in accordance with a combination of a bad pixel and a

hot pixel, after recognizing a bad pixel flag and a hot pixel flag generated by the first

flag generation unit and the second flag generation unit, respectively. The correction

unit corrects the subject pixel by calculating a logical sum of the bad pixel flag and the

hot pixel flag.

Another embodiment of the present invention features an apparatus for

detecting a hot pixel. The apparatus, disposed in a front part of an image processing

system, has a pre-application unit for performing at least one of gain operation and

gamma correction on a pixel of an inputted image, a hot pixel detection unit for

determining whether a pixel inputted from the pre-application unit is a hot pixel by

comparing the pixel with a neighboring pixel, and a flag generation unit for generating a

hot pixel flag in case the hot pixel detection unit detects a hot pixel.

The gain operation performed by the pre-application unit is an RGB operation,

and the gamma correction performed by the pre-application unit is performed by

applying a gamma curve.

[Description of Drawings]

FIG. 1 and FIG. 2 show how interpolation spreads a hot pixel to the

surrounding area during the image processing; FIG. 3 illustrates a gamma curve used in a gamma correction unit during the

image processing;

FIG. 4 and FIG. 5 show how hot pixels are formed by a process of the gamma

correction unit during the image processing;

FIG. 6 shows the structure of an image processing system in accordance with

an embodiment of the present invention;

FIG. 7 shows the structure of a bad pixel correction apparatus regarding a hot

pixel in accordance with an embodiment of the present invention; and

FIG. 8 to FIG. 10 show an example of how the correction unit of FIG. 7 makes

a correction.

510, 520: Delay buffer

530: Bad pixel detection unit

540, 570: Flag generation unit

550: Gamma/gain pre-application unit

560: Hot pixel detection unit

580: Correction unit

[Mode for Invention] The above objects, features and advantages will become more apparent through

the below description with reference to the accompanying drawings.

Since there can be a variety of permutations and embodiments of the present

invention, certain embodiments will be illustrated and described with reference to the

accompanying drawings. This, however, is by no means to restrict the present invention

to certain embodiments, and shall be construed as including all permutations,

equivalents and substitutes covered by the spirit and scope of the present invention.

Throughout the drawings, similar elements are given similar reference numerals.

Throughout the description of the present invention, when describing a certain

technology is determined to evade the point of the present invention, the pertinent

detailed description will be omitted.

Terms such as "first" and "second" can be used in describing various elements,

but the above elements shall not be restricted to the above terms. The above terms are

used only to distinguish one element from the other. For instance, the first element can

be named the second element, and vice versa, without departing the scope of claims of

the present invention. The term "and/or" shall include the combination of a plurality of

listed items or any of the plurality of listed items.

When one element is described as being "connected" or "accessed" to another

element, it shall be construed as being connected or accessed to the other element

directly but also as possibly having another element in between. On the other hand, if one element is described as being "directly connected" or "directly accessed" to another

element, it shall be construed that there is no other element in between.

The terms used in the description are intended to describe certain embodiments

only, and shall by no means restrict the present invention. Unless clearly used otherwise,

expressions in the singular number include a plural meaning. In the present description,

an expression such as "comprising" or "consisting of is intended to designate a

characteristic, a number, a step, an operation, an element, a part or combinations thereof,

and shall not be construed to preclude any presence or possibility of one or more other

characteristics, numbers, steps, operations, elements, parts or combinations thereof.

Unless otherwise defined, all terms, including technical terms and scientific

terms, used herein have the same meaning as how they are generally understood by

those of ordinary skill in the art to which the invention pertains. Any term that is

defined in a general dictionary shall be construed to have the same meaning in the

context of the relevant art, and, unless otherwise defined explicitly, shall not be

interpreted to have an idealistic or excessively formalistic meaning.

Hereinafter, preferred embodiments will be described in detail with reference

to the accompanying drawings. Identical or corresponding elements will be given the

same reference numerals, regardless of the figure number, and any redundant

description of the identical or corresponding elements will not be repeated. In describing the present invention, the cause of a hot pixel in an image sensor

and the problem of the hot pixel will be first described.

Generally, there are two types of bad pixel: hot pixel and dead pixel. A dead

pixel refers to a pixel that does not react to the light at all, and a hot pixel refers to a

pixel that reacts abnormally (i.e. either too sensitively or too insensitively) to the light.

The hot pixel usually occurs when the sensor is unable to output a normal value outside

the range of normal operation, for example, when the gain on the sensor is adjusted in

low luminosity or when the sensor is overheated due to the surrounding hardware

property. The hot pixel can also occur when the contrast is raised or when the gamma

correction unit is applied during the image processing.

In general, correction is made to the visible pixels that demonstrate a more than

a particular difference in brightness, without resulting in the distortion of the image.

Therefore, any hot pixels that are outside a particular range but become visible by

amplifying their values during the image processing as if they are bad pixels can not be

corrected. Another cause for making hot pixel detection and correction difficult is

performing the bad pixel detection and correction at the beginning of the image

processing. The details are described below.

Hot pixels usually become a problem after each pixel is applied with gain,

interpolated and gamma-corrected. In other words, hot pixels occur in the middle of the

image processing. Interpolation smears the hot pixel to the surrounding area, spreading the noise.

Thus, the image gets distorted after the RGB interpolation, making it impossible to

determine whether this image is a noise or a normal image. FIG. 1 and FIG. 2 show an

example of a hot pixel smearing to the surrounding area by the interpolation during the

image processing. As shown in the drawing, it can be inferred that the hot pixel (FIG. 1)

appearing during the image processing makes the surrounding pixels appear distorted by

the interpolation (FIG. 2).

If the gain is multiplied or the gamma correction unit is applied, the difference

in the pixels, which were not included in the range of bad pixels, is gained or expanded

as the gamma correction unit is applied, making these pixels appear to be bad pixels.

Thus, the image processing steps can cause a bad pixel. Let's take a look at the

drawings.

FIG. 3 illustrates a gamma curve used in the gamma correction unit during the

image processing. FIGS. 4 and 5 illustrate hot pixels occurred by a process of the

gamma correction unit during the image processing. For instance, Suppose that the bad

pixel is defined as a pixel that has a luminance difference of 30 over the surrounding

pixels. According to the conventional method of detecting a bad pixel, if the difference

(difference between the center pixel and surrounding pixels) in input values of the

image processing is 20 (i.e. A in FIG. 3, and FIG. 4), the output becomes 40 (i.e. B in

FIG. 3, and FIG. 5) after the gamma correction and gain are applied. Therefore, this pixel (i.e. C in FIG. 5), which is suddenly amplified, becomes much more visible over

other pixels, becoming a hot pixel causing image deterioration. Besides, this

phenomenon is discovered only after the noise is spread to the surrounding area, as

shown in FIG. 2, and thus applying a filter after this occurs will not solve the problem

completely.

Hereinafter, embodiments of the present invention will be described with

reference to the accompanying drawings.

FIG. 6 shows the structure of an image processing system to which the present

invention is applied.

As shown in the drawing, the image processing system of the present invention

comprises an image input unit 410, a bad/hot pixel correction unit 420, a gain

application unit 430, an interpolation unit 440, a gain application unit 450, a gamma

correction unit 460, a color correction unit 470, and an image output unit.

According to the present invention, the spread of the noise, as shown in FIG. 2,

can be prevented by placing the bad/hot pixel correction unit 420 in the front part of the

image processing system, and the size of hardware can be reduced by detecting and

correcting bad pixels and hot pixels at the same time.

Each element of FIG. 6 is well known, and thus the detailed description will

not be provided here. The detection and correction of bad/hot pixels by the bad/hot pixel correction unit 420 will be described with reference to FIG. 7.

FIG. 7 shows the structure of the bad pixel correction apparatus regarding a hot

pixel in accordance with an embodiment of the present invention.

As shown in the drawing, the correction apparatus of the present invention can

comprise a first delay buffer 510, a second delay buffer 520, a bad pixel detection unit

530, a first flag generation unit 540, a gain pre-application unit 550, a hot pixel

detection unit 560, a second flag generation unit 570 and a correction unit 580. hi the

present invention, once the Bayer image data is inputted, the corresponding image is

divided into three paths for detection and correction. For the sake of convenience, the

first delay buffer 510 will be referred to as a first path, the second delay buffer 520 a

second path, and a path to the gamma/gain pre-application unit 550 a third path.

The first delay buffer 510 is in charge of delaying image data inputted while an

operation to the second path and the third path is performed. The first delay buffer 510

is needed to bypass any normal input image data, which needs no correction, and correct

any bad pixel or hot pixel, which needs to be corrected, using surrounding values.

The gamma/gain pre-application unit 550 is in charge of multiplying in

advance an RGB gain, applied in the gain application unit 450 of FIG. 6, to RGB data

of an inputted image and performing in advance the correction by using the gamma

curve of the gamma correction unit 460 of FIG. 6, in order to match the white balance

with the gamma correction unit of FIG. 6. Through this, the hot pixel, which will occur near the end of the image processing system, can be generated in advance.

While the conventional image processing system detects a bad pixel and then

multiplies the gain to the corresponding data before proceeding with the correction

using a gamma curve, the gamma/gain pre-application unit 550 of the present invention

applies the gain in advance to the inputted image data and proceeds with the correction

to effectively detect a hot pixel early in the image processing system.

The second delay buffer 520 is in charge of delaying the data inputted through

the second path while the gamma/gain pre-application unit 550 proceeds with the

operation.

The bad pixel detection unit 530 and the hot pixel detection unit 560 are in

charge of detecting a bad pixel and a hot pixel, respectively, of the inputted image data.

These detection units 530 and 560 compare a subject pixel (i.e. the pixel subjected to

the determination of being a bad pixel or a hot pixel) and neighboring same-kind pixels,

located in the front and back of the subject pixel, for pixels that are sequentially

inputted, and determine the possibility of the subject pixel being a bad/hot pixel. To

determine a bad/hot pixel, various methods, including the threshold method, the

weighted method, and the threshold count method, are used. The threshold method

determines the presence of a bad/hot pixel by determining if the difference between the

subject pixel and the neighboring same-kind pixel exceeds a predetermined threshold.

The weighted method determines the presence of a bad/hot pixel by comparing the multiplication of the neighboring same-kind pixel with the subject pixel. In the

threshold count method, the difference between the same kind pixels that exceeds a

predetermined threshold is counted, and the pixel whose difference exceeds a

predetermined number is considered a bad/hot pixel. Any of the above methods can be

used in the present invention.

The first flag generation unit 540 is in charge of generating a bad pixel flag

when a bad pixel is detected by the bad pixel detection unit 530, and the second flag

generation unit 570 is in charge of generating a hot pixel flag when a hot pixel is

detected by the hot pixel detection unit 560.

The correction unit 580 recognizes the flag generated by the first flag

generation unit 540 or the second flag generation unit 570 to determine whether the

current pixel is a bad pixel, hot pixel or normal pixel. If the current pixel is deemed to

be a bad pixel or a hot pixel, the correction unit 580 corrects the pertinent pixel,

received from the first path, using a neighboring pixel value in accordance with the

combination of the two flags. In case it is desired that the hot pixel be ignored and only

obvious bad pixels be corrected, the pertinent pixel of the first path is corrected using

the neighboring pixel value if only the bad pixel flag is 1. In case it is desired that both

the hot pixel and the bad pixel be corrected, the logical sum (OR) of the bad pixel flag

and hot pixel flag is calculated, and if the logical sum is 1, the hot pixel and bad pixel

can be corrected using a neighboring pixel value. If the current pixel is neither a bad pixel nor a hot pixel, the pixel received from the first path is deemed normal and thus is

bypassed by the correction unit 580. The correction made by the correction unit 580 is

described with reference to FIG. 6.

FIGS. 8 through 10 are examples for describing how the correction unit makes

a correction as shown in FIG. 7. If the center pixel, shown in FIG. 8, is detected to be a

bad pixel or a hot pixel and thus is needed to be corrected, the correction unit 580 can

generate a correction value, using the surrounding 8 pixels. That is, in FIG. 8, the

correction unit 580 calculates the absolute value of the difference between the pixels

located diagonally, and corrects the value of the bad pixel or hot pixel to an average

value of neighboring pixels having the smallest absolute value.

In FIG. 9, the pixels connected by dotted lines are subjected to the calculation

smallest value. Therefore, the bad pixel or hot pixel having the value of 130 is corrected

to (90+92)/2, which is the average value of φ and (E).

The drawings and detailed description are only examples of the present

invention, serve only for describing the present invention and by no means limit or

restrict the spirit and scope of the present invention. Thus, any person of ordinary skill

in the art shall understand that a large number of permutations and other equivalent embodiments are possible. The true scope of the present invention must be defined only

by the spirit of the appended claims.

[Industrial Applicability]

As described above, the present invention can prevent the spread of noise in

advance to reduce the distortion and improve the quality of an image by detecting in

advance a hot pixel, which usually occurs in the middle of an image processing system.

The present invention can also reduce the size of hardware by detecting a hot

pixel and a bad pixel simultaneously.

Claims

[CLAIMS]

[Claim 1 ]

An apparatus for detecting a bad pixel regarding a hot pixel, the apparatus

comprising:

a bad pixel detection unit for determining whether a subject pixel is a bad pixel

by comparing the subject pixel with a neighboring pixel;

a first flag generation unit for generating a bad pixel flag in case the bad pixel

detection unit detects a bad pixel;

a pre-application unit for performing at least one of gain operation and gamma

correction on a pixel of an inputted image;

a hot pixel detection unit for determining whether a pixel inputted from the

pre-application unit is a hot pixel by comparing the pixel with a neighboring pixel; and

a second flag generation unit for generating a hot pixel flag in case the hot

pixel detection unit detects a hot pixel.

[Claim 2]

The apparatus of Claim 1, wherein the gain operation performed by the

pre-application unit is an RGB operation.

[Claim 3]

The apparatus of Claim 1 , wherein the gamma correction performed by the

pre-application unit is performed by applying a gamma curve.

[Claim 4]

The apparatus of Claim 1 , further comprising a correction unit for correcting

the subject pixel by using a neighboring pixel value in accordance with a combination

of a bad pixel and a hot pixel, after recognizing a bad pixel flag and a hot pixel flag

generated by the first flag generation unit and the second flag generation unit,

respectively.

[Claim 5]

The apparatus of Claim 4, wherein the correction unit corrects the subject pixel

by calculating a logical sum of the bad pixel flag and the hot pixel flag.

[Claim 6]

An apparatus for detecting a hot pixel, the apparatus being disposed in a front

part of an image processing system, the apparatus comprising:

a pre-application unit for performing at least one of gain operation and gamma

correction on a pixel of an inputted image; a hot pixel detection unit for determining whether a pixel inputted from the

a flag generation unit for generating a hot pixel flag in case the hot pixel

detection unit detects a hot pixel.

[Claim 7]

The apparatus of Claim 6, wherein the gain operation performed by the

pre-application unit is an RGB operation.

[Claim 8]

The apparatus of Claim 6, wherein the gamma correction performed by the

pre-application unit is performed by applying a gamma curve.