KR20100081886A - Adaptive tone mapping apparatus and method, and image processing system using the method - Google Patents

Abstract

The present invention, an adaptive tone mapping apparatus and method, and an image processing system using the same, perform preprocessing on an input high dynamic range image, and convert the preprocessed high dynamic range image into a basic image using linear bilateral filtering. The image is separated and extracted into a detail image, and post-processing is performed on the base image and the detail image to map a high dynamic range (HDR) image to a low dynamic range (LDR) image, thereby generating an existing method using a bilateral filter. By preventing the loss of contrast ratio of the tone-mapped high dynamic image, a natural original image can be displayed on a general display device, and high-speed computation can be realized.

Description

Adaptive tone mapping apparatus and method, and image processing system using the same {ADAPTIVE TONE MAPPING APPARATUS AND METHOD, AND IMAGE PROCESSING SYSTEM USING THE METHOD}

The present invention relates to tone mapping. More particularly, a high contrast image is separated into a base layer and a detail layer using a bilateral filter, and dynamic region compression is performed using the base image. An adaptive tone mapping apparatus and method for preserving detail by adding detail images after performing the same, and an image processing system using the same.

Due to the development of the multimedia industry and the rapid development of the image acquisition technology of digital cameras, real world scenes that humans see with visual information are represented. High dynamic range (HDR) images can express luminance information that is close to the real world, and the actual range of the luminance ranges from 10 ^-4 to 10 ⁵ cd / m ² . However, the actual luminance of the HDR image should be expressed within a limited low dynamic range (LDR) when expressed by a commonly used digital display device. Therefore, due to the inconsistency of the dynamic range, compression of the dynamic region occurs, thereby degrading contrast and detail of the image.

Therefore, in order to display a high contrast image in a general digital display device, a method of converting a high contrast image into a low contrast image is required, and this method is referred to as tone mapping or tone reproduction. . The tone mapping process converts a high contrast image from an image with a reduced dynamic range, so visual matching with the original image should be performed under the premise of preserving high contrast information as much as possible. Several methods for this have been proposed. Recently, researches on a method of minimizing the difference between an image of a general display device and an HDR image by modeling a human visual system have been actively conducted.

Among the conventional tone mapping techniques, there are a log function based tone mapping method and a zone system based tone mapping method.

Logarithmic tone mapping is a method that uses the characteristic that Weber's law, which is various visual characteristics of human, increases similarly to logarithmic function. Weber's law detects differences in the dark areas of the human eye, even if the brightness changes only a little, while even larger changes in the bright areas can't detect the difference. In the logarithmic tone mapping technique, the logarithmic tone mapping method estimates the feature value of the image using a histogram, and consequently maintains visual quality by preserving the contrast, brightness, and detail information of the original image. However, log-function-based tone mapping has the disadvantage of not preserving detail in the overall brightness and darkness of the input image, such as linear tone mapping, when applied to an image where the brightness distribution is concentrated in one place.

On the other hand, tonal mapping based on the zone system is used to perform the dosing and burning through the difference of the Gaussian filtering value according to the scale factor in order to preserve the detail of the dark and bright parts of the image, thereby reducing the detail in the dark part of the original image. detail) Maintains visual quality by preserving information and maintaining overall contrast ratio. However, tonal mapping based on zone system does not preserve detail in bright areas of an image, and has a disadvantage in that halo artifacts occur at extreme brightness values.

That is, the conventional tone mapping method does not preserve the overall brightness and brightness of the input image when applied to an image where the brightness distribution is concentrated in one place, or does not preserve the detail in the portion where the brightness value of the image is large, There is a disadvantage in that a halo effect occurs due to the determination of the low frequency component of the input image using the Gaussian filter in the region where the change is large. In addition, these methods require a large amount of data storage space during the Gaussian filtering operation of the input image for each scale factor in the zone system and a large amount of computation occurring in calculating the pixel probability distribution for all the brightness intervals. In addition, due to the complexity of the hardware and the large amount of computation in hardware implementation, there is a disadvantage that it cannot be applied to digital applications requiring real time processing.

Therefore, there is an urgent need for a study on tone mapping techniques that can preserve overall brightness and detail.

Therefore, the present invention separates a high contrast image into a base image and a detail image by using a bilateral filter, preserves the detail by adding a detail image after performing dynamic region compression using the base image, and prevents the loss of contrast ratio. The purpose of the present invention is to provide a natural original image and to facilitate high-speed operation and design.

The adaptive tone mapping method according to an aspect of the present invention for achieving the above object of the present invention is a method for mapping a high dynamic range (HDR) image to a low dynamic range (LDR) image, the input high dynamic range Performing preprocessing on the image, separating and extracting the preprocessed high dynamic range image into a base image and a detail image using linear bilateral filtering, and performing post-processing on the base image and the detail image It includes.

The linear bilateral filtering is performed using a domain weight filter and a range weight filter having linear characteristics.

At least one of the domain weight filter and the range weight filter uses a parameter referring to data stored in a lookup table and a parameter obtained by barrel shifting an input signal.

The preprocessing of the input high dynamic range image may include generating a brightness map for the input HDR image and performing linear log scaling on the brightness map of the image.

The linear log scaling is performed by using a parameter referring to data stored in the lookup table and a parameter obtained by barrel shifting the input signal.

Performing post-processing on the base image and the detail image includes compressing a dynamic region of the base image and outputting the dynamic region compressed base image by adding the detail image.

The compressing of the dynamic region of the base image may include compressing the dynamic region of the base image by using the maximum and minimum values of the base image.

The adaptive tone mapping method may further include converting the post-processed image to be suitable for the LDR dynamic region and performing gamma correction on the image converted to be suitable for the LDR dynamic region.

According to another aspect of the present invention, an adaptive tone mapping apparatus includes a linear log scaling unit and a linear bilateral filtering for performing a linear log scaling on a brightness map of an input high dynamic range image. The linear bilateral filtering unit extracts the brightness map of the log-scaled image into a base image and a detail image.

The image processing system according to another aspect of the present invention for achieving the above object of the present invention, the brightness map generator for generating a brightness map for the input high dynamic range image signal to the linear log scaling unit, input A linear log scaling unit for performing linear log scaling on the brightness map of the high dynamic range image, and a linear bilateral filtering unit for extracting the brightness map of the log-scaled image using the linear bilateral filtering into the base image and the detail image. And a base image and a detail image output from the linear bilateral filtering unit, and perform dynamic region compression of the base image using the maximum and minimum values of the received base image, and add the received image to the detailed image. Type tone reproducing unit.

According to the present invention, a natural original image can be displayed on a general display device by preventing the contrast ratio of a tone-mapped high dynamic image generated by a conventional method using a bilateral filter, and high speed computation can be realized.

As the invention allows for various changes and numerous embodiments, particular embodiments will be illustrated in the drawings and described in detail in the written description. However, this is not intended to limit the present invention to specific embodiments, it should be understood to include all changes, equivalents, and substitutes included in the spirit and scope of the present invention. Like reference numerals are used for like elements in describing each drawing.

Terms such as first and second may be used to describe various components, but the components should not be limited by the terms. The terms are used only for the purpose of distinguishing one component from another. For example, without departing from the scope of the present invention, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component. And / or < / RTI > includes any combination of a plurality of related listed items or any of a plurality of related listed items.

The terminology used herein is for the purpose of describing particular example embodiments only and is not intended to be limiting of the present invention. Singular expressions include plural expressions unless the context clearly indicates otherwise. In this application, the terms "comprise" or "have" are intended to indicate that there is a feature, number, step, operation, component, part, or combination thereof described in the specification, and one or more other features. It is to be understood that the present invention does not exclude the possibility of the presence or the addition of numbers, steps, operations, components, components, or a combination thereof.

The present invention proposes an adaptive tone mapping method using a bilateral filter having edge-preserving flattening characteristics to prevent the overall brightness and detail of the input image and to eliminate the halo effect. do. To this end, the present invention separates a high contrast image into a base layer and a detail layer using a bilateral filter, adds a detail image after performing dynamic region compression using the base image, Preserve detail. After the adaptive tone reproduction step, the resultant image of tone mapping to the dynamic range of the low dynamic range (LDR) display device is finally obtained. In addition, the present invention uses a linear function in order to minimize the amount of computation occurring in the hardware implementation.

Hereinafter, with reference to the accompanying drawings, it will be described in detail a preferred embodiment of the present invention. Hereinafter, the same reference numerals are used for the same components in the drawings, and duplicate descriptions of the same components are omitted.

1 shows a schematic operation sequence of the adaptive tone mapping method according to the present invention.

In the adaptive tone mapping method according to the present invention, first, a brightness map of the input HDR image is generated (S101). One preferred embodiment of the method for generating a brightness map from an HDR image uses Equation 1 below.

[Equation 1]

Here, R, G, and B represent each color channel value for red, green, and blue of the input HDR image, and L represents a luminance value of the real world image.

Next, linear log scaling is performed on the brightness map generated as described above (S102). By this log scaling, the brightness map has a log scaled value, and the scaled value is extracted and extracted into the base image and the detail image using linear bilateral filtering (S103). For the separated base image, compression is performed using the maximum and minimum values of the extracted base image (S104), and the dynamic region compressed HDR image is extracted. Then, the detail image separated and extracted in step 103 is added to the dynamic region compressed HDR image (S105). By performing steps 101 to 105, an adaptive tone reproduced output image in which global contrast (global contrast_) is preserved according to the present invention can be obtained (S106) LDR display representation for the adaptive tone reproduced output image. The output image is generated by normalizing the dynamic range or a digital value of 0 to 255 (S107), and performing final mapping indicating a gamma corrected image to the LDR display device for the output image. To express (S108).

Hereinafter, a detailed operation of linear log scaling, linear bilateral filtering, and adaptive tone reproduction of the brightness map used in the present invention will be described.

First, the linear log scaling of the brightness map according to the present invention will be described with reference to FIG. 2.

2 is a table illustrating a matching relationship between parameters used for linear log scaling according to the present invention.

The conventional log scaling formula used to perform log scaling on the brightness map is shown in Equation 2 below.

[Equation 2]

Here, L (x, y) means a brightness value at a specific coordinate, and I (x, y) is a log scaled value of brightness at a specific coordinate.

However, these conventional log scaling equations use nonlinear functions, which makes the actual hardware implementation complex and computationally expensive. Therefore, the present invention uses a linear function such as Equation 3 below, which can replace the existing log scaling equation.

&Quot; (3) "

,

을 나타내며, 도 2의 테이블은 입력 영상의 밝기 값의 각 영역에 대한 선형함수의 파라미터

및

의 매칭 관계를 나타낸다. 여기서, 파라미터 A는 룩업 테이블에 저장되는 데이터 값이며, 파라미터 B는 배럴 쉬프트한 값이다.here,

,

2 shows the parameters of the linear function for each region of the brightness value of the input image.

And

Indicates a matching relationship. Here, parameter A is a data value stored in the lookup table, and parameter B is a barrel shifted value.

That is, the log scaling according to the present invention is designed as a linear function using a look-up table (LUT) method and a barrel shift according to an input value, thereby facilitating hardware implementation in comparison with a conventional log scaling function.

Next, the linear bilateral filtering according to the present invention will be described with reference to FIGS. 3 and 4.

3 is a preferred embodiment of a table showing a mapping relationship between parameters used in a domain weight filter used in the linear bilateral filtering method according to the present invention, and FIG. 4 is used in the linear bilateral filtering method according to the present invention. An exemplary embodiment of a table representing a mapping relationship between parameters used in a range weight filter is shown.

In the log scaled brightness map according to the present invention, linear bilateral filtering is performed to generate a detail image representing the detail and local contrast of the high contrast image and a basic image representing the overall light and dark degree of the image.

)와 입력 값의 차에 따른 가중치를 적용한 레인지 가중치 필터(range weighting filter,

)로 구성되며, 아래 수학식 4와 같이 나타낼 수 있다.Conventional bilateral filters use the same domain weighting filter as the Gaussian filter,

Range weighting filter with weights based on the difference between

) And can be expressed as Equation 4 below.

&Quot; (4) "

Here, each element used in Equation 4 will be described below.

The domain weight filter used in the conventional bilateral filter is

And the range weight filter,

Can be expressed as: Meanwhile, here,

,

,

Lt; / RTI >

In addition, Ω denotes an area of the kernel, p and s denote pixel positions, (x, y) denote coordinate differences between two pixels, and I denote log-scaled pixel values. ks is a normalization factor, ss and sr are standard deviations of the domain weight filter and the range weight filter, and delta is the absolute value of the difference between pixel values.

As can be seen from Equation 4, conventionally used bilateral filtering uses a nonlinear function, which makes operation complicated in hardware implementation.

Accordingly, the present invention uses linear bilateral filtering, and the linear bilateral filtering method according to the present invention uses a domain weight filter obtained through Equation 5 below and a range weight filter obtained through Equation 6 below. Can be calculated by substituting Equation 4.

[Equation 5]

here,

이다.

to be.

,

이다. ego,

,

to be.

In relation to FIG. 3, parameters of the linear domain weight filtering function are shown according to pixel distances, parameter A represents data values stored in the lookup table, and parameter B represents barrel shifted values.

&Quot; (6) "

,

이고, 파라미터 A는 룩업 테이블에 저장되는 데이터 값이다.here,

,

And parameter A is a data value stored in the lookup table.

4 shows a matching relationship between parameters A and B of the linear range weight filter function according to an absolute value of two pixel differences.

 That is, by substituting the domain weight filter and the range weight filter according to the present invention into Equation 4, the base image can be obtained as shown in Equation 7 below.

[Equation 7]

In addition, a detail image may be obtained by Equation 8 below.

[Equation 8]

5A and 5B illustrate exemplary embodiments of the resultant image of the base image and the detail image obtained by the linear bilateral filter according to the present invention.

After acquiring the base image and the detail image, a mapping function shown in Equation 9 below is used to compress the dynamic region of the base image.

[Equation 9]

Where I ' _base (x, y) is the compressed base image, I _{base_value} is the base brightness of the compressed base image, I _{scale_range} is the size of the dynamic region of the compressed base image, and I _{base_max} and I _{base_min} are the base, respectively. The maximum and minimum values of the image are shown.

The compressed base image calculated by Equation 9 is added to the detail image as shown in Equation 10 below.

[Equation 10]

Here, L _{base_com} represents a high contrast image compressed with dynamic region, I ' _base represents a basic image compressed with dynamic region, and I _detail represents a detail image.

The detail was preserved by acquiring a high contrast image having a dynamic region compressed by the above-described method.

In the present invention, as shown in Equation 11 below, the brightness information of the original image is added to the high contrast image having the dynamic region compressed, thereby improving the overall brightness and contrast ratio.

[Equation 11]

이다.Here, I _disp is an image changed into the dynamic region of the LDR display device, I _{disp_ori} is a nonlinear normalization value of the brightness map (L (x, y)) that is luminance information of the original image from 0 to 1, and α is 0 to DRN _disp {} means a function that normalizes the user-defined scale factor of 1 to the dynamic range of the LDR display device,

to be.

Lastly, the image is expressed by multiplying the color channel of the high contrast image by the luminance component by the image changed into the dynamic region of the LDR display device, and this process may be expressed by Equation 12 below.

[Equation 12]

는 1∼4의 값을 갖는 감마조정 인자이다.Where R, G and B are the color channel values of the input high contrast image

Is a gamma adjustment factor with a value of 1-4.

6 is a block diagram of an adaptive tone mapping apparatus according to an exemplary embodiment of the present invention.

As shown in FIG. 6, the adaptive tone mapping apparatus 600 according to the present invention includes a brightness map generator 630, a linear log scaling unit 640, a controller 650, and a linear bilateral filtering unit 660. ), And an adaptive tone reproducing unit 670. The apparatus may further include a synchronization signal extractor 610 and an image format converter 620.

The brightness map generator 630 generates a brightness map of the input HDR image. The brightness map generator 630 converts the RGB color coordinates into Y values of the XYZ color coordinates in order to generate luminance components of the image from fixed-point data related to R, G, and B. , That is, converted into a value for the luminance component.

The linear log scaler 640 performs linear log scaling on the brightness map signal received from the brightness map generator 630. That is, the log scaled range is searched, the linear log scaling weight according to the found log scaling range is determined, and the input signal is barrel-shifted and added. The log scaling weight is determined with reference to the lookup table shown in FIG.

The linear bilateral filtering unit 660 performs linear bilateral filtering on the log scaled brightness map into a detailed image representing the detail and local contrast of the high contrast image and a basic image representing the overall light and dark degree of the image. do. The linear bilateral filtering unit 660 uses a linear domain weight filter and a linear range weight filter to separate the input image from the base image and the detail image. A base image signal is generated by multiplying an input image by a value obtained by normalizing a weight value generated by each of the linear domain weight filter and the linear range weight filter by a kernel weight block (not shown) in the linear bilateral filtering unit 660. In addition, a detailed video signal is generated using the input signal and the basic video signal.

The adaptive tone reproducer 670 receives the base image and the detail image separated and output from the linear bilateral filtering unit 660 and compresses the received image using the maximum and minimum values of the received basic image, thereby providing a dynamic range. Extract the compressed HDR image. In addition, the received detail image is added to the dynamic region compressed HDR image. The LDR image signal is generated through nonlinear normalization, adaptive tone reproduction, and gamma correction is performed on the generated LDR image signal.

Meanwhile, the controller 650 receives the brightness map information from the brightness map generator 630 and provides it to the adaptive tone reproducer 670 to control the operation of the adaptive tone reproducer, while the adaptive tone reproducer Information on the coordinates of the display screen to display the image output by the 670, that is, banks, rows, etc., commands for activating, reading, and writing the corresponding coordinates, and related controls Print the command.

In addition, the adaptive tone mapping apparatus according to the present invention may include a synchronization signal extractor 610, and the synchronization signal extractor 610 may provide a signal indicating an effective period between the input vertical sync signal and the horizontal sync signal. As a block for extracting, the input synchronization signal and the data clock are counted to generate a signal representing a data valid period and are supplied to the brightness map generator 630 and the linear log scaler 640.

The image format converter 620 converts the input high contrast image data into fixed-point data and provides it to the brightness map generator 630. According to an exemplary embodiment, the image format converter 620 extracts a color value and an exponent value to convert 2-bit RGBE image data into a 32-bit fixed point, and performs a barrel shift. We compute the 32-bit fixed-point values by shifting each color data value using.

The output image according to the adaptive tone reproduction method according to the present invention as described above, it can be seen that to preserve the detail in the dark portion of the original image and to maintain the contrast ratio of the original image, the adaptive tone mapping apparatus according to the present invention It can be applied to various types of image processing apparatuses.

Although described with reference to the above embodiments, those skilled in the art will understand that various modifications and changes can be made without departing from the spirit and scope of the invention as set forth in the claims below. Could be.

1 is a schematic operational flowchart of an adaptive tone mapping method according to the present invention;

2 is a table showing a matching relationship between parameters used for linear log scaling according to the present invention.

3 is a diagram illustrating a preferred embodiment of a table showing a mapping relationship between parameters used in a domain weight filter used in a linear bilateral filtering method according to the present invention.

4 is a diagram illustrating a preferred embodiment of a table showing a mapping relationship between parameters used in a range weight filter used in a linear bilateral filtering method according to the present invention.

5A and 5B illustrate exemplary embodiments of the resultant image of the base image and the detail image obtained by the linear bilateral filter according to the present invention.

6 is a block diagram of an adaptive tone mapping apparatus according to an embodiment of the present invention.

Claims (17)

In the method for mapping a high dynamic range (HDR) image to a low dynamic range (LDR) image, Performing preprocessing on the input high dynamic range image; Separating and extracting the preprocessed high dynamic range image into a base image and a detail image using linear bilateral filtering; And And performing post-processing on the base image and the detail image. The method according to claim 1, The linear bilateral filtering, Adaptive tone mapping method performed using domain weight filter and range weight filter with linear characteristics. The method according to claim 2, At least one of the domain weight filter and the range weight filter uses a parameter referring to data stored in a lookup table and a parameter obtained by barrel shifting an input signal. The method according to claim 1, Pre-processing the input high dynamic range image may include: Generating a brightness map for the input HDR image; Wow And performing linear log scaling on the brightness map of the image. The method according to claim 4, The linear log scaling is An adaptive tone mapping method performed by using a parameter referring to data stored in a lookup table and a parameter obtained by barrel shifting an input signal. The method according to claim 1, Post-processing the base image and the detail image may include: Compressing a dynamic region of the base image; And And outputting the dynamic region compressed base image in addition to the detail image. The method according to claim 6, Compressing the dynamic region of the base image, Adaptive tone mapping method characterized in that the dynamic region of the base image is compressed by using the maximum value and the minimum value of the base image. The method according to claim 1, Converting the post-processed image to fit a low dynamic range region; Wow And performing gamma correction on the image converted to be suitable for the low dynamic range region. A linear log scaling unit performing linear log scaling on the brightness map of the input high dynamic range image; Wow An adaptive tone mapping device including a linear bilateral filtering unit for separating and extracting a brightness map of the log-scaled image using linear bilateral filtering into a base image and a detail image. The method according to claim 9, The linear bilateral filtering unit, An adaptive tone mapping device that performs filtering using a domain weight filter and a range weight filter with linear characteristics. The method according to claim 10, And at least one of the domain weight filter and the range weight filter uses a parameter referring to data stored in a lookup table and a parameter obtained by barrel shifting an input signal. The method according to claim 9, An adaptive tone which receives the base image and the detail image output from the linear bilateral filtering unit, performs dynamic region compression of the base image using the maximum and minimum values of the received base image, and adds the received to the detail image. An adaptive tone mapping device further comprising a playback unit. The method according to claim 12, The adaptive tone playback unit, An adaptive tone, characterized in that to generate a low dynamic range video signal by converting the dynamic region compressed base picture and the detail picture plus the shape, and to perform gamma correction on the generated low dynamic range video signal Mapping device. The method according to claim 9, And a brightness map generator for generating a brightness map for the input high dynamic range video signal and providing the brightness map to the linear log scaling unit. A brightness map generator for generating a brightness map of the input high dynamic range video signal and providing the brightness map to the linear log scaling unit; A linear log scaling unit performing linear log scaling on the brightness map of the input high dynamic range image; A linear bilateral filtering unit configured to separately extract the brightness map of the log-scaled image using linear bilateral filtering into a base image and a detail image; And An adaptive tone which receives the base image and the detail image output from the linear bilateral filtering unit, performs dynamic region compression of the base image using the maximum and minimum values of the received base image, and adds the received to the detail image. An image processing system comprising a playback unit. The method according to claim 15, The linear bilateral filtering unit, An image processing system performing filtering using a domain weight filter and a range weight filter having linear characteristics. The method according to claim 15 or 16, At least one of the domain weight filter, the range weighting filter, and the linear log scaling unit may implement a linear characteristic by using a parameter referring to data stored in a lookup table and a parameter obtained by barrel shifting an input signal. Processing system.

Images