KR102730991B1 - Image processing method and display device using the same - Google Patents

Abstract

An image processing method and a display device using the same are provided. The image processing method includes a step of extracting a plurality of motion vectors from a reference image based on a reference image and a comparison image, a step of determining reliability of the plurality of motion vectors and extracting inaccurate motion vectors having reliability lower than a predetermined standard, a step of extracting a background motion vector for a predetermined background area from among the plurality of motion vectors, a step of extracting a plurality of representative region motion vectors from among the motion vectors included in each of a plurality of divided areas of the reference image, and a step of correcting the inaccurate motion vectors with replacement motion vectors determined based on the background motion vectors and the plurality of representative region motion vectors. The image processing method according to one embodiment of the present invention can improve image quality by improving the accuracy of motion vectors by replacing inaccurate motion vectors with more accurate motion vectors.

Description

{IMAGE PROCESSING METHOD AND DISPLAY DEVICE USING THE SAME}

The present invention relates to an image processing method and a display device using the same, and more specifically, to an image processing method capable of improving the image quality of a display device and a display device using the same.

Flat panel displays (FPDs) are being adopted for various electronic devices such as mobile phones, tablets, notebook computers, televisions, and monitors. Recently, liquid crystal display devices (LCDs) and organic light emitting diode displays (OLEDs) have been used as FPDs. Such display devices include a pixel array that displays an image and is composed of a plurality of pixels, and a driving circuit that controls light to be transmitted or emitted from each of the plurality of pixels. The driving circuit of the display device includes a data driving circuit that supplies data signals to data lines of the pixel array, a gate driving circuit (or scan driving circuit) that sequentially supplies gate signals (or scan signals) synchronized with the data signals to the gate lines (or scan lines) of the pixel array, and a timing controller that controls the data driving circuit and the gate driving circuit.

In recent display devices, Frame Rate Up Conversion (hereinafter referred to as FRUC) is being used to improve compatibility between video media and to output videos more naturally. Here, FRUC is a technology that increases the frame rate of an input video by inserting an interpolation image between the frames of the input video.

In the existing FRUC, a method was used to simply repeat the frames of the input image or insert the average value of two adjacent frames between the two frames. In this way, the problem of the output image being cut off or blurred occurred through the existing FRUC, resulting in a deterioration in the image quality.

Recent FRUC estimates motion between frames in an input image and inserts an interpolated image based on the estimated motion between frames. Specifically, in recent FRUC, motion is estimated from adjacent frames, a motion vector is extracted, and an interpolated image is inserted between frames using the motion vector.

When extracting motion vectors on a pixel basis, the amount of computation becomes too large, making the FRUC of the display device inefficient. Therefore, FRUC using motion vectors can use a method of estimating motion based on blocks. Specifically, by dividing each frame of the image into blocks of a certain size, and finding the most similar block among adjacent frames based on the divided blocks and connecting them into vectors, motion can be estimated between frames and motion vectors can be determined.

However, when estimating motion on a block-by-block basis, there is a problem in that the accuracy of the motion vector in block units is lower than that of the motion vector estimated on a pixel-by-pixel basis.

[Related technical literature]

Device and method for increasing frame rate of video signal (Korean Patent Publication No. 10-2013-0031132)

Accordingly, the problem to be solved by the present invention is to provide an image processing method capable of improving the accuracy of a motion vector by selectively replacing an inaccurate motion vector with an accurate motion vector, and a display device using the same.

In addition, another problem that the present invention seeks to solve is to provide an image processing method and a display device using the same capable of reducing the cost for improving the accuracy of a motion vector by selectively replacing only inaccurate motion vectors with accurate motion vectors.

The tasks of the present invention are not limited to the tasks mentioned above, and other tasks not mentioned will be clearly understood by those skilled in the art from the description below.

In order to solve the above-described problem, an image processing method according to an embodiment of the present invention includes the steps of extracting a plurality of motion vectors from a reference image based on a reference image and a comparison image, a step of determining the reliability of the plurality of motion vectors and extracting inaccurate motion vectors having a reliability lower than a predetermined standard, a step of extracting a background motion vector for a predetermined background area from among the plurality of motion vectors, a step of extracting a plurality of representative region motion vectors from among the motion vectors included in each of a plurality of divided areas of the reference image, and a step of correcting the inaccurate motion vectors with replacement motion vectors determined based on the background motion vectors and the plurality of representative region motion vectors. The image processing method according to an embodiment of the present invention can improve image quality by improving the accuracy of the motion vectors by replacing inaccurate motion vectors with more accurate motion vectors.

In order to solve the problem described above, a display device according to another embodiment of the present invention includes a display panel, and an image processing unit that processes an image input to the display panel. The image processing unit includes a motion vector extraction unit that extracts a plurality of motion vectors in a reference image based on a reference image and a comparison image, a motion vector reliability determination unit that determines the reliability of the plurality of motion vectors and extracts an inaccurate motion vector having a reliability lower than a predetermined standard, a background motion vector extraction unit that extracts a background motion vector for a predetermined background area from among the plurality of motion vectors, a representative region motion vector extraction unit that extracts a plurality of representative region motion vectors from among the motion vectors included in each of a plurality of divided regions of the reference image, and a motion vector correction unit that corrects an inaccurate motion vector to a substitute motion vector determined based on the background motion vector and the plurality of representative region motion vectors. The display device according to another embodiment of the present invention can selectively replace a motion vector by comparing a background motion vector and a region motion vector and replacing an inaccurate motion vector with a more accurate foreground motion vector.

Specific details of other embodiments are included in the detailed description and drawings.

The present invention provides an image processing method capable of improving image quality by improving the accuracy of a motion vector by replacing an inaccurate motion vector with a more accurate motion vector, and a display device using the same.

In addition, the present invention can produce an image processing method and a display device using the same capable of selectively replacing a motion vector by comparing a background motion vector and an area motion vector and replacing an inaccurate motion vector with a more accurate foreground motion vector.

In addition, the present invention can produce an image processing method and a display device using the same, which can reduce the cost for improving the accuracy of motion vectors by selectively replacing only inaccurate motion vectors with accurate motion vectors.

The effects according to the present invention are not limited to those exemplified above, and further diverse effects are included in the present specification.

FIG. 1 is a block diagram schematically illustrating a display device according to one embodiment of the present invention.
FIG. 2 is a block diagram schematically showing an image processing unit according to one embodiment of the present invention.
FIG. 3 illustrates a procedure for processing an image according to an image processing method according to one embodiment of the present invention.
FIG. 4a and FIG. 4b illustrate a process of calculating the cost and flatness of a motion vector according to one embodiment of the present invention.
FIGS. 5A and 5B illustrate a process for determining an inaccurate motion vector according to one embodiment of the present invention.
Figure 6 schematically illustrates a background area in an image according to one embodiment of the present invention.
Figure 7 is an exemplary class for extracting a background motion vector according to one embodiment of the present invention.
FIG. 8 illustrates an example of a process for extracting an area motion vector according to one embodiment of the present invention.
FIG. 9 is an exemplary class for extracting a region motion vector according to one embodiment of the present invention.
FIG. 10 illustrates an example of a process for correcting a replacement motion vector according to one embodiment of the present invention.
FIG. 11 is an exemplary image comparing before and after a motion vector is corrected according to an image processing method according to one embodiment of the present invention.

The advantages and features of the present invention, and the method for achieving them, will become clear with reference to the embodiments described in detail below together with the accompanying drawings. However, the present invention is not limited to the embodiments disclosed below, but may be implemented in various different forms, and these embodiments are provided only to make the disclosure of the present invention complete and to fully inform a person having ordinary skill in the art to which the present invention belongs of the scope of the invention, and the present invention is defined only by the scope of the claims.

The shapes, sizes, ratios, angles, numbers, etc. disclosed in the drawings for explaining embodiments of the present invention are exemplary, and therefore the present invention is not limited to the matters illustrated. In addition, in describing the present invention, if it is determined that a detailed description of a related known technology may unnecessarily obscure the gist of the present invention, the detailed description will be omitted. When the terms “includes,” “has,” “consists of,” etc. are used in this specification, other parts may be added unless “only” is used. When a component is expressed in singular, it includes a case where the plural is included unless there is a specifically explicit description.

When interpreting components, it is interpreted as including the error range even if there is no separate explicit description.

When describing a positional relationship, for example, when the positional relationship between two parts is described as 'on ~', 'upper ~', 'lower ~', 'next to ~', etc., one or more other parts may be located between the two parts, unless 'right' or 'directly' is used.

When an element or layer is referred to as being on another element or layer, this includes both cases where the other element is directly on top of the other element or layer, or where there is another layer or other element intervening therebetween.

Although the terms first, second, etc. are used to describe various components, these components are not limited by these terms. These terms are only used to distinguish one component from another. Accordingly, a first component referred to below may also be a second component within the technical concept of the present invention.

Throughout the specification, identical reference numerals refer to identical components.

The size and thickness of each component shown in the drawings are illustrated for convenience of explanation, and the present invention is not necessarily limited to the size and thickness of the illustrated components.

The individual features of the various embodiments of the present invention may be partially or wholly combined or combined with each other, and as can be fully understood by those skilled in the art, various technical connections and operations are possible, and each embodiment may be implemented independently of each other or may be implemented together in a related relationship.

Hereinafter, various embodiments of the present invention will be described in detail with reference to the attached drawings.

FIG. 1 is a block diagram schematically illustrating a display device according to one embodiment of the present invention. Referring to FIG. 1, a display device (100) includes a display panel (110), a gate driving circuit (120), a data driving circuit (130), a timing control unit (140), and an image processing unit (150).

Referring to FIG. 1, a display device (100) includes a display panel (110) including a plurality of pixels (P), a gate driving circuit (120) that supplies a gate signal to each of the plurality of pixels (P), a data driving circuit (130) that supplies a data signal to each of the plurality of pixels (P), and a timing control unit (140) that controls the gate driving circuit (120) and the data driving circuit (130). In addition, the display device (100) includes an image processing unit (150) that receives data on an image input to the display panel (110), converts it into a digital signal, and supplies the digital signal and the control signal to the timing control unit (140).

In a display panel (110), a plurality of gate lines (GL) and a plurality of data lines (DL) intersect with each other, and each of a plurality of pixels (P) is connected to the gate lines (GL) and the data lines (DL). Specifically, one pixel (P) receives a gate signal from a gate driving circuit (120) through a gate line (GL), receives a data signal from a data driving circuit (130) through a data line (DL), and receives various powers through a power supply line.

The gate driving circuit (120) supplies a gate signal to the gate line (GL) according to a gate control signal (GCS) supplied from the timing control unit (140). Here, the gate signal includes at least one scan signal and a light emission control signal. In FIG. 1, the gate driving circuit (120) is illustrated as being arranged spaced apart from one side of the display panel (110), but the number and arrangement positions of the gate driving circuits (120) are not limited thereto. That is, the gate driving circuit (120) may be arranged on one side or both sides of the display panel (110) in a GIP (Gate In Panel) manner.

The data driving circuit (130) converts image data (RGB) into data voltage according to a data control signal (DCS) supplied from a timing control unit (140) and supplies the converted data voltage to a pixel (P) through a data line (DL).

The timing control unit (140) controls the timing of driving signals input to the display panel (110). Specifically, the timing control unit (140) processes image data (RGB) input from the image processing unit (150) to be suitable for the size and resolution of the display panel (110) and supplies the processed image data to the data driving circuit (130). In addition, the timing control unit (140) generates a plurality of gate and data control signals (GCS, DCS) using synchronization signals (SYNC), which are timing control signals input from the image processing unit (150), for example, a dot clock signal (DCLK), a data enable signal (DE), a horizontal synchronization signal (Hsync), and a vertical synchronization signal (Vsync). By supplying the generated plurality of gate and data control signals (GCS, DCS) to the gate driving circuit (120) and the data driving circuit (130), the gate driving circuit (120) and the data driving circuit (130) are controlled.

The image processing unit (150) is connected to a timing control unit (140) that controls the timing of driving signals input to the display panel (110). The image processing unit (150) supplies image data (RGB) and a timing control signal to the timing control unit (140).

In addition, the image processing unit (150) includes a module capable of FRUC for improving image quality based on information about the input image. Specifically, the image processing unit (150) can generate an interpolated image using FRUC for improving the image quality of the input image, and can extract a motion vector for generating the interpolated image.

Here, the image processing unit (150) can process the image input to the display panel (110) in units of frames. That is, the image processing unit (150) can divide the input image into images in units of frames and generate an interpolated image through FRUC based on the images for each frame. In particular, the image processing unit (150) can compare images of adjacent frames and perform FRUC.

Hereinafter, one of the images of adjacent frames processed by the image processing unit (150) is defined as a reference image, and the remaining one of the images of adjacent frames is defined as a comparison image. The specific configuration and function of the image processing unit (150) will be described later with reference to FIG. 2.

A display device (100) according to one embodiment of the present invention includes an image processing unit (150) capable of generating an interpolated image and correcting an inaccurate motion vector into an accurate motion vector, thereby predicting an accurate motion and improving the image quality of an image output through a display panel (110) based on the predicted interpolated image.

FIG. 2 is a block diagram schematically showing an image processing unit according to an embodiment of the present invention. FIG. 3 illustrates a procedure for processing an image according to an image processing method according to an embodiment of the present invention. FIGS. 4A and 4B illustrate a process for calculating a cost and flatness of a motion vector according to an embodiment of the present invention. FIGS. 5A and 5B illustrate a process for determining an inaccurate motion vector according to an embodiment of the present invention. FIG. 6 illustrates a background area in an image according to an embodiment of the present invention. FIG. 7 is an exemplary class for extracting a background motion vector according to an embodiment of the present invention. FIG. 8 is an exemplary class for extracting a region motion vector according to an embodiment of the present invention. FIG. 9 is an exemplary class for extracting a region motion vector according to an embodiment of the present invention. FIG. 10 is an exemplary diagram for correcting a replacement motion vector according to an embodiment of the present invention. For convenience of explanation, it will be described below with reference to FIG. 1.

Referring to FIG. 2, the image processing unit (150) includes a motion vector extraction unit (151), a motion vector reliability judgment unit (152), a background motion vector extraction unit (153), a representative region motion vector extraction unit (154), and a motion vector correction unit (155). The image processing unit (150) extracts motion vectors from a reference image and a comparison image, determines the reliability of the extracted motion vectors, and corrects inaccurate motion vectors with low reliability with alternative motion vectors determined based on the background motion vector and the representative region motion vector.

Referring to FIGS. 2 and 3, multiple motion vectors are extracted from a reference image based on a reference image and a comparison image (S310).

Referring to FIG. 2, the motion vector extraction unit (151) receives a video image, divides it into images for each frame, sets each image as a reference image, and compares it with a comparison image, which is an image of a frame adjacent to the reference image. Accordingly, the motion vector extraction unit (151) extracts a plurality of motion vectors corresponding to each block of the reference image.

Referring to FIGS. 2 and 3, the motion vector reliability judgment unit (152) judges the reliability of multiple motion vectors and extracts inaccurate motion vectors having a reliability lower than a predetermined standard (S320).

Here, the predetermined criterion means the reliability that the motion vector can be used as is without being corrected, and means the case where the cost is greater than the critical cost value and the flatness is greater than the critical flatness value. Here, the critical cost value and the critical flatness value are arbitrary numbers that can be preset by the user through the motion vector reliability judgment unit (152) and can be changed. The critical cost value and the critical flatness value will be described later with reference to FIGS. 5A and 5B.

Referring to FIG. 4a, the motion vector reliability judgment unit (152) calculates the cost of the motion vector based on the SAD (Sum of Absolute Difference) between the reference image (410) and the comparison image (420). The SAD is calculated for each block of the reference image (410) by [Mathematical Formula 1].

Here, the representative value is a value representing the characteristics of a pixel in an image, and can be, for example, the luminance of the pixel, image data, or one of the R, G, B data. The following explanation is based on the SAD calculated based on the luminance of the pixel.

Referring to FIG. 4A, the cost is a value representing the difference between blocks of the comparison image (420) for each block of the reference image (410). The cost of one block of the reference image (410) can be determined as the minimum value of the SAD of the comparison image (420). For example, the cost of a specific block (411) of the reference image (410) is ‘390’, which is the minimum value of the SAD of the comparison image (420). In this way, the cost can be calculated for each block of all the reference images (410).

In addition, the motion vector reliability judgment unit (152) calculates the smoothness, which represents the similarity between the motion vector and the motion vector adjacent to the motion vector. The smoothness is calculated for each block of the reference image (410) by [Mathematical Formula 2].

Referring to FIG. 4b, the flatness is a value representing the degree of similarity between the motion vector of the reference image (410) and the surrounding motion vectors. Specifically, it is the absolute value of the difference between the motion vector of a specific block (411) and the average value of the motion vectors adjacent to the motion vector of the specific block (411) within the surrounding block set (415). Referring to [Mathematical Formula 2], if the vector and the average value of the motion vectors adjacent to the motion vector of the specific block (411) within the surrounding block set (415) are vectors of (4,10), and the motion vector of the specific block (411) is (7,6), the flatness is 5.

Next, the motion vector reliability judgment unit (152) determines a motion vector whose cost is greater than a threshold cost value and whose flatness is greater than a threshold flatness value as an inaccurate motion vector.

Referring to FIG. 5a, a threshold cost value is set with respect to the cost. The threshold cost value may include a primary threshold cost value (531) and a secondary threshold cost value (532). The primary threshold cost value (531) may be preset as an arbitrary number. An area having a cost value greater than the primary threshold cost value (531) may be set as an excess cost area (541).

If the cost calculated based on the reference image (420) is greater than the primary threshold cost value (531), the motion vector reliability judgment unit (152) can determine that the reliability of the motion vector corresponding to the cost is lower than a predetermined criterion. For example, if the primary threshold cost value (531) is ‘1200’, the excess cost region (541) is a region having a cost value greater than ‘1200’. If the cost of the motion vector is greater than ‘1200’, the motion vector is included in the excess cost region (541), and the reliability of the motion vector included in the excess cost region (541) is low. For example, if the cost of the motion vector of the left block of the specific block (411) in FIG. 4A is ‘8110’, the motion vector of the left block of the specific block (411) is a motion vector included in the excess cost region (541) and has low reliability. Motion vectors with low reliability like this are corrected to alternative motion vectors by the motion vector corrector (155).

On the other hand, if the cost of the motion vector is less than ‘1200’, the motion vector is not included in the excess cost area (541). The reliability of the motion vector not included in the excess cost area (541) is higher than the reliability of the motion vector included in the excess cost area (541). Therefore, the motion vector not included in the excess cost area (541) is not corrected to a substitute motion vector by the motion vector correction unit (155).

Referring to FIG. 5a, the auxiliary threshold cost value (532) may be set to a value smaller than the primary threshold cost value (531). An area having a cost value smaller than the auxiliary threshold cost value (532) may be set as a low-cost area (542). For example, if the auxiliary threshold cost value (532) is ‘800’, the low-cost area (542) is an area having a cost value smaller than ‘800’. Accordingly, if the cost of a motion vector is smaller than ‘800’, the motion vector is included in the low-cost area (542), and this motion vector has very high reliability. For example, if the cost of a motion vector of a specific block (411) in FIG. 4a is ‘390’, the motion vector of the specific block (411) has very high reliability as a motion vector included in the low-cost area (542). A motion vector with a very high reliability like this can be selected as a replacement motion vector in the motion vector correction unit (155) and can replace a motion vector with a low reliability.

Referring to FIG. 5b, a threshold flatness value is set with respect to flatness. The threshold flatness value may include a primary threshold flatness value (533) and an auxiliary threshold flatness value (534). The primary threshold flatness value (533) may be preset as an arbitrary number. An area having a flatness value greater than the primary threshold flatness value (533) may be set as an excess flatness area (543).

If the flatness calculated based on the reference image (420) is greater than the primary threshold flatness value (533), the motion vector reliability judgment unit (152) can determine that the reliability of the motion vector corresponding to the flatness is lower than a predetermined criterion. For example, if the primary threshold flatness value (533) is ‘30’, the excess flatness area (543) is an area having a flatness value greater than ‘30’. If the flatness of the motion vector is greater than ‘30’, the motion vector is included in the excess flatness area (543), and the reliability of the motion vector included in the excess flatness area (543) is low. For example, if the flatness of the motion vector of the left block of the specific block (411) in FIG. 4b is ‘70’, the motion vector of the left block of the specific block (411) is a motion vector included in the excess flatness area (543) and has low reliability. Motion vectors with low reliability like this are corrected to alternative motion vectors by the motion vector corrector (155).

On the other hand, if the flatness of the motion vector is less than ‘30’, the motion vector is not included in the excess flatness area (543). The reliability of the motion vector not included in the excess flatness area (543) is higher than the reliability of the motion vector included in the excess flatness area (543). Accordingly, the motion vector not included in the excess flatness area (543) is not corrected to a substitute motion vector by the motion vector correction unit (155).

Referring to FIG. 5b, the auxiliary threshold flatness value (534) may be set to a value smaller than the primary threshold flatness value (533). An area having a flatness value smaller than the auxiliary threshold flatness value (534) may be set as a low flatness area (544). For example, when the auxiliary threshold flatness value (534) is ‘15’, the low flatness area (544) is an area having a flatness value smaller than ‘15’. Accordingly, when the flatness of a motion vector is smaller than ‘15’, the motion vector is included in the low flatness area (544), and this motion vector has very high reliability. For example, when the flatness of a motion vector of a specific block (411) in FIG. 4a is ‘5’, the motion vector of the specific block (411) has very high reliability as a motion vector included in the low flatness area (544). A motion vector with a very high reliability like this can be selected as a replacement motion vector in the motion vector correction unit (155) and can replace a motion vector with a low reliability.

Furthermore, the motion vector reliability judgment unit (152) determines the motion vector as an inaccurate motion vector if the motion vector has a cost value higher than the primary threshold cost value (531) and a flatness value higher than the primary threshold flatness value (533). That is, if the motion vector does not satisfy a predetermined criterion for both the cost and the flatness, the motion vector reliability judgment unit (152) determines that the reliability of the motion vector is lower than the predetermined criterion and the motion vector is an inaccurate motion vector.

Accordingly, motion vectors with high reliability are used as they are without being changed, and motion vectors with very high reliability can be used as replacement motion vectors to change motion vectors with low reliability. In addition, by changing only motion vectors with low reliability, the reliability of multiple motion vectors in one reference image can be improved with a small amount of computation.

Referring to FIGS. 2 and 3, the background motion vector extraction unit (153) extracts a background motion vector for a predetermined background area from among a plurality of motion vectors (S330).

Referring to FIG. 6, the reference image (610) includes a background region (620) and a central region (630). The background region (620) includes an upper background region (621) and a side background region (622). The central region (630) refers to a region surrounded by the background region (620) in the reference image (610). In the background region (620), the upper background region (621) is closer to the upper edge than the lower edge of the reference image (610). The side background regions (622) in the background region (620) are closer to both side edges of the reference image (610), respectively. Specifically, the side background region (622) includes a left background region (622a) adjacent to the left edge and a right background region (622b) adjacent to the right edge. The upper background region (621) and the side background regions (622) may overlap each other. Additionally, each of the upper background area (621) and the side background area (622) may be set to a width of 2 to 8 blocks. For example, if one block includes 8 X 8 pixels, the width of each of the upper background area (621) and the side background area (622) may be a width of 16 to 64 pixels.

Referring to FIG. 6, the vertical distance (d2) from the upper edge to the background area (620) is shorter than the horizontal distance (d1) from the two side edges to the background area (620). Since the motion vector in the background area (620) has a higher probability of having a horizontal component greater than a vertical component, the side background area (622) can be determined as an area further away from the two side edges than the vertical distance (d2). That is, the horizontal distance (d1) of the background area (620) is set longer than the vertical distance (d2) so that the background motion vector having a higher probability of having a horizontal component greater than a vertical component can have high reliability.

The specific steps for extracting background motion vectors from a predetermined background area are described below with reference to FIGS. 6 and 7.

Referring to FIGS. 6 and 7, the background motion vector extraction unit (153) classifies motion vectors existing in a background area (620) among a plurality of motion vectors into a plurality of classes (710, 720, 730), and determines a motion vector having a mode in the plurality of classes (710, 720, 730) as a background motion vector. In FIG. 7, each of the plurality of classes (710, 720, 730) includes a plurality of items, and the plurality of items may be a horizontal component of the motion vector (MV_x), a vertical component of the motion vector (MV_y), a cost of the motion vector, a weight of the motion vector, and a frequency of the existence of the motion vector. However, each of the plurality of classes (710, 720, 730) is not limited to the items above, and may include more or fewer items. Here, the frequency at which a motion vector exists is a number that increases each time a motion vector is stored in each of multiple classes (710, 720, 730).

Specifically, the background motion vector extraction unit (153) compares the motion vector of the class having the mode with the motion vector of the class having the second highest frequency among the plurality of classes, and if the mode is greater than a predetermined multiple of the second highest frequency, the motion vector having the mode is determined as the background motion vector (790). For example, it is assumed that the first background motion vector class (710) has the mode, the second background motion vector class (720) has the second highest frequency, and the n-th background motion vector class (730) has the lowest frequency. Accordingly, if the frequency at which the motion vector exists in the first background motion vector class (710) is greater than a predetermined multiple of the frequency at which the motion vector exists in the second background motion vector class (720), the motion vector corresponding to the first background motion vector class (710) is determined as the background motion vector (790). For example, if the predetermined multiple is 3, the frequency at which a motion vector exists in the first background motion vector class (710) is 10, and the frequency at which a motion vector exists in the second background motion vector class (720) is 2, the motion vector corresponding to the first background motion vector class (710) is determined as the background motion vector (790).

Accordingly, a background motion vector (790) with high reliability can be easily extracted from a predetermined background area (620). That is, by determining a background area (620) in advance and classifying the motion vector extracted from the background area (620) into a class, a motion vector having a mode can be easily extracted. Furthermore, a motion vector that does not have an overwhelming frequency compared to other motion vectors extracted from the background area (620) even if it has a mode may not be determined as a background motion vector (790). Accordingly, the reliability of the background motion vector (790) can be improved. In addition, the extracted background motion vector (790) can be used to determine a replacement motion vector for correcting an inaccurate motion vector.

Referring to FIGS. 2 and 3, the representative region motion vector extraction unit (154) extracts a plurality of representative region motion vectors from among the motion vectors included in each of a plurality of divided regions of the reference image (S340).

Referring to FIG. 8, the reference image (810) may include a plurality of divided regions (R1 to R15). Here, the plurality of divided regions (R1 to R15) are obtained by dividing the reference image (810) into a plurality of regions having a constant size, and each of the plurality of divided regions (R1 to R15) may include a plurality of blocks. Each of the plurality of blocks includes a motion vector, and the motion vector constitutes a plurality of region motion vectors included in each of the plurality of divided regions. For example, in the reference image (810), a specific divided region (811) is an region divided into 15 regions, and one specific divided region (811) may include 20 blocks, and each of the blocks includes a region motion vector. Accordingly, one specific divided region (811) may include 20 region motion vectors. The number of divided regions and the number of blocks within the divided region are not limited by those illustrated in FIG. 8 and may be changed.

The specific steps for extracting multiple representative region motion vectors from multiple region motion vectors included in each of multiple divided regions are described below with reference to FIGS. 8 and 9.

Referring to FIGS. 8 and 9, the representative region motion vector extraction unit (154) classifies a plurality of region motion vectors included in each of a plurality of divided regions (R1 to R15) into a plurality of classes (910, 920), updates the plurality of classes (910, 920), and determines the region motion vectors stored in the updated plurality of classes (910, 920) as a plurality of representative region motion vectors (991, 992). In FIG. 9, the plurality of classes (910, 920) are illustrated as two, but are not limited thereto, and the plurality of classes may include two or more classes. Since each item of the plurality of classes (910, 920) illustrated in FIG. 9 is substantially the same as each item of the plurality of classes (710, 720, 730) illustrated in FIG. 7, a redundant description thereof will be omitted.

Specifically, the representative region motion vector extraction unit (154) stores each of the region motion vectors selected from the plurality of region motion vectors in the number of the plurality of classes (910, 920) in each of the plurality of classes (910, 920). For example, the plurality of classes (910, 920) include the first region motion vector class (910) and the second region motion vector class (920). That is, the number of the plurality of classes (910, 920) is two. Accordingly, two region motion vectors among the plurality of region motion vectors in a specific divided region (811) are selected and stored in the first region motion vector class (910) and the second region motion vector class (920), respectively.

In addition, the representative region motion vector extraction unit (154) compares the region motion vectors stored in each of the plurality of classes (910, 920) with the region motion vectors to be classified, and stores the region motion vectors to be classified in the class in which the region motion vector most similar to the region motion vector to be classified is stored among the region motion vectors stored in each of the plurality of classes (910, 920). Here, the region motion vectors to be classified are the remaining motion vectors excluding the region motion vectors already stored in the plurality of classes (910, 920). For example, the region motion vectors to be classified included in the classification target motion vector class (930) are compared with the motion vectors stored in the first region motion vector class (910) and the motion vectors stored in the second region motion vector class (920). That is, the items of the classification target motion vector class (930) and the items of each of the first region motion vector class (910) and the second region motion vector class (920) are compared. In other words, the representative region motion vector extraction unit (154) calculates the difference between the horizontal component (MV_x), the vertical component (MV_y), the cost and the weight of the region motion vector to be classified, which is included in the classification target motion vector class (930), and the horizontal component (MV_x), the vertical component (MV_y), the cost and the weight of the first region motion vector class (910) and the second region motion vector class (920), respectively. Here, the region motion vector stored in the class with the smaller calculated difference is most similar to the region motion vector to be classified.

Accordingly, the representative region motion vector extraction unit (154) updates multiple classes by storing the region motion vector to be classified in a class that stores more similar region motion vectors. That is, the region motion vector to be classified is stored in a class that includes a region motion vector with a small calculated difference, and the class is updated by storing the region motion vector to be classified in the class.

Accordingly, the representative region motion vector extraction unit (154) can compare the region motion vector to be classified with other region motion vectors through the class to determine the degree of similarity. In addition, the representative region motion vector extraction unit (154) can update the class by storing the region motion vector to be classified in a class that includes similar region motion vectors.

Referring to FIGS. 8 and 9, the representative region motion vector extraction unit (154) can update the plurality of classes (910, 920) until all region motion vectors in a specific segmented region (811) are stored in the plurality of classes (910, 920). When all region motion vectors in a specific segmented region (811) are stored in the plurality of classes (910, 920), each of the region motion vectors last stored in each of the plurality of classes (910, 920) is determined as the representative region motion vector (991, 992). For example, the region motion vector last stored in the first region motion vector class (910) is determined as the first representative region motion vector (991), and the region motion vector last stored in the second region motion vector class (920) is determined as the second representative region motion vector (992). Here, the first representative area motion vector (991) and the second representative area motion vector (992) are different from each other.

Accordingly, the representative region motion vector extraction unit (154) can easily extract the representative region motion vector (991, 992) by utilizing multiple classes (910, 920). The extracted representative region motion vector (991, 992) is compared with the background motion vector (790), and one of the representative region motion vectors (991, 992) can be determined as a substitute motion vector. A specific method for determining the substitute motion vector will be described later with reference to FIG. 10.

Referring to FIGS. 2 and 3, the motion vector correction unit (155) corrects an inaccurate motion vector with a replacement motion vector determined based on a background motion vector and a plurality of representative area motion vectors (S350).

Specifically, the motion vector correction unit (155) calculates the difference between each of the plurality of representative region motion vectors (991, 992) and the background motion vector (790), and determines the representative region motion vector having the largest difference from the background motion vector (790) among the plurality of representative region motion vectors (991, 992) as the replacement motion vector (1000).

Referring to FIG. 10, the background motion vector (790) is compared with each of a plurality of representative region motion vectors (991, 992). Specifically, the motion vector corrector (155) calculates the difference between the background motion vector (790) and the first representative region motion vector (991), and calculates the difference between the background motion vector (790) and the second representative region motion vector (992). For example, if the background motion vector (790) is a vector of (10,0), the first representative region motion vector (991) is a vector of (8,2), and the second representative region motion vector (992) is a vector of (10,0), then the difference between the background motion vector (790) and the first representative region motion vector (991) is a vector of (2,-2), and the difference between the background motion vector (790) and the second representative region motion vector (992) is a vector of (0,0). Accordingly, since the difference between the background motion vector (790) and the first representative region motion vector (991) is greater than the difference between the background motion vector (790) and the second representative region motion vector (992), the first representative region motion vector (991) is determined as the substitute motion vector (1000). In other words, the substitute motion vector (1000) is a vector having a large difference from the background motion vector (790) and thus has a high probability of being a foreground motion vector. In other words, the inaccurate motion vector is similar to the background motion vector (790) and has a high probability of having a relatively high reliability of the foreground motion vector, and if it is the background motion vector (790), the image quality may be significantly degraded. Accordingly, by correcting the motion vector having a low reliability due to the background motion vector (790) to the substitute motion vector (1000) that is close to the foreground motion vector having a relatively high reliability, the image quality may be improved.

Accordingly, the motion vector correction unit (155) can correct the inaccurate motion vector into an accurate motion vector by changing the inaccurate motion vector extracted by the motion vector reliability judgment unit (152) into a replacement motion vector (1000). Furthermore, by correcting the inaccurate motion vector in the reference image into an accurate motion vector, the accuracy of the motion vector of the reference image is improved, and accordingly, the accuracy of the interpolated image is also improved, so that the image quality can be improved.

FIG. 11 is an exemplary image comparing before and after a motion vector is corrected according to an image processing method according to one embodiment of the present invention.

Referring to Fig. 11, in the image (1110) before correction, the incorrect motion vector (1111) mainly exists at the boundary between the foreground and the background. The incorrect motion vector (1111) is substantially the same as the surrounding background motion vector, and shows a large difference from the foreground motion vector. In other words, the incorrect motion vector (1111) needs to be corrected to a substitute motion vector close to the foreground motion vector.

Accordingly, the post-correction image (1120) shows the result of correcting the incorrect motion vector (1111) based on the replacement motion vector. That is, the post-correction image (1120) includes the corrected motion vector (1121). Here, the corrected motion vector (1121) is very similar to the surrounding foreground motion vector and shows a large difference from the background motion vector. That is, the corrected motion vector (1121) is a result of being changed into a replacement motion vector that is similar to the foreground motion vector and different from the background motion vector according to the image processing method of the present invention. Accordingly, the corrected motion vector (1121) is a motion vector similar to the foreground motion vector, so that the reliability of the motion vector in the post-correction image (1120) is improved, and the image quality is also improved due to the improved reliability of the motion vector.

An organic light emitting display device according to embodiments of the present invention can be described as follows.

An image processing method according to one embodiment of the present invention includes the steps of extracting a plurality of motion vectors from a reference image based on a reference image and a comparison image, determining reliability of the plurality of motion vectors and extracting inaccurate motion vectors having reliability lower than a predetermined criterion, extracting a background motion vector for a predetermined background area from among the plurality of motion vectors, extracting a plurality of representative region motion vectors from among the motion vectors included in each of a plurality of divided areas of the reference image, and correcting the inaccurate motion vectors with replacement motion vectors determined based on the background motion vectors and the plurality of representative region motion vectors. The image processing method according to one embodiment of the present invention can improve image quality by improving the accuracy of motion vectors by replacing inaccurate motion vectors with more accurate motion vectors.

According to another feature of the present invention, the step of determining the reliability of a motion vector may include the step of calculating a cost of the motion vector based on a sum of absolute differences (SAD) between a reference image and a comparison image, the step of calculating a smoothness indicating a similarity between the motion vector and a motion vector adjacent to the motion vector, and the step of determining a motion vector in which the cost is greater than a threshold cost value and the smoothness is greater than a threshold smoothness value as an inaccurate motion vector.

According to another feature of the present invention, the step of extracting a background motion vector may include the step of classifying motion vectors existing in a background area among a plurality of motion vectors into a plurality of classes, and the step of determining a motion vector of a class having a mode value among the plurality of classes as the background motion vector.

According to another feature of the present invention, the step of determining a motion vector as a background motion vector may be a step of comparing a motion vector of a class having a mode with a motion vector of a class having a second highest frequency among a plurality of classes, and determining a motion vector having a mode as the background motion vector if the mode is greater than a predetermined multiple of the second highest frequency.

According to another feature of the present invention, the step of extracting a plurality of representative region motion vectors may include a step of classifying a plurality of region motion vectors included in each of a plurality of divided regions into a plurality of classes and updating the plurality of classes, and a step of determining region motion vectors stored in the updated plurality of classes as the plurality of representative region motion vectors.

According to another feature of the present invention, the step of updating the plurality of classes may include the step of storing, in each of the plurality of classes, each of the region motion vectors selected from the plurality of region motion vectors as many as the number of classes, and the step of comparing the region motion vectors stored in each of the plurality of classes with the region motion vectors to be classified, and storing the region motion vector to be classified in a class in which the region motion vector most similar to the region motion vector to be classified is stored among the region motion vectors stored in each of the plurality of classes.

According to another feature of the present invention, the correcting step may include the step of calculating a difference between each of the plurality of representative region motion vectors and a background motion vector, and the step of determining a representative region motion vector having the largest difference from the background motion vector among the plurality of representative region motion vectors as a replacement motion vector.

A display device according to another embodiment of the present invention includes a display panel, and an image processing unit which processes an image input to the display panel. The image processing unit includes a motion vector extraction unit which extracts a plurality of motion vectors in a reference image based on a reference image and a comparison image, a motion vector reliability determination unit which determines the reliability of the plurality of motion vectors and extracts an inaccurate motion vector having a reliability lower than a predetermined standard, a background motion vector extraction unit which extracts a background motion vector for a predetermined background area from among the plurality of motion vectors, a representative region motion vector extraction unit which extracts a plurality of representative region motion vectors from among the motion vectors included in each of a plurality of divided regions of the reference image, and a motion vector correction unit which corrects an inaccurate motion vector to a substitute motion vector determined based on the background motion vector and the plurality of representative region motion vectors. The display device according to another embodiment of the present invention can selectively replace a motion vector by comparing the background motion vector and the region motion vector and replacing the inaccurate motion vector with a more accurate foreground motion vector.

According to another feature of the present invention, the background region may include an upper background region adjacent to an upper edge rather than a lower edge of the reference image, and side background regions adjacent to each of the two side edges of the reference image.

According to another feature of the present invention, the vertical distance from the upper edge to the background area may be shorter than the horizontal distance from the two side edges to the background area.

According to another feature of the present invention, a motion vector reliability judgment unit may calculate a cost of a motion vector based on a SAD (Sum of Absolute Difference) between a reference image and a comparison image, calculate smoothness indicating a similarity between a motion vector and a motion vector adjacent to the motion vector, and determine a motion vector in which the cost is greater than a threshold cost value and the smoothness is greater than a threshold smoothness value as an inaccurate motion vector.

According to another feature of the present invention, the motion vector correction unit can calculate the difference between each of a plurality of representative region motion vectors and a background motion vector, and determine a representative region motion vector having the largest difference from the background motion vector among the plurality of representative region motion vectors as a replacement motion vector.

Although the embodiments of the present invention have been described in more detail with reference to the attached drawings, the present invention is not necessarily limited to these embodiments, and various modifications may be made within a scope that does not depart from the technical idea of the present invention. Therefore, the embodiments disclosed in the present invention are not intended to limit the technical idea of the present invention but to explain it, and the scope of the technical idea of the present invention is not limited by these embodiments. The protection scope of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

100: Display device
110: Display Panel
120: Gate drive circuit
130: Data drive circuit
140: Timing Control Unit
150: Image processing unit
151: Motion vector extraction unit
152: Motion vector reliability judgment unit
153: Background motion vector extraction unit
154: Representative area motion vector extraction unit
155: Motion vector correction unit
410: Reference image
411: Specific block
415: Set of surrounding blocks
420: Comparison Image
531: Main Critical Cost Value
532: Auxiliary critical cost value
533: Main critical flatness value
534: Auxiliary critical flatness value
541: Excess Cost Area
542: Low cost area
543: Excessive flatness area
544: That flat area
610: Reference image
620: Background area
621: Upper background area
622: Side background area
630: Central Area
710: First Background Motion Vector Class
720: 2nd background motion vector class
730: nth background motion vector class
790: Background motion vector
810: Reference image
811: Specific partition area
910: 1st domain motion vector class
920: Second domain motion vector class
930: Classification target motion vector class
991: 1st representative area motion vector
992: Second representative area motion vector
1000: Alternative motion vector
1110: Image before correction
1111: Incorrect motion vector
1120: Image after correction
1121: Corrected motion vector

Claims (12)

A step of extracting a plurality of motion vectors from a reference image based on a reference image and a comparison image;
A step of judging the reliability of the plurality of motion vectors and extracting inaccurate motion vectors having a reliability lower than a predetermined standard;
A step of extracting a background motion vector for a predetermined background area from among the plurality of motion vectors;
A step of extracting a plurality of representative region motion vectors from among the motion vectors included in each of a plurality of segmented regions of the above reference image; and
A step of correcting the above inaccurate motion vector with a replacement motion vector determined based on the background motion vector and the plurality of representative region motion vectors,
The step of extracting the above multiple representative region motion vectors is:
A step of classifying a plurality of region motion vectors included in each of the plurality of divided regions into a plurality of classes and updating the plurality of classes; and
An image processing method, comprising a step of determining a region motion vector stored in the updated plurality of classes as the plurality of representative region motion vectors. In the first paragraph,
The step of judging the reliability of the above motion vector is:
A step of calculating a cost of the motion vector based on the SAD (Sum of Absolute Difference) between the reference image and the comparison image;
A step of calculating a smoothness indicating a similarity between the above motion vector and a motion vector adjacent to the above motion vector; and
An image processing method, comprising a step of determining a motion vector as the inaccurate motion vector, wherein the cost is greater than a threshold cost value and the flatness is greater than a threshold flatness value. In the first paragraph,
The step of extracting the above background motion vector is:
A step of classifying motion vectors existing in the background area among the plurality of motion vectors into multiple classes; and
An image processing method, comprising a step of determining a motion vector of a class having a mode among the plurality of classes as the background motion vector. In the third paragraph,
The step of determining the background motion vector above is:
An image processing method, comprising: a step of comparing a motion vector of a class having the above mode with a motion vector of a class having the second highest frequency among the plurality of classes, and determining a motion vector having the above mode as the background motion vector if the mode is greater than a predetermined multiple of the second highest frequency. delete In the first paragraph,
The step of updating the above multiple classes is:
A step of storing each of the motion vectors of the regions selected from the plurality of motion vectors of the regions as many as the number of the plurality of classes in each of the plurality of classes; and
An image processing method, comprising the step of comparing a region motion vector to be classified with a region motion vector stored in each of the plurality of classes, and storing the region motion vector to be classified in a class in which a region motion vector most similar to the region motion vector to be classified is stored among the region motion vectors stored in each of the plurality of classes. In the first paragraph,
The above correction steps are:
A step of calculating the difference between each of the plurality of representative region motion vectors and the background motion vector; and
An image processing method, comprising a step of determining a representative region motion vector having the greatest difference from the background motion vector among the plurality of representative region motion vectors as the replacement motion vector. display panel;
Includes an image processing unit that processes an image input to the above display panel,
The above image processing unit,
A motion vector extraction unit that extracts a plurality of motion vectors from a reference image based on a reference image and a comparison image;
A motion vector reliability judgment unit for judging the reliability of the plurality of motion vectors and extracting inaccurate motion vectors having a reliability lower than a predetermined standard;
A background motion vector extraction unit for extracting a background motion vector for a predetermined background area from among the plurality of motion vectors;
A representative region motion vector extraction unit for extracting a plurality of representative region motion vectors from among the motion vectors included in each of a plurality of segmented regions of the above-mentioned reference image; and
Including a motion vector correction unit that corrects the above inaccurate motion vector with a replacement motion vector determined based on the background motion vector and the plurality of representative area motion vectors,
The representative area motion vector extraction unit above is,
A display device that classifies a plurality of region motion vectors included in each of the plurality of divided regions into a plurality of classes, updates the plurality of classes, and determines region motion vectors stored in the updated plurality of classes as the plurality of representative region motion vectors. In Article 8,
The above background area is,
A display device comprising an upper background region adjacent to an upper edge rather than a lower edge of the reference image, and side background regions adjacent to each of the two side edges of the reference image. In Article 9,
A display device, wherein the vertical distance from the upper edge to the background area is shorter than the horizontal distance from the both side edges to the background area. In Article 8,
The above motion vector reliability judgment unit is,
Calculate the cost of the motion vector based on the SAD (Sum of Absolute Difference) between the reference image and the comparison image,
Compute the smoothness, which represents the similarity between the above motion vector and the motion vector adjacent to the above motion vector,
A display device, wherein a motion vector is determined as an incorrect motion vector when the cost is greater than a threshold cost value and the flatness is greater than a threshold flatness value. In Article 8,
The above motion vector correction unit is,
Calculate the difference between each of the plurality of representative region motion vectors and the background motion vector,
A display device that determines a representative region motion vector having the greatest difference from the background motion vector among the plurality of representative region motion vectors as the alternative motion vector.

Images