RU2305377C2 - Method for decreasing distortion of compressed video image and device for realization of the method - Google Patents

Abstract

FIELD: radio engineering, possible use for digital processing of video signals, transferring the image.

SUBSTANCE: in accordance to the invention, the image being processed is divided on blocks with following transformation of each block using discontinuous quantum transformation, result coefficients are quantized and encoded, supporting points are computed and linear interpolation is performed, while before the stage of supporting point selection, one of the supporting points on edge limit of block is selected and a supporting point on opposite limit block is calculated using additional low frequency filters, after that linear interpolation is performed between thus computed supporting points.

EFFECT: improved quality of compressed video image with insignificant CPU resource costs.

2 cl, 4 dwg

Description

The invention relates to the field of radio engineering and can be used for digital processing of video signals transmitting an image.

The limited bandwidth of telecommunication channels impedes high-quality transmission of images, especially dynamic ones. Therefore, when transmitting video signals, they are resorted to their compression due to the removal of repeating or practically unchanging fragments with their subsequent recovery when playing video signals.

The current international standards for the presentation of a video signal (MPEG-1, MPEG-2, MPEG-4, H-261, H-263, etc.) do not regulate the methods of video compression, but only determines what the bitstream of the encoded video signal should look like, therefore, specific Algorithms are the intellectual property of equipment manufacturers. For example, Siemens AG patented the "Method for decoding compressed video data with reduced memory requirements" (published application-analogue RU 97104164) [1]. Also known is the American development of the “Method of low noise encoding and decoding” (Russian patent analogue RU 2201654) [2], in which the bandwidth requirements of the digital video decoder are reduced due to the fact that the standard MPEG2 encoder is preceded by an adaptive digital image processor that adaptively filters low frequency signal. Japanese experts proposed the “Image Quality Correction Scheme” described in the Russian application analogue RU 2000133250 [3], which implements a circuit containing a counter for the frequency of occurrence of the brightness level, a linear interpolator, and an image quality corrector. Known domestic development of methods for digital processing of video signals, in which the compression of video signals that transmit a dynamic image. RU 2131172 describes an “Interpolation method for compressing a television signal” [4], the essence of which is the artificial exclusion of line signals and their reconstruction by interpolation from fragments of non-excluded parts of lines, which reduces the redundancy of the television signal.

It is known that during the development of international standards the general principles for processing video signals and their compression were determined (see ISO-11172 and ISO-13818, parts 1, 2, 3, November 1994) [5]. In particular, it was found that the process of compressing a digital video signal can be divided into a number of sequential operations: converting an analog signal to digital, preprocessing, discrete cosine transform, quantization, coding.

The main idea of the MPEG standard is that only the selected (reference) frames are completely transmitted from the entire stream of video frames of the image, while for the rest, only their changes with respect to the reference ones are transmitted.

In fact, in a moving image from frame to frame in most cases only part of it changes. For example, when a speaker speaks in the news, only his facial expressions change. A complete frame change, when the next frame cannot be restored as a change to the previous one (in this case it is easier to transfer the frame itself), is relatively rare. For example, in American films it is usually 4-5 seconds, in European (and especially in domestic) this interval is much larger.

For this reason, MPEG-2 defines three types of frames:

I-frames (intra frames); P-frames (predicted frames); B-frames (bidirectional frames).

I-frames carry a full-fledged still image and in addition are used to build P- and B-frames. P-frames, that is, "predictable", are built on the basis of the last (from the receiver's point of view) received I- or P-frame. True, if it is very different from it (for example, a plan change occurred), then the P-frame is encoded as an I-frame.

B frames or “interpolated” are most difficult to recover. Such a frame can be constructed either as a continuation of the previous I (P) -frame, or as a predecessor of the next I (P) -frame, or as an interpolation between both. Moreover, if the B-frame is significantly different from the first and second, then it is encoded as an I-frame.

All types of frames are grouped in a certain sequence. A group of 12 frames forms the so-called GOP (Group of Pictures). Thus, at a frequency of 25 frames per second, a new I-frame arrives at a maximum of 12 × (1/25) = 0.48 seconds. Together with it, the full (in a sense) identity of the transmitted and received images is restored). Due to the fact that when decoding to obtain B-frames it is necessary to have the next P-frame, then during transmission the sequence of frames must be strictly defined.

There is also a special procedure for encoding individual frames. For example, to encode a color image, the YUV scheme used in conventional television broadcasting is used. In this case, the image is displayed not in three color channels (RGB scheme), but in two color channels (U, V) and in the luminance channel (Y).

An image in a luminance channel is essentially a black and white image. It is noted that one of the features of the perception of the image by the human eye is that it has a higher resolution in the luminance channel (Y) than in the color channels (U, V). Therefore, layering a color frame into these three components, we can subject the layers U and V to more compression than the layer Y. This principle was used even when creating color analog television, where U, V are transmitted not simultaneously, but alternately.

The I-frame is encoded as a static image as follows. Each layer of the frame is divided into blocks of 8 × 8 pixels and is subjected to a discrete cosine transform (DCT). DCT is a fully reversible transformation. In essence, DCT is a special case of the Fourier transform for an even function, when the function decomposes only into cosine harmonics.

When using DCT, instead of the pixel value (that is, the level of color and brightness), the DCT coefficient is set in the block cell. That is, the block is converted into its two-dimensional spectrum. As a rule, the energy spectrum of the image is concentrated in the low-frequency harmonics, so the coefficients located closer to the upper left corner are larger than the others. The smaller the neighboring pixels differ from each other in the source block, the closer to zero the values of most DCT coefficients.

For pixels of a monotonous image, the DCT coefficients are zero, except for the coefficient in the upper left corner, which sets the image intensity.

The resulting coefficients are quantized (i.e., rounded to a certain degree 2). The main task in this case is to increase the number of zero coefficients. In fact, high-frequency harmonics are discarded. As experience shows, usually this practically does not affect image quality.

The resulting set of binary vectors (coefficients) is compressed by the well-known Huffman code. Thus, a compressed I-frame is formed, which, with a known loss of quality, can be restored independently of other frames. P- and B-frames are encoded taking into account their difference from the reference I- and P-frames. Therefore, they are more compressible than I-frames. When encoding a P-frame (B-frames are encoded in almost the same way), it is also divided into 8x8 blocks and compared with the original frame (we assume that this is an I-frame, although there may be a previous P-frame). If some block in the encoded P-frame coincides with a similar block in the reference frame, then it is sufficient to indicate that it is the same. Another case is finding the exact same block in the reference I-frame, but in a different position, so instead of a P-frame block, you can only specify a link to another I-frame block in the form of an offset vector. The remaining blocks are encoded in the same way as in the case of the I-frame.

Note that if in a moving image some of the objects will move forward (and this happens often), then several blocks will be encoded with the same displacement vector. With subsequent compression using the Huffman method, this will give an additional increase in the degree of compression of the P-frame. On the other hand, with an increase in the compression ratio, most of the coefficients become equal to zero as a result of quantization and, thus, the block is encoded either only by the average brightness value or by a small number of low-frequency DCT coefficients. Since the blocks do not overlap, due to the difference in the average values of the brightness of the neighboring blocks, a clearly noticeable difference in brightness occurs between them, and the difference in brightness, as a rule, as a whole forms a regular cellular structure in the decoded image. If motion compensation was used in video compression, the block structure can change over time, losing the regularity of the blocks, due to the fact that blocks from the reference frames can be shifted in the predicted frames.

Each of the above developments [1-5] allows us to solve only some aspects of the complex problem “recording / playback of high-quality video”, while any, even slight, improvement in the quality of compressed video requires significant computational costs, which impedes the widespread adoption of these developments. Particularly difficult is the elimination of distortion of the dynamic image in the form of a cellular structure.

It seems that the combination of low-pass filtering with linear or bilinear interpolation can provide a significant improvement in compressed dynamic video at a fairly modest computational cost. The advantage of linear interpolation is the simplicity of the calculations, namely: it can be calculated using the half-sum operation or using pre-calculated tables. The disadvantage of linear interpolation is that it less efficiently eliminates block distortion compared to a low-pass filter, or compared to interpolation by higher order polynomials.

Closest to the proposed is the method described in the thesis of Joceli Mayer, "Blending Models for Image Enhancement and Coding", Ph.D. Thesis, University of California, Santa Cruz, Advisor: Prof. Glen G. Langdon, Ph.D., December 1999 [6]; which provides for the use of second-order interpolation (in particular, Bezier polynomials). However, the use of interpolating polynomials of more than first order requires significantly greater computational costs than linear interpolation (i.e., interpolation by a first order polynomial).

The present invention solves the problem of improving the quality of compressed video at a low computational cost.

To achieve the named technical result in the proposed method, which includes splitting the processed image into blocks with the subsequent transformation of each block using DCT, with quantization and coding of the resulting coefficients, the calculation of the correction characteristic points, usually called reference points, using low-pass filtering and then linear interpolation between reference points calculated in this way. This procedure somewhat resembles the choice of a reference frame in the MPEG standard. Moreover, since the number of interpolation reference points is less than the number of processed image points, the total computational complexity caused by the introduction of low-pass filtering increases slightly.

Distinctive features of the proposed method is the procedure for selecting control points using low-pass filtering and replacing the processing of image points by processing a significantly smaller number of reference points calculated in this way. In this case, the released computing power is directed to the implementation of one of the types of interpolation — by rows, columns, or even bilinear interpolation, which requires the greatest computational cost.

The proposed method is illustrated by drawings, which show:

Figure 1 - diagram of the division of the processed image into blocks and the processing order of the blocks.

Figure 2 - diagram of the use of linear interpolation to process the lines of the image block.

Figure 3 - diagram of the use of bilinear interpolation to process the image block.

Figure 4 is a functional diagram of a device that implements the present invention.

Figure 1 shows the scheme of dividing the processed image 1 into processing blocks 3 and the order 4 of processing blocks 3. During processing, the entire image 1 is divided into square blocks 3 with a size smaller than the size of the DCT block 2 used in the encoding. It is most advantageous to use block 3 with linear dimensions half the size of block 2 DCT (i.e., a 4 × 4 block with a block size of 2 DCT 8 × 8). Blocks 3 are processed sequentially, and the processed block 3 is replaced by the result of the calculations. Although the processing sequence 4 does not matter, everywhere in the future it is assumed that the processing sequence 4 of blocks 3 is from left to right, top to bottom.

Figure 2 shows a diagram of the use of linear interpolation to reduce block distortion on the row of the processed block 3. It is assumed that the block distortion on the columns are absent or negligible. The left reference point 7 is selected on the extreme right boundary of the left (possibly processed in the previous step) block 3. The right reference point 7 is calculated using low-pass filtering of pixels lying on both sides of the boundary of block 3. From the point of view of computational efficiency, it is advisable as a low-pass filter use a filter with a length of 2 and weights {0.5; 0.5}, or in other words, to calculate the half-sum of the brightness of pixels on both sides of the boundary of block 3. After calculating the left and right reference points 7, the pixel values between them are replaced by the values calculated by the linear interpolation formula:

Pixel [i] = L + (R-L) / n · i i = [1, n];

Where L is the value of the left reference point, P is the value of the right reference point, n is the number of interpolated points, i is the number of the interpolated image element.

This procedure is performed for each row of the processed block 3.

The reduction of block distortion on the image columns is carried out similarly, while it is assumed that the distortion on the rows is absent or insignificant.

Figure 3 shows a diagram of the use of bilinear interpolator 8 to reduce block distortion simultaneously in the rows and columns of the image block.

In the case of bilinear interpolation, the values of the four reference points are calculated. For this, low-pass filters 5 are also used, which are connected to the inputs of the bilinear interpolator 8. It is preferable to use low-pass filters 5 with the following coefficients:

Filter Left upper point coefficient Coefficient of the upper right point Left bottom point ratio Right bottom point ratio Lowpass filter 5 ₁ 0 0.5 0.5 0 Lowpass filter 5 ₂ 0.5 0 0.25 0.25 Lowpass filter 5 ₃ 0.5 0.25 0 0.25 Low pass filter 5 ₄ 0.5 0.25 0.25 0

With this choice of coefficients, the points processed in the previous step have large weights.

Figure 4 shows a functional diagram of a device that implements the proposed method.

The device consists of a block classifier 10, which analyzes blocks 3 and detects the presence and type of block distortion, and a set of low-pass filters 5 and interpolators 11-13 that process the received data. In the absence of block distortion in the processed block 3, interpolation is not applied. In the presence of block distortion, block 3 is processed depending on the type of block distortion with one of three interpolators: a linear interpolator 11 in rows, a linear interpolator 12 in columns, or a bilinear interpolator 13. The type of block classifier 10 is selected, for example, based on the following requirement:

- the presence of block distortions in rows (columns) is fixed if all rows (columns) of the processed block 3 contain no more than one brightness difference and the absolute value of this difference does not exceed the doubled step of quantization of DCT coefficients for this block 3.

In fact, the data from the processed block 3 located in the image buffer 9 is fed to the input of the classifier 10 and to the switch 14, which, like the switch 15, responds to the commands of the classifier 10. In the absence of block distortion, the switches 14 and 15 are set to the highest position ( see figure 4), while the signals from the processed block 3 bypass the filter block 5 and the block of interpolators 11-13. This means that block distortion correction is not required in this case. If the classifier 10 detects the presence of block distortion, then, depending on the type of distortion detected, the switches 14 and 15 activate the corresponding ruler: row, column interpolation or bilinear interpolation.

Claims (2)

1. A method of reducing the distortion of a compressed video image, including dividing the processed image into blocks and then converting each block using DCT, quantizing and coding the resulting coefficients, calculating reference points and performing linear interpolation, characterized in that before the step of calculating reference points, one of reference points at the extreme boundary of the block and the reference point on the opposite boundary of the block is calculated using additional low-pass filters, after which linear interpolation is carried out between the reference points calculated in this way. 2. A device for reducing distortion of a compressed video image, consisting of a block classifier that detects the type of block distortion on the processed block, and three interpolators, one of which performs linear interpolation by rows, the second performs linear interpolation by columns, the third performs bilinear interpolation by rows and columns characterized in that a low-pass filter is installed at the input of each interpolator and the classifier selects and switches on a specific interpolator in a dependent aws from the identified type of block distortion.

Images