RU2336661C2 - Method and device of adaptive choice of context model for entropy encoding - Google Patents

Abstract

FIELD: physics, video technics.

SUBSTANCE: invention concerns video signal encoding/decoding, particularly adaptive choice of context model for entropy encoding, and video decoder. The invention claims a method of residual prediction flag encoding, indicating prediction of residual data for enhanced layer unit of multilayer video signal, based on residual data from bottom layer unit correlating to residual data from enhanced layer unit. The method involves stages of residual data energy calculation for bottom layer unit, determination of residual prediction flag encoding method according to the energy, and residual prediction flag encoding by the determined encoding method.

EFFECT: method and device for efficient flag compression.

66 cl, 17 dwg

Description

FIELD OF THE INVENTION

Methods and devices compatible with the present invention relate to encoding and decoding a video signal and, more specifically, to adaptive selection of a context model for entropy encoding and a video decoder.

BACKGROUND

With the development of information communication technology, including the Internet, multimedia services are growing, containing various types of information, such as text, video, audio, etc. Multimedia data requires a large storage medium capacity and a wide bandwidth for transmission, since the volume of multimedia data is usually large. Therefore, a compression encoding method is mandatory for transmitting multimedia data, including text, video and audio.

The basic principle of data compression is to eliminate data redundancy. Data can be compressed by eliminating spatial redundancy, in which one color or object is repeated in the image, temporal redundancy, which provides for a slight change between adjacent frames in a moving image, or the same sound is repeated in audio or mental visual redundancy, which takes into account human vision and its limited perception of high frequency. In general video coding, temporal redundancy is eliminated by motion compensation based on motion estimation and compensation, and temporal redundancy is removed by transform coding.

In order to transmit multimedia generated after eliminating data redundancy, a transmission medium is required. Transmission performance varies depending on the transmission medium. The transmission medium currently used has various transmission rates. For example, an ultra-high-speed communication network can transmit data of several tens of megabits per second, while a mobile communication network has a transmission speed of 384 kilobits per second. Therefore, in order to support a transmission medium having different speeds, or to transmit multimedia at a data transfer rate suitable for the transmission medium, data encoding methods having scaling such as short-wave video encoding and video band encoding may be suitable for multimedia environments .

Scaled video coding is a technique that enables a compressed bitstream to be decoded at various resolutions, frame rates, and signal-to-noise ratio (SNR) by cutting off part of the bitstream according to external conditions, such as bit rate, frequency bit errors and system resources. The standardization of the Motion Picture 4 Expert Group (MPEG-4) Part 10 for scaled video coding is under development. In particular, significant efforts are being made to realize scalability based on a multilayer structure. For example, a bitstream may consist of several layers, i.e. base layer and the first and second advanced layers with different resolutions (QCIF, CIF and 2CIF) or frame rates.

Similarly, when a video is encoded in one layer, when a video is encoded in several layers, a motion vector (MV) is obtained for each of several layers in order to eliminate temporary redundancy. The motion vector search MV can be performed separately for each layer (in the first case), or the motion vector obtained by searching for the motion vector for one layer is used for another layer (without or after up / down sampling) (in the second case). In the second case, however, despite the advantages obtained from the exact motion vectors, there are losses due to the motion vectors generated for each layer. Thus, effectively eliminating redundancy between the motion vectors for each layer is a very difficult task.

Figure 1 shows an example of a scaled video codec using a multilayer structure. Referring to FIG. 1, the base layer has a resolution of a quarter of the common intermediate format (QCIF) and a frame rate of 15 Hz, the first enhanced layer has a resolution of the common intermediate format (CIF) and a frame rate of 30 Hz, and the second improved layer has standard resolution (SD) and a frame rate of 60 Hz. For example, in order to obtain a 0.5 Mbps CIF stream, the bit stream of the first enhanced layer (CIF_30Hz_0.7M) is truncated in order to correspond to a target transfer rate of 0.5 Mbps. Thus, spatio-temporal scalability and scalability of the signal-to-noise ratio (SNR) can be provided.

As shown in figure 1, frames (for example, 10, 20 and 30) at the same time position in each layer can be considered as similar images. One well-known coding technique involves predicting the texture of the current layer from the texture of the lower layer (immediately or after upsampling) and coding the difference between the predicted value and the actual texture of the current layer. This technique is defined as Intra_BL prediction in the scaled video 3.0 model of the scaled video encoding standard ISO / IEC 21000-13 ("SVM 3.0").

SVM 3.0 uses a technique to predict the current block by correlating between the current block and the corresponding block in the lower layer, in addition to the directional Intra prediction and Inter prediction used in the traditional H.264 standard to predict blocks or macroblocks in the current frame. The prediction method is called "Intra_BL prediction", and the encoding mode using the Intra_BL prediction is called the "Intra_BL mode".

Figure 2 is a schematic diagram for describing the above three prediction methods: 1) Intra prediction for macroblock 14 in the current frame 11; 2) Inter prediction using frame 12 at a different time position than the current frame 11; and 3) predicting Intra_BL using texture data from region 16 in frame 13 of the base layer corresponding to macroblock 14.

The scaled video coding standard selects an advantageous method from among three prediction methods for each macroblock.

To provide information about the selected prediction method or the data used for the selected prediction method, a plurality of flags may be used by the decoder. One bit, several bits, and several tens of bits can be used as flags, depending on whether encoding is performed on macroblocks, on a series of consecutive macroblocks, or on frames. The data size increases when flags are set for each macroblock, a series of consecutive macroblocks, or a frame in a common moving image.

Therefore, there is a need for a method and apparatus for efficiently compressing flags.

SUMMARY OF THE INVENTION

The present invention provides a method and apparatus for reducing the amount of data required for prediction using base layer data.

The above aspect, as well as other aspects, features and advantages of the present invention, will become apparent to those skilled in the art upon review of the following description.

According to an aspect of the present invention, there is provided a method for encoding a residual prediction flag indicating whether residual data is predicted for an enhanced layer layer of a multilayer video signal from residual data of a lower layer block corresponding to residual data for an enhanced layer block, the method comprising the steps of calculating the residual energy block of the lower layer, determine the method of encoding the residual prediction flag according to energy and encode the residual prediction flag with the power of a particular coding method.

According to another aspect of the present invention, there is provided a method for encoding a residual prediction flag indicating whether residual data is predicted for an enhanced layer layer of a multilayer video signal from residual data of a lower layer block corresponding to residual data for an enhanced layer block, the method comprising the steps of calculating an encoded pattern blocks (CBP) of the lower layer block, determine the method for encoding the residual prediction flag according to the calculated CBP value and encode the flag sufficiently prediction using a specific encoding method.

According to yet another aspect of the present invention, there is provided a method for decoding a residual prediction flag indicating whether residual data for an enhanced layer block of a multilayer video signal is predicted from residual data of a lower layer block corresponding to residual data for an enhanced layer block, the method comprising the steps of calculating energy the residual data of the lower layer block, determine the method for decoding the residual prediction flag according to the energy, and decode the residual flag predictions using a specific decoding method.

According to yet another aspect of the present invention, there is provided a method for decoding a residual prediction flag indicating whether residual data for an enhanced layer layer of a multi-layer video signal is predicted from residual data of a lower layer block corresponding to residual data for an enhanced layer block, the method comprising the steps of calculating a value CBP block of the lower layer, determine the method for decoding the residual prediction flag according to the calculated CBP value and decode the residual flag o predictions using a specific decoding method.

According to a further aspect of the present invention, there is provided a method for decoding a residual prediction flag indicating whether residual data for an enhanced layer block of a multi-layer video signal is predicted from residual data of a lower layer block corresponding to residual data for an enhanced layer block, the method comprising the steps of checking the CBP value in the block of the lower layer, determine whether or not each bit CBP in the block of the lower layer has a value of 1, determine the method of decoding the residual flag prediction according to whether or not the at least one bit CBP in the lower layer block 1, and decoding the residual prediction flag according to the determined decoding method.

According to yet a further aspect of the present invention, there is provided a method for decoding an enhanced layer of a multilayer video signal, the method comprising the steps of determining a method for decoding a residual prediction flag indicating whether the improved layer is predicted from the lower layer using the CBP value of the lower layer and decoding the residual flag predictions using the entropy decoding method selected according to the decoding method, and decode the enhanced layer using the flag but a residual prediction.

According to yet another aspect of the present invention, there is provided a residual prediction flag decoder indicating whether residual data for an enhanced layer layer of a multilayer video signal is predicted from residual data of a lower layer block corresponding to residual data for an enhanced layer block, wherein the decoder comprises an analyzer that analyzes the CBP value a lower layer unit, and a decoding unit that determines a method for decoding the residual prediction flag according to the CBP value and decodes the residual prediction flag using a decoding method specified.

According to a still further aspect of the present invention, there is provided a decoder for decoding an enhanced layer of a multilayer video signal, the decoder comprising an analyzer that analyzes the CBP value for a lower layer than the enhanced layer, an entropy decoding unit that determines a method for decoding a residual prediction flag indicating whether the improved layer from the lower layer is predicted using the CBP value, and performs entropy decoding of the residual prediction flag according to benefits and then decoding the enhancement layer, an inverse quantizer which inversely quantizes the enhanced layer and the inverse transformer which performs inverse transform inversely quantized enhancement layer and performs decoding result using prediction from the lower layer according to the residual prediction flag.

According to another aspect of the present invention, there is provided an encoding method performed by a multilayer video encoder, the encoding method comprising the steps of obtaining a base layer frame from an input frame, obtaining data for an enhanced layer from an input frame by referencing a base layer frame, and encoding data for an enhanced layer according to the result of determining whether the data for the enhanced layer should be predicted from the data for the base layer frame.

According to yet another aspect of the present invention, there is provided a decoding method performed by a multilayer video decoder, the decoding method comprising the steps of decoding an input frame of a base layer, determining whether data is needed for a frame of the base layer in order to predict enhanced layer data, and decode data for the enhanced layer according to the result of the determination.

According to still another aspect of the present invention, there is provided a video encoder comprising a base layer encoder that receives a base layer frame from an input frame, an enhanced layer encoder that generates data for an enhanced layer predicted from a base layer frame from an input frame, wherein the enhanced layer encoder comprises a converter that encodes the enhanced layer data according to the result of determining whether the enhanced layer data is predictable from the data of the base layer frame.

According to a further aspect of the present invention, there is provided a video decoder comprising a base layer decoder that decodes an input frame of a base layer, an enhanced layer decoder that decodes enhanced layer data by reference to a base layer frame, wherein the enhanced layer decoder comprises an inverter that decodes the enhanced layer data according to the result of determining whether data of a decoded frame of the base layer is required in order to predict the data of the enhanced layer, predicted from the frame of the base layer.

BRIEF DESCRIPTION OF THE DRAWINGS

The above and other features and advantages of the present invention will become more apparent through a detailed description of exemplary embodiments with reference to the accompanying drawings, of which:

Figure 1 illustrates an example of a scaled video codec using a multilayer structure;

2 is a diagram for describing Inter prediction, Intra prediction, Intra_BL prediction;

3 is a diagram showing an example of residual prediction in video encoding;

FIG. 4 is a flowchart illustrating a method for improving coding efficiency of a residual prediction flag according to an exemplary embodiment of the present invention; FIG.

FIG. 5 is a flowchart illustrating a method for decoding data encoded using the method illustrated in FIG. 4, according to an exemplary embodiment of the present invention;

6 is a diagram showing an example of motion prediction in video encoding according to an exemplary embodiment of the present invention;

7 is a flowchart illustrating an exemplary method for improving the coding efficiency of a motion prediction flag;

FIG. 8 is a flowchart illustrating a method of decoding data encoded using the method of FIG. 7 according to an exemplary embodiment of the present invention;

9 is a flowchart illustrating a process for encoding inverse residual prediction data according to an exemplary embodiment of the present invention;

FIG. 10 is a flowchart illustrating an example decoding process of data encoded using the process illustrated in FIG. 9;

11 is a flowchart illustrating a process for encoding a residual prediction flag using a modified context according to an exemplary embodiment of the present invention;

12 is a flowchart illustrating an exemplary process for decoding data encoded using the process illustrated in FIG. eleven;

13 is a flowchart illustrating an example in which the encoding mode varies depending on the residual data of the base layer;

14 is a flowchart illustrating a coding process using various context models when context adaptive binary arithmetic coding (CABAC) is performed according to an exemplary embodiment of the present invention;

FIG. 15 is a flowchart illustrating an example decoding process of video data encoded using the process illustrated in FIG. 14;

FIG. 16 is a block diagram of a video encoder according to an exemplary embodiment of the present invention; FIG. and

17 is a block diagram of a video decoder according to an exemplary embodiment of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention will now be described in more detail with reference to the accompanying drawings, in which exemplary embodiments of the invention are shown.

The advantages and features of the present invention and methods for its implementation can be more clearly understood by reference to the following detailed description of exemplary embodiments and the accompanying drawings. However, the present invention can be practiced in many different forms and should not be construed as limited by the exemplary embodiments set forth herein. Instead, these exemplary embodiments are provided so that this entity is exhaustive and complete, they fully express the idea of the invention to those skilled in the art, and the present invention is fully defined by the appended claims. Similar reference numbers mean similar elements throughout the specification.

The present invention will now be described with reference to flowcharts of methods of exemplary embodiments of the invention. It should be understood that each step in the illustrations of flowcharts and combinations of steps in the illustrations of flowcharts may be implemented by computational software instructions. These computing program instructions may be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing device in order to generate a machine so that instructions executed by a computer processor or other programmable data processing device provide means the implementation of the functions indicated in the step or steps of the flowchart.

These computational program instructions can also be stored in a computer-useable or machine-readable memory that can control a computer or other programmable data processing device for its operation in a certain way so that instructions stored in a computer-useable or machine-readable memory generate a production product containing means instruction, which implements the function indicated in the step or steps of the flowchart.

Computational program instructions can also be downloaded to a computer or other programmable data processing device to initiate the execution of a sequence of operational steps on a computer or other programmable device in order to generate a computer-executable process so that instructions that are executed on a computer or other programmable device , provided the stages of the implementation of the functions specified in the stage or stages of the flowchart of the sequence operations.

Each step of the illustrations of the flowchart may represent a module, segment, or piece of code that contains one or more executable instructions for implementing the indicated logical functions. It should also be noted that in some alternative implementations, the functions indicated in the steps may be performed in a different sequence. For example, two steps, shown one after another, can actually be executed almost simultaneously, or blocks can sometimes be executed in reverse order, depending on their functionality.

Information regarding the prediction methods shown in FIG. 2, i.e. data related to one selected method from the prediction methods shown in FIG. 2, or information regarding the reference data is set by the encoder and transmitted in order to be used by the decoder. Entropy encoding is carried out in order to finally and losslessly compress the data. Huffman coding is typically used as a method of entropy coding. Huffman coding is a form of coding along the length of the parameters, and it depends on the location of the data bits based on the data probabilities. Thus, in order to increase the overall bit efficiency, the data presentation method can be modified.

One way to notify the decoder of a prediction method selected from the prediction methods illustrated in FIG. 2 is to use a prediction flag that indicates that the selected method refers to base layer information, such as residual data or motion vectors of the base layer. As a prediction flag, a residual prediction flag or a motion prediction flag can be used to inform the decoder whether there were links to information of the base layer. Since prediction flags are always set for each macroblock, each series of consecutive macroblocks, or each frame, it is important to reduce the size of the flags or increase compression efficiency during encoding, such as entropy encoding. In order to achieve this goal, information is set for the decoder to predict the prediction flags and restore the prediction flags using the specified information.

Figure 3 illustrates an example of residual prediction in video coding. The residual prediction refers to the prediction performed with the residual data predicted using one of the selected prediction methods illustrated in FIG. 2. A macroblock (a series of consecutive macroblocks or a frame) 14 in the base layer may be composed of one of the selected prediction methods illustrated in FIG. 2, for example, Inter temporal prediction using residual data. In this case, residual prediction can also be performed for a macroblock, a series of consecutive macroblocks, or a frame in an enhanced layer using the residual data of the base layer. Although the invention is described below with respect to a macroblock, it is not limited to it and can be applied to a series of consecutive macroblocks or a frame.

In order for the macroblock in the enhanced layer to implement Inter temporal prediction, the remainder of the base layer may be mentioned. To notify the decoder that the remainder of the base layer is referred to in the Inter temporal prediction Inter, the residual prediction flag (residual prediction flag) is assigned the value 1. If all macroblock values in the base layer (i.e., macroblocks containing residual data) are set to zero, if the number of non-zero pixels is less than a predetermined threshold or the CBP value is zero or less than a predetermined threshold, it is not necessary to set the value of the residual prediction flag. This means that there is little movement as a result of temporal prediction in the base layer. Those. since there is no or insufficient reference data for the macroblock in the enhanced layer to be referenced, the encoding of the residual prediction flag is skipped, thereby preserving a certain number of bits required to set the value of the residual prediction flag.

The residual prediction makes sense when the motion vectors in the enhanced layer are similar to the vectors in the base layer. Thus, obtaining the difference between the motion vectors for two layers can be a fundamental issue in predicting the value of the residual prediction flag. For brevity, determination information for predicting the value of the residual prediction flag is hereinafter referred to as a “prediction determination flag”. In the general scaled video model (JSVM) 1, improved entropy coding efficiency can be achieved by coding the difference between the residual prediction flag and the prediction determination flag, designated as PrdRpFlag, instead of setting the residual prediction flag. Therefore, this residual prediction difference flag, designated as residual_prediction_flag_diff, is entered in order to encode the video information.

The following describes a method for obtaining a residual prediction difference flag.

First, if the number of non-zero pixels present in the remainder of the base layer is less than a threshold, the encoding of the residual prediction flag is skipped.

Conversely, if the number of non-zero pixels is greater than the threshold, the residual prediction difference flag is encoded, and not the residual prediction flag. To obtain the residual prediction difference flag, the remainder is used between the motion vector of the BaseMV base layer and the motion vector of the enhanced or current CurrMV layer. In order to encode the residual prediction difference flag, a determination flag must be obtained.

The above process can be summarized as follows.

If | BaseMV-CurrMV | less than a predetermined threshold, a value of 1 is assigned to the definition flag; otherwise, it is assigned a value of 0. Then, the residual prediction difference flag, which is the difference between the calculated determination flag and the residual difference flag, is encoded.

If the energy of the remainder of the base layer (or the number of nonzero pixels) is less than the Threshold _residual , the encoding of the residual prediction flag is skipped.

If | BaseMV-CurrMV | less than the threshold Threshold _MV , a value of one minus the residual prediction flag is encoded, i.e. 1 - residual_prediction_flag; otherwise, a value of zero minus the residual prediction flag is encoded, i.e. 0 - residual_prediction_flag, which is the same as the residual prediction flag.

4 is a flowchart illustrating a process for implementing the above concept. Referring to FIG. 4, in step S101, the residual data of the base layer is obtained by, for example, temporary intra-coding by referring to another frame or block. If the residual data of the base layer is less than a predetermined threshold, i.e. Threshold _residual , in step S105, for example, if the base residual data is 0 or all the energy is less than a predetermined threshold, the encoding of the residual reservation flag is skipped because it does not need to be set. Conversely, if the residual data of the base layer is greater than the Threshold _residual in step S105, the residual prediction flag is selectively 1 or 0. The basis for selecting the prediction determination flag, designated as PrdRpFlag, can be set.

If the difference between BaseMV and CurrMV is less than a predetermined threshold, i.e. Threshold _MV in step S110, which means that the probability of using the residual data of the base layer is high, the prediction determination flag is set to 1 in step S111. Conversely, if the difference between BaseMV and CurrMV is greater than ThresholdMv in step S110, the probability of using the residual data of the base layer is small. Thus, the prediction determination flag is set to 0 in step S112. In step S120, the prediction determination flag set in step S111 or S112 is encoded and the residual prediction difference flag corresponding to the difference between the residual prediction flag and the prediction determination flag. The encoder may encode the prediction determination flag and the residual prediction difference flag for macroblocks, for series of consecutive macroblocks, or for frames.

FIG. 5 is a flowchart illustrating a process of decoding data encoded using the method illustrated in FIG. 4, according to an exemplary embodiment of the present invention.

First, in step S151, the residual data of the base layer is obtained from the encoded data. On the one hand, if the residual data is not less than a predetermined threshold, i.e. Threshold _residual in step S155 (NO in step S155), the residual prediction flag is assigned a difference value between the encoded prediction determination flag and the residual prediction difference flag in step S161. As specified in the coding procedure illustrated in FIG. 4, the prediction determination flag and the residual prediction difference flag are designated as PrdRpFlag and residual_pred_flag_diff, respectively. In this case, residual_pred_flag_diff = PrdRpFlag - residual_prediction_flag. Thus, residual_prediction_flag can be obtained by subtracting residual_pred_flag_diff from PrdRpFlag.

On the other hand, if the residual data is less than the Threshold _residual in step S155 (YES in step S155), which means that the enhanced layer is not generated by referring to the residual prediction flag, the residual prediction flag is set to 0 in step S162.

In the above description, information other than a residual prediction flag is encoded indicating whether references to residual data of the base layer are contained, thereby increasing the encoding efficiency. Similarly, other information may be encoded instead of encoding a motion prediction flag that indicates whether or not motion information of the base layer is used to predict the motion vector in order to achieve the same goal.

6 is a diagram showing an example of motion prediction in video encoding according to an exemplary embodiment of the present invention. Motion prediction refers to the prediction of the motion vector of the enhanced layer or the current layer from the motion vector of the base layer. Thus, if the motion prediction flag designated as motion_prediction_flag is set to 1, the motion vector of the enhanced layer is predicted by reference to the motion vector of the base layer. Otherwise, if the motion prediction flag is set to 0, the motion vector of the base layer is not indicated by links. 21 or 25 in FIG. 6 denotes a macroblock (or subunit), a series of consecutive macroblocks, or a frame. For convenience of description, it is assumed that motion prediction is performed on macroblocks. The motion vector of the macroblock 21 of the base layer is the same as the motion vector of the macroblock 25 of the enhanced layer. In this case, the encoding of the motion prediction flag is not required. The encoding of the motion prediction flag may also be skipped if the remainder between the two motion vectors is less than a predetermined threshold, i.e. Threshold _Motion .

At the same time, the motion prediction flag can be determined by comparing the motion vector of the base layer with the motion vectors from spatially neighboring regions. Spatially adjacent areas can provide accurate motion vectors; however, in some cases, motion prediction may not be accurate. This can lead to a large remainder of the motion vector. Although the accuracy of motion prediction from the motion vector of the base layer is less than spatial motion prediction, the former provides acceptable accuracy in almost all cases. Thus, the remainder between two motion vectors can be used to predict a motion prediction flag.

If the remainder between the motion vectors from macroblocks 22 and 26 is greater than a predetermined threshold, i.e. Threshold _MV , the prediction determination flag prediction_determination_flag, is set to 1 because there is a high probability that the motion prediction flag is set to 1. On the other hand, if the remainder between the motion vectors from macroblocks 22 and 26 is less than Threshold _MV , the prediction determination flag is set to 0 since there is a high probability that the motion prediction flag is set to 0.

After the prediction determination flag value is set, the motion prediction difference flag flag, designated as motion_pred_flag_diff, corresponding to the difference between the prediction determination flag and the motion prediction flag, is encoded.

7 is a flowchart illustrating a method for improving the coding efficiency of a motion prediction flag.

Referring to FIG. 7, a motion vector predicted from a base layer designated as Predict_MV_From_BaseLayer and a motion vector predicted from a spatially adjacent area designated as Predict_MV_From_Spatia are obtained in step S201. In step S205, on the one hand, if the difference between the two motion vectors is less than a predetermined threshold, i.e. Threshold _Motion , the motion prediction flag is not encoded. On the other hand, if the difference between the two motion vectors is greater than Threshold _MV , the prediction determination flag is set to predict the motion prediction flag, instead of the motion prediction flag. The prediction determination flag is assigned a value of 1 or 0 depending on whether the difference is greater or not than Threshold _MV , in step 210.

Those. if in step S210 the difference is greater than Threshold _MV , the prediction determination flag is set to 1 in step S211. If the difference is greater than Threshold _MV , the prediction determination flag is set to 0 in step S212. In step S220, the value of the prediction determination flag set in step S211 or S212, 1 or 0, and the residual prediction difference flag corresponding to the difference between the residual prediction flag and the prediction determination flag are encoded.

FIG. 8 is a flowchart illustrating a method of decoding data encoded using the method illustrated in FIG. 7 according to an exemplary embodiment of the present invention.

Referring to FIG. 8, in step S251, a motion vector predicted from a base layer and a motion vector predicted from a spatially adjacent region are obtained based on encoded data.

In step S255, on the one hand, if the difference between the two motion vectors is less than a predetermined threshold, i.e. Threshold _Motion , the value of the motion prediction flag, designated as motion_prediction_flag, is set equal to the difference between the prediction determination flag and the motion prediction difference flag in step S261. As specified in the coding procedure illustrated above, the prediction determination flag and the residual prediction difference flag are designated as PrdRpFlag and residual_pred_flag_diff, respectively. In this case, motion_pred_flag_diff = PrdRpFlag - motion_prediction_flag. Thus, motion_prediction_flag can be obtained by subtracting motion_pred_flag_diff from PrdRpFlag.

On the other hand, in step S255, if the difference is less than the Threshold _Motion , the motion_prediction_flag is set to 0 in step S262.

As described in connection with FIGS. 3-8, the data to be encoded is predicted from the data of the base layer in order to reduce the size of the data to be encoded. The following describes a method of reducing the size of data to be encoded according to another exemplary embodiment of the present invention.

In the coding process illustrated in FIGS. 3-8, coding of a flag indicating whether references to the base layer data are contained is skipped when the base layer data is not needed for prediction or the value for predicting the flag is encoded instead of the flag itself.

In another exemplary embodiment, the encoding of the residual prediction flag is skipped using the base layer data, or the residual prediction flag can be converted to a new value for encoding.

When the remainder of the base layer does not contain non-zero pixels, or when the CBP of the base layer is zero, the encoding of the residual prediction flag may be skipped since the residual prediction flag may be predicted. Otherwise, the value is encoded one minus the residual prediction flag, i.e. 1 - residual_prediction_flag, since there is a high probability that the residual prediction flag has a value of 1. In this case, since the size of the data to be encoded can increase when encoded 1, 0 can be encoded instead of 1 in order to increase efficiency coding.

The above approach is used when a variable length coding (VLC) encoder is used, since this approach generates more zeros. When context adaptive binary arithmetic coding (CABAC) is used, it is possible to efficiently compress bits or a bit pattern often appearing in data to be encoded. Thus, the overall compression ratio can be increased by adjusting the data template containing units or zeros. CABAC achieves high compression performance by selecting a probabilistic model for each character based on the context of the character, adapting probability estimates corresponding to the probabilistic model based on local statistics, and performing arithmetic coding of the character. The process of encoding a data symbol consists of no more than four elementary steps: 1. Conversion to binary form; 2. Contextual modeling; 3. Arithmetic coding; and 4. Updating probabilities.

1. Binary Conversion

Among CABAC techniques, binary arithmetic coding allows a given non-binary character to be uniquely mapped to a binary sequence. In CABAC, only a binary solution is introduced into the encoding process. Symbols with non-binary values, such as conversion factors or motion vectors, are converted to binary codes before the actual arithmetic coding process. This process is similar to converting data characters into parameter length codes, except that binary codes are previously encoded by an arithmetic encoder prior to transmission.

The following elementary operations of context modeling, arithmetic modeling and updating of probabilities are performed recursively with the corresponding bits of the binary-converted codes, i.e. bins.

2. Contextual modeling

The context model, which is a probabilistic model for one or more binaries of binary-converted characters and is selected based on statistics of the last encoded data characters, saves the probability for each bin in order to be 1 or 0.

3. Arithmetic coding

An arithmetic encoder encodes each bin based on the selected probabilistic model. Each bin has only two probability subbands corresponding to the values 1 and 0, respectively.

4. Update probabilities

The selected probabilistic model is updated based on the actually encoded values. Those. if the bin value is 1, the number of occurrences of units is increased by one.

Since the context model is selected for each series of consecutive macroblocks according to the above CABAC, the probabilistic values of the probabilistic models are initialized into a table of constant values for a series of consecutive macroblocks. CABAC provides greater coding efficiency than traditional VLC, when a predetermined amount of information is accumulated, since the context model must be continuously updated based on statistics of the latest encoded data characters.

9 is a flowchart illustrating a process in which the encoding of the residual prediction flag, designated as residual_prediction_flag, is skipped using the remainder energy of the base layer or the CBP value, or the residual prediction flag is converted to a new encoding value.

In step S301, residual data of the base layer is obtained. If the energy of the residual data of the base layer or the CBP value is 0 in step S302, which means that the residual data of the base layer does not contain non-zero pixels, the residual data is not referenced in the enhanced layer. Thus, coding of the residual prediction flag is not necessary.

Conversely, if the energy of the residual data of the base layer or the CBP value is not 0, which means that the residual data of the base layer has nonzero pixels, the residual prediction flag makes sense, therefore, the value of the residual prediction flag is obtained in step S312. Then, in step S316, the inverse residual prediction flag is encoded, which is set to one minus the residual prediction flag, i.e. 1 - residual_prediction_flag.

FIG. 10 is a flowchart illustrating a process for decoding data encoded using the process illustrated in FIG. 9. Referring to FIG. 10, in step S351, residual data of the base layer is obtained. If the energy of the residual data of the base layer or the CBP value is 0 in step S352, which means that there is no data to be used for prediction in the enhanced layer, the residual prediction flag is set to 0 in step S364. Conversely, if the residual data of the base layer has nonzero pixels, the residual prediction flag makes sense. Since the data encoded according to the method illustrated in FIG. 9 is one minus the residual prediction flag, i.e. 1 - residual_prediction_flag, i.e. inverse residual prediction flag, the residual prediction flag may be set to one minus the inverse residual prediction flag, i.e. 1 is reversed_residual_prediction_flag, in step S370.

In the illustrated embodiment shown in FIGS. 9 and 10, the encoding process is described in such a way that it can be determined whether the residual prediction flag is encoded using the residual data of the base layer, and encoding is performed using the inverse value of the residual data of the base layer, thereby increasing coding efficiency.

The following describes a method for encoding a residual prediction flag using a modified context when using CABAC. Because CABAC is used to encode context-based characters as described above, it offers variable compression ratios depending on the context model shifted to 0 or 1. For example, when the context is shifted to 0, the compression ratio is high when data containing more zeros than ones are compressed by context. On the other hand, if the context is shifted to 1, the compression ratio is high if the data containing more units than zeros is compressed using the context. As described above with reference to FIG. 9, there is a very high probability that the residual prediction flag is 1 if it is encoded. Thus, the residual prediction flag is encoded according to the modified context shifted to the symbol.

11 is a flowchart illustrating a process for encoding a residual prediction flag, designated residual_prediction_flag, using a modified context according to an exemplary embodiment of the present invention. As described above, according to the encoding process, the encoding of the residual prediction flag is skipped using the remainder energy of the base layer or the CBP value, or the residual prediction flag is encoded using the context shifted to symbol 1.

Referring to FIG. 11, residual data of the base layer is obtained in step S401. If the energy of the residual data of the base layer or the CBP value is 0 in step S402, which means that the residual data of the base layer does not contain non-zero pixels, the residual data is not referenced in the enhanced layer. Thus, coding of the residual prediction flag is not necessary. In turn, if a multilayer structure is used, the CBP value can accumulate from the lowest layer. In this case, if the CBP value accumulated from the base layer is 0, the encoding of the residual prediction flag is also skipped.

If the energy of the residual data of the base layer or the accumulated CBP value is not 0, which means that the residual data of the base layer has nonzero pixels, the residual prediction flag makes sense, therefore, the value of the residual prediction flag is obtained in step S412. Since there is a high probability that the value of the residual prediction flag is 1, the residual prediction flag is encoded using the context shifted to symbol 1 in step S416.

12 is a flowchart illustrating a process for decoding data encoded using the process illustrated in FIG. 11. Referring to FIG. 12, in step S451, residual data of the base layer is obtained. If the energy of the residual data of the base layer or the accumulated CBP value is 0 in step S452, which means that there is no data to be used for prediction in the enhanced layer, the residual prediction flag is set to 0 in step S464. Conversely, if the residual data of the base layer has nonzero pixels in step S452, the residual prediction flag makes sense. Thus, the residual prediction flag is decoded using the context shifted to symbol 1 in step S462. Then, at step 470, an improved layer is obtained according to the value of the residual prediction flag.

Alternatively, it can be determined whether the residual prediction flag should be encoded using the residual energy of the base layer data or the CBP value. If the energy of the residual data of the base layer or the CBP value falls within a predetermined range, the encoding of the residual prediction flag may be skipped. The following is the following encoding pseudo-code:

If (the energy of the residual data of the base layer is 0 or the CBP value is 0)

then

skip encoding residual_prediction_flag

otherwise

encode residual_prediction_flag

If the energy of the residual data of the base layer or the CBP value is zero as a result of decoding, the decoder performs the decoding process without residual prediction on the base layer. Otherwise, the value of the residual prediction flag is decoded. The following pseudo decoding code is presented:

If (the energy of the residual data of the base layer is 0 or the CBP value is 0)

then

skip decoding residual_prediction_flag

otherwise

decode residual_prediction_flag

13 is a flowchart illustrating an example in which the encoding mode varies depending on the residual data of the base layer.

Referring to FIG. 13, in step S501, residual data of the base layer is obtained. In step 520, if the base layer residual data energy or CBP value is 0, which means that the residual prediction flag is 0, an encoding method is used that offers high encoding efficiency when the residual prediction flag is 0. For example, if CABAC is used for encoding , the residual prediction flag can be encoded using a probabilistic model for the CABAC context model, which is more likely to be 0 than be 1 .

Conversely, if the energy of the residual data of the base layer or the CBP value is not 0, there is a very high probability that the residual prediction flag is 1. Thus, in step S530, a coding method that offers high encoding efficiency when the residual prediction flag is 1 For example, if CABAC is used as the encoding method, the residual prediction flag can be encoded using a probabilistic model for the CABAC context model, which is more likely to be 1 than 0.

The following pseudo-code of the process illustrated in FIG. 13 is presented.

If (the energy of the residual data of the base layer is 0 or the CBP value is 0)

then

encode using a method that offers high encoding efficiency when the residual prediction flag is 0

otherwise

encode using a method that offers high encoding efficiency when the residual prediction flag is 1

In multi-layer video coding, when the video data is encoded on a base layer, a fine-grained scalability layer (FGS) and an enhanced layer, the residual data energy or CBP value may be referenced in the base layer or FGS layer.

The decoding process is performed in a manner similar to the encoding process. Those. after the residual data of the base layer is obtained, if the residual data energy or the CBP value is 0, which means that the residual prediction flag is 0, the encoded video data is decoded using a method offering high encoding efficiency when the residual prediction flag of 0 is used For example, if CABAC is used for encoding, video data can be decoded using the CABAC context model, which is more likely to be 0 than be 1 in order to obtain a residual flag th prediction.

Conversely, if the energy of the base layer residual data or the CBP value is not 0, which means that there is a high probability that the residual prediction flag is 1, the video signal encoded according to the encoding method that offers high encoding efficiency when the residual prediction flag is 1 is decoded in a manner similar to the encoding method. For example, if CABAC is used as the encoding method, the video data may be decoded using a probabilistic model for the CABAC context model having a higher probability of being 1 than being 0 in order to obtain a residual prediction flag. The following pseudo-code for the decoding process is presented:

If (the energy of the residual data of the base layer is 0 or the CBP value is 0)

then

decode according to a method offering high coding efficiency when the residual prediction flag is 0

otherwise

decode according to a method offering high coding efficiency when the value of the residual prediction flag is 1

Referring to FIG. 13, at steps 520 and 530, coding methods offering high coding efficiency when the residual prediction flag is 0 and when the residual prediction flag is 1, respectively, are used. One of the methods that offer high coding efficiency is to use various context models during entropy coding.

14 is a flowchart illustrating a coding process using various context models when using CABAC according to an exemplary embodiment of the present invention. Referring to FIG. 14, in step S601, residual data of the base layer is obtained. If the energy of the base layer residual data or the CBP value is 0 in step S610, which means that the residual prediction flag is 0, the residual prediction flag is encoded using the CABAC context model, which is more likely to be 0 than be 1, in step S620. Conversely, if the energy of the residual data of the base layer or the CBP value is not 0 in step S610, there is a high probability that the residual prediction flag is 1. Thus, the residual prediction flag is encoded using a CABAC context model having a higher probability of 0, than to be 1, in step S630.

FIG. 15 is a flowchart illustrating a process for decoding data encoded using the process illustrated in FIG. In step S651, residual data of the base layer is obtained. In step S652, bits of a coded block pattern (CBP) are obtained. In step S653, the 4 least significant bits in the CBP are allocated to the CBP value for the luminance signal designated as CodedBlockPatternLuma, and the 2 higher bits are allocated to the CBP value for the luminance signal designated as CodedBlockPatternChroma, respectively. The CBP values for the 4 sub-blocks in the macroblock can be known from the CBP value for CodedBlockPatternLuma. The CBP values for the DC and AC components of the current can be known from the CBP values for CodedBlockPatternChroma. The coded_block_pattern, CodedBlockPatternLuma, and CodedBlockPatternChroma formats are shown in the table below. The coded_block_pattern parameter provides information on whether the subblock or macroblock represented by each bit has nonzero pixels.

coded_block_pattern bit [5] bit [4] bit [3] bit [2] bit [1] bit [0] CodedBlockPatternChroma CodedBlockPatternLuma

Bit [0] has the value 1 when the upper left luminance block of 8 · 8 out of four blocks of luminance of 8 · 8 in the macroblock has non-zero pixels. Bit [1] and bit [2] have a value of 1 when the lower left and lower right blocks of brightness 8CH8 have non-zero pixels.

Bit [4] and bit [5] have a value of 1 when the constant and variable components of the current of color information in the macroblock have non-zero pixels.

Thus, the presence or absence of nonzero pixels can be determined by the CBP value. When at least one of the four bits making up the CodedBlockPatternLuma has a value of 1 or the CodedBlockPatternChroma value is greater than 0 in step S660, which means that the residual data has nonzero pixels, there is a high probability that the residual prediction flag is 1. Thus , the value for selecting the entropy coding context model designated as ctxIdxInc is set to 1 in step S670. After ctxIdxInc is set to 1, the encoded data is decoded using a decoding method that offers high encoding efficiency when the residual prediction flag is 1. For example, encoded data can be decoded using the CABAC context model, with a higher probability of 1 than be 0.

On the other hand, if none of the four bits making up the CodedBlockPatternLuma has a value of 1 and the CodedBlockPatternChroma value is 0 in step S660, which means that the residual data does not contain non-zero pixels, there is a high probability that the residual prediction flag is 0. Thus, the ctxIdxInc value is set to 0 in step S680 to select the entropy encoding context model. After ctxIdxInc is set to 0, the encoded data is decoded using a decoding method that offers high encoding efficiency when the residual prediction flag is 0. For example, encoded data can be decoded using a CABAC context model offering a higher probability of 1 than be 0.

After ctxIdxInc is selectively assigned a value of 1 in step S670 or 0 in step S680, entropy encoding is performed using the set value ctxIdxInc in step S690.

In step S660, it is determined whether a bit having a value of 1 is present in the CBP. As described above, the CBP value is allocated by CodedBlockPatternLuma and CodedBlockPatternChroma. The process for finding nonzero pixels using CodedBlockPatternLuma is as follows:

for (Luma8⊥ × 8BlkIdx = 0; i <= 3; i ++) {

if ((CodedBlockPatternLuma >> Luma8x8BlkIdx) & 1) and

(current macroblock is inter-macroblock))

then

residua AvailLuma8 × 8Blk = 1;

break;

else

residualAvailLuma8 × 8Blk = 0;

}

The AND (&) bitwise operation is performed for each Luma8 × 8Blkldx in order to determine whether each CodedBlockPatternLuma bit is 1 and a check is made as to whether the current macroblock is an internally encoded macroblock. If the current macroblock is an internally encoded macroblock, the value residualAvailLuma8 × 8Blk is set to 1; otherwise, it is set to 0. The residualAvailLuma8 × 8Blk parameter is used to set the baseResidualAvailLuma8 × 8Blk and determine the ctxIdxInc value, which defines the entropy encoding context model.

The process for finding nonzero pixels using CodedBlockPatternChroma is as follows:

if ((CodedBlockPatternChroma! = 0) and

(current macroblock is inter-macroblock))

then

residualAvailChroma = 1;

break;

else

residualAvailChroma = 0;

If the value of CodedBlockPatternChroma is not 0 and the current macroblock is an externally encoded macroblock, the value residualAvailChroma is set to 1; otherwise, it is set to 0. The residualAvailLuma8 × 8Blk parameter is used to set the baseResidualAvailLuma8 × 8Blk and determine the ctxIdxInc value, which defines the entropy encoding context model.

Thus, if at least one of the bits in CodedBlockPatternLuma or CodedBlockPatternChroma is set to 1, it is determined that the residual data has at least one nonzero pixel, so that ctxIdxInc is set to 1. Otherwise, ctxIdxInc is set to 0 for entropy coding.

In the exemplary embodiments described below, “block,” “part,” or “module” means a software or hardware component, such as a user programmable matrix LSI (FPGA) or application specific integrated circuit (ASIC). A block performs a specific function, but is not limited to software and hardware. The unit may be included in the address storage medium or may be configured to play one or more processors. Consequently, blocks can include components such as software components, object-oriented software components, class components and task components, processes, functions, attributes, procedures, auxiliary procedures, program code segments, drivers, firmware, microcodes, circuits, data, databases, data structures, tables, arrays and parameters. The components and features provided by the blocks can be combined into fewer components and fewer blocks, or can be divided into more components and more blocks. In addition, components and units can be implemented in such a way that they reproduce one or more central processing units (CPUs) in a device or a secure multimedia card (MMC).

FIG. 16 is a block diagram of a video encoder 500 according to an exemplary embodiment of the present invention, and FIG. 17 is a block diagram of a video decoder 550 according to an exemplary embodiment of the present invention. Although the present invention is described with reference to FIGS. 16 and 17 with respect to a scenario of using one base layer and one enhanced layer, two or more layers may be used. Specialists in the art should understand that the present invention can also be applied between the current layer and the lower layer.

Referring to FIG. 16, video encoder 500 mainly includes an enhanced layer encoder 400 and a base layer encoder 300. The configuration of the base layer encoder 300 is described below.

The downsampler 310 down-samples the input video according to the resolution, frame rate, or video size of the base layer. An MPEG downsampler or a shortwave downsampler can be used to downsample an input frame to the resolution of the base layer. A frame scheme or a frame interpolation scheme can simply be used to change the frame rate for downsampling. Downsampling the image to a smaller size can be done by removing information in the boundary region from the video information or reducing the size of the video information in order to fit the screen size. For example, downsampling can be performed to resize the original input video from a 16: 9 aspect ratio to a 4: 3 aspect ratio.

Block 350 motion estimation performs motion estimation in the frame of the base layer in order to obtain the motion vector mv for each segment in the frame of the base layer. Motion estimation is the process of finding an area in the previous reference frame Fr 'stored in a buffer of 380 frames, which is most similar to each section in the current frame Fc and has a minimal matching error. Various techniques, such as a hierarchical block size matching algorithm (HVSBM) or fixed size block matching, can be used to evaluate motion. The reference frame Fr ′ may be provided by frame buffer 380. Although in the above description, the base layer encoder 300 supports closed-loop encoding using the reconstructed frame as the reference frame, it can use an open-loop coding scheme using the original base layer frame provided by the downgrade sampler 310, as a reference frame.

The motion compensator 360 performs motion compensation in the reference frame using the motion vector mv. Subtractor 315 subtracts the motion-compensated reference frame from the current frame of the base layer Fc in order to generate a residual frame.

A converter 320 performs spatial transform of the residual frame in order to create a transform coefficient. The spatial transform method may include a discrete cosine transform (DCT) or a shortwave transform. Specifically, DCT coefficients can be created in the case where DCT is used, and short-wave coefficients can be created in the case where short-wavelength conversion is used.

Quantizer 330 quantizes the transform coefficient received from transformer 320. Quantization means the process of expressing transform coefficients generated by arbitrary real values, discrete values, and matching discrete values with indices according to a predetermined quantization table. A quantized resulting value is referred to as a quantized coefficient.

The lossless entropy encoding unit 340 encodes a quantization coefficient obtained by the quantizer 330 and the motion vector mv obtained by the motion estimation unit 350 in the bitstream of the base layer. There are many lossless coding methods, including arithmetic coding, length coding, etc.

The inverse quantizer 371 applies inverse quantization to the quantization coefficient received from the quantizer 330. Inverse quantization means the inverse quantization process in order to restore the values associated with the indices generated by the quantization according to a predetermined quantization table used in the quantization.

Inverse transformer 372 performs the inverse transform with the inverse quantized result. The inverse spatial transform may be, for example, an inverse DCT or an inverse shortwave transform performed in the reverse order to the transform performed by the transformer 320.

An adder 325 sums the output of the motion compensator 360 and the inverter 372 in order to restore the current frame Fc ', which is then sent to the frame buffer 380. A frame buffer 380 temporarily stores the recovered frame Fc 'and provides it as a reference frame for predicting Inter over a subsequent frame of the base layer.

The reconstructed frame Fc ′ is supplied to the enhanced layer encoder 400 after passing through the upsampler 395. Of course, if the base layer has the same resolution as the enhanced layer, upsampling may be skipped.

The following describes the configuration of the enhanced layer decoder 200. The frame received from the base layer encoder 300 and the input frame is supplied to the subtractor 410. The subtractor 410 calculates the difference between the input frame and the base layer frame containing the virtual area in order to generate a residual frame. The residual frame passes through a converter 420, a quantizer 430, and an entropy encoding unit 440 before being converted to an enhanced layer bitstream.

Transformer 420 performs a spatial transform with a residual signal between the macroblock in the input frame and the macroblock in the base layer in order to create a transform coefficient. The discrete cosine transform (DCT) or shortwave transform technique can be used for spatial transform. DCT coefficients (in the case of DCT) or short-wavelength coefficients (in the case of short-wavelength conversion) have similarities due to the macroblock characteristics of the enhanced layer. Thus, converter 420 removes the similarity of these coefficients in order to increase the compression ratio.

Since the quantizer 430 performs the same function / operation as its counterpart in the base layer encoder 300, its detailed description is not given. The output of the bit stream from the quantizer 430 is set by the encoding information setting unit 435 so as to effectively encode the residual prediction flag before encoding the entropy encoding unit 440.

As described above, with respect to information about whether or not the base layer frame data should be referenced in the enhanced layer data, the encoding information setting unit 435 determines the difference between the prediction data and the prediction result, thereby preventing loss of information that must be encoded and delivered from entropy encoding block 440. The process of extracting bits for compression performed in converter 420 is as described in the processes for predicting residual data (see FIGS. 4, 9, 11, 13, and 14) and predicting a motion vector (see FIG. 7). For example, if the energy of the residual data in the base layer or the FGS layer is 0 or the CBP value is 0, the encoding information setting unit 435 does not encode the residual prediction flag, since the prediction of the residual data of the base layer is not necessary; otherwise, it encodes a residual prediction flag. If the residual data energy in the base layer or the FGS layer or the CBP value is not 0, there is a high probability that the residual prediction flag is 1. As described above with reference to FIGS. 4, 9, 11, 13 and 14, the residual prediction flag encoded taking into account coding efficiency.

When the encoding process illustrated in FIG. 14 is applied if the residual data energy of the base layer or the CBP value is 0, which means that the residual prediction flag is 0, the encoding information setting unit 435 sets a probabilistic value for the CABAC context model so that so that the probability of being 0 was higher than the probability of being 1. On the contrary, if the energy of the residual data of the base layer or the CBP value is not 0, which means that there is a high probability that the residual prediction flag has a value of 1, the encoding information setting unit 435 sets a probabilistic value for the CABAC context model so that the probability of 1 is higher than being 0. In one exemplary embodiment, as described above, the encoding information setting unit 435 may set ctxIdxInc to 0 or 1 .

Entropy encoding unit 440 performs encoding of the enhanced layer data using information specified by encoding information setting unit 435. Since the entropy encoding unit 440 performs the same functions / operations as the entropy encoding unit 340 in the base layer encoder 300, with the exception of this operation, their description is omitted.

The enhanced layer encoder 400 illustrated in FIG. 16 can predict whether base layer frame data should be referenced during the encoding process in data that can refer to a base layer frame, such as residual data or a motion vector.

17 is a block diagram of a video decoder according to an embodiment of the present invention. Video decoder 550 mainly includes an enhanced layer encoder 700 and a base layer encoder 600. The configuration of the base layer encoder 600 is described below.

The lossless entropy decoding unit 610 decodes the bitstream of the base layer in order to extract texture data and motion data (for example, motion vector, segment information, reference frame number, etc.) in the base layer frame.

Inverse quantizer 620 inverse quantizes texture data. Inverse quantization is the inverse of quantization performed in the video encoder 500, and is the process of recovering values from the corresponding quantization indices created during the quantization process using the quantization table used during the quantization process.

Inverse transformer 630 performs inverse spatial transform of the inverse quantized result in order to recover the residual frame. The inverse spatial transform is the inverse spatial transform process performed by the converter 320 in the video encoder 500. The inverse DCT or inverse shortwave transform can be used for the inverse spatial transform.

Entropy decoding unit 610 also provides motion data, including motion vector mv, to motion compensator 660.

Motion compensator 660 performs motion compensation in a previously reconstructed video frame, i.e. a reference frame provided by a frame buffer 650 in order to generate a motion compensated frame.

An adder 615 summarizes the residual frame reconstructed by the inverse transformer 630 and the motion compensated frame generated by the motion compensator 660 in order to restore the video frame of the base layer. The reconstructed video frame may be temporarily stored in a buffer of 650 frames and may be provided to the motion compensator 660 to recover subsequent frames.

The reconstructed current frame Fc 'is supplied to the enhanced layer decoder 700 after passing through the upsampler 680. Thus, if the base layer has the same resolution as the enhanced layer, upsampling may be skipped. If the video information of the base layer does not contain area information that is presented in the video information of the enhanced layer, upsampling may also be skipped.

The following describes the configuration of the enhanced layer decoder 700. The lossless entropy decoding unit 710 decodes the enhanced layer bit stream in order to extract texture data for an unsynchronized frame.

Then, the extracted texture data is restored to the residual frame as it passes through the inverse quantizer 720 and the inverse transformer 730. The inverse quantizer 720 performs the same functions / operations as the inverse quantizer 620 in the base layer decoder 600.

An adder 715 adds the reconstructed residual frame to the base layer frame provided by the base layer decoder 600 in order to recover the frame.

The process illustrated in FIGS. 5, 8, 10, 12, or 15 may be performed for inverter 730 in enhanced layer decoder 700 in order to recover a residual frame. In order to determine whether the decoded data of the decoded frame of the base layer is referenced in the enhanced layer data, a check is performed with the residual data or motion data constituting the base layer frame. If the residual data of the base layer is less than the Threshold _residual , as described above with reference to FIG. 5, the residual prediction flag is set to 0, since the residual data of the base layer is not referenced in the residual data of the enhanced layer. Additionally, as described above with reference to Fig. 8, the motion prediction flag may be set to 0, since the motion vector of the base layer may not be indicated by links if the difference between the motion vector of the base layer and the motion vector from a spatially adjacent region is less than Threshold _Motion.

Alternatively, as described above with reference to FIGS. 10, 12 or 15, it can be determined whether the residual data of the base layer can be referenced in predicting the residual data of the enhanced layer depending on whether the energy of the residual data of the base layer or the CBP value is 0. Since the encoder performs encoding using various techniques in order to increase the encoding efficiency, the residual prediction flag can be obtained using a decoding method corresponding to the proper encoding method Nia.

If the decoding process illustrated in FIG. 15 is applied, the analyzer 705 obtains a CBP value from the enhanced layer bit stream in order to check if a value of 1 exists within the bit. An analyzer 705 may determine whether the residual data has nonzero pixels based on the encoded block pattern. Since this process is the same as that illustrated in FIG. 15, a detailed description thereof is not given.

Analyzer 705 can determine whether residual data has non-zero pixels using baseResidualAvailLuma8 · 8Blk [0 ... 3], as well as baseResidualAvailChroma. The 705 analyzer receives the CBP bit in order to allocate the 4 least significant bits and 2 higher bits to the CodedBlockPatternLuma and CodedBlockPatternChroma parameters, respectively. Then, analyzer 705 checks to see if at least one of the four bits in CodedBlockPatternLuma has a value of 1, and if CodedBlockPatternChroma is greater than 0. If at least one of the four bits has a value of 1 or the CodedBlockPatternChroma value is greater than 0, the analyzer 705 determines that all the residual data has at least one nonzero pixel, and sets the ctxIdxInc value to 1; otherwise, it sets the ctxIdxInc value to 0. Entropy decoding unit 710 performs entropy decoding using the ctxIdxInc value specified by the analyzer 705.

Although in the above description, the enhanced layer decoder 700 shown in FIG. 17 performs decoding using Intra_BL prediction from a base layer frame, those of ordinary skill in the art should understand that decoding may be performed by an Inter prediction technique or Intra prediction, as described with with reference to figure 2.

The present invention provides an increased compression ratio by determining whether enhanced layer data is predicted from a base layer frame.

Specialists in the art should understand that various modifications and changes can be made in the embodiments without departing from the scope of the present invention. Therefore, it should be understood that the above embodiment is not limiting, but illustrative in all aspects. The scope of the present invention is determined by the attached claims, and not the description of the invention. All modifications and changes derived from the scope and spirit of the claims and their equivalents should be construed as being included in the scope of the present invention.

Claims (66)

1. A method of encoding a residual prediction flag indicating whether residual data for the enhanced layer block of the multilayer video signal is predicted from the residual data of the lower layer block corresponding to the residual data for the enhanced layer block, the method comprising the steps of: calculating the characteristic data of the lower layer block , while the characteristic data contains at least one of the energy of the residual data of the block of the lower layer and the values of the pattern of coded blocks (CBP) of the block of the lower layer ; determining a method for encoding a residual prediction flag according to characteristic data; and encode the residual prediction flag using a specific encoding method. 2. The method according to claim 1, in which, if the characteristic data is less than a predetermined threshold, the encoding of the residual prediction flag is skipped, and if the characteristic data is not less than the predetermined threshold, the residual prediction encoding comprises encoding the difference between the predicted value of the residual prediction flag and the residual flag predictions. 3. The method according to claim 1, wherein if the characteristic data is not less than a predetermined threshold, the encoding of the residual prediction flag comprises encoding a value 1 minus the residual prediction flag. 4. The method according to claim 1, in which, if the characteristic data is less than a predetermined threshold, the encoding of the residual prediction flag is skipped, and if the characteristic data is not less than a predetermined threshold, the encoding of the residual prediction flag is performed using the context model shifted to symbol 1. 5. The method according to claim 1, in which, if the characteristic data is less than a predetermined threshold, the encoding of the residual prediction flag is performed using the first context model, and if the characteristic data is not less than the predetermined threshold, the encoding of the residual prediction flag is performed using the second context model , while the first context model is more likely to be 0 than to be 1, and the second context model is more likely to be 1 than to be 0. 6. The method according to claim 1, in which the lower layer contains one of the base layer and the fine-grained scalability layer (FGS). 7. The method according to claim 1, in which: the residual energy of the block of the lower layer contains the number of nonzero pixels in the block of the lower layer; and the CBP value of the bottom layer block contains the number of bits in the CBP having a value of 1. 8. The method according to claim 1, wherein the CBP value comprises the values of the four (4) LSB bits for luminance information, called CodedBlockPatternLuma, and the two (2) higher CBP bits for color information, called CodedBlockPatternChroma. 9. The method according to claim 1, in which the calculation of the energy of the residual data comprises the step of summing the energy of the lower layer, which is indicated by references in the improved layer. 10. The method according to claim 1, in which the calculation of the CBP of the block of the lower layer comprises the step of determining whether each bit of the CBP in the block of the lower layer has a value of 1. 11. A method for decoding a residual prediction flag indicating whether residual data for the enhanced layer block of the multi-layer video signal is predicted from the residual data of the lower layer block corresponding to the residual data for the enhanced layer block, the method comprising the steps of: calculating the characteristic data of the lower layer block , while the characteristic data contains at least one of the energy of the residual data of the block of the lower layer and the values of the pattern of coded blocks (CBP) of the block lower c Oja; determining a method for decoding the residual prediction flag according to characteristic data; and decode the residual prediction flag using a specific decoding method. 12. The method according to claim 11, in which: if the characteristic data is less than a predetermined threshold, the residual prediction flag is assigned a value of 0 without decoding the residual prediction flag; and if the characteristic data is not less than a predetermined threshold, the residual prediction flag is assigned a difference value between the residual prediction difference flag and the predicted residual prediction flag, the residual prediction difference flag corresponding to the difference between the predicted residual prediction flag and the residual prediction flag. 13. The method according to claim 11, in which, if the characteristic data is greater than a predetermined threshold, the residual prediction flag is set to 1 minus the inverse residual prediction flag. 14. The method according to claim 11, in which: if the characteristic data is less than a predetermined threshold, decoding the residual prediction flag is skipped; and if the characteristic data is not less than a predetermined threshold, the decoding of the residual prediction flag is performed using the context model shifted to symbol 1. 15. The method according to claim 11, in which, when the encoding of the residual prediction flag is performed using context adaptive binary arithmetic coding (SAVAS): if the characteristic data is less than a predetermined threshold, the residual prediction flag is decoded using the first context model; and if the characteristic data is not less than a predetermined threshold, the residual prediction flag is decoded using the second context model, the first context model being more likely to be 0 than being 1, and the second context model being more likely to be 1 than being 0. 16. The method according to claim 11, in which the lower layer contains one of the base layer and the fine-grained scalability layer (FGS). 17. The method according to claim 11, in which: the residual energy of the block of the lower layer contains the number of non-zero pixels in the block of the lower layer; and the CBP value of the bottom layer block contains the number of bits in the CBP having a value of 1. 18. The method of claim 11, wherein the CBP value comprises the values of the four (4) least significant bits of the CBP for luminance information called CodedBlockPatternLuma and the two (2) higher bits of the CBP for luminance information called CodedBlockPatternChroma. 19. The method according to claim 11, in which the calculation of the energy of the residual data comprises the step of summing the energy of the lower layer, which is indicated by references in the improved layer. 20. The method according to claim 11, in which the calculation of the CBP block of the lower layer contains a step on which it is determined whether each bit of the CBP in the block of the lower layer, which is indicated by the reference improved layer, value 1. 21. A method for decoding a residual prediction flag indicating whether residual data for the enhanced layer block of the multilayer video signal is predicted from the residual data of the lower layer block corresponding to the residual data for the enhanced layer block, the method comprising the steps of: checking the value of the encoded block pattern ( CBP) block of the lower layer; determining whether each CBP bit in the block of the lower layer has a value of 1; determining a method for decoding the residual prediction flag according to whether or not at least one CBP bit in the lower layer block has a value of 1; and decode the residual prediction flag according to a specific decoding method. 22. The method according to item 21, in which determining whether or not each bit of the CBP has a value of 1, comprises the steps of: setting the four (4) least significant bits of the CBP as the first parameter, which is used to determine the encoding luminance information, and two (2) higher bits as a second parameter, which is used to determine the encoding of the color information; perform a right shift operation, which shifts the bit of the first parameter to the right by an integer selected from the range 0-3 allocated to the macroblock address, and perform AND bitwise operation for 1 with the result of a right shift operation, i.e. 1 AND result of the shift operation to the right; and determining that at least one (1) CBP bit is 1 if the result of the AND bitwise operation is not 0. 23. The method according to item 21, in which the determination of the decoding method comprises setting a context model for decoding the residual prediction flag, and decoding the residual prediction flag by entropy decoding of the specified context model, while decoding by entropy is the inverse of SAVAS . 24. The method according to item 21, in which decoding by entropy is performed using context adaptive binary arithmetic coding (SAVAS). 25. The method according to item 21, in which the determination of the decoding method comprises the steps of: setting as the value of the selection parameter of the context model 1, if there is at least one (1) bit CBP, having a value of 1; and set as the value of the selection parameter of the context model 0, if there are no bits of the SVR having the value 1. 26. The method according A.25, in which when decoding the flag of the inverse prediction, the flag of the inverse prediction is decoded using: the first context model that is more likely to be 0 than to be 1 if the selection parameter of the context model is set to 0; and the second context model, which is more likely to be 1 than to be 0 if the parameter is set to 1. 27. The method according to item 21, in which the lower layer contains one of the base layer and the fine-grained scalability layer (FGS). 28. The method according to item 23, in which when confirming the CBP of the block of the lower layer, it is determined whether each bit of the CBP in the block of the lower layer, which is used to predict the improved layer, value 1. 29. A method for decoding an enhanced layer of a multilayer video signal, the method comprising the steps of: determining a method for decoding a residual prediction flag indicating whether the improved layer is predicted from the lower layer using the encoded block pattern (CBP) value of the lower layer; and decode the residual prediction flag using the entropy decoding method selected according to the decoding method, and the enhanced layer is decoded using the residual prediction flag. 30. A video encoder for encoding a residual prediction flag indicating whether residual data for the enhanced layer block of the multilayer video signal is predicted from the residual data of the lower layer block corresponding to the residual data for the enhanced layer block, wherein the encoder comprises: a lower layer encoding block that generates characteristic data for the block of the lower layer, while the characteristic data contains at least one of the energy of the residual data of the block of the lower layer and the value of the template is encoded ny blocks (SVR) of the lower layer block; an enhanced layer encoding unit that encodes a residual prediction flag using an encoding method for the residual prediction flag, the method being determined according to characteristic data. 31. The video encoder of claim 30, wherein, if the characteristic data is less than a predetermined threshold, the residual prediction flag encoding is skipped, and if the characteristic data is not less than a predetermined threshold, the residual prediction encoding comprises encoding the difference between the predicted residual prediction flag and the residual flag predictions. 32. The video encoder of claim 30, wherein, if the characteristic data is not less than a predetermined threshold, encoding the residual prediction flag comprises encoding a value 1 minus the residual prediction flag. 33. The video encoder of claim 30, wherein the coding of the residual prediction flag is performed using context adaptive binary arithmetic coding (SAVAS); and which, if the characteristic data is less than a predetermined threshold, the encoding of the residual prediction flag is skipped, and if the characteristic data is not less than the predetermined threshold, the encoding of the residual prediction flag is performed using the context model shifted to symbol 1. 34. The video encoder of claim 30, wherein the coding of the residual prediction flag is performed using context adaptive binary arithmetic coding (SAVAS); and in which, if the characteristic data is less than a predetermined threshold, the encoding of the residual prediction flag is performed using the first context model, and if the characteristic data is not less than the predetermined threshold, the encoding of the residual prediction flag is performed using the second context model, the first context model having more likely to be 0 than to be 1, and the second context model is more likely to be 1 than to be 0. 35. The video encoder of claim 30, wherein the bottom layer comprises one of a base layer and a fine grain scalability (FGS) layer. 36. The video encoder of claim 30, wherein the energy of the residual data of the lower layer block comprises a number of nonzero pixels in the lower layer block; and the CBP value of the bottom layer block contains the number of bits in the CBP having a value of 1. 37. The video encoder according to claim 30, wherein the CBP value comprises the values of the four (4) lowest bits of the CBP for luminance information called CodedBlockPatternLuma and the two (2) higher bits of the CBP for color information called CodedBlockPatternChroma. 38. A video decoder for decoding a residual prediction flag indicating whether residual data for the enhanced layer block of the multilayer video signal is predicted from the residual data of the lower layer block corresponding to the residual data for the enhanced layer block, wherein the decoder comprises: a lower layer decoding block that generates characteristic data for the block of the lower layer, while the characteristic data contains at least one of the energy of the residual data of the block of the lower layer and the value of the template odirovanny blocks (SVR) the block of the lower layer; and an enhanced layer decoding unit that decodes the residual prediction flag using a decoding method for the residual prediction flag, the method being determined according to characteristic data. 39. The video decoder of claim 38, wherein if the characteristic data is less than a predetermined threshold, the residual prediction flag is set to 0 without decoding the residual prediction flag; and if the characteristic data is not less than a predetermined threshold, the residual prediction flag is assigned a difference value between the residual prediction difference flag and the predicted residual prediction flag, the residual prediction difference flag corresponding to the difference between the predicted residual prediction flag and the residual prediction flag. 40. The video decoder of claim 38, wherein if the characteristic data is greater than a predetermined threshold, a residual prediction flag is set to 1 minus the inverse residual prediction flag. 41. The video decoder of claim 38, wherein when the residual prediction flag encoding is performed using context adaptive binary arithmetic coding (SAVAS): if the characteristic data is less than a predetermined threshold, decoding of the residual prediction flag is skipped; and if the characteristic data is not less than a predetermined threshold, the decoding of the residual prediction flag is performed using the context model shifted to symbol 1. 42. The video decoder of claim 38, wherein when the residual prediction flag encoding is performed using context adaptive binary arithmetic coding (SAVAS): if the characteristic data is less than a predetermined threshold, the residual prediction flag is decoded using the first context model; and if the characteristic data is not less than a predetermined threshold, the residual prediction flag is decoded using the second context model, the first context model being more likely to be 0 than being 1, and the second context model being more likely to be 1 than being 0. 43. The video decoder of claim 38, wherein the bottom layer comprises one of a base layer and a fine grain scalability (FGS) layer. 44. The video decoder according to clause 38, in which the energy of the residual data of the block of the lower layer contains the number of non-zero pixels in the block of the lower layer; and the CBP value of the bottom layer block contains the number of bits in the CBP having a value of 1. 45. The video decoder of claim 38, wherein the CBP value comprises the values of the four (4) lowest CBP bits for luminance information called CodedBlockPatternLuma and two (2) higher CBP bits for color information called CodedBlockPatternChroma. 46. A decoder for decoding a residual prediction flag indicating whether residual data for the enhanced layer block of the multilayer video signal is predicted from the residual data of the lower layer block corresponding to the residual data for the enhanced layer block, the decoder comprising: an analyzer that analyzes the value of the encoded block pattern ( CBP) block of the lower layer; a decoding unit that determines a method for decoding the residual prediction flag according to the CBP value and decodes the residual prediction flag using a specific decoding method. 47. The decoder according to item 46, in which the analyzer sets the four (4) LSB bits as the first parameter, which is used to determine the encoding of the brightness information, and two (2) LSBs as the second parameter, which is used to determine the encoding of the color information, performs a right shift operation, which shifts the bit of the first parameter to the right by an integer selected from the range 0-3 allocated to the macroblock address, and performs the AND bitwise operation for 1 with the result of the opera uu right shift, i.e. 1 AND the result of the shift operation to the right, and determines that at least one (1) bit of the CBP has a value of 1 if the result of the bitwise AND operation is not 0. 48. The decoder of claim 46, wherein the analyzer defines a context model for decoding the residual prediction flag, and the decoding unit performs entropy decoding using the specified context model. 49. The decoder of claim 48, wherein the entropy decoding is performed by context adaptive binary arithmetic coding (SAVAS). 50. The decoder according to item 46, in which the analyzer sets the value of the selection parameter of the context model 1, if there is at least one (1) bit CBP, having a value of 1, when determining whether each bit of CBP has a value of 1 , or sets the value of parameter 0 if there are no bits of the CBP having the value 1. 51. The decoder according to claim 50, wherein the decoding unit decodes the residual prediction flag with: a first context model having a higher probability of being 0 than being 1 if the context model selection parameter is set to 0; and the second context model, which is more likely to be 1 than to be 0 if the parameter is set to 1. 52. The decoder of claim 46, wherein the bottom layer comprises one of a base layer and a fine grain scalability (FGS) layer. 53. The decoder according to item 46, in which the analyzer determines whether each bit of the CBP block of the lower layer, which is indicated by a link in the enhanced layer, value 1. 54. A decoder for decoding an enhanced layer of a multilayer video signal, the decoder comprising: an analyzer that analyzes the value of the encoded block pattern (CBP) for the lower layer; an entropy decoding unit that determines a method for decoding a residual prediction flag indicating whether the improved layer from the lower layer is predicted using the CBP value and performs entropy decoding of the residual prediction flag according to the decoding method and then the improved layer; an inverse quantizer that inverse quantizes an improved layer; and an inverse transformer that performs the inverse transform of the inverse-quantized enhanced layer according to the residual prediction flag. 55. A method for decoding a residual prediction flag indicating whether residual data for the enhanced layer block of the multilayer video signal is predicted from the residual data of the lower layer block corresponding to the residual data for the enhanced layer block, the method comprising the steps of: determining the value of the encoded block pattern ( CBP) block of the lower layer; determining a method for decoding the residual prediction flag according to said specific CBP value; and decode the residual prediction flag according to the aforementioned specific decoding method. 56. The method of claim 55, wherein determining the decoding method comprises: if at least one bit of the CBP bits contains a value of 1, the residual prediction flag is decoded using the first context model; and if none of the CBP bits contains a value of 1, the residual prediction flag is decoded using the first context model. 57. The method of claim 55, wherein if any one brightness value in the CBP contains a value of 1, the residual prediction flag is decoded using the first context model. 58. The method of claim 55, wherein if one of the luminance values in the CBP contains a value of 1, the residual prediction flag is decoded using the first context model. 59. The method of claim 55, wherein if the number of bits in the CBP having a value of 1 is less than a predetermined threshold value, the residual prediction flag is decoded using the first context model, and if the number of bits in the CBP having a value of 1 is greater than the predetermined threshold value, the residual prediction flag is decoded using the second context model, the first context model having a higher probability of accepting the value “0” than the “1”, and the second context model having a higher probability of accepting the value “1”, h m "0". 60. The method of claim 55, wherein determining the CBP value of the lower layer block comprises setting four LSB and two LSB bits in a multilayer video signal as a CBP value for luminance information and a CBP value for color information, respectively. 61. A video decoder for decoding a residual prediction flag indicating whether residual data for the enhanced layer block of the multi-layer video signal is predicted from the residual data of the lower layer block corresponding to the residual data for the enhanced layer block, wherein the video decoder comprises: a block that determines the value of the encoded block pattern ( CBP) block of the lower layer; determines a method for decoding the residual prediction flag according to said specific CBP value; and decodes the residual prediction flag according to said specific decoding method. 62. The video decoder of claim 61, wherein if at least one bit of the CBP bits contains a value of 1, the residual prediction flag is decoded using the first context model; and if none of the CBP bits contains a value of 1, the residual prediction flag is decoded using the first context model. 63. The video decoder of claim 61, wherein if any one brightness value in the CBP contains a value of 1, the residual prediction flag is decoded using the first context model. 64. The video decoder of claim 61, wherein if one of the luminance values in the CBP contains a value of 1, the residual prediction flag is decoded using the first context model. 65. The video decoder of claim 61, wherein if the number of bits in the CBP having a value of 1 is less than a predetermined threshold value, the residual prediction flag is decoded using the first context model, and if the number of bits in the CBP having a value of 1 is greater than the predetermined threshold value, the residual prediction flag is decoded using the second context model, the first context model being more likely to accept the value “0” than “1”, and the second context model is more likely to accept the value e "1" rather than "0". 66. The video decoder of claim 61, wherein the four least significant bits and two high-order bits of the CBP in the multi-layer video signal are set as the CBP value for the luminance information and the CBP value for the color information, respectively.

Images