WO2017131475A1 - Method and device for encoding and decoding video by using prediction - Google Patents

Abstract

A method and a device for performing encoding and decoding by using prediction are disclosed. In the encoding of a current block, a residual signal of a current block is generated on the basis of the current block, a first prediction, and a second prediction. In addition, information on the residual signal encoded by encoding the residual signal is generated. The second prediction is a prediction for a part of the current block. In the decoding of the current block, a reconstructed residual signal for the current block is generated. A reconstructed block for the current block is generated on the basis of the reconstructed residual signal, the second prediction, and the first prediction. The second prediction is a prediction for a part of the current block.

Description

Method and apparatus for encoding and decoding video using prediction

The following embodiments relate to a video decoding method, a decoding device, an encoding method, and an encoding device, and more particularly, to a method and an apparatus for encoding and decoding a video using prediction on a target block.

Through the continuous development of the information and telecommunications industry, broadcasting services having high definition (HD) resolution have spread worldwide. This proliferation has resulted in many users becoming accustomed to high resolution, high quality images and / or video.

In order to satisfy users' demand for high image quality, many organizations are spurring the development of next generation video devices. Users are interested in Ultra High Definition (UHD) TVs, which have four times the resolution of FHD TVs, as well as High Definition TV (HDTV) and Full HD (FHD) TVs. This has increased, and as the interest increases, an image encoding / decoding technique for an image having higher resolution and image quality is required.

An image encoding / decoding apparatus and method include an inter prediction technique, an intra prediction technique, an entropy encoding technique, etc. in order to perform encoding / decoding of high resolution and high quality images. Can be used. The inter prediction technique may be a technique for predicting a value of a pixel included in a current picture using a temporally previous picture and / or temporally following picture. An intra prediction technique may be a technique of predicting a value of a pixel included in a current picture by using information of a pixel in a current picture. The entropy encoding technique may be a technique of allocating a short code to a symbol having a high appearance frequency and a long code to a symbol having a low appearance frequency.

In encoding and decoding of an image, prediction may mean generating a prediction signal similar to the original signal. Predictions can be broadly classified into predictions referring to spatially reconstructed images, predictions referring to temporal reconstructed images, and predictions for other symbols. In other words, a temporal reference may refer to a temporally reconstructed image, and a spatial reference may refer to a spatially reconstructed image.

The current block may be a block that is currently subjected to encoding or decoding. The current block may be named a target block or a target unit. In encoding, the current block may be called an encoding target block or an encoding target unit. In decoding, the current block may be called a decoding target block or a decoding target unit.

Inter prediction may be a technique for predicting the current block using temporal and spatial references. Intra prediction may be a technique for predicting the current block using only spatial references.

The image encoding / decoding technique encodes the current block using one prediction mode among a plurality of prediction modes when performing intra prediction to reduce spatial repetition. For example, High Efficiency Video Coding (HEVC) technology uses 35 intra prediction modes.

The encoding apparatus generates prediction blocks of the current block using all available prediction modes among the plurality of prediction modes, and selects one prediction modes that produce the best result as a prediction mode of encoding of the current block. However, even using the prediction mode that produces the best results, there is still an error between the original block and the prediction block. This error is represented as a residual block.

For some of the pixels, there is a large error between the original block and the prediction block. This large error may not result in sufficient spatial iteration reduction even after transform and / or quantization is applied to the residual block.

Since intra prediction is prediction in one direction for the entire block, intra prediction can cause a large error for a portion of the block.

One embodiment may provide a method and apparatus for reducing a prediction error of a prediction unit generated by the prediction.

One embodiment may provide a method and apparatus for reducing a prediction error caused by a prediction direction of a prediction mode.

An embodiment may provide a method and apparatus for performing a second prediction selectively in a direction of a first prediction mode with respect to a prediction error according to the first prediction.

One embodiment may provide a method and apparatus for dividing a current block into two parts having the same number of pixels, and performing a second prediction on the two parts.

One embodiment may provide a method and apparatus for using second prediction for a portion of a current block having a large prediction error according to the first prediction.

One embodiment may provide a method and apparatus for reducing a prediction error of a current block by using a second prediction.

One embodiment may provide a method and apparatus for providing a higher compression effect on a current block by using a second prediction.

The method of claim 1, further comprising: generating a residual signal of the current block based on the current block, the first prediction, and the second prediction; And generating information on the encoded residual signal by performing encoding on the residual signal, wherein the second prediction is a prediction on a portion of the current block.

A reconstructed residual signal generator for generating a reconstructed residual signal for the current block; And a reconstructed block generator that generates a reconstructed block for the current block based on the reconstructed residual signal, the second prediction, and the first prediction, wherein the second prediction is a prediction for a portion of the current block. A decoding method is provided.

Generating a reconstructed residual signal for the current block; And generating a reconstructed block for the current block based on the reconstructed residual signal, the second prediction, and the first prediction, wherein the second prediction is a prediction for a portion of the current block. Is provided.

The first prediction and the second prediction may be intra prediction.

The portion may be non-square.

The portion may be determined based on the prediction mode of the first prediction.

The portion may be determined based on a straight line passing through the center of the current block in the prediction direction of the first prediction.

The prediction direction of the first prediction and the prediction direction of the second prediction may be different from each other.

The part may be one of two parts generated by dividing the current block.

The number of pixels of the two portions may be the same.

Each of the two parts may be non-square.

The two portions may be determined based on a straight line passing through the center of the current block in the prediction direction of the first prediction.

The part may be selected according to the selected part information indicating one of the two parts.

If the prediction mode of the first prediction is a non-directional mode, the second prediction may not be used.

When the second prediction usage information indicating whether the second prediction is used in encoding the current block indicates that the second prediction is not used, the second prediction may not be used.

If the second prediction is not used, the first prediction may be a prediction of the entirety of the current block.

A second prediction signal generated by the second prediction for the portion may be added to the reconstructed residual signal.

The first prediction may be a prediction for a portion of the current block except for the portion.

The first prediction signal generated by the first prediction for the remainder may be added to the reconstructed residual signal.

The reconstructed block may be generated based on a second prediction signal generated by the second prediction for the portion and a first prediction signal generated by the first prediction on the remaining portion.

A method and apparatus are provided for reducing a prediction error of a prediction unit generated by prediction.

A method and apparatus are provided for reducing a prediction error caused by a prediction direction of a prediction mode.

A method and apparatus are provided for performing selective second prediction in a direction of a first prediction mode with respect to a prediction error according to the first prediction.

A method and apparatus are provided for dividing a current block into two parts having the same number of pixels, and performing a second prediction on the two parts.

A method and apparatus for using second prediction for a portion of a portion of a current block with a large prediction error according to the first prediction is provided.

One embodiment may provide a method and apparatus for reducing a prediction error of a current block by using a second prediction.

One embodiment may provide a method and apparatus for providing a higher compression effect on a current block by using a second prediction.

1 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment of the present invention.

2 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment of the present invention.

3 is a diagram schematically illustrating a division structure of an image when encoding and decoding an image.

4 is a diagram illustrating a form of a prediction unit PU that a coding unit CU may include.

FIG. 5 is a diagram illustrating a form of a transform unit (TU) that may be included in a coding unit (CU).

6 is a diagram for explaining an embodiment of an intra prediction process.

7 is a diagram for describing a position of a reference sample used in an intra prediction process.

8 is a diagram for explaining an embodiment of an inter prediction process.

9 illustrates a prediction error when the original image is predicted in the vertical direction according to an example.

10 is a structural diagram of an encoding apparatus according to an embodiment.

11 is a flowchart of an encoding method according to an embodiment.

12 illustrates first prediction and second prediction using intra prediction, according to an embodiment.

13 is a flowchart of a method of generating a residual block, according to an exemplary embodiment.

14 is a flowchart of a method of generating a restored block, according to an example.

15 illustrates an encoding process according to an embodiment.

16 is a structural diagram of a decoding apparatus according to an embodiment.

17 is a flowchart of a decoding method according to an embodiment.

DETAILED DESCRIPTION For the following detailed description of exemplary embodiments, reference is made to the accompanying drawings that show, by way of illustration, specific embodiments. These embodiments are described in sufficient detail to enable those skilled in the art to practice the embodiments. It should be understood that the various embodiments are different but need not be mutually exclusive. For example, certain shapes, structures, and characteristics described herein may be embodied in other embodiments without departing from the spirit and scope of the invention with respect to one embodiment. In addition, it is to be understood that the location or arrangement of individual components within each disclosed embodiment may be changed without departing from the spirit and scope of the embodiments. The following detailed description, therefore, is not to be taken in a limiting sense, and the scope of the exemplary embodiments, if properly described, is defined only by the appended claims, along with the full scope of equivalents to which such claims are entitled.

Like reference numerals in the drawings refer to the same or similar functions throughout the several aspects. Shape and size of the elements in the drawings may be exaggerated for clarity.

When a component is said to be "connected" or "connected" to another component, the above two components may be directly connected to or connected to each other, It is to be understood that other components may exist in the middle of the two components. In addition, the description "including" a specific configuration in the exemplary embodiments does not exclude a configuration other than the specific configuration described above, the additional configuration is the implementation of the exemplary embodiments or the technical spirit of the exemplary embodiments. It can be included in the range.

Terms such as first and second may be used to describe various components, but the above components should not be limited by the above terms. The above terms are used to distinguish one component from another component. For example, without departing from the scope of rights, the first component may be referred to as the second component, and similarly, the second component may also be referred to as the first component.

In addition, the components shown in the embodiments are shown independently to represent different characteristic functions, and do not mean that each component is composed of only separate hardware or one software component unit. That is, each component is listed as each component for convenience of description. For example, at least two of the components may be combined into one component. In addition, one component may be divided into a plurality of components. The integrated and separated embodiments of each of these components are also included in the scope of the present invention without departing from the essence.

In addition, some of the components may not be essential components for performing essential functions, but may be optional components for improving performance. Embodiments may be implemented including only components necessary to implement the nature of the embodiments, and structures including the optional components, such as, for example, components used only for performance improvement, are also included in the scope of rights.

Hereinafter, in order to enable those skilled in the art to easily implement the embodiments, embodiments will be described in detail with reference to the accompanying drawings. In describing the embodiments, when it is determined that the detailed description of the related well-known configuration or function may obscure the subject matter of the present specification, the detailed description thereof will be omitted.

In the following description, an image may mean one picture constituting a video and may represent a video itself. For example, "encoding and / or decoding of an image" may mean "encoding and / or decoding of a video" and may mean "encoding and / or decoding of one of images constituting the video." It may be.

Hereinafter, "video" and "motion picture" may be used interchangeably and may be used interchangeably.

Hereinafter, "image", "picture", "frame" and "screen" may be used in the same sense, and may be used interchangeably.

In embodiments, each of the specified information, data, flags and elements, attributes, etc. may have a value. The value "0" of information, data, flags and elements, attributes, etc. may represent a logical false or first predefined value. In other words, the value "0", logic false and the first predefined value can be used interchangeably. The value "1" of information, data, flags and elements, attributes, etc. may represent logical true or second predefined values. In other words, the value "1", the logical true and the second predefined value can be used interchangeably.

When a variable such as i or j is used to indicate a row, column, or index, the value of i may be an integer of 0 or more and may be an integer of 1 or more. In other words, in embodiments, rows, columns, indexes, etc. may be counted from zero and counted from one.

In the following, terms used in the embodiments are described.

Unit: A “unit” may represent a unit of encoding and decoding of an image. The meanings of the unit and the block may be the same. In addition, the terms "unit" and "block" may be used interchangeably.

The unit (or block) may be an M × N array of samples. M and N may each be a positive integer. A unit can often mean an array of two-dimensional samples. The sample may be a pixel or pixel value.

-The terms "pixel" and "sample" can be used interchangeably and can be used interchangeably.

In encoding and decoding of an image, a unit may be an area generated by division of one image. One image may be divided into a plurality of units. In the encoding and decoding of the image, a predefined process for the unit may be performed according to the type of the unit. Depending on the function, the type of unit may be classified into a macro unit, a coding unit (CU), a prediction unit (PU), a transform unit (TU), and the like. One unit may be further divided into subunits having a smaller size than the unit.

The unit division information may include information about the depth of the unit. The depth information may indicate the number and / or degree of division of the unit.

One unit may be divided into a plurality of sub-units hierarchically with depth information based on a tree structure. In other words, the unit and the lower unit generated by the division of the unit may correspond to the node and the child node of the node, respectively. Each divided subunit may have depth information. Since the depth information of the unit indicates the number and / or degree of division of the unit, the division information of the lower unit may include information about the size of the lower unit.

In the tree structure, the highest node may correspond to the first unit that is not split. The highest node may be referred to as a root node. In addition, the highest node may have a minimum depth value. At this time, the highest node may have a depth of level 0.

A node with a depth of level 1 may represent a unit created as the first unit is divided once. A node with a depth of level 2 may represent a unit created as the first unit is split twice.

A node with a depth of level n may represent a unit generated as the first unit is divided n times.

The leaf node may be the lowest node or may be a node that cannot be further divided. The depth of the leaf node may be at the maximum level. For example, the predefined value of the maximum level may be three.

Transform Unit: A transform unit may be a basic unit in residual signal coding and / or residual signal decoding such as transform, inverse transform, quantization, inverse quantization, transform coefficient encoding, and transform coefficient decoding. . One transform unit may be divided into a plurality of transform units having a smaller size.

Prediction Unit A prediction unit may be a basic unit in performing prediction or compensation. The prediction unit can be a number of partitions by partitioning. Multiple partitions may also be the basic unit in performing prediction or compensation. The partition generated by the partitioning of the prediction unit may also be the prediction unit.

Reconstructed Neighbor Unit: The reconstructed neighbor unit may be a unit that has already been encoded or decoded around the encoding target unit or the decoding target unit. The reconstructed neighbor unit may be a spatial neighbor unit or a temporal neighbor unit to the target unit.

Prediction unit partition: A prediction unit partition may mean a form in which a prediction unit is divided.

Parameter Set: A parameter set may correspond to header information among structures in the bitstream. For example, the parameter set may include a sequence parameter set, a picture parameter set, an adaptation parameter set, and the like.

Rate-distortion optimization: The encoding apparatus uses a combination of the size of the coding unit, the prediction mode, the size of the prediction unit, the motion information, and the size of the transform unit to provide high coding efficiency. Distortion optimization can be used.

The rate-distortion optimization method can calculate the rate-distortion cost of each combination in order to select the optimal combination among the above combinations. The rate-distortion cost may be calculated using Equation 1 below. In general, a combination in which the rate-distortion cost is minimized may be selected as an optimal combination in the rate-distortion optimization scheme.

D may represent distortion. D may be the mean square error of the squares of difference values between the original transform coefficients and the reconstructed transform coefficients in the transform block.

R can represent the rate. R may indicate a bit rate using the associated context information.

λ may represent a Lagrangian multiplier. R may include not only encoding parameter information such as a prediction mode, motion information, and a coded block flag, but also bits generated by encoding of transform coefficients.

The encoding apparatus performs processes such as inter prediction and / or intra prediction, transformation, quantization, entropy encoding, inverse quantization, and inverse transformation to calculate accurate D and R, which can greatly increase the complexity in the encoding apparatus. have.

Reference picture: The reference picture may be an image used for inter prediction or motion compensation. The reference picture may be a picture including a reference unit referenced by the target unit for inter prediction or motion compensation. The meanings of the picture and the image may be the same. In addition, the terms "picture" and "image" may be used interchangeably.

Reference picture list: The reference picture list may be a list including reference pictures used for inter prediction or motion compensation. The type of the reference picture list may be List Combined (LC), List 0 (List 0; L0), List 1 (List 1; L1), and the like.

Motion Vector (MV): The motion vector may be a two-dimensional vector used in inter prediction. For example, MV may be expressed in the form of (mv _x , mv _y ). mv _x may represent a horizontal component and mv _y may represent a vertical component.

The MV may indicate an offset between the target picture and the reference picture.

Search range: The search range may be a two-dimensional area in which a search for MV is performed during inter prediction. For example, the size of the search region may be M × N. M and N may each be a positive integer.

1 is a block diagram illustrating a configuration of an encoding apparatus according to an embodiment of the present invention.

The encoding apparatus 100 may be a video encoding apparatus or an image encoding apparatus. The video may include one or more images. The encoding apparatus 110 may sequentially encode one or more images of the video over time.

Referring to FIG. 1, the encoding apparatus 100 may include an inter predictor 110, an intra predictor 120, a switch 115, a subtractor 125, a transformer 130, a quantizer 140, and entropy decoding. The unit 150 may include an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

The encoding apparatus 100 may encode the input image in an intra mode and / or an inter mode. The input image may be referred to as a current image that is a target of current encoding.

In addition, the encoding apparatus 100 may generate a bitstream including encoding information by encoding the input image, and may output the generated bitstream.

When the intra mode is used, the switch 115 can be switched to intra. When the inter mode is used, the switch 115 can be switched to inter.

The encoding apparatus 100 may generate a prediction block for the input block of the input image. In addition, after the prediction block is generated, the encoding apparatus 100 may encode a residual between the input block and the prediction block. The input block may be referred to as a current block that is a target of current encoding.

When the prediction mode is the intra mode, the intra prediction unit 120 may use a pixel value of an already encoded block in the vicinity of the current block as a reference pixel. The intra predictor 120 may perform spatial prediction on the current block by using the reference pixel, and generate prediction samples on the current block through spatial prediction.

The inter predictor 110 may include a motion predictor and a motion compensator.

When the prediction mode is the inter mode, the motion predictor may search an area that best matches the current block from the reference image in the motion prediction process, and may derive a motion vector for the current block and the searched area. The reference picture may be stored in the reference picture buffer 190 and may be stored in the reference picture buffer 190 when encoding and / or decoding of the reference picture is processed.

The motion compensator may generate a prediction block by performing motion compensation using a motion vector. Here, the motion vector may be a two-dimensional vector used for inter prediction. In addition, the motion vector may indicate an offset between the current picture and the reference picture.

The subtractor 125 may generate a residual block that is a difference between the input block and the prediction block. The residual block may be referred to as a residual signal.

The transform unit 130 may generate transform coefficients by performing transform on the residual block, and output the generated transform coefficients. Here, the transform coefficient may be a coefficient value generated by performing transform on the residual block. When the transform skip mode is applied, the transform unit 130 may omit the transform on the residual block.

Quantized transform coefficient levels may be generated by applying quantization to the transform coefficients. In the following embodiments, the quantized transform coefficient level may also be referred to as transform coefficient.

The quantization unit 140 may generate a quantized transform coefficient level by quantizing the transform coefficients according to the quantization parameter. The quantization unit 140 may output the generated quantized transform coefficient level. In this case, the quantization unit 140 may quantize the transform coefficients using the quantization matrix.

The entropy decoder 150 may generate a bitstream by performing entropy encoding according to a probability distribution based on the values calculated by the quantizer 140 and / or encoding parameter values calculated in the encoding process. . The entropy decoder 150 may output the generated bitstream.

The entropy decoder 150 may perform entropy encoding on information for decoding an image in addition to information about pixels of an image. For example, the information for decoding the image may include a syntax element.

The encoding parameter may be information required for encoding and / or decoding. The encoding parameter may include information encoded by the encoding apparatus and transmitted to the decoding apparatus, and may include information that may be inferred in the encoding or decoding process. For example, there is a syntax element as information transmitted to the decoding apparatus.

For example, coding parameters include prediction modes, motion vectors, reference picture indexes, coding block patterns, residual signals, transform coefficients, quantized transform coefficients, quantization parameters, block sizes, block partitions. It may include values or statistics such as information. The prediction mode may indicate an intra prediction mode or an inter prediction mode.

The residual signal may mean a difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming a difference between the original signal and the prediction signal. Alternatively, the residual signal may be a signal generated by transforming and quantizing the difference between the original signal and the prediction signal. The residual block may be a residual signal in block units.

When entropy coding is applied, a small number of bits may be allocated to a symbol having a high occurrence probability, and a large number of bits may be allocated to a symbol having a low occurrence probability. As the symbol is represented through this assignment, the size of the bitstring for the symbols to be encoded may be reduced. Therefore, compression performance of image encoding may be improved through entropy encoding.

In addition, for entropy coding, such as exponential golomb, context-adaptive variable length coding (CAVLC) and context-adaptive binary arithmetic coding (CABAC), etc. An encoding method can be used. For example, the entropy decoder 150 may perform entropy encoding by using a variable length coding (VLC) table. For example, the entropy decoder 150 may derive a binarization method for the target symbol. In addition, the entropy decoder 150 may derive a probability model of the target symbol / bin. The entropy decoder 150 may perform entropy encoding using the derived binarization method or the probability model.

Since encoding is performed through inter prediction by the encoding apparatus 100, the encoded current image may be used as a reference image with respect to other image (s) to be processed later. Therefore, the encoding apparatus 100 may decode the encoded current image again and store the decoded image as a reference image. Inverse quantization and inverse transform on the encoded current image may be processed for decoding.

The quantized coefficients may be inversely quantized in the inverse quantization unit 160 and inversely transformed in the inverse transformer 170. The inverse quantized and inverse transformed coefficients may be summed with the prediction block via the adder 175. A reconstructed block may be generated by adding the inverse quantized and inverse transformed coefficients and the prediction block.

The restored block may pass through the filter unit 180. The filter unit 180 may apply at least one or more of a deblocking filter, a sample adaptive offset (SAO), and an adaptive loop filter (ALF) to the reconstructed block or the reconstructed picture. have. The filter unit 180 may be referred to as an adaptive in-loop filter.

The deblocking filter may remove block distortion generated at boundaries between blocks. The SAO may add an appropriate offset value to the pixel value to compensate for coding errors. The ALF may perform filtering based on a value obtained by comparing the reconstructed image and the original image. The reconstructed block that has passed through the filter unit 180 may be stored in the reference picture buffer 190.

2 is a block diagram illustrating a configuration of a decoding apparatus according to an embodiment of the present invention.

The decoding apparatus 200 may be a video decoding apparatus or an image decoding apparatus.

Referring to FIG. 2, the decoding apparatus 200 may include an entropy decoder 210, an inverse quantizer 220, an inverse transformer 230, an intra predictor 240, an inter predictor 250, and an adder 255. The filter unit 260 may include a reference picture buffer 270.

The decoding apparatus 200 may receive a bitstream output from the encoding apparatus 100. The decoding apparatus 200 may perform intra mode and / or inter mode decoding on the bitstream. In addition, the decoding apparatus 200 may generate a reconstructed image by decoding, and output the generated reconstructed image.

For example, switching to the intra mode or the inter mode according to the prediction mode used for decoding may be made by a switch. When the prediction mode used for decoding is an intra mode, the switch may be switched to intra. When the prediction mode used for decoding is an inter mode, the switch may be switched to inter.

The decoding apparatus 200 may obtain a reconstructed residual block from the input bitstream, and generate a prediction block. When the reconstructed residual block and the prediction block are obtained, the decoding apparatus 200 may generate the reconstructed block by adding the reconstructed residual block and the prediction block.

The entropy decoder 210 may generate symbols by performing entropy decoding on the bitstream based on the probability distribution. The generated symbols may include symbols in the form of quantized coefficients. Here, the entropy decoding method may be similar to the entropy encoding method described above. For example, the entropy decoding method may be an inverse process of the above-described entropy encoding method.

The quantized coefficient may be inverse quantized by the inverse quantization unit 220. In addition, the inverse quantized coefficient may be inversely transformed by the inverse transformer 230. As a result of inverse quantization and inverse transformation of the quantized coefficients, a reconstructed residual block may be generated. In this case, the inverse quantization unit 220 may apply a quantization matrix to the quantized coefficients.

When the intra mode is used, the intra predictor 240 may generate a predictive block by performing spatial prediction using pixel values of blocks already decoded around the current block.

The inter predictor 250 may include a motion compensator. When the inter mode is used, the motion compensator may generate a prediction block by performing motion compensation using a motion vector and a reference image. The reference picture may be stored in the reference picture buffer 270.

The reconstructed residual block and the prediction block may be added through the adder 255. The adder 255 may generate the reconstructed block by adding the reconstructed residual block and the predictive block.

The restored block may pass through the filter unit 260. The filter unit 260 may apply at least one or more of the deblocking filter, SAO, and ALF to the reconstructed block or the reconstructed picture. The filter unit 260 may output the restored image. The reconstructed picture may be stored in the reference picture buffer 270 and used for inter prediction.

3 is a diagram schematically illustrating a division structure of an image when encoding and decoding an image.

In order to efficiently divide an image, a coding unit (CU) may be used in encoding and decoding. A unit may be a term that collectively refers to 1) a block including image samples and 2) a syntax element. For example, "division of a unit" may mean "division of a block corresponding to a unit".

Referring to FIG. 3, the image 300 may be sequentially divided in units of a largest coding unit (LCU), and the division structure of the image 300 may be determined according to the LCU. Here, the LCU may be used as the same meaning as a coding tree unit (CTU).

The partition structure may mean a distribution of a coding unit (CU) for efficiently encoding an image in the LCU 310. This distribution may be determined according to whether to divide one CU into four CUs. The horizontal size and the vertical size of the CU generated by the split may be half of the horizontal size and half of the vertical size, respectively, before the split. The partitioned CU may be recursively divided into four CUs whose width and length are reduced by half in the same manner.

At this time, partitioning of the CU may be performed recursively up to a predetermined depth. Depth information may be information indicating the size of a CU. Depth information may be stored for each CU. For example, the depth of the LCU may be zero, and the depth of the smallest coding unit (SCU) may be a predefined maximum depth. Here, the LCU may be a CU having a maximum coding unit size as described above, and the SCU may be a CU having a minimum coding unit size.

The division may start from the LCU 310, and the depth of the CU may increase by one whenever the horizontal and vertical sizes of the CU are reduced by half by the division. For each depth, the CU that is not divided may have a size of 2N × 2N. In addition, in the case of a partitioned CU, a CU of 2N × 2N size may be divided into four CUs having an N × N size. The size of N can be reduced by half for every 1 increase in depth.

Referring to FIG. 3, an LCU having a depth of 0 may be 64 × 64 pixels. 0 may be the minimum depth. An SCU of depth 3 may be 8x8 pixels. 3 may be the maximum depth. At this time, a CU of 64x64 pixels, which is an LCU, may be represented by a depth of zero. A CU of 32x32 pixels may be represented by depth one. A CU of 16 × 16 pixels may be represented by depth two. A CU of 8x8 pixels, which is an SCU, may be represented by depth 3.

In addition, information on whether the CU is split may be expressed through split information of the CU. The split information may be 1 bit of information. All CUs except the SCU may include partition information. For example, when the CU is not split, the value of partition information of the CU may be 0, and when the CU is split, the value of partition information of the CU may be 1.

4 is a diagram illustrating a form of a prediction unit PU that a coding unit CU may include.

A CU that is no longer split among CUs split from the LCU may be split into one or more prediction units (PUs). This partition may also be called a partition.

The PU may be a basic unit for prediction. The PU may be encoded and decoded in any one of a skip mode, an inter mode, and an intra mode. PU may be divided into various types according to each mode.

In skip mode, there may be no partition in the CU. In the skip mode, 2N × 2N mode 410 having the same size of PU and CU without splitting may be supported.

In the inter mode, eight divided forms in a CU may be supported. For example, in the inter mode, 2Nx2N mode 410, 2NxN mode 415, Nx2N mode 420, NxN mode 425, 2NxnU mode 430, 2NxnD mode 435, nLx2N mode 440, and nRx2N Mode 445 may be supported.

In intra mode, 2Nx2N mode 410 and NxN mode 425 may be supported.

In the 2Nx2N mode 410, a PU having a size of 2Nx2N may be encoded. A PU having a size of 2N × 2N may mean a PU having a size equal to the size of a CU. For example, a PU having a size of 2N × 2N may have a size of 64 × 64, 32 × 32, 16 × 16, or 8 × 8.

In the NxN mode 425, a PU having a size of NxN may be encoded.

For example, in intra prediction, when the size of the PU is 8x8, four divided PUs may be encoded. The size of the partitioned PU may be 4 × 4.

When the PU is encoded by the intra mode, the PU may be encoded using one intra prediction mode among the plurality of intra prediction modes. For example, High Efficiency Video Coding (HEVC) technology can provide 35 intra prediction modes, and the PU can be coded in one of the 35 intra prediction modes.

Which of the 2Nx2N mode 410 and NxN mode 425 is to be coded may be determined by the rate-distortion cost.

The encoding apparatus 100 may perform an encoding operation on a PU having a size of 2N × 2N. Here, the encoding operation may be to encode the PU in each of a plurality of intra prediction modes that the encoding apparatus 100 may use. Through the coding operation, an optimal intra prediction mode for a 2N × 2N size PU may be derived. The optimal intra prediction mode may be an intra prediction mode that generates a minimum rate-distortion cost for encoding a 2N × 2N size PU among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

In addition, the encoding apparatus 100 may sequentially perform encoding operations on each PU of the PUs divided by N × N. Here, the encoding operation may be to encode the PU in each of a plurality of intra prediction modes that the encoding apparatus 100 may use. Through the coding operation, an optimal intra prediction mode for a N × N size PU may be derived. The optimal intra prediction mode may be an intra prediction mode that generates a minimum rate-distortion cost for encoding of a PU of an N × N size among a plurality of intra prediction modes that can be used by the encoding apparatus 100.

The encoding apparatus 100 may determine which of 2Nx2N size PU and NxN size PU to encode based on a comparison of the rate-distortion cost of the 2Nx2N size PU and the rate-distortion costs of the NxN size PUs.

FIG. 5 is a diagram illustrating a form of a transform unit (TU) that may be included in a coding unit (CU).

A transform unit (TU) may be a basic unit used for a process of transform, quantization, inverse transform, inverse quantization, entropy encoding, and entropy decoding in a CU. The TU may have a square shape or a rectangular shape.

Of the CUs partitioned from the LCU, a CU that is no longer split into CUs may be split into one or more TUs. In this case, the partition structure of the TU may be a quad-tree structure. For example, as shown in FIG. 5, one CU 510 may be divided one or more times according to the quad-tree structure. Through division, one CU 510 may be configured with TUs of various sizes.

In the encoding apparatus 100, a 64x64 coding tree unit (CTU) may be split into a plurality of smaller CUs by a recursive quad-tree structure. One CU may be divided into four CUs having the same sizes. CUs may be recursively split, and each CU may have a quad tree structure.

The CU may have a depth. If a CU is split, the CUs created by splitting may have a depth increased by one from the depth of the split CU.

For example, the depth of the CU may have a value of 0 to 3. The size of the CU may be from 64x64 to 8x8 depending on the depth of the CU.

Through recursive partitioning for a CU, an optimal partitioning method can be selected that produces the smallest rate-distortion ratio.

6 is a diagram for explaining an embodiment of an intra prediction process.

Arrows outward from the center of the graph of FIG. 6 may indicate prediction directions of intra prediction modes. In addition, the number displayed near the arrow may represent an example of a mode value allocated to the intra prediction mode or the prediction direction of the intra prediction mode.

Intra encoding and / or decoding may be performed using reference samples of units around the target unit. Peripheral units may be peripheral restored units. For example, intra encoding and / or decoding may be performed using a value or encoding parameter of a reference sample included in a neighboring reconstructed unit.

The encoding apparatus 100 and / or the decoding apparatus 200 may generate the prediction block by performing intra prediction on the target unit based on the information of the sample in the current picture. When performing intra prediction, the encoding apparatus 100 and / or the decoding apparatus 200 may generate a prediction block for a target unit by performing intra prediction based on information of a sample in a current picture. When performing intra prediction, the encoding apparatus 100 and / or the decoding apparatus 200 may perform directional prediction and / or non-directional prediction based on at least one reconstructed reference sample.

The prediction block may mean a block generated as a result of performing intra prediction. The prediction block may correspond to at least one of a CU, a PU, and a TU.

The unit of a prediction block may be the size of at least one of a CU, a PU, and a TU. The prediction block may have a square shape, having a size of 2N × 2N or a size of N × N. The size of NxN may include 4x4, 8x8, 16x16, 32x32 and 64x64.

Alternatively, the prediction block may be a block in the form of a square having a size of 2x2, 4x4, 16x16, 32x32, or 64x64, or a rectangular block having a size of 2x8, 4x8, 2x16, 4x16, and 8x16.

Intra prediction may be performed according to an intra prediction mode for the target unit. The number of intra prediction modes that the target unit may have may be a predefined fixed value or may be a value determined differently according to the attributes of the prediction block. For example, the attributes of the prediction block may include the size of the prediction block and the type of the prediction block.

For example, the number of intra prediction modes may be fixed to 35 regardless of the size of the prediction unit. Or, for example, the number of intra prediction modes may be 3, 5, 9, 17, 34, 35, 36, or the like.

The intra prediction mode may include two non-directional modes and 33 directional modes as shown in FIG. 6. Two non-directional modes may include a DC mode and a planar mode.

For example, in the vertical mode having a mode value of 26, prediction may be performed in the vertical direction based on the pixel value of the reference sample. For example, in the horizontal mode having a mode value of 10, prediction may be performed in the horizontal direction based on the pixel value of the reference sample. For example, in the vertical mode having a mode value of 26, prediction may be performed in the vertical direction based on the pixel value of the reference sample.

Even in the directional mode other than the above-described mode, the encoding apparatus 100 and the decoding apparatus 200 may perform intra prediction on the target unit using the reference sample according to the angle corresponding to the directional mode.

The intra prediction mode located on the right side of the vertical mode may be referred to as a vertical right mode. The intra prediction mode located at the bottom of the horizontal mode may be referred to as a horizontal-below mode. For example, in FIG. 6, intra prediction modes in which the mode value is one of 27, 28, 29, 30, 31, 32, 33, and 34 may be vertical right modes 613. Intra prediction modes with a mode value of one of 2, 3, 4, 5, 6, 7, 8, and 9 may be horizontal bottom modes 616.

The non-directional mode may include a DC mode and a planar mode. For example, the mode value of the DC mode may be 1. The mode value of the planner mode may be zero.

The directional mode may include an angular mode. Among the plurality of intra prediction modes, a mode other than the DC mode and the planner mode may be a directional mode.

In the DC mode, a prediction block may be generated based on an average of pixel values of the plurality of reference samples. For example, the value of a pixel of the prediction block may be determined based on an average of pixel values of the plurality of reference samples.

The number of intra prediction modes described above and the mode value of each intra prediction modes may be exemplary only. The number of intra prediction modes described above and the mode value of each intra prediction modes may be defined differently according to an embodiment, implementation, and / or need.

The number of intra prediction modes may differ depending on the type of color component. For example, the number of prediction modes may vary depending on whether the color component is a luma signal or a chroma signal.

7 is a diagram for describing a position of a reference sample used in an intra prediction process.

7 shows the location of a reference sample used for intra prediction of a target unit. Referring to FIG. 7, a reconstructed reference pixel used for intra prediction of a current block includes, for example, lower-left reference samples 731 and left reference samples 733. , Upper-left corner reference sample 735, upper-reference samples 737, upper-right reference samples 739, and the like.

For example, the left reference samples 733 may refer to a reconstructed reference pixel adjacent to the left side of the target unit. The top reference samples 737 may refer to a reconstructed reference pixel adjacent to the top of the target unit. The upper left corner reference pixel 735 may mean a restored reference pixel located at the upper left corner of the target unit. Also, the lower left reference samples 731 may refer to a reference sample located at the bottom of the left sample line among samples positioned on the same line as the left sample line composed of the left reference samples 733. The upper right reference samples 739 may refer to reference samples positioned to the right of the upper pixel line among samples positioned on the same line as the upper sample line formed of the upper reference samples 737.

When the size of the target unit is N × N, the lower left reference samples 731, the left reference samples 733, the upper reference samples 737, and the upper right reference samples 739 may each be N pieces.

The prediction block may be generated through intra prediction on the target unit. Generation of the predictive block may include determining a value of pixels of the predictive block. The size of the target unit and the prediction block may be the same.

The reference sample used for intra prediction of the target unit may vary according to the intra prediction mode of the target unit. The direction of the intra prediction mode may indicate a dependency relationship between the reference samples and the pixels of the prediction block. For example, the value of the specified reference sample can be used as the value of the specified one or more pixels of the prediction block. In this case, the specified one or more specified pixels of the specified reference sample and prediction block may be samples and pixels designated by a straight line in the direction of the intra prediction mode. In other words, the value of the specified reference sample may be copied to the value of the pixel located in the reverse direction of the intra prediction mode. Alternatively, the pixel value of the prediction block may be a value of a reference sample located in the direction of the intra prediction mode based on the position of the pixel.

For example, when the intra prediction mode of the target unit is a vertical mode having a mode value of 26, the upper reference samples 737 may be used for intra prediction. When the intra prediction mode is the vertical mode, the value of the pixel of the prediction block may be the value of the reference pixel located vertically above the position of the pixel. Thus, the top reference samples 737 adjacent to the top of the target unit can be used for intra prediction. Also, the values of the pixels of one row of the prediction block may be the same as the values of the top reference samples 737.

For example, when the intra prediction mode of the current block is a horizontal mode with a mode value of 10, the left reference samples 733 may be used for intra prediction. When the intra prediction mode is the horizontal mode, the pixel value of the prediction block may be a value of a reference pixel located horizontally on the left side with respect to the pixel. Thus, left reference samples 733 adjacent to the target unit to the left may be used for intra prediction. In addition, the values of the pixels of one column of the prediction block may be the same as the values of the left reference samples 733.

For example, if the mode value of the intra prediction mode of the current block is 18, at least some of the left reference samples 733, the upper left corner reference sample 735, and the at least some intra prediction of the top reference samples 737 are included. Can be used. When the mode value of the intra prediction mode is 18, the value of the pixel of the prediction block may be the value of the reference pixel located at the top left diagonally with respect to the pixel.

In addition, when an intra prediction mode having a mode value of 27, 28, 29, 30, 31, 32, 33, or 34 is used, at least some of the upper right reference pixels 439 may be used for intra prediction.

In addition, when an intra prediction mode having a mode value of 2, 3, 4, 5, 6, 7, 8, or 9 is used, at least some of the lower left reference pixels 431 may be used for intra prediction.

In addition, when an intra prediction mode having a mode value of 11 to 25 is used, the upper left corner reference samples 735 may be used for intra prediction.

The reference sample used to determine the pixel value of one pixel of the prediction block may be one, or may be two or more.

As described above, the pixel value of the pixel of the prediction block may be determined according to the position of the reference sample indicated by the position of the pixel and the direction of the intra prediction mode. If the position of the reference sample indicated by the position of the pixel and the direction of the intra prediction mode is an integer position, the value of one reference sample indicated by the integer position may be used to determine the pixel value of the pixel of the prediction block.

If the position of the reference sample indicated by the position of the pixel and the direction of the intra prediction mode is not an integer position, an interpolated reference sample may be generated based on the two reference samples closest to the position of the reference sample. have. The value of the interpolated reference sample can be used to determine the pixel value of the pixel of the prediction block. In other words, when the position of the reference sample indicated by the position of the pixel of the prediction block and the direction of the intra prediction mode indicates between the two reference samples, an interpolated value is generated based on the values of the two samples. Can be.

The prediction block generated by the prediction may not be the same as the original target unit. In other words, a prediction error that is a difference between the target unit and the prediction block may exist, and the prediction error may exist between the pixel of the target unit and the pixel of the prediction block. For example, in the case of directional intra prediction, the greater the distance between the pixel and the reference sample of the prediction block, the larger prediction error may occur. Discontinuity may occur between the prediction block and the neighboring block generated by such a prediction error.

Filtering on the prediction block may be used to reduce the prediction error. The filtering may be to adaptively apply a filter to a region that is considered to have a large prediction error in the prediction block. For example, an area considered to have a large prediction error may be a boundary of a prediction block. In addition, according to the intra prediction mode, an area considered to have a large prediction error among the prediction blocks may be different, and characteristics of the filter may be different.

8 is a diagram for explaining an embodiment of an inter prediction process.

The rectangle illustrated in FIG. 8 may represent an image (or picture). In addition, arrows in FIG. 8 may indicate prediction directions. That is, the image may be encoded and / or decoded according to the prediction direction.

Each picture (or picture) may be classified into an I picture (Intra Picture), a P picture (Uni-prediction Picture), and a B picture (Bi-prediction Picture) according to an encoding type. Each picture may be encoded according to an encoding type of each picture.

When the image to be encoded is an I picture, the image may be encoded with respect to the image itself without inter prediction. When the image to be encoded is a P picture, the image may be encoded through inter prediction using a reference picture only in the forward direction. When the image to be encoded is a B picture, it may be encoded through inter prediction using reference pictures in both the forward and reverse directions, and may be encoded through inter prediction using the reference picture in one of the forward and reverse directions.

The P picture and the B picture encoded and / or decoded using the reference picture may be regarded as an image using inter prediction.

In the following, inter prediction in inter mode according to an embodiment is described in detail.

In the inter mode, the encoding apparatus 100 and the decoding apparatus 200 may perform prediction and / or motion compensation on the encoding target unit and the decoding target unit. For example, the encoding apparatus 100 or the decoding apparatus 200 may perform prediction and / or motion compensation by using the reconstructed motion information of the neighboring unit as the motion information of the encoding target unit or the decoding target unit. Here, the encoding target unit or the decoding target unit may mean a prediction unit and / or a prediction unit partition.

Inter prediction may be performed using a reference picture and motion information. In addition, inter prediction may use the skip mode described above.

The reference picture may be at least one of a previous picture of the current picture or a subsequent picture of the current picture. In this case, the inter prediction may perform prediction on a block of the current picture based on the reference picture. Here, the reference picture may mean an image used for prediction of a block.

In this case, an area in the reference picture may be specified by using a reference picture index refIdx indicating a reference picture, a motion vector to be described later, and the like.

The inter prediction may select a reference picture and a reference block corresponding to the current block within the reference picture, and generate the prediction block for the current block using the selected reference block. The current block may be a block that is a target of current encoding or decoding among blocks of the current picture.

The motion information may be derived during inter prediction by each of the encoding apparatus 100 and the decoding apparatus 200. In addition, the derived motion information may be used to perform inter prediction.

In this case, the encoding apparatus 100 and the decoding apparatus 200 may use encoding information and / or decoding efficiency by using motion information of a restored neighboring block and / or motion information of a collocated block (col block). Can improve. The call block may be a block corresponding to the current block in a collocated picture (col picture).

The reconstructed neighboring block may be a block in the current picture and may be a block already reconstructed through encoding and / or decoding. The reconstructed block may be a neighboring block adjacent to the current block and / or a block located at an outer corner of the current block. Here, the block located at the outer corner of the current block may be a block vertically adjacent to a neighboring block horizontally adjacent to the current block or a block horizontally adjacent to a neighboring block vertically adjacent to the current block.

For example, a restored peripheral unit may be a unit located to the left of the target unit, a unit located at the top of the target unit, a unit located at the lower left corner of the target unit, a unit located at the upper right corner of the target unit, or an upper left of the target unit. It may be a unit located at the corner.

Each of the encoding apparatus 100 and the decoding apparatus 200 may determine a block existing at a position corresponding to a current block spatially in the call picture, and may determine a predetermined relative position based on the determined block. The predefined relative position may be a position inside and / or outside of a block that exists spatially at a position corresponding to the current block. In addition, each of the encoding apparatus 100 and the decoding apparatus 200 may derive a call block based on the determined predetermined relative position. Here, the call picture may be one picture among at least one reference picture included in the reference picture list.

The block in the reference picture may exist at a position spatially corresponding to the position of the current block in the reconstructed reference picture. In other words, the position of the current block in the current picture and the position of the block in the reference picture may correspond to each other. Hereinafter, motion information of a block included in the reference picture may be referred to as temporal motion information.

The method of deriving the motion information may vary according to the prediction mode of the current block. For example, as a prediction mode applied for inter prediction, there may be an advanced motion vector predictor (AMVP) and merge.

For example, when AMVP is applied as the prediction mode, each of the encoding apparatus 100 and the decoding apparatus 200 may predict the motion vector candidate using the motion vector of the reconstructed neighboring block and / or the motion vector of the call block. You can create a list. The motion vector of the reconstructed neighboring block and / or the motion vector of the collocated block may be used as a prediction motion vector candidate.

The bitstream generated by the encoding apparatus 100 may include a predicted motion vector index. The prediction motion vector index may indicate an optimal prediction motion vector selected from the prediction motion vector candidates included in the prediction motion vector candidate list. The predicted motion vector index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through the bitstream.

The decoding apparatus 200 may select the prediction motion vector of the current block from the prediction motion vector candidates included in the prediction motion vector candidate list by using the prediction motion vector index.

The encoding apparatus 100 may calculate a motion vector difference (MVD) between the motion vector and the predictive motion vector of the current block, and may encode the MVD. The bitstream may include encoded MVD. The MVD may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. At this time, the decoding apparatus 200 may decode the received MVD. The decoding apparatus 200 may derive the motion vector of the current block through the sum of the decoded MVD and the predictive motion vector.

The bitstream may include a reference picture index and the like indicating the reference picture. The reference picture index may be transmitted from the encoding apparatus 100 to the decoding apparatus 200 through a bitstream. The decoding apparatus 200 may predict the motion vector of the current block by using the motion information of the neighboring block, and may derive the motion vector of the current block by using the predicted motion vector and the motion vector difference. The decoding apparatus 200 may generate a prediction block for the current block based on the derived motion vector and the reference picture index information.

Since the motion information of the neighboring unit reconstructed with respect to the encoding target unit and the decoding target unit may be used, in a specific inter prediction mode, the encoding apparatus 100 may not separately encode the motion information for the target unit. If the motion information of the target unit is not encoded, the amount of bits transmitted to the decoding apparatus 200 may be reduced, and encoding efficiency may be improved. For example, the inter prediction mode in which the motion information of the target unit is not encoded may include a skip mode and / or a merge mode. In this case, the encoding apparatus 100 and the decoding apparatus 200 may use an identifier and / or an index indicating which unit of the reconstructed neighboring units is used as the movement information of the target unit.

Another example of a method of deriving motion information is merge. Merge may mean merging of motions for a plurality of blocks. Merge may mean applying motion information of one block to other blocks. When the merge is applied, each of the encoding apparatus 100 and the decoding apparatus 200 may generate a merge candidate list using the motion information of the reconstructed neighboring block and / or the motion information of the call block. have. The motion information may include at least one of 1) a motion vector, 2) an index for a reference image, and 3) a prediction direction. The prediction direction may be unidirectional or bidirectional.

In this case, the merge may be applied in a CU unit or a PU unit. When merging is performed in a CU unit or a PU unit, the encoding apparatus 100 may transmit predefined information to the decoding apparatus 200 through a bitstream. The bitstream may include predefined information. The predefined information may include 1) information indicating whether to merge for each block partition, and 2) information about which one of neighboring blocks adjacent to the current block to merge with. For example, the neighboring blocks of the current block may include a left neighboring block of the current block, a top neighboring block of the current block, a temporal neighboring block of the current block, and the like.

The merge candidate list may represent a list in which motion information is stored. In addition, the merge candidate list may be generated before the merge is performed. The motion information stored in the merge candidate list may be 1) motion information of a neighboring block adjacent to the current block or 2) collocated block motion information corresponding to the current block in the reference image. In addition, the motion information stored in the merge candidate list may be new motion information generated by a combination of motion information already present in the merge candidate list.

The skip mode may be a mode in which information of neighboring blocks is applied to the current block as it is. The skip mode may be one of modes used for inter prediction. When the skip mode is used, the encoding apparatus 100 may transmit only information on which block motion information to use as the motion information of the current block to the decoding apparatus 200 through the bitstream. The encoding apparatus 100 may not transmit other information to the decoding apparatus 200. For example, the other information may be syntax information. The syntax information may include motion vector difference information.

9 illustrates second prediction for a portion of a current block according to an example.

The current block shown in FIG. 9 may be a PU.

When the optimal intra prediction for the current block is performed, a large prediction error may occur in a part because it is predicted in only one direction with respect to the entirety of the current block.

For a portion where a large prediction error occurs, a method of dividing the current block into a non-sqaure type may be used.

In other words, after the first prediction is performed on a square prediction unit, if a second prediction of another prediction mode is performed on a portion having a large prediction error, the prediction error for the current block may be reduced.

For example, the first prediction mode and the second prediction mode may be an intra prediction mode.

In FIG. 9, the current block may be square. The prediction direction of the first prediction for the whole of the current block is shown. For example, the prediction mode of the first prediction may be a vertical mode having a value of 26.

The current block may be divided into a first portion and a second portion. Each of the first portion and the second portion may be non-square.

In FIG. 9, the prediction direction of the second prediction for the first part is shown, and the prediction direction of the second prediction for the second part is shown. For example, the prediction mode of the second prediction for the first portion may be a horizontal mode having a value of ten. The prediction mode of the second prediction for the second portion may be the lowest diagonal mode with a value of 34.

The shape of the portion with the large prediction error may correspond to the direction of the optimal prediction mode of the current block. In the following embodiments, a method and apparatus for improving the performance of prediction and reducing the prediction error by efficiently using the tendency that the shape of the portion having the large prediction error corresponds to the direction of the optimal prediction mode of the current block is described. Can be.

10 is a structural diagram of an encoding apparatus according to an embodiment.

The encoding apparatus 1000 may be a general-purpose computer system that performs encoding.

As shown in FIG. 10, the encoding apparatus 1000 may include at least one processor 1010, a memory 1030, a user interface (UI) input device 1050, which communicates with each other via a bus 1090, and the like. UI output device 1060 and storage 1040. In addition, the encoding apparatus 1000 may further include a communication unit 1020 connected to the network 1099. The processor 1010 may be a semiconductor device that executes processing instructions stored in a central processing unit (CPU), a memory 1030, or a storage 1040. Memory 1030 and storage 1040 may be various types of volatile or nonvolatile storage media. For example, the memory may include at least one of a ROM 1031 and a RAM 1032.

The processor 1010 is an inter predictor 110, an intra predictor 120, a switch 115, a subtractor 125, a transformer 130, a quantizer 140, and an entropy decoder of the encoding apparatus 100. 150, an inverse quantization unit 160, an inverse transform unit 170, an adder 175, a filter unit 180, and a reference picture buffer 190.

The recording medium may store at least one module for the operation of the encoding apparatus 1000. The memory 1030 may store at least one module and may be configured to be executed by the at least one processor 1010.

According to an embodiment, the inter prediction unit 110, the intra prediction unit 120, the switch 115, the subtractor 125, the transform unit 130, the quantization unit 140, and the entropy decoding of the encoding apparatus 100 may be used. At least some of the unit 150, the inverse quantizer 160, the inverse transform unit 170, the adder 175, the filter unit 180, and the reference picture buffer 190 may be program modules, and may be an external device or system. Communicate with The program modules may be included in the encoding apparatus 1000 in the form of an operating system, an application module, and other program modules.

The program modules may be physically stored on various known storage devices. In addition, at least some of these program modules may be stored in a remote storage device that can communicate with the encoding apparatus 1000.

Program modules perform routines or subroutines, programs, objects, components, and data to perform functions or operations, or to implement abstract data types, according to one embodiment. Data structures and the like, but is not limited thereto.

The program modules may be composed of instructions or code performed by the processor 1010.

Functions related to communication of data or information of the encoding apparatus 1000 may be performed through the communication unit 1020.

11 is a flowchart of an encoding method according to an embodiment.

The encoding method of the embodiment may be performed by the encoding apparatus 100 or the encoding apparatus 1000.

In operation 1110, the prediction unit may generate a residual signal of the current block based on the current block, the first prediction, and the second prediction.

The predictor may include an inter predictor 110, an intra predictor 120, and a subtractor 125.

For example, the residual signal may represent a residual block.

For example, the residual block may be the difference between the current block and the prediction block. Alternatively, the residual block may be generated based on the difference between the current block and the prediction block.

For example, the prediction block may be the sum of the first prediction block and the second prediction block. The residual block may be obtained by subtracting the sum of the first prediction block and the second prediction block from the current block.

The first prediction block may be a block generated by the first prediction. The second prediction block may be a block generated by the second prediction. If the second prediction is not used, the second prediction block may be a null block. The null block may be a block in which the values of all pixels of the block are "0".

The prediction unit may generate a residual block of the current block by performing inter prediction or intra prediction.

In an embodiment, the prediction unit may generate a second prediction block based on the current block and / or the first prediction block.

In one embodiment, the prediction unit may generate a residual signal of the current block by performing a first prediction on the current block and performing a second prediction on the first prediction error that is a result of the first prediction. Here, the first prediction error may be a difference between the current block and the first prediction signal. The residual signal may be a difference between the first prediction error and the second prediction signal.

In one embodiment, the first prediction and the second prediction may be different kinds of predictions.

In one embodiment, the first prediction and the second prediction may be the same kind of predictions. For example, each of the first prediction and the second prediction may be intra prediction. If the first prediction is intra prediction, the second prediction may also be set to intra prediction.

In one embodiment, the prediction direction of the first prediction and the prediction direction of the second prediction may be the same. The prediction direction of the second prediction may be set to be the same as the prediction direction of the first prediction.

For example, in one embodiment, each of the first prediction and the second prediction may be inter prediction.

In one embodiment, a block that is the target of the first prediction and a block that is the target of the second prediction may be different from each other. Here, the "block to be predicted" may mean to which block the "prediction" generates a prediction block.

For example, the type of the block targeted for the first prediction and the type of the block targeted for the second prediction may be different from each other. The type of block may be an original block, a luma block, a chroma block, a depth block, a residual block, or the like.

For example, the block that is the target of the first prediction may be the current block or the original block.

For example, the first prediction error generated by the first prediction may represent the first prediction error block. The first prediction error may be a first residual signal. Also, the first residual signal may represent a first residual block. The block that is the target of the second prediction may be a first prediction error block or a first residual block. In this case, the first residual signal may be generated by the first prediction, and the second prediction error may be generated by the second prediction. The second prediction error may be a second residual signal. The second residual signal may represent a second residual block.

Reference blocks may be used for prediction. In one embodiment, the first reference block of the first prediction and the second reference block of the second prediction may be different from each other. The reference blocks may differ from each other in terms of type and / or location.

In one embodiment, the position of the first reference block of the first prediction and the position of the second reference block of the second prediction may be different from each other. Here, the position of the first reference block may be a position relative to the block that is the target of the first prediction. The position of the second reference block may be a position relative to the block that is the target of the second prediction.

In one embodiment, the first reference block of the first prediction may be plural. The second reference block of the second prediction may be plural. At least a portion of the plurality of first reference blocks of the first prediction and the plurality of second reference blocks of the second prediction may be different from each other.

The position of at least one of the plurality of first reference blocks of the first prediction and the plurality of second reference blocks of the second prediction is the positions of the plurality of first reference blocks and the positions of the plurality of second reference blocks. Can only be included in one of the

In one embodiment, the type of the first reference block of the first prediction and the type of the second reference block of the second prediction may be different from each other. The type of block may be a reconstructed block, a reconstructed luma block, a reconstructed chroma block, a reconstructed depth block, a reconstructed first residual block, a reconstructed second residual block, or the like.

For example, the first reference block of the first prediction may be a reconstructed block. The second reference block of the second prediction may be a reconstructed first residual block. The first reference sample of the first prediction may be a pixel of the reconstructed block. The second reference sample of the second prediction may be a pixel of the reconstructed first residual block.

For example, the first reference block of the first prediction may be a reconstructed neighboring block neighboring the current block. The second reference block of the second prediction may be a reconstructed first residual block neighboring the current block.

The reconstructed neighbor residual block neighboring the current block may be obtained by adding the reconstructed residual block of the reconstructed neighboring block to the second prediction block of the reconstructed neighboring block. Alternatively, the reconstructed neighbor residual block neighboring the current block may be a difference between the reconstructed neighboring block and the first prediction block of the reconstructed neighboring block.

For example, when the object of the first prediction is the current block, a reconstructed block around the current block may be used as a reference block for prediction of the current block. When the target of the second prediction is the first residual block, a reconstructed first residual block around the current block or the first residual block may be used as the second reference block for prediction on the first residual block.

In one embodiment, the region of the first prediction and the region of the second prediction may be different from each other. Here, the "prediction area" may indicate an area in which a prediction value is generated among blocks that are to be predicted. Alternatively, the “area of prediction” may indicate an area to which a prediction value generated by prediction is allocated among prediction blocks corresponding to a block that is a prediction target.

In one embodiment, the second prediction may be a prediction for a portion of the current block. The first prediction may be a prediction of the entirety of the current block. Alternatively, the first prediction may be a prediction for a part of the current block except for the part predicted by the second prediction.

For example, the region of the second prediction may be part of the current block. The area of the first prediction may be the whole of the current block. Alternatively, the region of the first prediction may be a portion of the current block except for the portion of the second prediction.

For example, the region of the second prediction may be a non-square portion of the current block. The area of the first prediction may be the whole of the current block. Alternatively, the region of the first prediction may be a non-square portion of the current block except for the portion of the second prediction.

For example, prediction values determined by the first prediction may be assigned only to pixels in the region of the first prediction among blocks that are the targets of the first prediction. The prediction value according to the first prediction may not be allocated to pixels outside the region of the first prediction among the blocks that are the targets of the first prediction. Alternatively, a predefined value may be assigned to pixels outside the region of the first prediction among blocks that are the target of the first prediction. The predefined value may be "0".

For example, prediction values determined by the second prediction may be assigned only to pixels in the region of the second prediction among blocks that are the target of the second prediction. Pixels outside the region of the second prediction among the blocks that are the target of the second prediction may not be assigned a prediction value by the second prediction. Alternatively, a predefined value may be assigned to pixels outside the region of the second prediction among blocks that are the target of the second prediction. For example, the predefined value may be "0".

In one embodiment, the region of the first prediction may be determined based on the type of the first prediction. For example, the region of the first prediction may be determined based on whether the first prediction is inter prediction. Alternatively, the region of the first prediction may be determined based on whether the first prediction is intra prediction. Alternatively, the region of the first prediction may be determined based on the prediction direction of the first prediction.

In one embodiment, the region of the second prediction may be determined based on the type of the second prediction. For example, the region of the second prediction may be determined based on whether the second prediction is inter prediction. Alternatively, the region of the second prediction may be determined based on whether the second prediction is intra prediction. Alternatively, the region of the second prediction may be determined based on the prediction direction of the second prediction.

In operation 1120, the encoder may perform encoding on the residual signal. The encoder may generate information about the encoded residual signal by performing encoding on the residual signal.

The encoder may include a transformer 130, a quantizer 140, and an entropy encoder 150.

Step 1120 may include steps 1121, 1122, and 1123.

In operation 1121, the transformer 130 may generate transform coefficients of the residual signal.

In operation 1122, the quantization unit 140 may generate quantized transform coefficients by performing quantization on the transform coefficients.

In operation 1123, the entropy encoder 150 may generate information on the encoded residual signal by performing entropy encoding on the quantized transform coefficients.

In operation 1130, the entropy encoder 150 may generate a bitstream including information about the encoded residual signal.

The bitstream may include information about the encoded residual signal, and may include information related to prediction.

The entropy encoder 150 may include information related to prediction in the bitstream. Related information of the prediction may be entropy coded.

For example, the information related to the prediction may include prediction scheme information indicating how the current block is encoded.

For example, the prediction scheme information may indicate whether the current block is encoded by intra prediction or intra prediction. Alternatively, the prediction scheme information may indicate whether the current block is encoded by intra prediction. Alternatively, the prediction scheme information may indicate whether the current block is encoded by inter prediction.

For example, the information related to the prediction may include intra prediction mode information indicating a prediction mode of the intra prediction.

For example, the information related to the prediction may include second prediction usage information indicating whether the second prediction is used in encoding the current block.

For example, the information related to the prediction may include first prediction mode information indicating the prediction mode of the first prediction and second prediction mode information indicating the prediction mode of the second prediction.

For example, the information related to the prediction may include selected part information indicating a selected part to which the second prediction is applied among the two parts generated by the division of the current block.

For example, the information related to the prediction includes current block type information indicating the type of the current block, first prediction block type information indicating the type of the first prediction block, second prediction block type information indicating the type of the second prediction block, First reference block type information indicating a type of a reference block, second reference block type information indicating a type of a second reference block, first reference sample type information indicating a type of a first reference sample, and a type of a second reference sample Second reference sample type information indicating the information may be included.

For example, the information related to the prediction may include first prediction region information indicating the region of the first prediction and / or second prediction region information indicating the second prediction region of the second prediction.

Information related to the prediction may include information used for the prediction described in the embodiments. The entropy encoder 150 may include the information related to the prediction in the bitstream in the order described in the embodiment or the generation order according to the embodiment.

In operation 1140, the communication unit 1020 may transmit the bitstream to the decoding apparatus 200 or the decoding apparatus 1700.

In operation 1150, the reconstructed residual signal generator may generate a reconstructed residual signal based on the information about the residual signal.

The reconstructed residual signal generator may include an inverse quantizer 160 and an inverse transformer 170.

Step 1150 may include steps 1151 and 1152.

In operation 1151, the inverse quantization unit 160 may generate the reconstructed transform coefficients by performing inverse quantization on the quantized transform coefficients.

In operation 1152, the transformer 130 may generate the reconstructed residual signal by performing transform on the reconstructed transform coefficients.

In operation 1160, the reconstructed block generator may generate a reconstructed block based on the reconstructed residual signal, the first prediction, and the second prediction.

The restored block generator may include an adder 175.

For example, the sum of the reconstructed residual signal and the prediction signal may represent a reconstructed block. Alternatively, the reconstructed block may be generated based on the sum of the reconstructed residual signal and the prediction signal.

For example, the reconstructed block may be the sum of the reconstructed residual block and the prediction block. The prediction block may be the sum of the first prediction block and the second prediction block. The second prediction signal may be a signal generated by the second prediction for the portion of the current block. The first prediction signal may be a signal generated by the first prediction on the current block or a portion of the current block except for the portion of the second prediction.

Alternatively, the reconstructed block may be generated based on the sum of the reconstructed residual block and the prediction block.

For example, the reconstructed block may be a block generated by the reconstructed residual block, the first prediction block, and the second prediction block. The reconstructed block generator may generate the reconstructed first residual signal by adding the reconstructed residual signal and the second prediction signal. The reconstructed block generator may generate the reconstructed block by adding the reconstructed first residual signal and the first prediction signal.

Alternatively, the reconstructed block generator may generate the reconstructed first residual block by adding the reconstructed residual block and the second prediction block. The reconstructed block generator may generate the reconstructed block by adding the reconstructed first residual block and the first prediction block.

The reconstructed block and the reconstructed first residual signal may be used as reference blocks for encoding other blocks.

12 illustrates splitting of a current block according to an embodiment.

In Fig. 12, the current block of the size of NxN is shown.

The current block can be divided into two parts. The two parts may be determined based on a straight line passing through the center of the current block in the prediction direction of the first prediction. In FIG. 12, the prediction direction of the first prediction may be the prediction direction of the prediction mode having a mode value of three.

The first prediction may be a prediction using a first optimal prediction mode obtained through intra prediction for the current block.

The current block may be split into two parts such that the number of pixels in the two parts is the same using the prediction mode of the first prediction.

Here, the division of the current block in the prediction direction of the first prediction may be because a large prediction error tends to occur in a section specified in the prediction direction when the prediction on the current block is performed by intra prediction. .

In Figure 12, two parts are shown as S1 and S2. Among the points at which the value of the x coordinate is i, a point at which the value of the y coordinate is j ₂ may be included in S1. Among the points at which the value of the x coordinate is i, a point at which the value of the y coordinate is j ₁ may be included in S2.

The prediction direction of the prediction mode may be calculated according to Equation 2, Equation 3, Equation 4, Equation 5, Equation 6, and Table 1 below.

A prediction mode with a mode value of 0 or 1 may be excluded from division because it is a non-directional prediction mode.

predMode may indicate the number of prediction modes. vertical may indicate that the prediction mode corresponds to the vertical mode. horizontal may indicate that the prediction mode corresponds to the horizontal mode.

The value of absAngle can be defined as shown in Table 1 below.

absAngle 0 One 2 3 4 5 6 7 8 value 0 2 5 9 13 17 21 26 32

When the value of Tan of the prediction mode is calculated, the pixel of the coordinate (i, j) of the current block may be included in at least one of the portion S0 and the portion S1 according to Equation 7 below.

When the mode value of the prediction mode PredMode is 3, 18, or 34, the value of Tan may be -1 or +1. In this case, the pixel on the straight line dividing the current block may be included in each of the two parts S0 and S1.

In other words, when the current block is divided into two parts by a straight line passing through the center of the current block, pixels lying on the straight line of the current block may be included in each of the two parts.

13 is a flowchart of a method of generating a residual block, according to an exemplary embodiment.

Step 1110 described above with reference to FIG. 11 may include steps 1310, 1320, 1330, 1335, 1340, 1345, 1350, 1355, 1360, 1370, and 1380.

In one embodiment, the block that is the target of the first prediction may be the current block. The first prediction may be intra prediction.

In operation 1310, the prediction unit may generate a first prediction signal by performing a first prediction on the current block.

The first prediction signal may represent the first prediction block.

The prediction unit may perform first prediction on each prediction mode of the plurality of prediction modes to find a prediction mode of the first prediction that is optimal for encoding the current block. In other words, step 1310 may be performed for each prediction mode of the plurality of prediction modes.

The plurality of prediction modes may correspond to different prediction directions.

The prediction unit may generate a first prediction error based on the current block and the first prediction signal.

The first prediction error may represent the first residual block.

The first residual block may represent a residual of the current block and the first prediction block. In other words, the prediction unit may generate the first prediction error by subtracting the first prediction block indicated by the first prediction signal from the current block.

The prediction unit may calculate the rate-distortion cost of the first prediction using the first prediction error.

The prediction unit may store the calculated first prediction error and / or the rate-distortion cost of the first prediction in the memory 1030, the storage 1040, or a buffer.

The prediction unit may perform first prediction on each prediction mode of the plurality of prediction modes to find a prediction mode of the first prediction that is optimal for encoding the current block. The prediction unit may calculate rate-distortion costs of the plurality of prediction modes using the first prediction errors of the plurality of prediction modes. The prediction unit may store the minimum rate-distortion cost of the calculated rate-distortion costs as the minimum first rate-distortion cost RDcost_1. For example, RDcost_1 may be the rate-distortion cost of the mode with the minimum rate-distortion cost in the first intra picture prediction.

The prediction unit may store first prediction errors corresponding to RDcost_1 and RDcost_1.

In operation 1320, the prediction unit may determine a first prediction mode of the first prediction. The first prediction mode may indicate a prediction direction of the first prediction.

The prediction unit may determine a prediction mode having a minimum rate-distortion cost among the plurality of prediction modes as the first prediction mode. Hereinafter, the prediction mode of the first prediction may be a prediction mode having a minimum rate-distortion cost among the plurality of prediction modes. The prediction direction of the first prediction may be the prediction direction of the prediction mode of the first prediction.

In step 1330, it may be determined whether to perform second prediction on the current block.

In one embodiment, the determination of step 1330 may not ultimately determine that the current block is encoded using the second prediction. In operation 1130, the prediction unit may determine whether the second prediction is possible for the current block.

For example, if it is determined not to perform the second prediction, the current block may be encoded without using the second prediction. When it is determined to perform the second prediction, the second prediction signal and the second prediction error may be generated by the second prediction, but in encoding of the current block through comparison of the rate-distortion costs in step 1360. It may be determined whether to use the second prediction.

If it is determined not to perform the second prediction, the residual block of the current block may be a block indicated by the first prediction error. In other words, the first prediction error may be provided as a residual block.

When it is determined to perform the second prediction, the residual block of the current block may be a block indicated by the second prediction error, which will be described later. In other words, the second prediction error may be provided as a residual block.

The prediction unit may determine whether to perform the second prediction based on the predefined condition.

In an embodiment, the prediction unit may determine whether to perform the second prediction based on the prediction mode of the first prediction.

For example, if the prediction mode of the first prediction is the non-directional mode, the prediction unit may not perform the second prediction. Alternatively, the prediction unit may perform the second prediction when the prediction mode of the first prediction is the directional mode.

When the prediction mode of the first prediction is a non-directional mode among the plurality of prediction modes, the current block may not be split according to the prediction direction because there is no direction of the prediction mode.

If it is determined to perform the second prediction, step 1335 may be performed.

If it is determined not to perform the second prediction, step 1370 or step 1120 may be performed. When step 1120 is performed, the residual signal of step 1120 may be a first prediction error. In other words, when only the first prediction is performed without the second prediction, the first prediction error generated by the first prediction may be used as the residual signal of the current block.

In operation 1335, the prediction unit may split the current block based on the first prediction mode. The prediction unit may generate two portions by dividing the current block based on the first prediction mode.

In one embodiment, the prediction unit may generate two non-square portions by dividing the current block into a straight line passing through the center of the current block in the prediction direction of the first prediction. In the following, the two non-square portions are called the first portion and the second portion, respectively.

If the current block is split, second prediction may be performed for each of the portions generated by the split.

In operation 1340, the prediction unit may generate a second prediction signal of the first portion by performing a second prediction on the first portion.

The second prediction signal of the first portion may represent the second prediction block of the first portion.

The second prediction of the first portion may be intra prediction.

The prediction unit may perform second prediction on each prediction mode of the plurality of prediction modes to find an optimal prediction mode for encoding the first portion. In other words, step 1340 may be performed for each prediction mode of the plurality of prediction modes.

The plurality of prediction modes may correspond to different prediction directions.

The prediction unit may generate a second prediction error of the first part based on the first part and the second prediction signal.

The second prediction error of the first portion may represent the second residual block of the first portion.

The second residual block of the first portion may represent a residual of the first portion and the second prediction block. In other words, the prediction unit may generate the second prediction error of the first part by subtracting the second prediction block of the first part indicated by the second prediction signal of the first part from the first part.

The prediction unit may calculate the rate-distortion cost of the second prediction of the first portion using the second prediction error of the first portion.

The prediction unit may store the calculated second prediction error of the first portion and / or the rate-distortion cost of the second prediction of the first portion in the memory 1030, the storage 1040, or a buffer.

The prediction unit may perform second prediction on each prediction mode of the plurality of prediction modes to find a prediction mode of the second prediction of the first portion that is optimal for encoding the first portion. The prediction unit may calculate rate-distortion costs of the plurality of prediction modes using the second prediction errors of the plurality of prediction modes. The prediction unit may store the minimum rate-distortion cost of the calculated rate-distortion costs as the minimum second rate-distortion cost RDcost_2 of the first portion. For example, RDcost_2 may be the rate-distortion cost of the mode with the minimum rate-distortion cost in the second intra-picture prediction of the first portion.

The prediction unit may store the second prediction error of the first portion corresponding to the RDcost_2 of the first portion and the RDcost_2 of the first portion.

In operation 1345, the prediction unit may determine second prediction modes of the second prediction of the first portion. The second prediction mode of the first portion may indicate a prediction direction of the second prediction of the first portion.

The prediction unit may determine a prediction mode having the minimum rate-distortion cost among the plurality of prediction modes as the second prediction mode of the first portion. Hereinafter, the prediction mode of the second prediction of the first portion may be a prediction mode having a minimum rate-distortion cost among the plurality of prediction modes. The prediction direction of the second prediction of the first part may be the prediction direction of the prediction mode of the second prediction of the first part.

In operation 1350, the predictor may generate a second prediction signal of the second part by performing a second prediction on the second part.

The second prediction signal of the second part may represent the second prediction block of the second part.

The second prediction of the second portion may be intra prediction.

The prediction unit may perform second prediction on each prediction mode of the plurality of prediction modes to find an optimal prediction mode for encoding the second portion. In other words, step 1340 may be performed for each prediction mode of the plurality of prediction modes.

The plurality of prediction modes may correspond to different prediction directions.

The prediction unit may generate a second prediction error of the second part based on the second part and the second prediction signal.

The second prediction error of the second portion may represent the second residual block of the second portion.

The second residual block of the second portion may represent a residual of the second portion and the second prediction block. In other words, the prediction unit may generate the second prediction error of the second part by subtracting the second prediction block of the second part indicated by the second prediction signal of the second part from the second part.

The prediction unit may calculate the rate-distortion cost of the second prediction of the second part using the second prediction error of the second part.

The prediction unit may store the calculated second prediction error of the second portion and / or the rate-distortion cost of the second prediction of the second portion in the memory 1030, the storage 1040, or a buffer.

The prediction unit may perform second prediction on each prediction mode of the plurality of prediction modes to find a prediction mode of the second prediction of the second optimal portion for encoding the second portion. The prediction unit may calculate rate-distortion costs of the plurality of prediction modes using the second prediction errors of the plurality of prediction modes. The prediction unit may store the minimum rate-distortion cost of the calculated rate-distortion costs as the minimum second rate-distortion cost RDcost_2 of the second portion. For example, RDcost_2 may be the rate-distortion cost of the mode with the minimum rate-distortion cost in the second intra-picture prediction of the second portion.

The prediction unit may store a second prediction error of the second part corresponding to RDcost_2 of the second part and RDcost_2 of the second part.

In operation 1355, the prediction unit may determine second prediction modes of the second prediction of the second portion. The second prediction mode of the second part may indicate a prediction direction of the second prediction of the second part.

The prediction unit may determine a prediction mode having the minimum rate-distortion cost among the plurality of prediction modes as the second prediction mode of the second portion. Hereinafter, the prediction mode of the second prediction of the second portion may be a prediction mode having a minimum rate-distortion cost among the plurality of prediction modes. The prediction direction of the second prediction of the second part may be the prediction direction of the prediction mode of the second prediction of the second part.

In operation 1360, the prediction unit may determine whether to use the second prediction in encoding of the current block.

The prediction unit may determine whether to use the second prediction in encoding of the current block based on the predefined condition.

For example, the prediction unit may determine to use the second prediction if the rate-distortion cost is further reduced by using the second prediction. The prediction unit may determine not to use the second prediction if the rate-distortion cost is not further reduced even if the second prediction is used.

For example, the prediction unit may determine to use the second prediction when the minimum rate-distortion cost when the second prediction is used is smaller than the minimum rate-distortion cost when the second prediction is not used.

In one embodiment, the predictor is a rate-distortion cost of the prediction mode of the first prediction, a rate-distortion cost of the prediction mode of the second prediction of the first portion and a rate-distortion cost of the prediction mode of the second prediction of the second portion It may be determined whether to use the second prediction based on the result of the comparison.

In an embodiment, the prediction unit may select a portion to be encoded from the first portion and the second portion. The prediction unit may select a part to be encoded among the first part and the second part based on a comparison between the rate-distortion cost of the second prediction mode of the first part and the rate-distortion cost of the second prediction mode of the second part.

For example, the prediction unit may select a portion having a higher rate-distortion cost among the first portion and the second portion. The prediction unit may select the first portion when the rate-distortion cost of the second prediction mode of the first portion is greater than the rate-distortion cost of the second prediction mode of the second portion. The prediction unit may select the second portion when the rate-distortion cost of the second prediction mode of the first portion is smaller than the rate-distortion cost of the second prediction mode of the second portion.

For example, when the rate-distortion cost of the second prediction mode of the first portion and the rate-distortion cost of the second prediction mode of the second portion are the same, the prediction unit may be the first prediction mode. If different from the first portion can be selected, if the second prediction mode of the second portion is different from the first prediction mode, the first portion can be selected. In other words, the prediction unit may select one of the first part and the second part based on the rate-distortion cost as the first criterion and the prediction mode as the next criterion.

Hereinafter, a portion selected by the prediction unit among the first portion and the second portion is referred to as a selected portion. Also, the second prediction means a second prediction of a selected portion of the second prediction of the first portion and the second prediction of the second portion. The portion of the first portion and the second portion that the prediction unit does not select is referred to as the remaining portion.

In an embodiment, the prediction unit may determine whether to use the second prediction based on the second prediction mode of the selected portion and the first prediction mode of the current block.

For example, if the second prediction mode of the selected portion and the first prediction mode of the current block are the same, the result when the second prediction is used and the result when the second prediction is not used may be the same. Thus, in this case, there may be no effect due to the second prediction. The prediction unit may determine not to use the second prediction for the selected portion if the second prediction mode of the selected portion and the first prediction mode of the current block are the same.

In one embodiment, the prediction unit may determine to use the second prediction for the selected portion if the rate-distortion cost for the prediction of the current block is reduced by the second prediction for the selected portion. The prediction unit may determine to use the second prediction for the selected portion unless the rate-distortion cost for the prediction of the current block is reduced by the second prediction for the selected portion.

For example, the rate-distortion cost by the second prediction for the selected part is RDcost_SS, the rate-distortion cost by the first prediction for the remaining part is RDcost_SNS, and the rate-distortion by the first prediction for the current block. When the cost is RDcost, the prediction unit may determine to use the second prediction for the selected portion if the sum of RDcost_SS and RDcost_SNS is less than RD_cost. In addition, the prediction unit may determine not to use the second prediction for the selected portion when the sum of the RDcost_SS and the RDcost_SNS is equal to or greater than the RD_cost.

For example, if the rate-distortion cost by the second prediction for the selected portion is less than the rate-distortion cost by the first prediction for the selected portion, it may be determined to use the second prediction for the selected portion. If the rate-distortion cost by the second prediction for the selected portion is more than the rate-distortion cost by the first prediction for the selected portion, it may be determined not to use the second prediction for the selected portion.

If the second prediction is not used for encoding the current block, step 1370 may be performed.

When the second prediction is used for encoding the current block, step 1380 may be performed.

In operation 1370, the prediction unit may perform a setting indicating that the second prediction is not used.

The prediction unit may set a value of the second prediction usage information to indicate that the second prediction usage information does not use the second prediction. For example, the second prediction usage information may be an Additional Secondary Intra Prediction (ASIP) flag.

For example, if the value of the second prediction usage information is "0", the second prediction usage information may indicate that the second prediction usage information is not used.

The prediction unit may set the first prediction mode information as the prediction mode of the first prediction.

When step 1370 is performed, the residual signal of step 1120 may be a residual signal of the current block. The residual signal of the current block may be a first prediction error. In other words, when the second prediction is not performed on the current block, the first prediction error generated based on the first prediction may be used as the residual signal of the current block.

In operation 1380, the prediction unit may perform setting indicating to use the second prediction.

In an embodiment, the prediction unit may set a value of the second prediction usage information to indicate that the second prediction usage information uses the second prediction.

In one embodiment, for example, if the value of the second prediction usage information is "1", the second prediction usage information may indicate that the second prediction usage information is used.

The prediction unit may set a value of the selected portion information so that the selected portion information indicates the selected portion to which the second prediction is applied. For example, the selected partial information may be a select section flag.

For example, the prediction unit may set the value of the selected partial information such that the selected partial information represents one of two parts generated by the division of the current block. If the selected part is the part S1 described above with reference to FIG. 11, the predictor may set the value of the selected part information to “0”. If the selected part is the part S2 described above with reference to FIG. 11, the predictor may set the value of the selected part information to “1”.

The prediction unit may set the first prediction mode information as the prediction mode of the first prediction. The prediction unit may set the second prediction mode information as the prediction mode of the second prediction.

When step 1370 is performed, the residual signal of step 1120 may be a residual signal of the current block. The residual signal of the current block may be a signal corresponding to the difference between the current block and the prediction block. The prediction block may be the sum of the prediction block generated by the second prediction for the selected portion and the prediction block generated by the first prediction for the remaining portion.

In other words, when the second prediction is performed on the current block, the prediction error generated based on the first prediction and the second prediction may be used as the residual signal.

Alternatively, when step 1370 is performed, the residual signal of step 1120 may be a residual signal of the selected portion and a residual signal of the remaining portion. The residual signal of the selected portion may be a signal corresponding to the difference between the selected portion and the second prediction block generated by the second prediction for the selected portion. The residual signal of the remaining portion may be a signal corresponding to the difference between the remaining portion and the third prediction block generated by the first prediction for the remaining portion.

14 is a flowchart of a method of generating a restored block, according to an example.

Step 1160 described above with reference to FIG. 11 may include steps 1410, 1420, and 1430 below.

In operation 1410, it may be determined whether a second prediction is used for encoding of the reconstructed block generator current block.

If second prediction is used for encoding of the current block, step 1420 may be performed.

If the second prediction is not used for the encoding of the current block, step 1430 may be performed.

In operation 1420, the reconstructed block generator may add a second prediction signal for the selected portion to the reconstructed residual signal.

The second prediction signal may be a signal generated by the second prediction for the selected portion.

In operation 1430, the reconstructed block generator may generate the reconstructed block by adding the first prediction signal for the current block or the remaining part to the reconstructed residual signal.

In one embodiment, when the second prediction is not used, the reconstructed block generator may generate the reconstructed block by adding the first prediction signal for the current block to the reconstructed residual signal. The first prediction signal may be a signal generated by the first prediction for the current block.

In one embodiment, when the second prediction is not used, the reconstructed block generator may generate the reconstructed block by adding the first prediction signal for the remaining part to the reconstructed residual signal. The first prediction signal may be a signal generated by the first prediction for the remaining part.

For example, when the second prediction is used, the signal representing the reconstructed block may be the sum of the reconstructed residual signal, the second prediction signal for the selected portion, and the first prediction signal for the remainder. As described in steps 1420 and 1430, when second prediction is used, a second prediction signal for the selected portion and a first prediction signal for the remainder may be added to the reconstructed residual signal. Alternatively, when the second prediction is used, the reconstructed block may be generated based on the reconstructed residual signal, the second prediction signal for the selected portion, and the first prediction signal for the remaining portion.

For example, if the second prediction is not used, the signal representing the reconstructed block may be the sum of the reconstructed residual signal and the first prediction signal for the current block. Or, if the second prediction is not used, the reconstructed block may be generated based on the reconstructed residual signal and the first prediction signal for the current block.

15 is a structural diagram of a decoding apparatus according to an embodiment.

The decoding apparatus 1500 may be a general purpose computer system for performing decryption.

As shown in FIG. 15, the decoding apparatus 1500 may include at least one processor 1510, a memory 1530, a user interface (UI) input device 1550, which communicates with each other through a bus 1590, UI output device 1560 and storage 1540. In addition, the decoding apparatus 1500 may further include a communication unit 1520 connected to the network 1599. The processor 1510 may be a semiconductor device that executes processing instructions stored in a central processing unit (CPU), a memory 1530, or a storage 1540. Memory 1530 and storage 1540 may be various forms of volatile or nonvolatile storage media. For example, the memory may include at least one of a ROM 1531 and a RAM 1532.

The processor 1510 may include an entropy decoder 210, an inverse quantizer 220, an inverse transformer 230, an intra predictor 240, an inter predictor 250, an adder 255 of the decoding apparatus 200. The filter unit 260 and the reference picture buffer 270 may be included.

The recording medium may store at least one module for operating the decoding apparatus 1500. The memory 1530 may store at least one module and may be configured to be executed by the at least one processor 1510.

According to an embodiment, the entropy decoder 210, the inverse quantizer 220, the inverse transformer 230, the intra predictor 240, the inter predictor 250, and the adder 255 of the decoding apparatus 1500. At least some of the filter unit 260 and the reference picture buffer 270 may be program modules, and may communicate with an external device or system. The program modules may be included in the decoding apparatus 1500 in the form of an operating system, an application program module, and other program modules.

The program modules may be physically stored on various known storage devices. In addition, at least some of these program modules may be stored in a remote storage device that can communicate with the decryption apparatus 1500.

Program modules perform routines or subroutines, programs, objects, components, and data to perform functions or operations, or to implement abstract data types, according to one embodiment. Data structures and the like, but is not limited thereto.

The program modules may be composed of instructions or code performed by the processor 1510.

Functions related to communication of data or information of the decoding apparatus 1500 may be performed through the communication unit 1520.

16 is a flowchart of a decoding method according to an embodiment.

The decoding method of the embodiment may be performed by the decoding apparatus 200 or the decoding apparatus 1500.

In operation 1610, the communication unit 1520 may receive a bitstream from the encoding apparatus 100 or the encoding apparatus 1100.

The bitstream may include information about the encoded residual signal, and may include information related to prediction.

The information about the encoded residual signal may include entropy coded quantized transform coefficients.

Information related to the prediction may be entropy coded.

In operation 1620, the entropy decoder 210 may generate quantized transform coefficients by performing entropy decoding on the bitstream. In addition, the entropy decoding unit 210 may generate information related to prediction by performing entropy decoding on the bitstream.

For example, the information related to the prediction may include prediction scheme information indicating how the current block is encoded.

For example, the prediction scheme information may indicate whether the current block is encoded by intra prediction or intra prediction. Alternatively, the prediction scheme information may indicate whether the current block is encoded by intra prediction. Alternatively, the prediction scheme information may indicate whether the current block is encoded by inter prediction.

For example, the information related to the prediction may include intra prediction mode information indicating a prediction mode of the intra prediction.

For example, the information related to the prediction may include second prediction usage information indicating whether the second prediction is used in encoding the current block.

For example, the information related to the prediction may include first prediction mode information indicating the prediction mode of the first prediction and second prediction mode information indicating the prediction mode of the second prediction.

For example, the information related to the prediction may include selected part information indicating a selected part to which the second prediction is applied among the two parts generated by the division of the current block.

For example, the information related to the prediction includes current block type information indicating the type of the current block, first prediction block type information indicating the type of the first prediction block, second prediction block type information indicating the type of the second prediction block, First reference block type information indicating a type of a reference block, second reference block type information indicating a type of a second reference block, first reference sample type information indicating a type of a first reference sample, and a type of a second reference sample Second reference sample type information indicating the information may be included.

For example, the information related to the prediction may include first prediction region information indicating the region of the first prediction and / or second prediction region information indicating the second prediction region of the second prediction.

In addition, the information related to the prediction may include information used for the prediction described in the embodiments. Information related to the prediction may be included in the bitstream according to the order described in the embodiment or the order of generation according to the embodiment. The reconstructed residual signal generator may obtain information related to the prediction from the bitstream in the order described in the embodiment or the generation order according to the embodiment.

In operation 1630, the reconstructed residual signal generator may generate a reconstructed residual signal for the current block based on the quantized transform coefficients.

The reconstructed residual signal generator may include an inverse quantizer 220 and an inverse transformer 230.

Step 1630 may include steps 1631 and 1732.

In operation 1163, the inverse quantization unit 220 may generate the reconstructed transform coefficients by performing inverse quantization on the quantized transform coefficients.

In operation 1632, the transform unit 230 may generate the reconstructed residual signal by performing an inverse transform on the inverse quantized transform coefficients.

Reconstructed residual signal for the current block may be generated through steps 1610, 1720, and 1730.

In operation 1640, the reconstructed block generator may generate a reconstructed block based on the reconstructed residual signal, the second prediction, and the first prediction.

The reconstructed block generator may include an adder 225, an intra predictor 240, an inter predictor 250, a filter 260, and a reference picture buffer 270.

The recovered residual signal may be a recovered residual block.

For example, the reconstructed block may be the sum of the reconstructed residual block and the prediction block. Alternatively, the reconstructed block may be generated based on the sum of the reconstructed residual block and the prediction block.

For example, the prediction block may be the sum of the first prediction block and the second prediction block. Alternatively, the prediction signal may be the sum of the first prediction signal and the second prediction signal. The second prediction signal may be a signal generated by the second prediction for the selected portion of the current block. The first prediction signal may be a signal generated by the first prediction for the current block or the remainder of the current block.

Alternatively, the reconstructed block may be generated based on the sum of the reconstructed residual block and the prediction block.

For example, the reconstructed block may be a block generated by the reconstructed residual block, the first prediction block, and the second prediction block. The reconstructed block generator may generate the reconstructed first residual signal by adding the reconstructed residual signal and the second prediction signal. The reconstructed block generator may generate the reconstructed block by adding the reconstructed first residual signal and the first prediction signal.

Alternatively, the reconstructed block generator may generate the reconstructed first residual block by adding the reconstructed residual block and the second prediction block. The reconstructed block generator may generate the reconstructed block by adding the reconstructed first residual block and the first prediction block.

In one embodiment, when the second prediction is not used, the second prediction signal may be a null signal and the second prediction block may be a null block. The null block may be a block in which the values of all pixels of the block are "0".

The reconstructed block generator may generate a reconstructed block by performing inter prediction or intra prediction.

In one embodiment, the first prediction and the second prediction may be different kinds of predictions.

In one embodiment, the first prediction and the second prediction may be the same kind of predictions. For example, each of the first prediction and the second prediction may be intra prediction. If the first prediction is intra prediction, the second prediction may also be set to intra prediction.

In one embodiment, the prediction direction of the first prediction and the prediction direction of the second prediction may be the same. The prediction direction of the second prediction may be set to be the same as the prediction direction of the first prediction.

For example, in one embodiment, each of the first prediction and the second prediction may be inter prediction.

In one embodiment, a block that is the target of the first prediction and a block that is the target of the second prediction may be different from each other. Here, the "block to be predicted" may mean to which block the "prediction" generates a prediction block.

For example, the type of the block targeted for the first prediction and the type of the block targeted for the second prediction may be different from each other. The type of block may be an original block, a luma block, a chroma block, a depth block, a residual block, or the like.

For example, the block targeted for the first prediction may be a reconstructed block.

Reference blocks may be used for prediction. In one embodiment, the first reference block of the first prediction and the second reference block of the second prediction may be different from each other. The reference blocks may differ from each other in terms of type and / or location.

In one embodiment, the position of the first reference block of the first prediction and the position of the second reference block of the second prediction may be different from each other. Here, the position of the first reference block may be a position relative to the block that is the target of the first prediction. The position of the second reference block may be a position relative to the block that is the target of the second prediction.

In one embodiment, the first reference block of the first prediction may be plural. The second reference block of the second prediction may be plural. At least a portion of the plurality of first reference blocks of the first prediction and the plurality of second reference blocks of the second prediction may be different from each other.

The position of at least one of the plurality of first reference blocks of the first prediction and the plurality of second reference blocks of the second prediction is the positions of the plurality of first reference blocks and the positions of the plurality of second reference blocks. Can only be included in one of the

In one embodiment, the type of the first reference block of the first prediction and the type of the second reference block of the second prediction may be different from each other. The type of block may be a reconstructed block, a reconstructed luma block, a reconstructed chroma block, a reconstructed depth block, a reconstructed first residual block, a reconstructed second residual block, or the like.

For example, the first reference block of the first prediction may be a reconstructed block. The second reference block of the second prediction may be a reconstructed first residual block. The first reference sample of the first prediction may be a pixel of the reconstructed block. The second reference sample of the second prediction may be a pixel of the reconstructed first residual block.

For example, the first reference block of the first prediction may be a reconstructed neighboring block neighboring the current block. The second reference block of the second prediction may be a reconstructed first residual block neighboring the current block.

The reconstructed neighbor residual block neighboring the current block may be obtained by adding the reconstructed residual block of the reconstructed neighboring block to the second prediction block of the reconstructed neighboring block. Alternatively, the reconstructed neighbor residual block neighboring the current block may be a difference between the reconstructed neighboring block and the first prediction block of the reconstructed neighboring block.

For example, when the object of the first prediction is the current block, a reconstructed block around the current block may be used as a reference block for prediction of the current block. When the target of the second prediction is the first residual block, a reconstructed first residual block around the current block or the first residual block may be used as the second reference block for prediction on the first residual block.

In one embodiment, the region of the first prediction and the region of the second prediction may be different from each other. Here, the "prediction area" may indicate an area in which a prediction value is generated among blocks that are to be predicted. Alternatively, the “area of prediction” may indicate an area to which a prediction value generated by prediction is allocated among prediction blocks corresponding to a block that is a prediction target.

In one embodiment, the second prediction may be a prediction for a selected portion of the current block. The first prediction may be a prediction of the entirety of the current block. Alternatively, the first prediction may be a prediction for a portion of the current block except for the portion selected for the second prediction.

For example, the region of the second prediction may be a selected portion of the current block. The area of the first prediction may be the whole of the current block. Alternatively, the region of the first prediction may be a portion of the current block except for the selected portion of the second prediction.

For example, the region of the second prediction may be a non-square portion of the current block. The area of the first prediction may be the whole of the current block. Alternatively, the region of the first prediction may be a non-square portion of the current block except for the portion of the second prediction.

For example, prediction values determined by the first prediction may be assigned only to pixels in the region of the first prediction among blocks that are the targets of the first prediction. The prediction value according to the first prediction may not be allocated to pixels outside the region of the first prediction among the blocks that are the targets of the first prediction. Alternatively, a predefined value may be assigned to pixels outside the region of the first prediction among blocks that are the target of the first prediction. For example, the predefined value may be "0".

For example, prediction values determined by the second prediction may be assigned only to pixels in the region of the second prediction among blocks that are the target of the second prediction. Pixels outside the region of the second prediction among the blocks that are the target of the second prediction may not be assigned a prediction value by the second prediction. Alternatively, a predefined value may be assigned to pixels outside the region of the second prediction among blocks that are the target of the second prediction. For example, the predefined value may be "0".

In one embodiment, the region of the first prediction may be determined based on the type of the first prediction. For example, the region of the first prediction may be determined based on whether the first prediction is inter prediction. Alternatively, the region of the first prediction may be determined based on whether the first prediction is intra prediction. Alternatively, the region of the first prediction may be determined based on the prediction direction of the first prediction.

In one embodiment, the region of the second prediction may be determined based on the type of the second prediction. For example, the region of the second prediction may be determined based on whether the second prediction is inter prediction. Alternatively, the region of the second prediction may be determined based on whether the second prediction is intra prediction. Alternatively, the region of the second prediction may be determined based on the prediction direction of the second prediction.

17 is a flowchart of a method of generating a restored block, according to an embodiment.

Step 1640 described above with reference to FIG. 16 may include the following steps 1710, 1720, 1730, 1740, and 1750.

The second prediction can optionally be performed based on the predefined conditions.

In operation 1710, the reconstructed block generator may determine whether to use the second prediction for generation of the reconstructed block.

The reconstructed block generator may determine whether to use the second prediction for the generation of the reconstructed block based on a predefined condition.

In an embodiment, the reconstructed block generator may determine whether to use the second prediction based on the prediction mode of the first prediction.

For example, the reconstructed block generator may obtain first prediction mode information indicating a prediction mode of the first prediction from the bitstream. The reconstructed block generator may not use the second prediction if the prediction mode of the first prediction is a non-directional mode.

In one embodiment, after the determination is made whether to use the second prediction by the prediction mode, the position of the current block, and / or the number of adjacent reconstructed blocks, the reconstructed block generator then performs a second prediction on the encoding of the current block. It may be determined whether to use the second prediction based on whether it is used.

For example, the reconstructed block generator may obtain second prediction usage information from the bitstream. The reconstructed block generator may use the second prediction when the second prediction usage information indicates that the second prediction use information is used. The reconstructed block generator may not use the second prediction when the second prediction usage information indicates that the second prediction usage information does not use the second prediction.

When the second prediction is used for encoding of the current block, step 1720 may be performed.

If second prediction is not used for encoding of the current block, step 1740 may be performed.

In operation 1720, the reconstructed block generator may generate a second prediction signal by performing a second prediction.

The reconstructed block generator may generate a second prediction signal by performing a second prediction on the selected portion.

The second prediction may correspond to the second prediction for the selected portion in the encoding of the current block described above with reference to FIGS. 11 and 13. For example, the second prediction may be a prediction for the selected portion of the current block. The second prediction signal may correspond to the second prediction for the selected portion described above with reference to FIGS. 11 and 13.

The second prediction signal may represent the second prediction block.

The first prediction and the second prediction may be intra prediction, and the second prediction mode of the second prediction may be different from the first prediction mode of the first prediction. Alternatively, the prediction direction and the second prediction direction of the second prediction may be different from each other.

The reference block of the first prediction and the reference block of the second prediction may be reconstructed neighboring blocks neighboring the current block.

In one embodiment, the selected part where the second prediction is performed may be part of one of two parts generated by partitioning of the current block.

In one embodiment, each of the two portions may be non-square. The selected portion may be non-square.

The selected portion where the second prediction is performed may be determined based on the prediction direction of the prediction mode of the first prediction.

In one embodiment, the partitioning of the current block may be performed according to the prediction direction in the manner described above with reference to FIG. 12. The two parts may be determined based on a straight line passing through the center of the current block in the prediction direction of the first prediction. The two parts may be one of two parts generated by dividing the current block by a straight line passing through the center of the current block in the prediction direction of the first prediction.

In one embodiment, the sizes of the two parts produced by the division may be the same. Or, the number of pixels of the two portions may be the same.

In one embodiment, each of the two portions produced by the division may be non-square.

In one embodiment, the selected part to which the second prediction is to be applied may be selected according to the selected part information. The reconstructed block generator may acquire the selected partial information from the bitstream. The selected part may be selected according to the selected part information indicating one of the two parts. The part of the two parts that the selected part information does not point to may be the remaining part.

For example, the value of the selected portion information may indicate a portion of the two portions to which the second prediction is applied. If the value of the selected portion information is "0", the selected portion may be the portion S1 described above with reference to FIG. 11. If the value of the selected portion information is "1", the selected portion may be the portion S2 described above with reference to FIG. 11.

In operation 1730, the reconstructed block generator may add a second prediction signal for the selected portion to the reconstructed residual signal.

The second prediction signal may be a signal generated by the second prediction for the selected portion.

In operation 1740, the reconstructed block generator may generate a first prediction signal for the current block or the remaining part.

The reconstructed block generator may generate a first prediction signal by performing a first prediction on the current block or the remaining part.

The first prediction signal may represent the first prediction block.

In the first prediction, reconstructed blocks around the current block can be used as reference blocks. Also, pixels of reconstructed blocks around the current block can be used as reference samples.

In an embodiment, when the second prediction is not used, the first prediction may be performed on the current block to generate a first prediction signal for the current block. The first prediction signal may be a signal generated by the first prediction for the current block. If the second prediction is not used for decoding of the current block, the first prediction may be a prediction of the entirety of the current block.

In one embodiment, when the second prediction is not used, the first prediction may be performed on the remaining portion to generate a first prediction signal for the remaining portion. The first prediction signal may be a signal generated by the first prediction for the remaining part. When the second prediction is used for decoding of the current block, the first prediction may be a prediction for a portion of the current block except for the portion selected for the second prediction.

In operation 1750, the reconstructed block generator may generate the reconstructed block by adding the first prediction signal for the current block or the remaining part to the reconstructed residual signal.

In one embodiment, when the second prediction is not used, the reconstructed block generator may generate the reconstructed block by adding the first prediction signal for the current block to the reconstructed residual signal. The first prediction signal may be a signal generated by the first prediction for the current block.

In one embodiment, when the second prediction is not used, the reconstructed block generator may generate the reconstructed block by adding the first prediction signal for the remaining part to the reconstructed residual signal. The first prediction signal may be a signal generated by the first prediction for the remaining part.

According to the above described steps 1710, 1720, 1730, 1740, and 1750, when the second prediction is used, the signal representing the reconstructed block is a reconstructed residual signal, a second prediction signal for the selected portion, and the remainder of the portion. It may be the sum of the first prediction signals for. As described in steps 1730 and 1750, when the second prediction is used, the second prediction signal for the selected portion and the first prediction signal for the remainder may be added to the reconstructed residual signal. Alternatively, when the second prediction is used, the reconstructed block may be generated based on the reconstructed residual signal, the second prediction signal for the selected portion, and the first prediction signal for the remaining portion.

According to the above described steps 1710, 1720, 1730, 1740, and 1750, when the second prediction is not used, the signal representing the reconstructed block may be the sum of the reconstructed residual signal and the first prediction signal for the current block. have. Or, if the second prediction is not used, the reconstructed block may be generated based on the reconstructed residual signal and the first prediction signal for the current block.

The reconstructed block can be used as a reference block for decoding another block.

According to the above-described embodiment, when the second prediction is used, the selected part to which the first prediction is applied and the remaining part to which the second prediction is applied may be separated.

In one embodiment, when the second prediction is used, the first prediction may be applied to the entirety of the current block. In this case, the reconstructed block for the selected portion to which the second prediction is applied is based on the sum of the reconstructed residual block, the second prediction signal generated by the second prediction, and the first prediction signal generated by the first prediction. Can be generated. In addition, a reconstructed block for the remaining part to which the second prediction is not applied may be generated based on the sum of the reconstructed residual block and the first prediction signal generated by the first prediction.

In this case, the second prediction may be a prediction for the residual. The second prediction may be a prediction for a first residual signal that is a difference between the current block and the first prediction signal. The reference block of the second prediction may be a reconstructed residual block, and the reference pixel of the second prediction may be a pixel of the reconstructed residual block. In other words, a second residual signal, which is a difference between the first residual signal and the second prediction signal, may be generated through the second prediction on the first residual signal, and encoded information of the current block is generated using the second residual signal. Can be.

Alternatively, in this case, the reconstructed block for the selected portion to which the second prediction is applied may be a weighted-sum of the reconstructed residual block, the second prediction signal, and the first prediction signal. The information related to the prediction may include a first weight of the reconstructed residual block, a second weight of the second prediction signal, and a third weight of the first prediction signal. The fact that the second prediction is not used may be considered that the second weight of the second prediction signal is set to zero. Alternatively, the second weight of the second prediction signal may be considered to be set to 0 with respect to the remaining part to which the second prediction is not applied.

The above description of encoding of the current block may also be applied to decoding of the current block. Duplicate explanations are omitted. In addition, the above description of decoding of the current block may be applied to encoding of the current block. Duplicate explanations are omitted.

In the above-described embodiments, the methods are described based on a flowchart as a series of steps or units, but the present invention is not limited to the order of steps, and certain steps may occur in a different order or simultaneously from other steps as described above. Can be. Also, one of ordinary skill in the art appreciates that the steps shown in the flowcharts are not exclusive, that other steps may be included, or that one or more steps in the flowcharts may be deleted without affecting the scope of the present invention. I can understand.

Embodiments according to the present invention described above may be implemented in the form of program instructions that may be executed by various computer components, and may be recorded in a computer-readable recording medium. The computer-readable recording medium may include program instructions, data files, data structures, etc. alone or in combination. Program instructions recorded on the computer-readable recording medium may be those specially designed and configured for the present invention, or may be known and available to those skilled in the computer software arts. Examples of computer-readable recording media include magnetic media such as hard disks, floppy disks and magnetic tape, optical recording media such as CD-ROMs, DVDs, and magneto-optical media such as floptical disks. media), and hardware devices specifically configured to store and execute program instructions, such as ROM, RAM, flash memory, and the like. Examples of program instructions include not only machine code generated by a compiler, but also high-level language code that can be executed by a computer using an interpreter or the like. The hardware device may be configured to operate as one or more software modules to perform the process according to the invention, and vice versa.

Although the present invention has been described by specific embodiments such as specific components and the like, but the embodiments and the drawings are provided to assist in a more general understanding of the present invention, the present invention is not limited to the above embodiments. For those skilled in the art, various modifications and variations can be made from these descriptions.

Accordingly, the spirit of the present invention should not be limited to the above-described embodiments, and all of the equivalents or equivalents of the claims, as well as the appended claims, fall within the scope of the spirit of the present invention. I will say.

Claims (20)

Generating a residual signal of the current block based on the current block, the first prediction, and the second prediction; And Generating information on the encoded residual signal by performing encoding on the residual signal Including, And the second prediction is a prediction for a portion of the current block. A reconstructed residual signal generator for generating a reconstructed residual signal for the current block; And A reconstructed block generator for generating a reconstructed block for the current block based on the reconstructed residual signal, the second prediction, and the first prediction Including, And the second prediction is a prediction for a portion of the current block. Generating a reconstructed residual signal for the current block; And Generating a reconstructed block for the current block based on the reconstructed residual signal, a second prediction and a first prediction Including, And the second prediction is a prediction for a portion of the current block. The method of claim 3, And the first prediction and the second prediction are intra prediction. The method of claim 3, And said portion is non-square. The method of claim 3, And the portion is determined based on a prediction mode of the first prediction. The method of claim 6, And the portion is determined based on a straight line passing through the center of the current block in the prediction direction of the first prediction. The method of claim 3, And a prediction direction of the first prediction and a prediction direction of the second prediction are different from each other. The method of claim 3, And the portion is one of two portions generated by dividing the current block. The method of claim 9, And the number of pixels of the two portions is the same. The method of claim 9, Wherein each of the two portions is non-square. The method of claim 9, And the two portions are determined based on a straight line passing through the center of the current block in the prediction direction of the first prediction. The method of claim 9, And the portion is selected according to selected portion information indicating one of the two portions. The method of claim 3, If the prediction mode of the first prediction is a non-directional mode, the second prediction is not used. The method of claim 3, And the second prediction is not used when the second prediction usage information indicating whether or not the second prediction is used in encoding the current block indicates that the second prediction is not used. The method of claim 3, And if the second prediction is not used, the first prediction is a prediction of the entirety of the current block. The method of claim 3, And a second prediction signal generated by the second prediction for the portion is added to the reconstructed residual signal. The method of claim 3, And the first prediction is a prediction for a portion of the current block except for the portion. The method of claim 18, And a first prediction signal generated by the first prediction for the remainder is added to the reconstructed residual signal. The method of claim 18, And the reconstructed block is generated based on a second prediction signal generated by the second prediction for the portion and a first prediction signal generated by the first prediction on the remaining portion.

Images