WO2025198453A1 - Video coding method and device, and recording medium storing bitstream - Google Patents

Abstract

The image encoding/decoding method and device according to the present disclosure may configure a candidate list including a plurality of candidates with regard to the current block, may calculate costs with regard to respective candidates in the candidate list, and may generate a predicted block of the current block on the basis of the costs. The candidate list may include at least one of a candidate derived on the basis of a regular mode and a candidate derived on the basis of a merge mode.

Description

Video coding method and device, and recording medium storing bitstream

The present invention relates to a video signal processing method and device.

The market demand for high-resolution video is growing, necessitating technologies capable of efficiently compressing high-resolution images. To address this market need, the ISO/IEC's Moving Picture Expert Group (MPEG) and the ITU-T's Video Coding Expert Group (VCEG) jointly formed the Joint Collaborative Team on Video Coding (JCT-VC). They completed development of the HEVC (High Efficiency Video Coding) video compression standard in January 2013 and have been actively conducting research and development on next-generation compression standards.

Video compression largely consists of intraprediction, interprediction, transform, quantization, entropy coding, and in-loop filtering. Among these, intraprediction refers to a technique that generates a prediction block for the current block using reconstructed pixels surrounding the current block. The encoder encodes the intraprediction mode used for intraprediction, and the decoder performs intraprediction by reconstructing the encoded intraprediction mode.

According to the present disclosure, a TIMD-based intra prediction method and device are provided.

The present disclosure provides a method and device for determining a reference sample line for MIP mode.

According to the present disclosure, a method and device for implicitly signaling a MIP mode are provided.

The video decoding method and device according to the present disclosure can construct a candidate list including multiple candidates for a current block, calculate a cost for each candidate in the candidate list, and generate a prediction block of the current block based on the cost. Here, the candidate list can include at least one of a candidate derived based on a regular mode or a candidate derived based on a merge mode.

In the image decoding method and device according to the present disclosure, a candidate derived based on the Regular mode can be derived based on an intra prediction mode of a neighboring block adjacent to the current block.

In the image decoding method and device according to the present disclosure, the candidate derived based on the merge mode can be derived based on a surrounding block to which template-based intra mode derivation (TIMD) is applied.

In the video decoding method and device according to the present disclosure, the cost can be calculated based on the difference between the predicted value and the restored value of the template of the current block, and the predicted value of the template of the current block can be generated based on the intra prediction mode of each candidate.

In the video decoding method and device according to the present disclosure, the top two candidates can be selected from the candidate list in ascending order of the calculated cost.

In the video decoding method and device according to the present disclosure, two prediction blocks can be generated based on the intra prediction modes of the two candidates, and the prediction block of the current block can be generated based on the weighted sum of the two prediction blocks.

In the image decoding method and device according to the present disclosure, the weight of the weighted sum can be derived based on the cost corresponding to the two candidates.

In the video decoding method and device according to the present disclosure, if the candidate with the smallest cost among the two candidates is significantly smaller than the cost of the other candidates, the prediction block of the current block can be generated using only the candidate with the smallest cost.

The video encoding method and device according to the present disclosure can construct a candidate list including a plurality of candidates for a current block, calculate a cost for each candidate in the candidate list, and generate a prediction block of the current block based on the cost. Here, the candidate list can include at least one of a candidate derived based on a regular mode or a candidate derived based on a merge mode.

A computer-readable recording medium according to the present disclosure can store a bitstream encoded by the image encoding method.

According to the present disclosure, the encoding/decoding complexity of TIMD, an intra prediction technique, can be reduced.

According to the present disclosure, in MIP-based prediction, the accuracy of intra prediction can be improved by adaptively utilizing reference sample lines.

According to the present disclosure, compression performance of a decoder/sub-decoder can be improved by implicitly transmitting a MIP mode to the decoder.

FIG. 1 is a block diagram showing an image encoding device according to the present disclosure.

FIG. 2 is a block diagram showing an image decoding device according to the present disclosure.

Figure 3 illustrates a TIMD-based prediction method according to the present disclosure.

FIG. 4 is an example according to the present disclosure, showing the location of non-adjacent blocks available to the current block.

FIG. 5 illustrates a matrix-based intra prediction method according to the present disclosure.

The video decoding method and device according to the present disclosure can construct a candidate list including multiple candidates for a current block, calculate a cost for each candidate in the candidate list, and generate a prediction block of the current block based on the cost. Here, the candidate list can include at least one of a candidate derived based on a regular mode or a candidate derived based on a merge mode.

In the image decoding method and device according to the present disclosure, a candidate derived based on the Regular mode can be derived based on an intra prediction mode of a neighboring block adjacent to the current block.

In the image decoding method and device according to the present disclosure, the candidate derived based on the merge mode can be derived based on a surrounding block to which template-based intra mode derivation (TIMD) is applied.

In the video decoding method and device according to the present disclosure, the cost can be calculated based on the difference between the predicted value and the restored value of the template of the current block, and the predicted value of the template of the current block can be generated based on the intra prediction mode of each candidate.

In the video decoding method and device according to the present disclosure, the top two candidates can be selected from the candidate list in ascending order of the calculated cost.

In the video decoding method and device according to the present disclosure, two prediction blocks can be generated based on the intra prediction modes of the two candidates, and the prediction block of the current block can be generated based on the weighted sum of the two prediction blocks.

In the image decoding method and device according to the present disclosure, the weight of the weighted sum can be derived based on the cost corresponding to the two candidates.

In the video decoding method and device according to the present disclosure, if the candidate with the smallest cost among the two candidates is significantly smaller than the cost of the other candidates, the prediction block of the current block can be generated using only the candidate with the smallest cost.

The video encoding method and device according to the present disclosure can construct a candidate list including a plurality of candidates for a current block, calculate a cost for each candidate in the candidate list, and generate a prediction block of the current block based on the cost. Here, the candidate list can include at least one of a candidate derived based on a regular mode or a candidate derived based on a merge mode.

A computer-readable recording medium according to the present disclosure can store a bitstream encoded by the image encoding method.

Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings attached to this specification so that those skilled in the art can easily implement the present invention. However, the present invention may be implemented in various different forms and is not limited to the embodiments described herein. In addition, in the drawings, parts irrelevant to the description have been omitted to clearly explain the present invention, and similar parts have been designated with similar reference numerals throughout the specification.

Throughout this specification, when a part is said to be 'connected' to another part, this includes not only cases where they are directly connected, but also cases where they are electrically connected with another element in between.

Additionally, whenever a part throughout this specification is said to "include" a component, this does not mean that other components are excluded, but rather that other components may be included, unless specifically stated otherwise.

Additionally, while terms such as "first," "second," etc. may be used to describe various components, these components should not be limited by these terms. These terms are used solely to distinguish one component from another.

Additionally, in the embodiments of the devices and methods described herein, some components of the devices or some steps of the methods may be omitted. Furthermore, the order of some components of the devices or some steps of the methods may be changed. Furthermore, other components or other steps may be inserted into some components of the devices or some steps of the methods.

Additionally, some components or some steps of the first embodiment of the present invention may be added to the second embodiment of the present invention, or some components or some steps of the second embodiment may be replaced.

In addition, the components shown in the embodiments of the present invention are independently depicted to represent different characteristic functions, and this does not mean that each component is composed of separate hardware or a single software component. That is, each component is described by listing each component for convenience of explanation, and at least two components among each component may be combined to form a single component, or a single component may be divided into multiple components to perform a function. Such integrated and separate embodiments of each component are also included in the scope of the present invention as long as they do not deviate from the essence of the present invention.

In this specification, a block can be variously expressed as a unit, an area, a unit, a partition, etc., and a sample can be variously expressed as a pixel, a pel, a pixel, etc.

Hereinafter, embodiments of the present invention will be described in more detail with reference to the attached drawings. In describing the present invention, duplicate descriptions of identical components will be omitted.

FIG. 1 is a block diagram showing an image encoding device according to the present disclosure.

Referring to FIG. 1, a video encoding device (100) may include a picture segmentation unit (110), a prediction unit (120, 125), a transformation unit (130), a quantization unit (135), a reordering unit (160), an entropy encoding unit (165), an inverse quantization unit (140), an inverse transformation unit (145), a filter unit (150), and a memory (155).

The picture segmentation unit (110) can segment the input picture into at least one processing unit. At this time, the processing unit may be a prediction unit (PU), a transform unit (TU), or a coding unit (CU). Hereinafter, in the embodiments of the present disclosure, the coding unit may be used to mean a unit that performs encoding or a unit that performs decoding.

A prediction unit may be divided into at least one square or rectangular shape of the same size within a single coding unit, or may be divided such that one prediction unit among the divided prediction units within a single coding unit has a different shape and/or size from another prediction unit. When a prediction unit that performs intra prediction based on a coding unit is generated and is not the minimum coding unit, intra prediction can be performed without being divided into a plurality of NxN prediction units.

The prediction unit (120, 125) may include an inter prediction unit (120) that performs inter prediction or inter prediction, and an intra prediction unit (125) that performs intra prediction or intra prediction. It may determine whether to use inter prediction or intra prediction for a prediction unit, and determine specific information (e.g., intra prediction mode, motion vector, reference picture, etc.) according to each prediction method. A residual value (residual block) between the generated prediction block and the original block may be input to the transformation unit (130). In addition, prediction mode information, motion vector information, etc. used for prediction may be encoded together with the residual value by the entropy encoding unit (165) and transmitted to the decoder.

The inter prediction unit (120) may predict a prediction unit based on information of at least one picture among the previous or subsequent pictures of the current picture, and in some cases, may predict a prediction unit based on information of a portion of an encoded region within the current picture. The inter prediction unit (120) may include a reference picture interpolation unit, a motion prediction unit, and a motion compensation unit.

The reference picture interpolation unit can receive reference picture information from the memory (155) and generate pixel information less than an integer pixel from the reference picture. In the case of luminance pixels, a DCT-based 8-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1/4 pixels. In the case of a chrominance signal, a DCT-based 4-tap interpolation filter with different filter coefficients can be used to generate pixel information less than an integer pixel in units of 1/8 pixels.

The motion prediction unit can perform motion prediction based on a reference picture interpolated by the reference picture interpolation unit. Various methods such as FBMA (Full search-based Block Matching Algorithm), TSS (Three Step Search), and NTS (New Three-Step Search Algorithm) can be used to derive a motion vector. The motion vector can have a motion vector value in units of 1/2 or 1/4 pixels based on the interpolated pixel. The motion prediction unit can predict the current prediction unit by using different motion prediction methods. Various methods such as Skip Mode, Merge Mode, AMVP Mode, Intra Block Copy Mode, and Affine Mode can be used as motion prediction methods.

The intra prediction unit (125) can generate a prediction unit based on reference pixel information surrounding the current block, which is pixel information within the current picture. If the surrounding block of the current prediction unit is a block on which inter prediction has been performed and the reference pixel is a pixel on which inter prediction has been performed, the reference pixel included in the block on which inter prediction has been performed can be replaced and used with reference pixel information of the surrounding block on which intra prediction has been performed. That is, if the reference pixel is not available, the unavailable reference pixel information can be replaced and used with at least one reference pixel among the available reference pixels.

Additionally, a residual block containing residual value information, which is the difference between the prediction unit that performed the prediction based on the prediction unit generated in the prediction unit (120, 125) and the original block of the prediction unit, can be generated. The generated residual block can be input to the transformation unit (130).

In the transformation unit (130), the residual block including the residual value information of the prediction unit generated through the original block and the prediction unit (120, 125) can be transformed using a transformation method such as DCT (Discrete Cosine Transform), DST (Discrete Sine Transform), or KLT. Whether to apply DCT, DST, or KLT to transform the residual block can be determined based on the intra prediction mode information of the prediction unit used to generate the residual block.

The quantization unit (135) can quantize values converted to the frequency domain by the transformation unit (130). The quantization coefficients can vary depending on the block or the importance of the image. The values produced by the quantization unit (135) can be provided to the dequantization unit (140) and the reordering unit (160).

The rearrangement unit (160) can perform rearrangement of coefficient values for quantized residual values.

The rearrangement unit (160) can change a two-dimensional block-shaped coefficient into a one-dimensional vector form through a coefficient scanning method. For example, the rearrangement unit (160) can change the two-dimensional block-shaped coefficient into a one-dimensional vector form by scanning from the DC coefficient to the coefficient of the high-frequency region using a zig-zag scan method. Depending on the size of the transformation unit and the intra prediction mode, a vertical scan that scans the two-dimensional block-shaped coefficient in the column direction or a horizontal scan that scans the two-dimensional block-shaped coefficient in the row direction may be used instead of the zig-zag scan. That is, depending on the size of the transformation unit and the intra prediction mode, it is possible to determine which scan method among the zig-zag scan, the vertical scan, and the horizontal scan is to be used.

The entropy encoding unit (165) can perform entropy encoding based on the values produced by the rearrangement unit (160). Entropy encoding can use various encoding methods such as, for example, Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC). In this regard, the entropy encoding unit (165) can encode residual value coefficient information of the encoding unit from the rearrangement unit (160) and the prediction units (120, 125). In addition, according to the present disclosure, it is possible to signal and transmit information indicating that motion information is derived and used on the decoder side and information on a technique used to derive motion information.

The inverse quantization unit (140) and the inverse transformation unit (145) inversely quantize the values quantized in the quantization unit (135) and inversely transform the values transformed in the transformation unit (130). The residual values generated in the inverse quantization unit (140) and the inverse transformation unit (145) can be combined with the predicted prediction units predicted through the motion estimation unit, motion compensation unit, and intra prediction unit included in the prediction unit (120, 125) to generate a reconstructed block.

The filter unit (150) may include at least one of a deblocking filter, an offset correction unit, and an ALF (Adaptive Loop Filter). The deblocking filter may remove block distortion caused by boundaries between blocks in a restored picture. The offset correction unit may correct the offset from the original image on a pixel-by-pixel basis for the image on which deblocking has been performed. In order to perform offset correction for a specific picture, a method may be used in which the pixels included in the image are divided into a certain number of regions, the regions to be offset are determined, and the offset is applied to the regions, or the offset is applied by considering edge information of each pixel. The ALF (Adaptive Loop Filtering) may be performed based on a value obtained by comparing the filtered restored image with the original image. After dividing the pixels included in the image into a predetermined group, one filter to be applied to the group is determined, and filtering may be performed differentially for each group.

The memory (155) can store a restored block or picture produced through the filter unit (150), and the stored restored block or picture can be provided to the prediction unit (120, 125) when performing inter prediction.

FIG. 2 is a block diagram showing an image decoding device according to the present disclosure.

Referring to FIG. 2, the image decoding device (200) may include an entropy decoding unit (210), a rearrangement unit (215), an inverse quantization unit (220), an inverse transformation unit (225), a prediction unit (230, 235), a filter unit (240), and a memory (245).

When a video bitstream is input to a video encoding device, the input bitstream can be decoded in the opposite procedure to that of the video encoding device.

The entropy decoding unit (210) can perform entropy decoding in a procedure opposite to that of the entropy encoding unit of the video encoder. For example, various methods such as Exponential Golomb, Context-Adaptive Variable Length Coding (CAVLC), and Context-Adaptive Binary Arithmetic Coding (CABAC) can be applied in response to the method performed in the video encoder.

The entropy decoding unit (210) can decode information related to intra prediction and inter prediction performed in the encoder.

The reordering unit (215) can perform reordering based on the method by which the bitstream entropy-decoded by the entropy decoding unit (210) is reordered by the encoding unit. The coefficients expressed in the form of a one-dimensional vector can be reordered by restoring them back to coefficients in the form of a two-dimensional block.

The inverse quantization unit (220) can perform inverse quantization based on the quantization parameters provided by the encoder and the coefficient values of the rearranged block.

The inverse transform unit (225) can perform inverse transform, i.e., inverse DCT, inverse DST, and inverse KLT, on the transforms performed by the transform unit, i.e., DCT, DST, and KLT, on the quantization result performed by the image encoder. The inverse transform can be performed based on the transmission unit determined by the image encoder. In the inverse transform unit (225) of the image decoder, a transform technique (e.g., DCT, DST, KLT) can be selectively performed according to a plurality of pieces of information, such as a prediction method, the size of the current block, and the prediction direction.

The prediction unit (230, 235) can generate a prediction block based on the prediction block generation related information provided by the entropy decoding unit (210) and the previously decoded block or picture information provided by the memory (245).

As described above, when performing intra prediction or intra prediction in the same manner as the operation in the image encoder, if the size of the prediction unit and the size of the transformation unit are the same, intra prediction for the prediction unit is performed based on the pixels on the left side of the prediction unit, the pixels on the upper left side, and the pixels on the upper side. However, when performing intra prediction, if the size of the prediction unit and the size of the transformation unit are different, intra prediction can be performed using reference pixels based on the transformation unit. In addition, intra prediction using NxN division only for the minimum coding unit can be used.

The prediction unit (230, 235) may include a prediction unit determination unit, an inter prediction unit, and an intra prediction unit. The prediction unit determination unit may receive various information such as prediction unit information input from the entropy decoding unit (210), prediction mode information of an intra prediction method, and motion prediction-related information of an inter prediction method, and may distinguish a prediction unit from a current encoding unit and determine whether the prediction unit performs inter prediction or intra prediction. On the other hand, if the encoder (100) does not transmit motion prediction-related information for the inter prediction, but instead transmits information indicating that motion information is to be derived and used on the decoder side and information on a technique used to derive motion information, the prediction unit determination unit determines whether the inter prediction unit (230) performs prediction based on the information transmitted from the encoder (100).

The inter prediction unit (230) can perform inter prediction on the current prediction unit based on information included in at least one picture among the previous picture or the subsequent picture of the current picture including the current prediction unit, using information required for inter prediction of the current prediction unit provided by the image encoder. In order to perform inter prediction, it can be determined based on the encoding unit whether the motion prediction method of the prediction unit included in the corresponding encoding unit is one of Skip Mode, Merge Mode, AMVP Mode, Intra Block Copy Mode, and Affine Mode.

The intra prediction unit (235) can generate a prediction block based on pixel information within the current picture. If the prediction unit is a prediction unit that has performed intra prediction, intra prediction can be performed based on intra prediction mode information of the prediction unit provided by the image encoder.

The intra prediction unit (235) may include an Adaptive Intra Smoothing (AIS) filter, a reference pixel interpolation unit, and a DC filter. The AIS filter is a unit that performs filtering on the reference pixels of the current block and can determine whether to apply the filter based on the prediction mode of the current prediction unit and apply it. AIS filtering can be performed on the reference pixels of the current block using the prediction mode and AIS filter information of the prediction unit provided by the image encoder. If the prediction mode of the current block is a mode that does not perform AIS filtering, the AIS filter may not be applied.

The reference pixel interpolation unit can interpolate the reference pixel to generate a reference pixel of a pixel unit less than an integer value when the prediction mode of the prediction unit is a prediction unit that performs intra prediction based on the pixel value interpolated from the reference pixel. When the prediction mode of the current prediction unit is a prediction mode that generates a prediction block without interpolating the reference pixel, the reference pixel may not be interpolated. The DC filter can generate a prediction block through filtering when the prediction mode of the current block is the DC mode.

The restored block or picture may be provided to a filter unit (240). The filter unit (240) may include a deblocking filter, an offset correction unit, and an ALF.

Information about whether a deblocking filter has been applied to a corresponding block or picture can be received from a video encoding device, and if a deblocking filter has been applied, information about whether a strong or weak filter has been applied. The deblocking filter of the video decoder can receive information related to the deblocking filter provided by the video encoder, and the video decoder can perform deblocking filtering on the corresponding block.

The offset correction unit can perform offset correction on the restored image based on the type of offset correction applied to the image during encoding and information on the offset value. ALF can be applied to the encoding unit based on information on whether ALF is applied and ALF coefficient information provided from the encoder. This ALF information can be provided by being included in a specific parameter set.

The memory (245) can store a restored picture or block so that it can be used as a reference picture or reference block, and can also provide the restored picture to an output unit.

The present disclosure relates to a prediction method based on Template-based Intra Mode Deriation (TIMD). TIMD may be a method for calculating a cost for a given intra prediction mode based on a template of a current block, and deriving an optimal intra prediction mode based on the calculated cost. Here, a predicted value of the template may be generated based on the given intra prediction mode, and a difference (e.g., SAD, SATD) between the predicted value of the template and a reconstructed value may be defined as the cost. The given intra prediction mode may be an intra prediction mode of a candidate belonging to a given candidate list, and the candidate may be derived based on one or more blocks belonging to a pre-decoded region prior to the current block.

Based on TIMD, one or more intra prediction modes can be derived. If one intra prediction mode is derived, the prediction block of the current block can be generated based on that intra prediction mode. Alternatively, if two or more intra prediction modes are derived, two or more prediction blocks can be generated based on the two or more intra prediction modes, and the final prediction block can be generated through a weighted sum of the two or more prediction blocks.

The left and/or top surrounding areas adjacent to the current block can be used as templates. If the left and/or top surrounding areas extend beyond the picture boundary, the surrounding areas may not be used as templates. For example, if the left surrounding area extends beyond the picture boundary but the top surrounding area does not, the top surrounding area may be used as a template. Alternatively, if the top surrounding area extends beyond the picture boundary but the left surrounding area does not, the left surrounding area may be used as a template. If the left and top surrounding areas extend beyond the picture boundary, the intra prediction mode of the current block may be set to a predefined mode (e.g., Planar mode).

The upper peripheral area constituting the template can be defined as a WxN area, and the left peripheral area constituting the template can be defined as a NxH area. Here, W can be equal to or less than the width of the current block. However, the present invention is not limited thereto, and the W of the template can be set to be greater than the width of the current block. H can be equal to or less than the height of the current block. However, the present invention is not limited thereto, and the H of the template can be set to be greater than the height of the current block. The size (N) of the template can be a value predefined for the encoder and decoder. For example, the predefined value can be an integer of 1, 2, 3, 4, or more. Alternatively, the size (N) of the template can be variable based on the size of the current block. For example, when the width and height of the current block are less than or equal to 8, the size (N) of the template can be 2, and otherwise, the size (N) of the template can be 4. Alternatively, if the current block is a 4x16 block, the upper peripheral region constituting the template may be a 4x4 region, and the left peripheral region constituting the template may be a 2x16 region. Alternatively, if the current block is a 16x4 block, the upper peripheral region constituting the template may be a 16x2 region, and the left peripheral region constituting the template may be a 4x4 region.

Below, with reference to Fig. 3, we will examine a method for constructing a candidate list and a method for generating a prediction block based on the same.

Figure 3 illustrates a TIMD-based prediction method according to the present disclosure.

Referring to FIG. 3, a candidate list including multiple candidates for the current block can be constructed (S300).

Example 1: Regular mode

Candidates in the candidate list according to the present disclosure can be derived based on the intra prediction modes of neighboring blocks adjacent to the current block. Here, the neighboring blocks may include at least one of a left neighboring block, an upper neighboring block, an upper-left neighboring block, a lower-left neighboring block, or an upper-right neighboring block. The candidate list and candidates according to Embodiment 1 may also be referred to as an MPM list and an MPM candidate.

If the DC mode, horizontal mode, and/or vertical mode are not present in the pre-configured candidate list, these modes may be added to the candidate list. The candidate list may consist of a minimum of 22 candidates and a maximum of 25 candidates.

Example 2: Merge Mode

Candidates in the candidate list according to the present disclosure can be derived based on surrounding blocks to which TIMD is applied. Each candidate may have at least one of the intra prediction modes derived based on TIMD or weighting information for weighted summation between predicted blocks. The candidate list and candidates according to Embodiment 2 may also be referred to as a TIMD merge list and merge candidates.

The above-mentioned neighboring blocks may include at least one of adjacent blocks or non-adjacent blocks of the current block. The adjacent blocks may include at least one of an upper neighboring block, a left neighboring block, a lower left neighboring block, an upper right neighboring block, or an upper left neighboring block. A non-adjacent block may refer to a block encoded before the current block and not adjacent to the current block. For example, the positions of non-adjacent blocks available to the current block may be defined as shown in FIG. 4. The numbers in FIG. 4 may indicate the priority order of blocks referenced to construct a candidate list.

A TIMD merge list can have at most N merge candidates, where N can be an integer greater than or equal to 1. For example, a TIMD merge list can be structured as shown in Table 1 below.

candidate Intra prediction mode Weight information 1 (IPM _1,1 , IPM _1,2 ) (W _1,1 , W _1,2 ) 2 (IPM _2.1 , IPM _2.2 ) (W _2,1 , W _2,2 ) 3 (IPM _3.1 , IPM _3.2 ) (W _3,1 , W _3,2 ) ... ... ... 10 (IPM _10.1 , IPM _10.2 ) (W _10.1 , W _10.2 )

Example 3: Combination Mode 1

The candidate list according to the present disclosure may include candidates derived based on the Regular mode and candidates derived based on the Merge mode. This may be referred to as a combination list, and the combination list may have M candidates, where M may be an integer greater than or equal to 1. The candidates based on the Regular mode and the candidates based on the Merge mode have been previously discussed, and any duplicate description will be omitted here.

For example, the candidate list according to the combination mode can be structured as shown in Table 2 below.

candidate Intra prediction mode Weight information 1 (IPM _1,1 , IPM _1,2 ) (W _1,1 , W _1,2 ) 2 (IPM _2.1 , IPM _2.2 ) (W _2,1 , W _2,2 ) 3 (IPM _3.1 , IPM _3.2 ) (W _3,1 , W _3,2 ) ... ... ... 10 (IPM _10.1 , IPM _10.2 ) (W _10.1 , W _10.2 ) 11 (IPM _11.1 , IPM _11.2 ) (W _11,1 , W _11,2 )

In Table 2, candidates 1 to 10 may be derived based on the merge mode, and candidate 11 may be derived based on the regular mode. Candidate 11 may be a combination of the top two MPM candidates in ascending order of cost. That is, IPM _11,1 may represent the intra prediction mode of the first MPM candidate in ascending order of cost, and IPM _11,2 may represent the intra prediction mode of the other MPM candidate. In addition, the weight information of candidate 11 may be derived based on the cost of IPM _11,1 and the cost of IPM _11,2 , and for example, this may be derived as shown in Equation 2 described below.

Example 4: MIP mode

A candidate list according to the present disclosure may include multiple intra prediction modes for matrix-based intra prediction (MIP). Here, the matrix may be a matrix trained on an arbitrary data set. The matrix may have a size of NxM, where N and M may each be integers greater than or equal to 1. The matrix for MIP may be selected explicitly or implicitly.

The reference region for matrix operations can have a size equal to the width (Width) x N of the current block or the height (Height) x N of the current block. N can be an integer greater than or equal to 1. The number of reference sample lines (M) that constitute the reference region can be variable. M can be an integer greater than or equal to 1.

Example 5: Combination Mode 2

The candidate list according to the present disclosure may include a candidate based on the aforementioned MIP mode and a candidate based on at least one of Embodiments 1 to 2.

The encoder and decoder may define one or more TIMD modes. Here, the one or more TIMD modes may include at least one of the aforementioned Regular mode, Merge mode, Combination mode 1, MIP mode, or Combination mode 2. When multiple TIMD modes are defined, the encoder may determine an optimal mode among the multiple TIMD modes and encode an index for specifying the optimal mode into the bitstream. The decoder may determine the TIMD mode of the current block based on the corresponding index signaled through the bitstream.

The cost can be calculated for each candidate in the candidate list (S310).

For example, a prediction value for the template of the current block can be generated based on the intra prediction mode of each candidate. The difference (e.g., SAD, SATD) between the generated prediction value and the restored value of the template can be calculated. A cost can be derived based on the calculated difference.

Alternatively, each candidate may have two or more intra prediction modes. For convenience of explanation, it is assumed that each candidate has two intra prediction modes. In this case, the first difference between the predicted value of the template and the reconstructed value can be calculated based on one of the two intra prediction modes (IPM ₁ ), and the second difference between the predicted value of the template and the reconstructed value can be calculated based on the other (IPM ₂ ). The cost of the candidate can be calculated by a weighted sum of the first and second differences. Here, the weights for the weighted sum can be set based on the weight information of the candidate. For example, the cost of each candidate can be calculated as in the following mathematical equation (1).

[Mathematical Formula 1]

Cost = (cost1 * W1) + (cost2 * W2) >> shift

In mathematical expression 1, cost1 may represent the first difference between the predicted value and the restored value of the template calculated based on IPM _1. cost2 may represent the second difference between the predicted value and the restored value of the template calculated based on IPM _2. W1 is weight information stored in the candidate, and may represent the weight applied to the prediction block generated based on IPM _1. W2 is weight information stored in the candidate, and may represent the weight applied to the prediction block generated based on IPM ₂ .

Alternatively, weights can be applied to calculate costs for candidates based on merge mode, while costs for candidates based on regular mode can be calculated without weights. For example, the cost for a candidate based on regular mode can be calculated as the sum of the costs calculated for each intra prediction mode.

Alternatively, costs can be calculated without applying weights to both merge-mode and regular-mode candidates. For example, the cost of each candidate can be calculated as the sum of the costs calculated for each intra prediction mode.

A prediction block of the current block can be generated based on the calculated cost (S320).

Specifically, the top K candidates can be selected in ascending order of their calculated costs. The candidates within the candidate list can also be reordered in ascending order of their calculated costs. Prediction blocks can be generated based on the intra-prediction modes of the selected candidates. Here, K can be an integer of 1, 2, or a larger number.

For example, if one candidate is selected from the candidate list, the prediction block for the current block can be generated based on the intra prediction mode of that candidate. If the selected candidate has two intra prediction modes, two prediction blocks can be generated based on the two intra prediction modes. The final prediction block can also be generated by weighting the two prediction blocks based on the weight information of the selected candidate.

For example, if two candidates are selected from a candidate list, two prediction blocks can be generated based on the intra prediction modes of the two candidates, and a final prediction block can be generated through a weighted sum of the two generated prediction blocks. The weights for the weighted sum can be derived based on the costs of the selected candidates, as shown in the following mathematical expression 2. In this case, the final prediction block can be generated as shown in the following mathematical expression 3.

[Equation 2]

[Equation 3]

In mathematical expressions 2 and 3, M1 and M2 may denote intra prediction modes corresponding to the top two candidates in ascending order of cost.

However, if, as an exception, the cost of the candidate (M1) with the smallest cost among the top two candidates is determined to be significantly smaller than the cost of the other candidate (M2), the prediction block of the current block may be generated using only the candidate (M1) with the smallest cost. For example, if the condition of mathematical expression 4 is satisfied, the cost of M1 may be determined to be significantly smaller than the cost of M2. In this case, a weight of 1 may be applied to the prediction block (Pred _M1 ) generated based on M1, and a weight of 0 may be applied to the prediction block (Pred _M2 ) generated based on M2.

[Equation 4]

According to the method described above, one full-RD process can be performed in the encoder using the final derived prediction block.

Alternatively, one of the top two candidates in ascending order of the calculated cost can be selected, and a prediction block for the current block can be generated based on the selected candidate. For example, two prediction blocks can be generated based on the two intra prediction modes (IPM ₁ and IPM ₂ ) of the selected candidate, and the final prediction block can be generated by weighting the two prediction blocks based on the weight information of the selected candidate.

To this end, the encoder can select the optimal candidate among the top two candidates and encode an index to specify it into the bitstream. The decoder can then select one of the top two candidates based on the index signaled through the bitstream.

Alternatively, if the cost of the second candidate in ascending order within the candidate list is greater than a predetermined threshold, the predicted block for the current block can be generated based on the candidate with the smallest cost (i.e., the first candidate) among the top two candidates in ascending order. Here, the candidate list may refer to a TIMD merge list or a combination list. In this case, the signaling of an index for specifying one of the top two candidates may be omitted.

Here, the threshold can be derived by applying an arbitrary weight to the value of the smallest cost within the candidate list. The weight can be a real number greater than or equal to 1. For example, if the value of the smallest cost is assumed to be 1000, the threshold can be derived through operations such as (1000*1.2) or (1000+1.2).

Alternatively, the threshold may be derived based on the costs of candidates in the MPM list. For example, the threshold may be derived by applying a specific weight to the cost of the top candidate in ascending order of cost within the MPM list. The threshold may be derived by applying a specific weight to the average of the costs of the top two candidates in ascending order of cost within the MPM list. The threshold may be derived by applying a specific weight to the cost of the top one candidate in descending order of cost within the MPM list. The threshold may be derived by applying a specific weight to the average of the costs of the top two candidates in descending order of cost within the MPM list. The specific weight here may be a real number greater than or equal to 1.

Alternatively, the threshold can be derived by applying a specific weight to the cost of the merge mode-based candidate without referencing the Regular mode-based candidate. For example, within the merge mode-based candidate list, the threshold can be derived by applying a specific weight to the cost of the first candidate in ascending cost order. The weight can be a real number greater than or equal to 1. For example, the weight can be 1.2. Within the candidate list, if the cost of the second candidate in ascending cost order is greater than the predetermined threshold, the prediction block of the current block can be generated based on the first candidate. In this case, the signaling of the index for specifying either of the top two candidates in ascending cost order within the merge mode-based candidate list can be omitted.

Alternatively, the top N candidates can be selected by sorting the MIP mode-based candidates in the candidate list in ascending order of cost. Furthermore, the top M candidates can be selected by sorting the remaining candidates in the candidate list in ascending order of cost. Here, N and M can be integers greater than or equal to 1. N and M can be the same value or different values. In this case, the final prediction block of the current block can be generated by a weighted sum of the prediction blocks generated based on the top N candidates and the prediction blocks generated based on the top M candidates.

FIG. 5 illustrates a matrix-based intra prediction method according to the present disclosure.

Matrix-based intra prediction (MIP) may be a method of generating a prediction block by applying a pre-defined matrix (or weight) to a reference region adjacent to the current block.

The following mathematical expression 5 is an example of generating prediction samples through MIP.

[Equation 5]

In Equation 5, r(k) represents a reference sample, F(x,y) represents a matrix applied to the reference sample, and P(x,y) may represent a prediction sample. k may represent an index of a reference sample within the reference region. The reference sample may be a pre-reconstructed sample within the reference region, or may be derived through a combination of two or more pre-reconstructed samples. Some prediction samples within the current block can be generated through MIP, and the remaining prediction samples within the current block can be generated by performing linear interpolation based on these.

A reference region for MIP may include one or more sample lines. For example, the reference region may include at least one of an adjacent sample line or a non-adjacent sample line of the current block. Here, the non-adjacent sample line may be a sample line that is N samples away from the boundary of the current block. N may be an integer greater than or equal to 1, and the number of non-adjacent sample lines constituting the reference region may be 1 or more.

A candidate list for the current block can be constructed (S500).

The method of constructing the candidate list is as described with reference to Fig. 3.

An extended candidate list can be generated based on the candidate list (S510).

An extended candidate list can be generated by combining candidates (i.e., intra prediction modes) in the candidate list with two or more reference sample lines adjacent to the current block. Here, the reference sample line may include at least one of a reference sample line adjacent to the current block (reference sample line 0) or K consecutive reference sample lines adjacent to reference sample line 0 (reference sample lines 1 to K). For example, all of the reference sample lines 0 to K may be used, or some of the reference sample lines 0 to K may be selectively used.

For example, for convenience of explanation, it is assumed that the candidate list of the current block is structured as shown in Table 3 below, and sample lines 1, 3, 5, 7, and 12 are used to construct the extended candidate list. The candidate numbers in Table 3 may represent the numbers of the intra prediction modes.

Index candidate 0 39 1 38 2 40 3 37 ... ... 9 5

At this time, a first extended candidate list consisting of 50 combinations can be generated as shown in Table 4 below through a combination between the intra prediction mode of the candidate list and the reference sample line.

Index Reference sample line index candidate 0 1 39 1 1 38 2 1 40 3 1 37 ... ... ... 9 1 5 10 1 39 11 1 38 12 1 40 13 1 37 ... ... ... 19 1 5 20 1 39 21 1 38 22 1 40 23 1 37 ... ... ... 29 1 5 30 1 39 31 1 38 32 1 40 33 1 37 ... ... ... 39 1 5 40 1 39 41 1 38 42 1 40 43 1 37 ... ... ... 49 1 5

For each of the 50 combinations in the first extended candidate list, intra prediction can be performed on a given template region based on that combination to generate a predicted value for that template region. The cost (e.g., SAD, SATD) for that combination can be calculated based on the difference between the predicted value for the template region and the base-reconstructed value.

Among the 50 combinations, the top M combinations can be selected in ascending order of the calculated costs, and a second expanded candidate list can be generated based on the selected top M combinations. The top M combinations can be arranged in the second expanded candidate list in ascending order of costs.

Alternatively, combinations with the same intra prediction mode can be eliminated through a redundancy check among the M combinations that make up the second extended candidate list. For example, among combinations with the same intra prediction mode, all combinations except the one with the lowest cost can be eliminated from the second extended candidate list.

A reference area for MIP can be determined based on the extended candidate list (S520).

An index for specifying at least one of the combinations constituting the aforementioned extended candidate list (i.e., the first or second extended candidate list) may be signaled via the bitstream. A reference region for MIP may be determined based on a reference sample line of the combination corresponding to the index.

For example, if the second extended candidate list consists of 20 combinations, an index for specifying at least one of the 20 combinations can be encoded based on Truncated Golomb-Rice Coding. The index can be encoded into up to six bins to which context coding is applied. For example, the binarization of the index can be defined as shown in Table 5 below.

Index empty string (prefix) empty string (suffix) 0 0 00 1 0 01 2 0 10 3 0 11 4 10 00 ... ... ... 18 1111 10 19 1111 11

Alternatively, combinations within the first or second extended candidate list may be grouped into multiple groups. Each group may consist of one or more combinations. For example, T consecutive combinations within the first or second extended candidate list may be designated as a group. Here, T may be an integer of 2, 3, 4, 5, or more.

For each combination in each group, it can be checked whether the cost of the combination exceeds a predetermined threshold. Combinations with a cost greater than the predetermined threshold can be excluded from the group. The excluded combination can be used as the combination corresponding to the first index of the next group. Here, the threshold can be derived based on the cost of the combination corresponding to the first index within the group (i.e., the combination with the lowest cost). For example, the threshold can be set to a value equal to 2 times the cost of the combination corresponding to the first index within the group.

Using the aforementioned method, combinations of the second extended candidate list can be grouped into multiple groups, and a reference region for MIP can be determined based on any one of the multiple groups. For example, a reference region for MIP can be determined based on a reference sample line of a combination belonging to any of the selected groups.

A prediction block of the current block can be generated based on the reference area (S530).

Meanwhile, the template area for generating the extended candidate list may include reference sample line 0. Alternatively, the template area may include reference sample lines other than reference sample line 0. The template area may be composed of one or more reference sample lines.

For example, a template area can be set by extending from reference sample line 0 to consecutive reference sample lines.

For example, the area corresponding to reference sample lines 0 and 1 may be set as a template area. In this case, an extended candidate list may be generated based on reference sample lines 3, 5, 7, and 12.

For example, multiple reference sample lines including reference sample lines 1, 3, 5, 7, and 12 can be set as template areas.

For example, if the width and height of the current block are W and H, the width of the top template of the current block may be less than or equal to W, and the height of the left template of the current block may be less than or equal to H. However, this is not limited thereto, and the width of the top template of the current block may be greater than W, and the height of the left template of the current block may be greater than H.

For example, an extended candidate list can be generated without using the top or left template, based on the structure of the current block. Here, the structure can refer to the current block's position within a picture, slice, tile, or CTU row, the shape of the current block, or the partitioning structure of the current block.

For example, if the width and height of the current block are W and H, and W is N times or more than H, only the top template can be used. N can be a real number greater than or equal to 1.

For example, if the width and height of the current block are W and H, and H is N times or more greater than W, only the left template can be used. N can be a real number greater than or equal to 1.

An extended candidate list may be generated based on all or part of the configuration method of the template area described above, and a reference area for the MIP may be determined.

In addition, the above-described prediction method can be equally applied to an image encoding device and an image decoding device.

The various embodiments of the present disclosure are not intended to list all possible combinations but rather to illustrate representative aspects of the present disclosure, and the matters described in the various embodiments may be applied independently or in combinations of two or more.

Additionally, various embodiments of the present disclosure may be implemented by hardware, firmware, software, or a combination thereof. In the case of hardware implementation, the embodiments may be implemented by one or more ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), DSPDs (Digital Signal Processing Devices), PLDs (Programmable Logic Devices), FPGAs (Field Programmable Gate Arrays), general processors, controllers, microcontrollers, microprocessors, etc.

The scope of the present disclosure includes software or machine-executable instructions (e.g., operating systems, applications, firmware, programs, etc.) that cause operations according to the methods of various embodiments to be executed on a device or a computer, and a non-transitory computer-readable medium having such software or instructions stored thereon and executable on the device or computer.

Claims (10)

A step of constructing a candidate list including multiple candidates for the current block; A step of calculating a cost for each candidate in the above candidate list; and Including a step of generating a prediction block of the current block based on the above cost, A method for decoding an image, wherein the above candidate list includes at least one of a candidate derived based on a regular mode or a candidate derived based on a merge mode. In the first paragraph, A method for decoding an image, wherein a candidate derived based on the above Regular mode is derived based on an intra prediction mode of a neighboring block adjacent to the current block. In the first paragraph, A method for decoding an image, wherein a candidate derived based on the above merge mode is derived based on a surrounding block to which TIMD (template-based intra mode derivation) is applied. In the first paragraph, The above cost is calculated based on the difference between the predicted value and the restored value of the template of the current block, A method for decoding an image, wherein the predicted value of the template of the current block is generated based on the intra prediction mode of each candidate. In the first paragraph, A video decoding method in which the top two candidates are selected from the above candidate list in ascending order of the calculated cost. In paragraph 5, Based on the intra prediction modes of the two candidates above, two prediction blocks are generated respectively. A video decoding method in which a prediction block of the current block is generated based on a weighted sum of the above two prediction blocks. In paragraph 6, A method for decoding an image, wherein the weights of the above weighted sum are derived based on the costs corresponding to the two candidates. In paragraph 6, An image decoding method in which, if the candidate with the smallest cost among the two candidates above is significantly smaller than the cost of the other candidates, the prediction block of the current block is generated using only the candidate with the smallest cost. A step of constructing a candidate list including multiple candidates for the current block; A step of calculating a cost for each candidate in the above candidate list; and Including a step of generating a prediction block of the current block based on the above cost, A video encoding method, wherein the above candidate list includes at least one of a candidate derived based on a regular mode or a candidate derived based on a merge mode. A computer-readable storage medium for storing a bitstream generated by an image encoding method according to Article 9.